1. Apache Hadoop
The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.The project includes these modules:
- Hadoop Common: The common utilities that support the other Hadoop modules.
- Hadoop Distributed File System : A distributed file system that provides high-throughput access to application data.
- Hadoop YARN: A framework for job scheduling and cluster resource management.
- Hadoop MapReduce: A YARN-based system for parallel processing of large data sets.
Other Hadoop-related projects at Apache include:
Ambari: A web-based tool for provisioning, managing, and monitoring Apache Hadoop clusters which includes support for Hadoop HDFS, Hadoop MapReduce, Hive, HCatalog, HBase, ZooKeeper, Oozie, Pig and Sqoop. Ambari also provides a dashboard for viewing cluster health such as heatmaps and ability to view MapReduce, Pig and Hive applications visually alongwith features to diagnose their performance characteristics in a user-friendly manner.
Avro: A data serialization system.
Cassandra: A scalable multi-master database with no single points of failure.
Chukwa: A data collection system for managing large distributed systems.
HBase: A scalable, distributed database that supports structured data storage for large tables.
Hive: A data warehouse infrastructure that provides data summarization and ad hoc querying.
Mahout: A Scalable machine learning and data mining library.
Pig: A high-level data-flow language and execution framework for parallel computation.
ZooKeeper: A high-performance coordination service for distributed applications.
2. Software-defined networks
One of key value propositions of SDN is the network abstraction that in turn allows for the kind of network programmability that accelerates automation.If network capabilities could be properly abstracted, IT administrators could design networks and define policies in IT-centric and application-friendly terms. Once designed, they could apply them many times, and enforce appropriate policies throughout their datacenter assets.
SDN does just that by instantiating connectivity in line with policies.
SDN also provides the foundation for services chaining, a method of programmatically linking networking functions as they are needed, allowing developers to incorporate networking functions directly into applications
Data center networks have grown calcified over time. While servers and storage have benefited from software abstractions, networks have remained hardware-bound and static, making them a major roadblock to cloud computing. Enter SDN (software-defined networking), which drapes a software layer over switch and router hardware to serve as both a centrally managed control plane and a platform for innovation.
SDN isn't network virtualization; rather it is a way to "program the network" -- that is, it allows cloud providers and ISVs to build new networking capabilities the rest of us can draw on. The leading example of SDN today is OpenFlow, the brainchild of university researchers who wanted to experiment with new network protocols on large production networks.
3. Distributed storage tiering
Vastly faster than disk and many times cheaper than DRAM, NAND flash memory is a hot commodity that will heat up further when storage management software catches up with its potential in the data center. Its combination of high speed and low cost makes it excellent for server-side cache and a natural choice for tier-one SAN storage.With the cost of flash dropping and the capacities of SSDs on the rise, the days of disk drives in servers and SANs appear to be numbered. The best part: Flash storage will enable server-side storage to be managed as an extension of the SAN, storing the most frequently accessed or I/O-intensive data closer to the app. It's like caching, but smarter and more cost-effective.
4. JavaScript replacements
JavaScript is often used in a way that was never intended: as a platform for the development of large Web applications that are hosted in the browser. If that had been its intended use -- rather than simply for adding simple dynamism to Web pages - it would almost certainly have been designed differently.
That's because there are two main problems with the language when creating large Web applications.
First, there's the performance problem: There's no getting around the fact that large applications written in JavaScript are relatively slow, and that impacts the end users of the applications.
Second, there's the problem of the structure of JavaScript itself: It's a language that's difficult for large teams to develop in. That's because it lacks structure and strong ties between modules, and because it's hard to communicate the intent of the code within the code itself. For a solo developer that's not necessarily a problem, but for a team of developers working on a large project, communicating the intent of the code is key -- especially over time as the original developers move on.
Large applications are usually also modular, and different people are responsible for developing different pieces. But JavaScript is dynamic, and the behavior of objects can change over time. That's very different to what's the case with C or C++, or even Java or C#, and it means you generally need to execute your code to verify that it works.
JavaScript may be the most commonly executed code on the planet, but for all its success, everyone is moving on to the next thing. Some want to build entirely new languages that fix all of the troubles with JavaScript; others just want to translate their code into JavaScript, so they can pretend they don't use it.
Translated code is all the rage. Google's Web Toolkit cross-compiles Java into JavaScript so the developer types only properly typed Java code. Some translations are cosmetic: Programmers who write in CoffeeScript don't need to worry about much of JavaScript's punctuation because the cross-compiler inserts it before it runs. Other translations, such as Google's Dart, are more ambitious, pointing to a future of ever more options.
5. Trust on a chip (The TPM)
The Trusted Platform Module (TPM) Work Group has been chartered to create the TPM specification.The definition of the TPM architecture comes from the Technical Committee and the TPM Work Group defines the implementation of that architecture. A working knowledge of of security in relation to the design and usage of cryptographic modules as well as cryptographic techniques including public-key cryptography, cryptographic algorithms and protocols is recommended.Assuring security at the highest application level requires verification at every layer, including the physical construction of the computing device. Enter trust on a chip.
The TPM (Trusted Platform Module) from the TCG (Trusted Computing Group) was the first popularly adopted hardware chip to assure trusted hardware and boot sequences. Last year, Intel combined the TPM chip and a hardware hypervisor layer to protect boot sequences, memory, and other components. Any software vendor can take advantage of it. Hardware trust solutions aren't perfectly secure, as the Princeton memory freeze and electron microscope attacks showed, but they beat software-only protection solutions. The hardware protection schemes will only get better. Soon enough, every computer device you can use will have a hardware/software protection solution running.
6. Client-side hypervisors
Talk about virtualisation often centres on virtually hosted desktops, in which the entire desktop is run on a back-end server. But this is by no means the only way to operate.Amid all the other options, such as application virtualisation and offloading computational tasks onto rich clients, there is one model that is altogether different: client-side virtualisation.
Many users want the features offered by different operating systems. Perhaps your users’ favourite office apps are Mac-based but one integrated development environment is available only on Windows.
Forensics experts might be happy with Windows but may need some vital tools that are available only with BackTrack, the ethical hacker’s Ubuntu distro. Developers may need to have access to IE6 in addition to IE9.
Desktop virtualization faltered for two reasons: It requires continuous connection between client and server, as well as a beefy server to run desktop VMs. Client hypervisors solve both problems. Install one on an ordinary machine and leverage the processing power of the client. Laptop users can take a "business VM" with them containing the OS, apps, and personal configuration settings.
That VM is secure and separate from whatever else may be running on that machine -including malware accidentally downloaded and you get all the virtualization management advantages, such as VM snapshots, portability, and easy recovery. Client hypervisors point to a future where we bring our own computers to work and download or sync our business VMs to start the day.
7. Advanced synchronization
Advanced synchronization features include various options to manage bandwidth and store updates. Apple and Microsoft agree: It's time to say good-bye to single-user environments, where each device is a separate island from the rest of the user's computing world. Apple paved the way with iOS and iCloud, introducing a cloud-based syncing service across devices. Microsoft's Windows 8 takes the concept further, keeping not just data but application state in sync.This shift will drastically change how people work on computers, giving applications dramatic new utility. Automatic data syncing coupled with context, such as location, available input methods, connectivity, and sensor-driven data, will give rise to truly user-centric computing, profoundly altering how IT approaches applications, security models, and other tech policies and strategies centered on user productivity.
8. HTML5
HTML5 is the ubiquitous platform for the web. Whether you're a mobile web developer, an enterprise with specific business needs, or a serious game dev looking to explore the web as a new platform, HTML5 has something for you! Choose your path:Naysayers point out that we've been putting tags together to form Web pages since the beginning of the World Wide Web. HTML5 has simply added new ones. But while HTML5 looks similar to old-fashioned HTML, the tasks it accomplishes are dramatically different. Local data storage, make it possible to do much more than pour words and images into a rectangle. Plus, the new HTML5 WebSockets spec defines a new way to conduct full-duplex communication for event-driven Web apps. And with Adobe's decision to end development of mobile Flash, an entire sector of the Web development industry is going to retool as we move to HTML5 from Flash -- a tectonic shift for Web developers.
9. Private Cloud Optimization
Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (eg, networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.With a private cloud, IT managers can borrow technologies pioneered by public cloud providers and apply them to their own data center. These clouds have many moving parts -- virtualization management, chargeback systems, self-service provisioning -- hence the need for orchestration. Open source project OpenStack has gained considerable momentum offering a core set of cloud orchestration services. Eucalyptus is another alternative, offering essentially a private cloud implementation of Amazon Web Services.
It may be easy to be cynical about any technology attached to the term "cloud," but no one questions the benefits of pooling resources for greater economies of scale. Paradigm changes demand new ways of working -- and the emerging collection of cloud orchestration software supplies the means.
10. Continuous build tools
The more collaboratively minded developers among us like the way continuous build tools like Jenkins, Hudson, and other "continuous integration" servers help us work together for the betterment of the whole. These tools put all code through a continuous stream of tests, alerting developers about problems with code they checked in some 10 seconds ago, keeping everybody moving toward the same goal.
Tools like Hudson or Jenkins aren't new -- there have been a number of slick proprietary continuous integration tools for some time, but the emergence of open source solutions encourages the kind of experimentation and innovation that comes when programmers are given the chance to make their tools better.