Computing as a Service in the Cloud Computing Stack: 1st Revisit

After the previous post about "A Speculation of Computing as a Service in the Cloud Computing Stack", I keep a close observation in the blogosphere to see if anyone in the world has similar ideas on what I proposed in the speculation. Two articles that I found today shed similar lights to the speculation: "The HPC Software Conundrum" and "Engineering Parallel Applications with Tunable Architectures" . The former article says

"... multi-core and streaming-core needs software in a big way. And, there is no sense in bragging about how well the hardware works unless there is an adequate software base to drive the hardware to its intended performance."

The author further lists out a number of solutions that are available today for programmers to use the great hardware and they are OpenMP, CUDA, OpenCL and Ct. However, the author points out that none of these solutions actually tell the programmers how should the code be updated/rewritten using this new tools, "no one size fits all or no silver bullet" as the author would say. He also says

"I still get the feel that it is a kludge of some sort and there should be a way to express what you want to do in a parallel cluster environment using a clear concise standard level of abstraction."

in which I agree with him. There is a need for a layer of abstraction that takes care of the hardware, be it SMP multi-core with or without GPU architecture. The layer of abstraction should be intelligent enough to make use of all the available hardware in the machine to improve the utilization.

The second article suggests a new approach to address the difficulty in adapting a serial program to a parallel program on the new hardware architecture. The new approach will automate the architecture adaptation of parallel programs and uses an auto-tuner to find the best-performing architectural choice for a given machine by introducing a new architecture description language based on parallel patterns and a framework for expressing architecture variants in a generic way. This is a step toward multi-core virtualization.

I wish to see more contributions on the software side to bring the programmers to this multi-core era!

A Speculation of Computing as a Service in the Cloud Computing Stack

Borrowing from Wikipedia, Computing is usually defined as the activity of using and improving computer technology, computer hardware and software. It is clear that both computer hardware and software are important to the development of Computing, but the pace of their development is significant different. Due to the physical limits of the technology, for the past decade, the Chip industry has shifted their focus of developing a single powerful CPU from one single processor to many microprocessor cores. While the development of multi-core processor is underway, there is a new paradigm in Computing (in terms of computer hardware) and that is the GPU. Although the concept is similar (i.e. they are both based on the multi-core architecture), the underlying hardware is fundamentally different and it is this difference which allows them to serve different needs in the application (i.e. some applications are better to be executed on CPU and some are better to be executed on GPU). In order to make the new computer hardware useful, the industry is now looking for better software to take advantage of the new processing powers they offer.

Currently, many applications fail to take advantage of the multi-core architecture because it requires developers to learn a new set of skills in order to make use of them. Programming on the multi-core architecture is challenging for many developers because it requires the developers to think in "parallel" and to handle many pitfalls inherited from parallel programming that are not present in serial programming. Not to mention that developing applications for multi-core CPU is different than multi-core GPU and therefore, it requires them 2X of the effort to take the full advantage of what is available today in the computer hardware. Many universities have recognized the gap between the development of the computer hardware and the computer software and have decided to teach the required skill for programming parallel processors in school and a new textbook has been published recently to support this movement. Personally, I really think this is a good move and I would love to see more universities to engage in this movement. However, I would like to argue that this is not going to help much for the general public unless the movement focuses on solving the real pain point in the Computing world.

The real pain point is that developers should spend most of their development time on delivering business values rather than working with the underlying hardware. Developers are looking for a better way which can facilitate them to develop software on the multi-core architecture without a steep learning curve on the underlying hardware. In fact, software engineers experienced a similar problem in the past and they found a solution for it. The solution is well-known in the Engineering domain and it is called "Virtualization". Virtualization has been used in many places. For example, by virtualizing operating system, a physical machine can be sliced into multiple virtual machines which can then allow to deploy several applications on the same physical machines. I think "Virtualization" can also be applied on the multi-core architecture so that multi-core applications can be developed with "less" pain. It is also a logical and economical way to make use of the available resources. With a group of experts specialized in the multi-core virtualization development, a large population of developers can make use of the multi-core architecture to build high-responsive applications.

One solution available today, that I know of, approaches the concept of multi-core virtualization and it is called Ct Technology. Code can be written once using the development platform provided by Ct Technology, then run in parallel on any of the processors that the Ct Technology Platform supports. The current supported processors are Cell BE, GPUs, and CPUs. I'm hoping to see more of this kind of technology in the future because I believe that it revolutionalizes the computing industry. It abstracts the complexity of the hardware from the application code, freeing the developers to implement applications that matter to the user.

In the future, as blogged in DevCentral, virtualization is becoming the most disruptive infrastructure technology. Especially when Cloud Computing takes off, virtualization will become even more important than one can imagine. Therefore, I'm positive that one day, we will have something called "Computing as a Service" available in the Cloud Computing stack and everyone can access it without knowing what is the underlying computer hardware to execute the actual instructions (be it Cell, GPU or Molecules).

References:

2. My Mental Image of the Technological Landscape for the next 10 Years

Design Pattern: GigaSpaces Messaging for Apama

One of the nice things about GigaSpaces is that heterogeneous independent applications can make use of GigaSpaces as a high performance fault-tolerant messaging hub. It has support for JMS in which applications can create topics and queues for exchanging messages. In this post, I would like to describe a design pattern that I have used for the design of a GigaSpaces Messaging for Apama.

In one of my current projects, I need to design a high performance, highly-available and scalable market data system (MDS) which is used by many different types of trading systems in a bank. We chose GigaSpaces as the middleware to build the MDS due to its simplicity; it provides messaging capability, high availability, scalability and performance in one single platform which simplifies a lot of development overhead, deployment issues and maintenance efforts. One of the requirements of the MDS is to integrate with Progress Apama. Apama provides a robust integration framework which facilitates the integration with many different event sources into Apama. Apama offer many popular connections out-of-the-box such as JMS, TIBCO Rendezvous, just to name a few. I decide to use JMS as the messaging transport to stream data from GigaSpaces to Apama. This requires us to develop a GigaSpaces JMS channel adapter for Apama.

GigaSpaces channel adapter

The GigaSpaces JMS channel adapter acts as a messaging client to the MDS and invokes Appama functions via the supplied interface. The adapter in this case is unidirectional since data is only streamed in one direction (from GigaSpaces MDS to Apama). GigaSpaces JMS already provides all the necessary mechanics to build the channel adapter so that Apama can interact with it similar to a JMS provider. Before the message can be delivered to Apama, we need to convert the market data format into Apama specific message. Therefore, a message translator is designed. This way, we decouple the adapter from the Apama internal structure.

Another implementation consideration of the channel adapter design is about transaction semantics. Since the message is streamed from the MDS to the message channel specific for Apama, we need to ensure that the message is delivered to Apama even during the event of partial failure. GigaSpaces JMS provides transactions so that message consumers can receive messages transactionally through the JMS session. We decide to stick with local transactions because it is more efficient. Hence, we will need to design a message channel that receives all messages that are relevant to Apama from GigaSpaces MDS.

GigaSpaces Message Channel

Once we have the GigaSpaces JMS channel adapter ready, we need to build a message channel where the channel adapter can consume market data from. The market data are fed into a GigaSpaces IMDG with a primary backup partitioning topology. This means that the market data are spread across multiple machines. The market data are partitioned based on the stock symbol (Content-Based Routing). This topology is essential for a high performance load-balanced MDS. Unfortunately, we can't use the market data IMDG directly as the message channel for Apama. As noted in the GigaSpaces documentation, GigaSpaces JMS does not support destination partitioning. Currently, all messages of a JMS destination are routed to the same partition. The partition where the messages go is resolved according to the index field of the message which, currently, is the destination name. In other words, we can't use the market data IMDG directly as the message channel for Apama because the market data are partitioned across multiple machines. In order to allow Apama to consume JMS message, we create another space which is uniquely designed for Apama (Apama Space Message Channel). This space will store only JMS market data messages that Apama is interested. In order to reduce the number of network hops for delivering market data to Apama, the Apama space message channel is built-in to the GigaSpaces channel adapter. To deliver a market data of interest from the Market Data IMDG to the Apama space message channel, a messaging bridge is designed. Basically, When a message is delivered on the market data IMDG, the bridge consumes the message of interest and sends another with the same contents on the Apama space message channel. The notify event container is implemented in the bridge to capture all updates for the market data of interest and the market data POJO message is transformed into a JMS message before it is written into the Apama space message channel. Note that in order to guarantee that the notification of a new update is received, the Market data IMDG will send the notification at least once in case of failover. This is fine for us as the trading algorithm is idempotent to the duplicate market data message.

With the current design, new applications which are interested in the market data messages can tab into the MDS in similar manner without affecting other existing applications. Applications are loosely-coupled by using GigaSpaces as a high performance message hub.