My Thought on Extreme Transaction Processing (XTP) in Financial Services

Today, I have a discussion with a customer through emails regarding Extreme Transaction Processing (XTP) system for Order Management System (OMS) using GigaSpaces XAP. I found the discussion interesting and would like to also share it with you all. The discussion was about measuring system performance in XTP which usually involves Latency and Throughput. Here is the detail in the email:

Although latency is an important performance metric for an OMS, we also need to consider other aspects such as the system throughput. Measuring independently the latency and the throughput using different test cases will not reflect the performance of the system in the real world as the test case might aim to optimize the performance of a specific system parameter (in this case the latency) by trading off other important aspects of performance (such as the throughput and the ACID properties). In optimization theory, the optimization complexity increases with the number of variable in the system. In the real-world application, this complexity arises frequently and GigaSpaces does a good job, if not the best, to solve this problem.

We believe that the single-threaded approach to optimize out the object locking in order to shave off any possible latency will actually impact the throughput of the system as this approach can only update ONE order information at a time; limiting the system concurrency and utilization. Also, the single-threaded approach DOES NOT satisfy all the ACID properties even in this simple test case which affects the system reliability.

In fact, GigaSpaces can achieve the same latency using the single-threaded approach (i.e. polling container with single consumer in GigaSpaces). It can achieve even lower latency using embedded space. Also, in addition to the single-threaded approach, GigaSpaces provides standard transaction support. It has several implementation of Spring's PlatformTransactionManager which allow user to utilize Spring's rich support for declarative transaction management (which is reliable and standard), instead of coding an in-house transaction manager which might be error-prone and complex.

In reality, there will be more than 1 application using the XTP. The denominator for all these applications is the same which boils down to data consistency. It is very easy to achieve weak consistency in which many XTP solutions can/only provide. However, for some applications, strong consistency is a must. Therefore, we need to evaluate an enterprise XTP solution from a broader perspective and how much flexibility that it can provide in order to achieve the desired performance, manageability and security. When the usage of a XTP solution is beyond the basic and many different applications rely on it, these functionalities are not just "nice-to-have" but essential foundations for any enterprise application.

At the end of the day, what we would like to achieve in Extreme Transaction Processing is to keep latency low and throughput high while the processing (the business logic) is done transactionally (i.e. to provide ACID guarantee). Therefore, the performance of a XTP should be measured against all 3 properties as a whole in order to get a better idea of the capability of the solution.