Virtualization Technology News and Information
In-Memory Computing: A Key Enabler of Digital Transformation
Article Written by Nikita Ivanov, founder and CTO, GridGain Systems 

Many companies are undergoing a "digital transformation" as they deploy new technologies to implement revolutionary changes to their business processes and dramatically improve the experience of their end users. Digital transformation takes many forms, but some common trends include the creation of web-scale applications for external or internal use cases, the shift towards software-as-a-service (SaaS) business models, and the deployment of connected devices for Internet of Things (IoT) use cases.

Web-Scale Applications and SaaS

To improve customer experience, companies are integrating and performing real-time analysis on sales data, website clicks, social media streams, help desk activity, vehicle usage data, wearables and more to gain a better understanding of customer needs, tendencies and preferences. The goal is to be able to respond in real-time to user activities in ways that are an immediate win-win for the company and the customer. In retail and ecommerce, this capability may drive improved decision making related to pricing, inventory, recommendations, advertising and more. In B2B use cases, it can help sales teams optimize customer engagements in order to increase loyalty and improve upsell opportunities.

In the transition to service-based business models, thousands of companies, including icons like Netflix, Salesforce and Uber, have taken radically different approaches to their offerings. They have created web-scale applications that deliver their solutions using a modern technology infrastructure that must ensure 24/7 availability and high performance while enabling unlimited scalability.

For both of these trends, success depends on the ability to simultaneously transact and analyze huge amounts of data with extreme speed and scale. A number of technology approaches are currently being employed to analyze and respond to the incoming data including streaming analytics, machine learning and artificial intelligence, advanced modeling and scenario analysis, and more. The performance of these technologies, however, can suffer if they rely on disk-based storage. The slowest part of any computing platform, disk-based storage introduces unavoidable latency as data is written to and read from the disk.

The Benefits of In-Memory Computing

Today, the most effective and least obtrusive way to eliminate latency and dramatically improve application performance is by deploying an in-memory computing (IMC) platform. IMC platforms offer ACID transaction support and massive parallel processing across a distributed computing cluster. The IMC platform is inserted between the application and data layers. The data in the underlying RDBMS, NoSQL or Hadoop database also resides in the RAM of the distributed IMC platform cluster, delivering a tremendous performance boost and extreme scalability to the application.

With an IMC platform, the total system RAM can be increased by adding new nodes to the cluster. Distributed processing offers redundancy and massive scalability without compromising performance. When new nodes are added to the IMC cluster, the system can automatically rebalance the data across the nodes, adding the processing power and RAM of the new nodes to seamlessly add to the existing cluster capacity. Since utilization of the new memory is automatic, scaling the platform ahead of demand is hassle-free.

Companies that use IMC platforms can see a 1,000x or more increase in OLTP and OLAP processing speeds compared to applications built directly on disk-based databases. In many cases, IMC platforms are able to support hundreds of millions of transactions per second while ensuring high availability.

In-memory computing is also more cost-effective than ever. Memory costs have dropped roughly 30 percent per year since the 1960s, and it is now only slightly more expensive than disk-based storage. When we factor in a tremendous increase in performance, easy scalability, dramatic improvements in customer experience, and creating a foundation for all future digital transformation initiatives, IMC offers an extraordinary value proposition.

Advancing IoT and More with In-Memory Computing and HTAP

In-memory computing will also play a critical role in the evolution of IoT. With relatively low-cost memory and simple scalability, an IMC platform can power hybrid transactional and analytical processing (HTAP) use cases. HTAP is the ability to perform analytics on transactional data without impacting the performance of the transactional system, on one unified OLTP and OLAP database. This application of "fast data" can drive a new generation of ground-breaking IoT applications that rely on analyzing sensor data to enable immediate responses to changing real-world conditions, such as for smart cities, energy distribution, weather tracking, healthcare and much more. HTAP can also provide tremendous technical and cost advantages for other applications where real-time decisions based on incoming transactions are required, such as reacting in real-time to sales and inventory data, patient and security monitoring, routing driverless cars, and monitoring manufacturing equipment and processes.

The Role of Open Source Software

Another aspect of the role of in-memory computing in digital transformation is that most of the innovative development of IMC platforms is taking place using the open source model. Today's enterprises increasingly want greater control over their own destinies. They reject vendor lock-in and want input on the roadmaps of the technologies they use. As a result, they are embracing open source for even mission-critical applications.

The importance of this cannot be overstated. Digital transformation is ultimately about technology changes that enable a far more personal and nuanced experience. Achieving these polar goals via the traditional vendor model - with slow development cycles and a fundamental aversion to the risks associated with "transformation" - is doubtful. The community approach of open source fuels rapid innovation and is at the core of every major area of disruptive change. And with its ability to dramatically improve the performance of data platforms, in-memory computing is now the foundation upon which future digital transformation will be built.


About the Author

Nikita Ivanov

Nikita Ivanov is founder and CTO of GridGain Systems, started in 2007 and funded by RTP Ventures and Almaz Capital. Nikita has led GridGain to develop advanced and distributed in-memory data processing technologies - the top in-memory data fabric starting two times per second every day around the world.

Nikita has over 20 years of experience in software application development, building HPC and middleware platforms, contributing to the efforts of other startups and notable companies including Adaptec, Visa and BEA Systems. Nikita was one of the pioneers in using Java technology for server side middleware development while working for one of Europe's largest system integrators in 1996.

He is an active member of Java middleware community, contributor to the Java specification, and holds a Master's degree in Electro Mechanics from Baltic State Technical University, Saint Petersburg, Russia.

Published Wednesday, June 14, 2017 7:52 AM by David Marshall
Filed under: ,
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<June 2017>