Article Written by Nikita Ivanov,
founder and CTO, GridGain Systems
Many companies are undergoing a "digital transformation" as
they deploy new technologies to implement revolutionary changes to their
business processes and dramatically improve the experience of their end users. Digital
transformation takes many forms, but some common trends include the creation of
web-scale applications for external or internal use cases, the shift towards software-as-a-service
(SaaS) business models, and the deployment of connected devices for Internet of
Things (IoT) use cases.
Web-Scale
Applications and SaaS
To improve customer experience, companies are integrating
and performing real-time analysis on sales data, website clicks, social media
streams, help desk activity, vehicle usage data, wearables and more to gain a
better understanding of customer needs, tendencies and preferences. The goal is
to be able to respond in real-time to user activities in ways that are an
immediate win-win for the company and the customer. In retail and ecommerce,
this capability may drive improved decision making related to pricing, inventory,
recommendations, advertising and more. In B2B use cases, it can help sales
teams optimize customer engagements in order to increase loyalty and improve
upsell opportunities.
In the transition to service-based business models,
thousands of companies, including icons like Netflix, Salesforce and Uber, have
taken radically different approaches to their offerings. They have created
web-scale applications that deliver their solutions using a modern technology
infrastructure that must ensure 24/7 availability and high performance while
enabling unlimited scalability.
For both of these trends, success depends on the ability to simultaneously
transact and analyze huge amounts of data with extreme speed and scale. A
number of technology approaches are currently being employed to analyze and
respond to the incoming data including streaming analytics, machine learning
and artificial intelligence, advanced modeling and scenario analysis, and more.
The performance of these technologies, however, can suffer if they rely on
disk-based storage. The slowest part of any computing platform, disk-based
storage introduces unavoidable latency as data is written to and read from the disk.
The Benefits of In-Memory
Computing
Today, the most effective and least obtrusive way to
eliminate latency and dramatically improve application performance is by
deploying an in-memory computing (IMC) platform. IMC platforms offer ACID
transaction support and massive parallel processing across a distributed
computing cluster. The IMC platform is inserted between the application and data
layers. The data in the underlying RDBMS, NoSQL or Hadoop database also resides
in the RAM of the distributed IMC platform cluster, delivering a tremendous performance
boost and extreme scalability to the application.
With an IMC platform, the total system RAM can be increased
by adding new nodes to the cluster. Distributed processing offers redundancy
and massive scalability without compromising performance. When new nodes are added to
the IMC cluster, the system can automatically rebalance the data across the
nodes, adding the processing power and RAM of the new nodes to seamlessly add
to the existing cluster capacity. Since utilization of the new memory is
automatic, scaling the platform ahead of demand is hassle-free.
Companies that use IMC platforms can see a 1,000x or more
increase in OLTP and OLAP processing speeds compared to applications built
directly on disk-based databases. In many cases, IMC platforms are able to
support hundreds of millions of transactions per second while ensuring high
availability.
In-memory computing is also more cost-effective than ever.
Memory costs have dropped roughly 30 percent per year since the 1960s, and it is
now only slightly more expensive than disk-based storage. When we factor in a tremendous
increase in performance, easy scalability, dramatic improvements in customer
experience, and creating a foundation for all future digital transformation
initiatives, IMC offers an extraordinary value proposition.
Advancing IoT and
More with In-Memory Computing and HTAP
In-memory computing will also play a critical role in the
evolution of IoT. With relatively low-cost memory and simple scalability, an IMC
platform can power hybrid
transactional and analytical processing (HTAP) use cases. HTAP is the ability
to perform analytics on transactional data without impacting the performance of
the transactional system, on one unified OLTP and OLAP database. This application
of "fast data" can drive a new generation of ground-breaking IoT applications
that rely on analyzing sensor data to enable immediate responses to changing
real-world conditions, such as for smart cities, energy distribution, weather
tracking, healthcare and much more. HTAP can also provide tremendous technical
and cost advantages for other applications where real-time decisions based on
incoming transactions are required, such as reacting in real-time to sales and
inventory data, patient and security monitoring, routing driverless cars, and
monitoring manufacturing equipment and processes.
The Role of Open
Source Software
Another
aspect of the role of in-memory computing in digital transformation is that most
of the innovative development of IMC platforms is taking place using the open
source model. Today's enterprises increasingly want greater control over their
own destinies. They reject vendor lock-in and want input on the roadmaps of the
technologies they use. As a result, they are embracing open source for even
mission-critical applications.
The importance of this cannot be overstated. Digital
transformation is ultimately about technology changes that enable a far more
personal and nuanced experience. Achieving these polar goals via the
traditional vendor model - with slow development cycles and a fundamental aversion
to the risks associated with "transformation" - is doubtful. The community
approach of open source fuels rapid innovation and is at the core of every
major area of disruptive change. And with its ability to dramatically improve
the performance of data platforms, in-memory computing is now the foundation
upon which future digital transformation will be built.
##
About the Author

Nikita Ivanov is founder and CTO of GridGain Systems, started in 2007 and funded by RTP Ventures and Almaz Capital. Nikita has led GridGain to develop advanced and distributed in-memory data processing technologies - the top in-memory data fabric starting two times per second every day around the world.
Nikita has over 20 years of experience in software application development, building HPC and middleware platforms, contributing to the efforts of other startups and notable companies including Adaptec, Visa and BEA Systems. Nikita was one of the pioneers in using Java technology for server side middleware development while working for one of Europe's largest system integrators in 1996.
He is an active member of Java middleware community, contributor to the Java specification, and holds a Master's degree in Electro Mechanics from Baltic State Technical University, Saint Petersburg, Russia.