Has your legacy
system been worn down by many business applications?
By Fred Chien, Chief System Architect, Brobridge
Does your company have serious performance problems with the
legacy system under the traditional SOA architecture? In addition to being
quite inefficient in cross-system data exchange and query between different
business systems, is it even complained by end users that the data update is
too slow and the delay is too high? More and more external systems require the
data you have, causing your database systems to burn up, which in turn affects
normal business tasks? Then you find that expanding the size of the database
and expanding the hardware still cannot solve the problem?
This has not been mentioned yet, whenever there is data
access across business systems, different responsible teams always set up
shields, and are afraid that the other parties will crash their own system when
they demand a large amount of data. No one is willing to take the
responsibility for system crashes, so everyone has a hard time, and what
ultimately hurts is the user experience of the application and the efficiency
of business execution.
What is even more annoying is that the situation over the
years was barely able to survive, but with the advent of the digital age, the
growth of the applications and the integration of a large number of digital
systems have caused major enterprises to experience unprecedented pain.
In the end, how can we save these legacy systems painlessly?
Isolate the legacy system to avoid heavy data
access pressure
A large amount of external system access is equivalent to
attacking the databases of the legacy system
The reason why the legacy system is difficult to support the
development of new applications is because the application data pressure is
concentrated on the database systems. However, in the design of the legacy
system in the past, the high-concurrency and huge query requirements may not
have been considered originally, which resulted in the system not being
designed for this from the beginning. Even all query performance depends
entirely on the performance of the database system.
When external applications are created one by one, the
demand for access increases rapidly, causing the legacy system to bear the
pressure of data query from all directions. The original limited capacity
database system collapses or the performance becomes extremely poor, even the
original business can't be executed.
Introduce data mesh technology to completely isolate the
legacy system from the application
Therefore, the key is how to isolate and protect the
database of the legacy system so that it is not affected by external
application access requirements.
Expand the data supply capacity without
changing the legacy system
The data supply capacity of the legacy system must be
expanded to meet many needs
The legacy system database with limited capacity simply
cannot handle such a large number of queries and data exchange loading when
many external application services demand data from the legacy system. This
means that we must improve the ability of the legacy system to provide data to
meet the various needs of these external application services.
Amplify the data supply capacity through the data mesh
solution to supply huge external application demands
Because the legacy system has a long history or high
complexity, it is not easy to modify the system programs or database structure,
so we must find a way to expand the data supply capacity without changing the
legacy system.
Do not use files for data exchange, because of
the loss of performance and not real-time
Cross-system data exchange or synchronization through
files seriously hurts database performance and with high latency
Using files to exchange or transfer data in an expanded
system is an alternative move. This approach allows the development and
maintenance personnel of different systems to have relatively little
responsibility. The team of the legacy system does not have to worry about the
external system directly accessing the database, nor be afraid of the problem
of system crash caused by excessive concurrent queries.
However, this method usually executes batch processing at
regular intervals and downloads data to files, so that the real-time
performance is often not high. When batch processing involves more complex
queries and high data volume, it will also hurt the current performance of the
database and cause serious system loading.
Some companies do this work once every few minutes or once
every one or two days long, and try to arrange it during off-peak business
hours to avoid performance impact. But this approach often makes external
business application users complain that the data update speed is too slow (for
example, it takes another day to check new orders).
Using data mesh technology, various data can be instantly
transferred to other data nodes or databases
The use of data mesh technology can instantly transfer data
without affecting different systems, or even merge data from multiple sources,
which greatly reduces the work of data customization processing. More
importantly, there is no need to be afraid of the responsibility of system
crashes and ineffectiveness.
How to introduce data mesh technology? Is it a
big project?
Data mesh needs to meet the infrastructure
Data mesh requires many automated data processing
mechanisms. In addition to realizing data virtualization, data routing and
relaying, data caching, and data provisioning, it also needs to implement a
management mechanism for data processing rules. If you want to piece together
or implement it yourself, it is time-consuming and labor-intensive.
Having said that, if you choose a mature data mesh platform,
the actual application of the introduction is not a big project, and there is
no need to change the program architecture of the existing systems. You only
need to build a set of data management infrastructure of data mesh, and then
set up a few data mesh processing pipelines, you can immediately enhance and
protect the data access performance of existing legacy systems.
The basic theory may seem complicated, but in fact it is
extremely easy to import and use.
---
Have more interest in Data Mesh?
The relevant information and materials in this article are from the Brobridge
Gravity data platform solution. This product is used to solve enterprise
cross-system data exchange, microservice architecture data supply and AIoT data
platform. It assists IT personnel and developers to achieve the optimal data
exchange architecture, and to meet the needs of all types of digital
transformation. If you also have related questions or needs, you are welcomed
to contact us at Brobridge.
##
To learn more about
cloud native technology innovation, join us at KubeCon + CloudNativeCon Europe 2021 - Virtual, which will
take place from May 4-7.
ABOUT THE AUTHOR
Fred Chien (錢逢祥), Chief System Architect,
Brobridge
Fred is currently the chief architect of Brobridge, an
open source developer who has been obsessed with the latest technology for many
years, the core founder of Hackathon Taiwan, and the well-known desktop
environment project LXDE (Raspberry Pi presets initiated by Taiwan Desktop
environment) core developer. He was starting business, working as a consultant
and developing products in major companies, he cross-domain from the bottom to
the end of the application, involved in Web, Internet and mobile phone
applications, Linux system applications, drivers and embedded systems. In
response to the advent of the massive cloud digital era, online game
applications, IoT systems, blockchain technology, and data system design have
been invested in recent years. Currently focusing on product development and design
in the financial system industry, various cloud technologies, and microservice
architecture, helping to lower the threshold for companies and application
developers to design high-throughput and large-scale systems.
With the rise of microservice architecture, he started to publish articles on
the Brobridge Blog(https://medium.com/brobridge)
from time to time to share relevant experience in microservice architecture,
and hope that interested parties can communicate together.
Photo by JOHN TOWNER on Unsplash