Virtualization Technology News and Information
Article
RSS
Data Mesh Saves Your Legacy System!

 

Has your legacy system been worn down by many business applications?

By Fred Chien, Chief System Architect, Brobridge

Does your company have serious performance problems with the legacy system under the traditional SOA architecture? In addition to being quite inefficient in cross-system data exchange and query between different business systems, is it even complained by end users that the data update is too slow and the delay is too high? More and more external systems require the data you have, causing your database systems to burn up, which in turn affects normal business tasks? Then you find that expanding the size of the database and expanding the hardware still cannot solve the problem?

This has not been mentioned yet, whenever there is data access across business systems, different responsible teams always set up shields, and are afraid that the other parties will crash their own system when they demand a large amount of data. No one is willing to take the responsibility for system crashes, so everyone has a hard time, and what ultimately hurts is the user experience of the application and the efficiency of business execution.

What is even more annoying is that the situation over the years was barely able to survive, but with the advent of the digital age, the growth of the applications and the integration of a large number of digital systems have caused major enterprises to experience unprecedented pain.

In the end, how can we save these legacy systems painlessly?

Isolate the legacy system to avoid heavy data access pressure

 

A large amount of external system access is equivalent to attacking the databases of the legacy system

The reason why the legacy system is difficult to support the development of new applications is because the application data pressure is concentrated on the database systems. However, in the design of the legacy system in the past, the high-concurrency and huge query requirements may not have been considered originally, which resulted in the system not being designed for this from the beginning. Even all query performance depends entirely on the performance of the database system.

When external applications are created one by one, the demand for access increases rapidly, causing the legacy system to bear the pressure of data query from all directions. The original limited capacity database system collapses or the performance becomes extremely poor, even the original business can't be executed.

 

Introduce data mesh technology to completely isolate the legacy system from the application

Therefore, the key is how to isolate and protect the database of the legacy system so that it is not affected by external application access requirements.

Expand the data supply capacity without changing the legacy system

 

The data supply capacity of the legacy system must be expanded to meet many needs

The legacy system database with limited capacity simply cannot handle such a large number of queries and data exchange loading when many external application services demand data from the legacy system. This means that we must improve the ability of the legacy system to provide data to meet the various needs of these external application services.

 

Amplify the data supply capacity through the data mesh solution to supply huge external application demands

Because the legacy system has a long history or high complexity, it is not easy to modify the system programs or database structure, so we must find a way to expand the data supply capacity without changing the legacy system.

Do not use files for data exchange, because of the loss of performance and not real-time

 

Cross-system data exchange or synchronization through files seriously hurts database performance and with high latency

Using files to exchange or transfer data in an expanded system is an alternative move. This approach allows the development and maintenance personnel of different systems to have relatively little responsibility. The team of the legacy system does not have to worry about the external system directly accessing the database, nor be afraid of the problem of system crash caused by excessive concurrent queries.

However, this method usually executes batch processing at regular intervals and downloads data to files, so that the real-time performance is often not high. When batch processing involves more complex queries and high data volume, it will also hurt the current performance of the database and cause serious system loading.

Some companies do this work once every few minutes or once every one or two days long, and try to arrange it during off-peak business hours to avoid performance impact. But this approach often makes external business application users complain that the data update speed is too slow (for example, it takes another day to check new orders).

 

Using data mesh technology, various data can be instantly transferred to other data nodes or databases

The use of data mesh technology can instantly transfer data without affecting different systems, or even merge data from multiple sources, which greatly reduces the work of data customization processing. More importantly, there is no need to be afraid of the responsibility of system crashes and ineffectiveness.

How to introduce data mesh technology? Is it a big project?

 

Data mesh needs to meet the infrastructure

Data mesh requires many automated data processing mechanisms. In addition to realizing data virtualization, data routing and relaying, data caching, and data provisioning, it also needs to implement a management mechanism for data processing rules. If you want to piece together or implement it yourself, it is time-consuming and labor-intensive.

Having said that, if you choose a mature data mesh platform, the actual application of the introduction is not a big project, and there is no need to change the program architecture of the existing systems. You only need to build a set of data management infrastructure of data mesh, and then set up a few data mesh processing pipelines, you can immediately enhance and protect the data access performance of existing legacy systems.

 

The basic theory may seem complicated, but in fact it is extremely easy to import and use.

---

Have more interest in Data Mesh? The relevant information and materials in this article are from the Brobridge Gravity data platform solution. This product is used to solve enterprise cross-system data exchange, microservice architecture data supply and AIoT data platform. It assists IT personnel and developers to achieve the optimal data exchange architecture, and to meet the needs of all types of digital transformation. If you also have related questions or needs, you are welcomed to contact us at Brobridge.

##

To learn more about cloud native technology innovation, join us at KubeCon + CloudNativeCon Europe 2021 - Virtual, which will take place from May 4-7.    

ABOUT THE AUTHOR

Fred Chien (錢逢祥), Chief System Architect, Brobridge 

Fred Chien 

Fred is currently the chief architect of Brobridge, an open source developer who has been obsessed with the latest technology for many years, the core founder of Hackathon Taiwan, and the well-known desktop environment project LXDE (Raspberry Pi presets initiated by Taiwan Desktop environment) core developer. He was starting business, working as a consultant and developing products in major companies, he cross-domain from the bottom to the end of the application, involved in Web, Internet and mobile phone applications, Linux system applications, drivers and embedded systems. In response to the advent of the massive cloud digital era, online game applications, IoT systems, blockchain technology, and data system design have been invested in recent years. Currently focusing on product development and design in the financial system industry, various cloud technologies, and microservice architecture, helping to lower the threshold for companies and application developers to design high-throughput and large-scale systems.

With the rise of microservice architecture, he started to publish articles on the Brobridge Blog(https://medium.com/brobridge) from time to time to share relevant experience in microservice architecture, and hope that interested parties can communicate together.

 

Photo by JOHN TOWNER on Unsplash 

Published Monday, April 26, 2021 7:30 AM by David Marshall
Filed under: ,
Comments
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
top25
Calendar
<April 2021>
SuMoTuWeThFrSa
28293031123
45678910
11121314151617
18192021222324
2526272829301
2345678