By: Lewis Carr, Senior Director of Marketing at Actian

Enterprises are using the increasing amounts of data to generate richer and more timely insights for better business outcomes. In fact, an IDC model projects that the global datasphere will roughly triple in size by 2025, offering business leaders a plethora of business intelligence at their fingertips.

While the term ‘big data' has been around for a while, the difference this time is where the data will be generated and sourced, and how fluid it will be. Mobile and IoT, or the edge, will drive a signifigant portion of data creation, and processing and analysis will happen at various points at the gateways, from on device, and across the cloud, flowing in largely from the network's edge. Perhaps big data is no longer the right term, and ‘fluid distributed data' would be a better fit.

To fully take advantage of the increasing amounts of data available to them, businesses need a way to manage it more efficiently across platforms, from the edge to the cloud and back. They need to process, store, and optimize various types of data coming from different sources with different levels of cleanliness, value, and validity to connect it to internal applications and apply business process logic, increasingly aided by artificial intelligence and machine learning models.

Enterprises are looking to tackle this challenge by adopting a new solution - a data fabric. As data volumes continue to grow at the network's edge, this solution will grow further into what will be known as an ‘edge data fabric.'

A data fabric acts as both the plumbing and translator for data moving onto and off different platforms, including public and private clouds, data centers, and the many types of devices and gateways operating at the edge. It allows us to easily and transparently access data distributed across different areas - in real time in a unifying data layer, under the same management - and enables operators to move and access data across different data processes, deployment platforms, structural approaches, and geographical locations.

Data fabric and the edge

Edge computing provides a unique set of challenges for data being generated and processed outside the network core, and the devices themselves operating at the edge are becoming increasingly more powerful, networked, and complex. While most processing used to take place in an enterprise's data center, now a larger portion is in the virtual cloud data center; in either case, with exception of front-end client server thick client applications and office productivity and collaboration, the vast majority of business processing happens on one side of a gateway.

Cloud is about fluidity and removing locality, but like the fixed data center, it's also about processing data associated with applications far from the point of action. By point of action, I mean the factory floor, the retail storefront, and the dock where cargo is being shipped and received, for example. We may not care where a particular cloud is located, but we do care that our data must transit between various clouds and persist in each of them for use in different operations.  With the introduction of more powerful networks like 5G and new standards for bringing Cloud architectures directly to the network's edge like Multi-Access Edge Computing or MEC, it's time to rethink how much computing is done on the other side of the gateway, at the edge.

This added level of complexity requires organizations to determine which pieces of the processing are done at which level. There's an application for each, and for each application there's a manipulation. For each manipulation, there's processing of data and memory management.

A data fabric handles essentially all the data management complexity, and the edge is starting to become a new cloud, leveraging the same technologies and standards along with edge-specific networks, such as 5G and WLAN 6. Each device and gateway offers richer, more intelligent applications, and offers the equivalent of what would have been a data center running on a factory floor, on a cargo ship, or in an airplane. It stands to reason you will need an analogous edge data fabric to the one that is solidifying in the core cloud.

Key elements of an edge data fabric

An edge data fabric must perform several important functions to take on the growing number of data requirements posed by edge devices, including:

• Run on multiple operating environments: Most importantly POSIX compliant.
• Access many different interfaces: http, mttp, radio networks, manufacturing and other industry-specific networks.
• Handle streaming data: Through standards such as Spark and Kafka.
• Provide JDBC/ODBC database connectivity: For legacy applications and a quick and dirty connection between databases.
• Work with key protocols and APIs: Including more recent ones with REST API.
  

Edge data fabric is at a turning point

The key drivers for edge computing have changed since its origin in content delivery networks back in the 1990s, and we're reaching a market tipping point for an edge data fabric.

To truly harness all the intelligence and processing done at the edge, its crucial to shed the client-server mentality. The days of single-location - physical or virtual - data centralization are gone; most data is going to stay at the edge. More intelligence at the edge means directly executing automated routines and allowing machine learning to run in an unsupervised fashion at the edge.

You may be asking: Why not move it all into the cloud? Here's why:

• Bandwidth: It would take too much to move back to the cloud. Every jump from 2G to 3G to 4G to LTE to 5G comes with an accompanying bandwidth surge, and with each bandwidth surge, you can do less and less in the cloud. You keep getting more data outpacing the new bandwidth - call it the bandwidth paradox.
• Latency: Even if you could move all the data there, with an automated process, decisions need to be made in real time. Making that decision and sending that decision back to the point of action would create too much latency - even with the speed of 5G.
• Privacy & Security: Considering all the risks organizations face, it's best to build your historical baseline, run it locally, get rid of the back-end historical baseline over time, keep it limited to the data you need, and throw away data as fast as you can. If you're going to do that, why not do everything locally?
  

Historical data from the edge will, of course, need to flow up-front to the machine learning algorithm developers for design, tuning, and drift adjustment. Key pieces of data from the edge will flow to the core cloud just as pertinent, but relatively small data sets will flow from core systems to the edge. This displays the fluidity of data and the importance of seamlessly connecting the edge data fabric to the core cloud data fabric. Given that standards like MEC are now migrating cloud technologies to the edge, the outlook for moving cloud data fabric to the edge looks promising.

##

ABOUT THE AUTHOR

Lewis Carr is Senior Director of Product Marketing at Actian. In his role, Lewis leads product management, marketing and solutions strategies and execution. Lewis has extensive experience in Cloud, Big Data Analytics, IoT, Mobility and Security, as well as a background in original content development and diverse team management. He is an individual contributor and manager in engineering, pre-sales, business development, and most areas of marketing targeted at Enterprise, Government, OEM, and embedded marketplaces.