Industry executives and experts share their predictions for 2023. Read them in this 15th annual VMblog.com series exclusive.
Organizations Will Design for Observability at the Architectural Level
By Avi Freedman, CEO and Co-founder, Kentik
As enterprise organizations continue to adopt a myriad of services and platforms into the different layers of their systems, enabling visibility into how these systems function is paramount.
In 2023, more organizations will achieve success by designing for observability at the architectural level. For a modern system to function at scale, you must be able to see what’s happening inside of it. After all, a properly designed architecture should illuminate and not obfuscate.
Using telemetry to detect and isolate issues
The easiest way to avoid eventual visibility problems is to design the different layers around providing rich telemetry – whether OpenTelemetry or not.
When things go wrong with a piece of a complex system, issues tend to spill out and cause problems in other areas. Telemetry data gives you the data that can help detect issues and isolate the root cause of these issues, making remediation much easier (and less panic-inducing.)
An example – services meshes can be great for visibility. If built correctly, service meshes help run microservices at scale by providing a more flexible release process, availability and resiliency, and secure communications between services. However, unnecessarily using service meshes, or without the right telemetry, can make it harder to observe and understand your applications and infrastructure.
Implementing telemetry pipelines
A second related trend we expect to grow in 2023 is the use of telemetry pipelines to underpin a broad range of operational systems.
No “system or platform to rule them all” is available today, so designing your architecture to enable sending telemetry to multiple systems is necessary. Between infrastructure-focused platforms, application-focused platforms, and operational systems, there is no shortage of services that benefit from telemetry. This includes data lakes, internal AI/ML projects, observability platforms, BI platforms that look at revenue impact, and many more.
A system designed to accept and collect broad ranges of telemetry can copy, massage, and send it to many different systems, adapting it as needed for the advantages and limitations of those platforms. There is a huge amount of innovation in the open source and commercial world around this design pattern, and it is part of most observability architectures we are seeing now.
Enabling systems to be self-driven
A third related trend that great telemetry collection enables is the ability to enable systems to be increasingly (though still not totally) self-driven. Whether it’s applications, or enterprise or service provider networks, we’re seeing an increasing number of customers use telemetry collected and shared on busses to carefully enable control loops for auto-scaling and remediation.
Of course – you need to be careful with all feedback loops. It can be tricky to only add stability. The best architects always remember the Tacoma Narrows Bridge, and complex distributed systems have their own harmonic frequencies to avoid.
Observability as a means for success
I’ve said this before, but as observability solidifies and modern architecture scales out to using even more services, it’s more apt than ever. Designing for observability at the architectural level is crucial for understanding how systems function. This empowers organizations to understand and fix problems faster and ultimately achieve better results, which is critical to staying competitive in the market.
##
ABOUT THE AUTHOR
Avi Freedman is the co-founder and CEO of Kentik, with decades of experience as a leading technologist and executive in networking. Freedman was with Akamai for over a decade as VP of Network Infrastructure and then as Chief Network Scientist. Before that, he started Philadelphia’s first ISP (netaxs) in 1992, later running the network at AboveNet and serving as CTO for ServerCentral.
