Virtualization Technology News and Information
Seeing through the cloud with seamless observability

By Abdi Essa, RVP UK&I, Dynatrace

Digital transformation is big business. Worldwide spending on digitalisation projects is set to total $6.8 trillion between 2020 and 2023, with technology investment at the top of most organisations' agendas. However, it can be extremely difficult for organisations to maintain visibility into the dynamic, hybrid, cloud-native environments that result from digital transformation. These cloud ecosystems are inherently hard to observe, way beyond the capabilities of traditional manual approaches, monitoring tools, and human operators. The scale and complexity of today's enterprise cloud environments will only continue to rise, so organisations must find new ways of monitoring and managing their digital service performance. If not, they face the growing risk that they'll be blindsided by unforeseen problems and the consequences that follow.

To observability and beyond

IT teams need complete and seamless visibility into every single corner of their digital environment; including multicloud infrastructures, container orchestration platforms like Kubernetes, service meshes, functions-as-a-service, and container workloads. This visibility is critical if teams are to optimise the user experience. Organisations traditionally maintained visibility by instrumenting their environment for a predefined set of problems. Any issues would be detected when the performance and availability of components within their IT ecosystem began to degrade. While this approach was fine for previous architectures that were relatively static, it doesn't work in the dynamically scaling and unpredictable environments organisations now rely on.

The challenge for IT teams is to be able to identify the unknown unknowns and anticipate system anomalies that may only ever occur once. Observability offers a new and better approach to achieving complete, seamless visibility, by collecting data from all system components across dynamic cloud-native environments. It accomplishes this by focusing on the collection of three principal data types-metrics, logs, and traces-the so called three pillars of observability. Observability is strengthened with distributed tracing, entity models, code-level detail, and user experience data, and data from the latest open-source standards, including OpenTelemetry. The key is to ingest all this data and keep it in context and tied to business impact.  

As recent research shows, 86% of organisations are now using cloud-native technologies, including hybrid, multicloud architectures, Kubernetes, microservices, and containers. With this shift, it is crucial that cloud-native software and infrastructure becomes more observable. Industry-led efforts are working to ensure this, with the OpenTelemetry project spearheaded by the Cloud Native Computing Foundation (CNCF) proving a notable example. However, while observability is much more effective than traditional methods of monitoring, data is still often collected and stored in silos. This makes it difficult for organisations to understand the context behind anomalies, unlock contextualised insights, and drive better business outcomes.

The powerful duo of AI and automation

Unfortunately, most approaches to observability simply serve up more data in dashboards. This can be a time drain, as it requires developers to manually manipulate that data to uncover the actionable insights needed to drive more informed business decisions. If teams are to leverage observability effectively when dealing with the scale and dynamic nature of cloud-native environments, automation is critical. Organisations need to automate the discovery and instrumentation of all IT components across the full stack, as well as the collection and analysis of data from anywhere in the business. Automation dramatically reduces manual-intensive tasks, through continuous discovery, as well as automatic configuration and instrumentation. 

That data should then be analysed in a common data model, as this helps to remove silos and unlock the full context behind business-impacting anomalies.

Artificial Intelligence (AI) is also a crucial ingredient. Using AI enables IT teams to automatically set baselines as their environment changes. They will be able to understand what ‘normal' behaviour looks like, with the system instantly identifying problems as they arise. AI further enables observability to deliver precise answers that allow IT teams to continuously optimise the user experience by ensuring issues are proactively resolved before any impact is felt.

Looking ahead to a more promising future

Digital transformation was already accelerating pre-pandemic; now, as we enter a new digital age, the scale and speed is greater than most organisations could have foreseen. Many are scrambling to maintain seamless visibility across the dynamically changing IT environments they are increasingly reliant on. With the sheer volume of data increasing, it's also increasingly challenging to turn monitoring data into actionable answers. This leaves IT teams struggling to manage digital service performance effectively and resolve issues in real-time, before the business feels the impact.

Moving beyond visibility to observability is the next crucial step for organisations as they work to overcome these challenges. When combined with AI and automation, observability provides the groundwork for organisations to effectively monitor and manage their highly dynamic and complex IT environments. As a result, IT teams can stop worrying about what they can't see and focus on investing their time more wisely in projects that drive innovation for the business and its customers.


To learn more about cloud native technology innovation, join us at KubeCon + CloudNativeCon Europe 2021 - Virtual, which will take place from May 4-7.    


Abdi Essa 

Abdi Essa is Regional VP, UK&I at Dynatrace, where he has worked for more than 8 years. During his time at the company, Abdi has headed up the financial services and public sector team, as well as working on partner relations. He holds a degree from the University of London in aeronautical engineering.

Published Wednesday, April 21, 2021 7:40 AM by David Marshall
Filed under: ,
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<April 2021>