As we approach another year of amazing
CNCF-backed get-togethers at the annual KubeCon + CloudNativeCon events in Europe and
North America, along with an exciting list of (new for 2023) KCDs, it's useful to take an assessment of the
overall K8s monitoring landscape.
Over the years that I've been directly
involved in cloud observability - from my time as a pure user of open source
tools employed to monitor my company's products, to leading strategy at an
observability provider - a lot has changed. The massive adoption of Kubernetes serves as a
perfect example of how the challenge of monitoring our environments today is
radically different than it was just five years ago.
Whereas a loose knit stack of semi-integrated
capabilities, or even leading Applications Performance Monitoring solutions
were once sufficient in helping us understand enough about our systems to
ensure that they were sufficiently healthy - the increased velocity of today's
CI/CD practices and the ephemeral, short-lived nature of our microservices
architectures have significantly changed the game.
Monitoring (or more accurately, observing)
Kubernetes - along with understanding related security implications - have
increasingly surfaced as concerns among today's engineering and DevOps
practitioners. This is validated in recent research that Logz.io conducted with
around 500 respondents spanning roles from developers to SREs, along with IT
directors and executives.
According to our 2023 Logz.io DevOps Pulse
Survey, use of Kubernetes and related observability challenges are growing
noticeably each year. Our findings include:
- When asked about the main
challenges in gaining full observability, the largest category of respondents
(almost 50%) stated that Kubernetes posed one of their main challenges.
- Over 41% of survey respondents
cited monitoring and observability of K8s as a primary challenge of running
Kubernetes in production. Compared to last year, this represents a noticeable
increase as only 31% of respondents highlighted this issue in 2022.
- On the whole, leading
observability challenges for those running Kubernetes in production were:
aggregating relevant data during troubleshooting, cluster networking, and
security. Data collection and visualization also remain pervasive issues.
- In fact, the research clearly
evidenced growing concern about K8s-related security as roughly 50% of
respondents agree it's one of the most difficult components of running Kubernetes
in production.
As an indicator of how broadly survey
respondents are using Kubernetes, in general, over 67% said that they had
already implemented it in production with another 23% working to do so.
Moreover, the survey finds that the biggest issue facing today's monitoring and
observability teams is that - despite perceptions of growing cloud, DevOps and
observability maturity - the all-important metric of mean time to resolution
(MTTR) continues to head in the wrong direction. The survey specifically found
that MTTR is taking more than a few hours for over 73% of respondents, up
noticeably from 64% in 2022 and dramatically from only 47% in 2021.
Clearly this is a very troubling trend, and
one might defensibly suggest (I will) that these listed Kubernetes concerns are
contributing to this problem. As we continue to embrace and accelerate our use
of Kubernetes we need to find better solutions for effectively monitoring,
troubleshooting and securing our applications and infrastructure.
In our work with customers at Logz.io, we also
see security monitoring for Kubernetes becoming even more deeply ingrained into
observability practice and believe that in the not-so-distant future these
requirements will increasingly be tackled in an integrated manner (we may have
something to say about this at KubeCon Europe, in fact!).
So, now what? What actions can we, as a
community of passionate believers in Kubernetes and open source, undertake to
help address this problem? (It's worth noting our survey does find that a
whopping 93% of respondents utilize open source tools of some kind for
observability.)
Let's talk about it! Let's meet up in person
at these fantastic events and bring our creative thinking and innovation to the
table. Beyond anything that we as vendors contribute (and Logz.io is leaning in
on easing and simplifying K8s observability and security)... let's work as a
community to keep raising the bar.
Thanks for reading this, I hope the data is
helpful and please look out for the full DevOps Pulse Report. Let's connect in
person at KubeCon + CloudNative in Amsterdam in April!
##
To learn more about the transformative
nature of cloud native applications and open source software, join us at KubeCon + CloudNativeCon Europe 2023, hosted by the
Cloud Native Computing Foundation, which takes place from April 18-21.
ABOUT THE AUTHOR
Asaf Yigal Co-Founder and CTO, Logz.io
Asaf Yigal leads Logz.io's product vision
and strategic direction. Prior to launching Logz.io in 2014, Yigal was
Co-Founder and VP of Product Development at Currensee, acquired by OANDA in
2013, where he served as VP of Product. He holds an Electrical Engineering
degree from the Israel Institute of Technology.