Virtualization Technology News and Information
It's 2023: How come we still have no Observability?

The year is 2023. The digital transformation is almost over. Cloud-native is the norm. Suddenly, AI is the new buzzword. And yet, most of us developers still can't get our hands on decent Observability tools, can we?

If you are reading this, you know not only what Observability is, but also how crucial it is to deliver and operate fantastic software applications. Most developers I speak with understand that Observability is for more than just IT Ops and SRE teams.

Seeing how our code behaves across the software development lifecycle (SDLC) from development to production has a tremendous positive impact on every aspect of our jobs. Insights into the performance and correctness of the code keep us focused on matters and push us towards higher standards. If you zoom out, you will almost certainly find that Observability for developers positively impacts engineering culture, driving accountability and ownership - a real Shift-Left.

Those effects are all magnified by several orders in cloud-native environments where the code execution on our local machines is light-years apart from what's happening in those remote Kubernetes clusters.

It's Not What You Look At That Matters, It's What You See

Thoreau got it right with that saying.Since the dawn of time, evaluating and procuring Observability solutions has been the purgatory of operations teams. For them, only one thing matters - keep the system running smoothly. This creates a very peculiar set of requirements. First, detect when things go wrong as quickly as possible, and second, help identify the root cause. The rest of the time, try not to bother anyone. These requirements are best met using Traces and Metrics, with a large portion of AI poured on top.

For developers, it's a whole different ballgame. We are the endless tinkerers, and for us, there's always something that needs to be improved on. Our infinite bucket list of desired changes includes (but isn't limited to) adding and removing features, fixing bugs, improving performance and cost, or just rewriting things to make them nicer. For each task, we need our Observability tools to answer different questions. We also tend to go deeper into the application code than other stakeholders. Our questions are best answered using Dynamic Snapshots and Logs. We also find great exploration and visualization tools often win over AI.

There's no right or wrong here. Ideally, a single Observability platform meets all of those requirements. For that, developers ought to participate in the vendor selection process.

So where's the value in that?

Negligible performance impact. Perpetual availability. Uncompromised security. Real-time insights. Operations have draconian requirements for their Observability tools, and that comes at a steep cost.

We developers are no different. We have environments to monitor. We are users to be included. We need various features and use cases. We require new data points. However simple or complex the pricing scheme is, there will always be an additional cost.

Organizations understand the paramount value of developer-focused Observability tools. To make it a reality, we now need to put our money where our mouth is.

Putting Your Money Where Your Mouth Is

As Steven Magee once said:

Security isn't the absence of danger, but the presence of better solutions.

As every seasoned developer knows, we ultimately find our way into production. We team with an operator and use their machine. We are provided a host or database backup to look under the hood. We get mirrored traffic for testing. The options are truly endless, but, at some point, there's no avoiding it.

Developer Observability pulls back the curtain on what we've kept hidden for so long. We don't have to learn obscure tools. We can skip working around organizational rules and procedures. It's as if anyone and everyone can do it. With great power comes great responsibility, and that can raise many objections.  

Luckily, great Observability tools come with stringent security guarantees, and developer use cases are no different. This is a massive opportunity for security, privacy, and compliance. Suddenly, multi-factor authentication (MFA), role-based access controls (RBAC), audit logs, and so much more are at our fingertips. We can configure any required policy directly into the tools we are using. Risk-free production access can now be a daily part of our lives.

Give me six hours to chop down a tree and I will spend the first four sharpening the axe.

-- "Abraham Lincoln"

As developers, we love to dive into the code and let our keyboards sing. Unfortunately, great Observability requires a different tune. The time has come to step away from our screens.

The tools we need are out there. Operations, security, and finance are well-acquainted with the importance of getting them in place. Let's make sure our voice is heard. Great Observability is life-changing for every software development team out there.

If you got this far and are not a software developer, it's not in vain. Do your part to ensure software developers are engaged with your Observability architecture and solutions. The results will speak for themselves.


Join us at KubeCon + CloudNativeCon North America this November 6 - 9 in Chicago for more on Kubernetes and the cloud native ecosystem. 



Liran Haimovitch


Liran is an Observability and Instrumentation expert with a deep understanding of Java, Python, Node, and C++. Liran has broad experience in cybersecurity and compliance from his past roles. When not coding, you can find Liran hosting his podcast, speaking at conferences, writing about his tech adventures, and trying out the local cuisine when traveling.

Published Wednesday, October 18, 2023 7:31 AM by David Marshall
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<October 2023>