It’s no secret that the amount of data within enterprises is rapidly increasing due to factors like the widespread adoption of IoT devices, cloud storage, and real-time data collection, leading to a significant rise in the volume of information businesses need to manage and analyze, and at the same time, organizations and teams are being given more key performance indicators (KPIs) to become data-driven.
Recent surveys have placed the average number of observability tools used in the enterprise at five to eight different vendors. To better understand how enterprises can keep up with, and get the most relevant information from their existing observability tools, VMblog caught up with Logan Rohloff, the Technical Lead for Cloud and Observability at RapDev, a solutions provider focusing on cloud-native best practices and observability consulting to Fortune 500 organizations.
++
VMblog: Observability has been one of the most significant topics in recent years, and is rapidly increasing. What is your approach to observability and thoughts on what companies should consider?
Logan Rohloff: I look at observability as a forethought. Too often I’ve seen enterprises have this thought process of needing to get product out the door and that observability is something that can be added later. That is until they have a business impacting outage, which causes them to speed through implementation after the damage is done. While it’s great that observability is there now, they’re now playing catch-up because they have no historical frame of reference to know if their infrastructure or software has gotten faster, has less bugs, or is more secure than it was before. Building observability into technology solutions from the beginning helps develop a frame of reference early on, which ultimately leads to all around better apps, which means happier customers.
VMblog: How does RapDev work with companies to optimize observability?
Rohloff: We have developed a process for implementations and have seen it work in a number of organizations of varying sizes and verticals. The key challenges with observability for enterprises often follow similar patterns and/or themes, and include: managing large volumes of data, filtering noise from useful insights, real-time data processing complexities, data privacy concerns, instrumentation overhead impacting performance, integrating diverse systems, lack of skilled personnel to analyze data, and aligning observability goals with business objectives; all requiring sophisticated tools and strategies to effectively monitor and troubleshoot complex systems.
VMblog: How else does RapDev help organizations fill gaps in their observability strategies and tooling?
Rohloff: We have developed more than 40 integrations and added them to the Datadog Marketplace to help businesses monitor their infrastructure, applications and workflows.The integrations are designed to help enterprises simplify their complex observability systems while getting the most out of their existing infrastructure. We offer more Datadog integrations than any other partner, and our 40+ integrations offer support for true end-to-end infrastructure monitoring. We can also offer custom integrations to fill gaps in native Datadog support, and automate DevOps to eliminate manual approvals, reduce human error and optimize workflows. Datadog vets all Marketplace integrations and applications to ensure that they are seamlessly integrated.
VMblog: What else should our readers think about?
Rohloff: The extent to which observability is configured or enabled within an environment, in my opinion, is ultimately a business decision. It’s about factoring in risks and the impact to the organization if certain pieces of infrastructure or applications become unavailable. Given this, it’s important to determine if having a lack of visibility into certain technologies can impact the organization’s bottom line either directly or indirectly. A direct impact would be if an application that accepts customer payments goes down, causing lost revenue into the business. An indirect impact could be an important internally used tool going down, which may not be customer facing. This can still indirectly affect an organization when it comes to productivity, especially if the tool is widely used to make employees jobs easier or help them get things done faster. While it can be more difficult to directly correlate that to dollars lost against the bottom line, productivity loss can stack up to be very expensive to the organization.
So many companies, even if they are not “tech companies,” are becoming more and more reliant on technology, cloud providers, AI, etc. In my opinion it would be difficult to argue that any piece of infrastructure or any application used within an organization can be properly operated or maintained with absolutely no observability or security measures in place. Operating blindly when trying to fix an issue in production just increases the amount of time it takes to discover and resolve those issues, impacting revenue, productivity, or the overall trust that end users have in the product or service provided by the company. The tricky part is making the business case to spend money on the implementation of proper observability tooling, as some organizations still view this as a cost rather than an investment into the stability of a product. It’s ultimately a balancing act requiring weighing risks, costs, and benefits, and can be a major culture shift to convince the leaders of an organization to make the investment. Speaking from personal experience, you don’t want to be the one that finds out how those risks might manifest while troubleshooting a major production issue, because at that point it’s almost too late.
##
