
Virtualization and Cloud executives share their predictions for 2014. Read them in this VMblog.com series exclusive.
Contributed article by Olivier Pomel, CEO and Co-Founder of Datadog.
Host-level Management becomes Irrelevant
As we look back on 2013, the shift to cloud
computing dominated tech headlines. From our vantage point at Datadog, where a
large number of our customers extensively use cloud-based infrastructure, we
observed these movements to be quite tangible: A number of Global 2000 firms
that we work with are abandoning their on-premise hardware to run applications
on public clouds. Notably, this infrastructure automatically scales to match a customer's
workload demand.
Auto-scaling cloud infrastructure provides
engineering teams with more capabilities to grow computing resources. Engineers
can now deploy "services" that are comprised of multiple hosts. These services
automatically scale up or down at a moment's notice. Thus, specific hosts
become trivial commodities. The performance of the aggregate is what matters.
In 2014, this evolution will become complete and render host-level management
irrelevant.
Public cloud infrastructure and automated
configuration management as catalysts
Host-level management has been losing
importance for some time. A decade ago, an IT department would agonize over
which specific physical servers to buy for a project. Once installed, system
administrators would monitor each server throughout its useful life. The
widespread adoption of virtualization technology de-emphasized the physical
server as hosts were decoupled from their hardware. Yet, virtual machines (VMs)
were still managed as individual servers. VMs had to be deployed and overseen
manually.
The ability for a public cloud to auto-deploy
and parallelize an "infinite" amount of new servers made it feasible to level
up to the "service". However, the catalyst for managing infrastructure at the
service level has been the increasing adoption of automated configuration
management systems such as Chef and Puppet. This software has allowed system
administrators to liberally utilize auto-scaling while manageably overseeing
countless hosts.
Infrastructure disposability and
containerization accelerate this shift
As cloud auto-scaling gains increasing
mainstream adoption, applications are being developed to take advantage of
these features. Most notably, applications now treat infrastructure as
disposable. Compute capacity can be built up or torn down at a moment's notice.
Because few infrastructure components are ever permanent, it makes little sense
to expend manual effort on any specific host. This new approach forces
automated configuration management adoption. In turn,
further use of automated configuration management produces a virtuous cycle,
which increasingly enables auto-scaling.
Also, in 2013, new containerization projects
such as Docker began to gain significant adoption. As this technology seeps
into the mainstream, the host will become increasingly more marginalized. Containerization allows for applications and hosts to be
recombined effortlessly, regardless of what on-premise data center or
cloud the physical server is residing in. Thus, which host is being used by an
application will be a negligible concern, as that host could change at any
time.
The rest of the stack will have to adjust
A major consequence of host-level irrelevance is that the rest of the application stack will have to
adjust. Systems that are downstream from the application will have to handle
data from the infrastructure at the service level. In particular, monitoring
systems will be forced to evolve. These systems have for decades required that
new hosts be manually instrumented. Now, monitoring solutions will be left
behind if they cannot auto-instrument new hosts which have been auto-deployed.
Likewise, these systems' unit of measure will have to progress past individual
servers so that performance can be assessed at the service level.
In conclusion, hosts in 2014 will become as
irrelevant as processes in an application. Do you know how many processes are
running your web app? No. It should be and will be the same with your hosts.
The data center will scale-up one level.
##
About the Author
Olivier
Pomel is the CEO and Co-Founder
of Datadog. Prior
to Datadog, Olivier built data
systems for K-12 teachers as the VP of Technology for Wireless Generation,
Before Wireless Generation, Olivier held software engineering positions at IBM
Research and several internet startups. Olivier is an original author of the
VLC media player and holds a MS in CS from the Ecole Centrale Paris.