I started out my professional career as a software developer then became a DevOps engineer. (Get it? It’s a joke.)
The irony is that DevOps is not an engineering discipline at all, it’s actually a set of principles. In fact, the notion of a “DevOps engineer” runs directly contrary to those very principles. Developers and operators are simply supposed to share responsibilities; operations must be in scope for development. Introducing a middleman that has been steeped in both cultures was never the intent. Rather, it was steeping developers and operators in their respective cultures.
The reason that DevOps has become a title-worthy classification is largely due to the state of the art. Whereas industry-leading technologies that dominate the DevSecOps landscape, like Terraform and Ansible, have led the way in demonstrating how infrastructure ought to be provisioned and configured with the click of a button, they are niche skills because their language interfaces are domain-specific. Despite their extraordinary functionality, in some ways, they are the worst of both worlds. Only a small subset of developers have the necessary skills required to write the code, which perhaps even fewer operators can read. Now, I happen to love Terraform and Ansible, but the sheer number of companies that require dedicated teams of highly paid engineers on staff to maintain similar proprietary infrastructure codebases is criminally inefficient. This anti-pattern is unsustainable as there is simply not enough domain-specific talent to satisfy the market’s demand. Nor should there need to be!
“…The sheer number of companies that require dedicated teams of highly paid engineers on staff to maintain similar proprietary infrastructure codebases is criminally inefficient.”
The proverbial icing on this irony-cake is that I have found myself losing touch with my software engineering background along my path toward becoming an expert in “DevOps engineering”. I have become a developer without a language. Sure, I can make myself useful with various languages, but I have no expertise. I can develop infrastructure and configuration management solutions for complex application infrastructures, but I can barely write an application. But for my own extracurricular interests, my training in programming language design concepts would languish.
Along came Pulumi Automation API
When my friend and former colleague first introduced me to Pulumi, I was intrigued yet highly skeptical. A language-agnostic infrastructure platform? At the time, I saw no novel problem that Pulumi was solving. Terraform’s infrastructure as code solution was already very elegant, and it seemed to me that trying to make the same functionality sets accessible from imperative language constructs was misguided. Why sacrifice the robustness of a declarative configuration engine for the familiar comforts of what seemed like a syntactically sugared, imperative runtime environment? Since it was also fairly new at the time, I decided to put a pin in it.
Then a few years later, a colleague at Anitian mentioned it to me. So, I gave it a closer look, speed-reading through a bit of the documentation. I was impressed, it was clearly more well-designed and thought out than I had initially realized. There was no trade-off of a declarative resource engine for the expressivity of the supported languages. There was still a declarative engine. In fact, it was using Terraform’s under the hood! I realized that I had made an unfair mischaracterization when I had considered this to be syntactic sugar. It was elegant innovation. That said, the team at Anitian had well over a decade of combined experience with Terraform at the time and none with Pulumi. There would have been far too much risk inherent in trying to switch tooling and with no clear benefit. But in all its cliché glory, the third time was the charm.
“…I realized that I had made an unfair mischaracterization when I had considered this to be syntactic sugar. It was elegant innovation.”
Anitian was in the early stages of development for a new infrastructure platform. Its design was the product of a year of research on how to improve upon our existing product. After a few months of development, it was really coming together. But there was a problem: the tech stack was mammoth! We had carefully chosen each tool so that functionally, it was spectacular. But from an accessibility standpoint, it was extremely challenging. Interfacing the platform required fluency in HashiCorp Configuration Language (HCL), Golang, and Ruby at a bare minimum. Meanwhile, a new technical manager had recently joined the team and had asked about using Pulumi as an alternative. We still had no experience with Pulumi, but frankly, we were having a heck of a time recruiting polyglots to help to contribute to what we already had. Pulumi would let us choose a single, common programming language for defining our infrastructure, which meant that the talent pool from which we could recruit would be comparatively massive. Eureka! Chalk up one solution to a novel problem for Pulumi.
Since most of the team at Anitian was familiar with Python, we quickly realized that its object-oriented design, comprehensible syntax and feature sets like multiple inheritances were a perfect fit for implementing our strategy. Then there are the runtime dependencies. We had resorted to packaging our code in Docker images for the sake of consistency, if not sanity. By contrast, Pulumi would let us publish infrastructure code as proper artifact packages. Terraform, eat your heart out! (And for the record, nobody is going to give you money just to get a private module registry.) Finally, Pulumi’s CrossGuard policy-as-code was the kicker. It would allow us to govern and validate our infrastructure with the same programming language in which it had been defined. It’s very difficult to make a sane argument against sanity itself.
The proof-of-concept phase went well. The platform surpassed the team’s expectations. First, having IntelliSense work out of the box was refreshing. The packaging of modularized infrastructure code was as trivial as expected. No more repo-spaghetti, no infrastructure mono-repo, no complex SCM integration — just a repository directory with a package declaration. At any time, the latest published version of the infrastructure code could be installed into virtually any environment with just one authenticated pip command. Diagnostic logging capabilities were also very nice to have. I also cannot overemphasize the sense of freedom that comes from having trivial programming language design constructs accessible again for implementing non-trivial dynamic behaviors. Nested loops with conditional logic, for example, are no longer implemented as excruciatingly complex lambda evaluations. And when a logical error is introduced into the code, it is far easier to identify a root cause. Stack traces, exception handling, and standard debugging utilities are far more conducive to my sanity than having to peel back layer after layer of an onion of red-herring error messages.
Benefits of Pulumi for DevOps
But the most substantial benefit that Pulumi had to offer Anitian, in my view, came as a surprise. It completely dissolved any distinction between application and infrastructure; we were now developing an infrastructure application. Having had development experience with several enterprise-scale projects, I had come to accept a marginal amount of rubeosity in general application design. I considered some moderate level of hackery to be a necessary evil, inherent to any practical innovation. But with Pulumi, there was no need to develop a Goldbergian integration for operations, just an operational interface for the application. And the Pulumi Automation API allows Anitian to codify the infrastructure operations themselves. In other words, operations are developed in a common language that most operators can read — the very same language in which the infrastructure is defined, governed, and validated. Holy Toledo! The landscape has changed. Again!
DevOps has meant different things to different people ever since it first became a buzzword. To some, it has meant specially trained engineers. To others, it has meant trading one set of information silos for another. To many, it has been… Well, suffice it to say that the human imagination is awesome. But to those who really understand what DevOps is all about, it means sanity. Plain and simple.
Anitian’s SecureCloud for Compliance Automation and Enterprise Cloud Security is the fastest path to PCI, ISO 27001, and FedRAMP compliance.
