By Kasey Best, senior director of threat hunting, DNSFilter
Rapid enterprise AI adoption has outpaced operational maturity. Many organizations are deploying AI because of market pressure rather than clearly defined business outcomes. There’s a growing proliferation of “AI products” that are little more than thin wrappers around existing large language models.
The truth is that there’s a widening gap between AI hype and operational reality, underscoring how quickly today’s innovations can become obsolete, and even a risk to security, as core capabilities are absorbed by major platforms.
In this article, I’ll explain why sustainable AI depends on engineering rigor, security architecture and deliberate implementation, not saturation, and practical tips to achieve that sustainability.
The obsolescence problem
Many startups and enterprise tools rely almost entirely on third-party foundational models. Features built on top of commodity LLM capabilities can disappear overnight as major platforms absorb them natively. As such, there’s a strategic risk of building enterprise workflows on unstable or non-differentiated AI layers.
Examples of fragile deployment patterns include:
- Minimal proprietary logic
- Weak governance controls
- No domain-specific intelligence
- Limited security validation
There’s a difference between “AI-enabled products” and true AI engineering. If a product can be recreated in minutes using public APIs, it is unlikely to provide lasting competitive or security value.
It’s important to understand the difference between experimentation and production-grade AI systems. Successful AI deployments should combine the following components:
- Domain-specific knowledge: The domain, field of study, expertise, professional background and experience that an individual has in any given domain or field like a programmer with specific knowledge in coding.
- Human validation layers: Commonly referred to as “human-in-the-loop”, a human expert validates an output instead of an algorithm, function, rule or model.
- Machine Learning Orchestration: Layering machine learning models and algorithms together in a stable, coherent flow to achieve a given outcome.
- Security-by-design principles: The practice and means of embedding security directly in the development process.
- Deterministic controls: Controls with explicitly set and defined rules that deliver the same outcome every time. A deterministic control, such as machine learning (ML), is one example. It might check to see if a value is lower than 5, and a 4 would pass whereas a 6 would not – and it will output those results every time. Thus the outcomes it provides can be relied upon as deterministic. Whereas a non-deterministic output is what AI models deliver: if you ask it what color the room is, a given model might say blue, purple, or baseball bat (depending on context and model capability). Thus its outcome is non-deterministic (i.e. it can change).
Scalable AI systems require continuous tuning, governance and operational oversight. Relying on a single AI model to detect and mitigate threats is the same as relying on any other single point of failure. Moving from a prototype to a production-grade system requires a defense-in-depth architecture, as no single model is 100% accurate, and AI itself introduces new attack surfaces, such as prompt injection or data poisoning.
Layering, or stacking the above layers and orchestrations on top of one another and in sequence, ensures that if a clever adversary evades one model, subsequent validation layers, deterministic controls or human overrides will catch the slip. That’s why AI remains an augmentation capability rather than a replacement for expertise. Effective AI systems design results in carefully engineered ecosystems, not standalone models.
Ideally, a production-grade system implements layered detection and verification through several distinct tiers: diverse detection layers, cross-verification mechanisms, the “human-in-the-loop” (HITL) layer and continuously refined input/output guardrails.
From AI hype to operational reality
When evaluating current operational issues, teams should assess whether AI is necessary at all and if so, build governance and security controls prior to scaling deployments. The aim should always be interoperability and resilience rather than sheer novelty for its own sake.
Balancing automation with human expertise and accountability is key here. Organizations who establish proper controls and treat AI as they might any other professional-use tool are the ones who can achieve this balance. Such a process involves conducting security audits, implementing permission restrictions, investing in governance and monitoring tools, offering regular employee education, and validating outputs.
The biggest issue I come across with automation, however, is when AI is paired with a novice, or worse, a confident non-expert in a particular domain. They tend to fail to account for their own lack of expertise, which turns their automated “output” into a cascading series of errors that spread at machine scale, resulting in quite the opposite of the efficiency boost they were originally looking for. More time is then spent on cleanup when it could have been spent on education or training beforehand. An ounce of prevention here goes a long way. AI tooling is a fantastic enabler, but it needs to be paired with domain expertise and validation to enable actual, enterprise-grade outcomes.
Build a strong AI foundation
While AI tools promise rapid innovation, many lack the engineering rigor required to deliver long-term value or security and relying on the capability of a single model’s performance (much less a single frozen version of that model) isn’t a competitive moat. The next phase of enterprise AI maturity should, if not must, reward operational discipline over unchecked experimentation at scale. Security leaders must move beyond the cycles of hype that embrace possibility as evidence of delivery, and instead focus on systems that are explainable (i.e. iterable), resilient, and maintainable.
In practice, real AI engineering oft comes down to a blend: the integration of domain-specific logic with embedded security and scalability, ideally married with a purpose-driven systems design approach. AI should (and can!) act as an enhancement upon human capability and operational efficiency, not a replacement for the foundational engineering practices or security rigors that have been tried and tested for so long. The strongest AI deployments will be those built upon those very same bricks, as they will inherently weather platform shifts, evolving threats and real-world operational pressures to deliver on their original (and future) goals.
In the Security and Tech space, the steady path towards sustainable AI adoption comes from leaders on both sides coming together with a mature understanding of its inherent strengths and limitations as a toolset. Because AI is not a toolset that is going away. It’s never been a question of if but rather when threat actors fully embrace this latest set of shiny hammers (as many already have). Those of us in the security community should mark ourselves down in the when column as well, out of necessity if nothing else.
In summation: future-proof AI strategies depend on adaptability, transparency, and infrastructure ownership. Those organizations who will succeed with AI will not be the ones deploying it everywhere in pursuit of leading the latest token-burning leaderboard, but the ones who apply it deliberately. Decisively… and, dare I say it, in pursuit of the very same deterministic outcomes so many rely upon us to provide.
##
ABOUT THE AUTHOR
Kasey Best is the Senior Director, Threat Hunting at DNSFilter, bringing over 15 years of intelligence and cybersecurity expertise to the team. Kasey leads a team of global threat researchers, providing our customers with IOFAs at scale as well as extensively detailed reports on the malicious network infrastructure used by a variety of state-backed APTs and large-scale cybercrime actors. His expertise is in threat intelligence, network analysis, malware analysis, cloud forensics, technical project leadership, and project management. He speaks four languages, is both CISSP & PMP certified and is working towards his Masters in Cloud Computing Systems. Previously, he was a Senior Analyst for the U.S. Department of Defense where he led interdisciplinary and multi-agency teams to mitigate cyber threats to US infrastructure.
