By Nathan Hamiel, Senior Director of Research at Kudelski Security
For over a year, artificial intelligence has dominated the headlines. You’d be hard-pressed to find people who haven’t heard of ChatGPT, and in the tech world, that number drops to zero. The hype continues to build, and hope is high, but there are looming concerns from companies attempting to operationalize generative AI in their own environments, and investors in AI companies are eager to see returns. So where does that leave Generative AI in 2024? In this post, I’d like to call attention to some lesser-discussed topics around generative AI and call out a few things I believe are in store for the near future.
Security and Privacy Risks Will Hinder Wider Adoption
Deep integration of Generative AI into systems and applications can negatively impact security and privacy. This isn’t immediately obvious when experimenting with a use case, but when deployed into a production environment, the attack surface becomes more exposed and can be manipulated in unexpected ways by an attacker. There are inherent security issues like Prompt Injection in technologies like Large Language Models that haven’t been resolved, and it can also be easy to leak sensitive data inadvertently. Ultimately, Generative AI approaches are experimental technologies, and the attack surface isn’t completely known.
From a privacy perspective, it may not be obvious why files are accessed, or why data leaves a system. Data is a valuable asset for training AI systems, and there is a relentless appetite for more of it. The temptation will be there to use this previously private data to train and update models. Another privacy concern is that logs and interactions with generative AI systems are more likely to be reviewed by a human to evaluate performance.
Of course, any discussion of risk and privacy has to be situational and depend on the use case. The more critical and exposed the use case, the more important security and privacy aspects are. Applying generative AI to use cases with a high cost of failure should be avoided.
Sustainability and Environmental Factors Become Part of the Conversation
While the topic of generative AI has dominated headlines, what’s received less attention is the sustainability and environmental cost of these technologies. There isn’t a precise understanding of the amount of power and water consumption for both training and generating output from Generative AI systems. However, there is evidence to believe that it’s significant. There was a backlash against proof of work cryptocurrencies over energy consumption, but the same criticism hasn’t been lobbed in the direction of AI.
There is still more work to do in understanding the true cost of generative AI systems, but companies should take notice as it can affect the future cost of new products and services and the planet.
Organizations Continue to Struggle with Generative AI Use Cases
According to a November 2023 O’Reilly Radar Report, the most significant hurdle to Generative AI adoption is the lack of appropriate business use cases. In addition, once you’ve identified a use case, operationalizing and getting the solution into production is another challenge. Use cases rarely operationalize as easily as they seem, and edge cases and outliers further confound systems confronted with production data and the realities of real-world situations.
Solutions often seem promising in small tests and experiments, but the real challenges don’t present themselves until they get launched into production and are confronted with the complexities of the real world. Before the generative AI craze, it was known that many AI experiments didn’t make it into production, but for some reason, people treated generative AI as the exception. When the biggest companies in the world struggle to operationalize these technologies, we should expect that others will struggle as well.
Wider Development Outside the Development Team
New approaches to development based on using natural language instead of programming languages offer interesting new possibilities. Examples of this approach are OpenAI’s GPTs and Microsoft’s Copilot Studio. The value proposition of these tools is enabling people to build the solutions they need without needing to know how to code. However, the downside is increased risk. As organizations struggle with traditional development security challenges, it’s now possible for anyone at the organization to deploy systems and use sensitive data, increasing attack surface and opening the door for compromise. Once again, these situations will be use-case dependent and also depend on how these systems are deployed.
Organizations should attempt to get ahead of this early by defining policies and processes that should be communicated to their organizations, covering the development and deployment of solutions built with these technologies.
##
ABOUT THE AUTHOR
Nathan Hamiel, Senior Director of Research at Kudelski Security
Nathan leads the Fundamental and Applied Research team. To support the innovation of future products and services, his team focuses mainly on privacy, advanced cryptography, emerging technologies, and on internal and external collaboration. Nathan is a regular public speaker at global security events, including Black Hat, DEF CON, HOPE, ShmooCon, SecTor, ToorCon, and many others. He is also a veteran member of the Black Hat review board, where he serves as the track lead for AI, ML, and data science.
