By Jim Broome, President and CTO, DirectDefense
As generative AI becomes more prevalent in consumer and business settings, its safety and security are being called into question. As security professionals, we must exercise caution regarding the data security risks with generative AI, such as privacy threats, AI-caused copyright problems, and the potential for intellectual property theft.
In a recent study on The State of AI and Security Survey, 55% of organizations plan to adopt generative AI solutions by the end of 2024. While there is promise for AI to enhance security measures, especially in threat detection and response capabilities, the question remains whether we are rushing to adopt these technologies without ensuring they meet our own uptime and security requirements.
Data Risks with Generative AI
Last year, our primary concern was implementing strategies to monitor what employees were voluntarily or involuntarily submitting through AI platforms or web browsers and a whole suite of technology emerged to help monitor and control this input. The landscape has evolved significantly since then, with generative AI being integrated into operating systems and functioning in the background beyond our control.
Initially, users had to provide content but recent updates from vendors like Microsoft and Apple have introduced native support into their solutions for generative AI that could be exploited by attackers or the broader service ecosystem. This raises serious questions about what filters and limits are in place to prevent the automatic extraction of intellectual property by AI-driven services to vendors such as Microsoft and Apple (among others).
Design decisions made by these vendors who promise to safeguard user data have not resolved these concerns. Some features like Microsoft Copilot have already been circumvented using post exploitation and forensic tools that capture generative AI created data such as desktop screenshots and all search queries performed by the user on systems running the latest version of Windows.
Impact on Privacy Concerns
Currently, the broader concern is with these AI-infused tools natively integrated into the devices we use to create our own (you the individual or you the company) intellectual property (IP). These vendors are scraping and leveraging data from devices where IP is generated and incorporating our unique content into their models, and neither vendor has yet to directly address this new impact on intellectual property rights. (There are major legal disputes over copyright ownership that have yet to be settled.) Even though they say that they do not sample such data, there is a legitimate concern about IP potentially leaking on these platforms.
Operational Reliability Issues
While these solutions are adding advanced functionality, few AI vendors including OpenAI have yet to publish their service level agreements for uptime or show proof via certification documentation like the trust services criteria documentation provided by SOC2 certification. In certain cases, AI solution providers have yet to achieve the five nines uptime most customers expect of a solution – just check their status pages.
So, as a developer and vendor you have to ask, should we be adopting technologies that fail to meet our own uptime requirements? Or is their current availability within acceptable operational limits for your solution?
Three Recommendations for Securing the Enterprise
1. Know How to Disable and Monitor New Applications
With every new operating system, IT personnel should prioritize training on how to disable or implement security enterprise security configurations based on their operational requirements. As an example, in Windows 10 and 11, one common practice was to disable all ads through the Windows notification settings. Carefully consider, control, and monitor these features in a test environment before widely adopting operating systems that enable them by default. Resources from Tom’s Guide and The RSnake Show can assist you here.
2. Leverage Data Leak Protection (DLP) Tools
Implement solutions like Nightfall that employ DLP to intercept and block sensitive information from being accessed or transmitted via AI-driven platforms like chatbots or ChatGPT. Agent-based web content filter can aid in detection with work from home employees, but a broader approach will be required to monitor OS-enabled AI solutions, such as implementing an enterprise wise DLP enforcement solution
3. Enhance and Update User Awareness Training
On a quarterly basis, update your user awareness training to educate employees on the latest best practices. Threats are constantly changing, and your user awareness content should be evolving with it. The spam training you gave two years ago is largely irrelevant today due to threat actors leveraging ChatGPT to craft more accurate and targeted campaigns against you.
##
ABOUT THE AUTHOR
Jim Broome is a seasoned IT/IS veteran with more than 20 years of information security experience in both consultative and operational roles. Jim leads DirectDefense, where he is responsible for the day-to-day management of the company, as well as providing guidance and direction for our service offerings.
Previously, Jim was a Director with AccuvantLABS where he managed, developed, and performed information security assessments for organizations across multiple industries, while also developing and growing a team of consultants in his charge.
Prior to AccuvantLABS, Jim was a Principal Security Consultant with Internet Security Systems (ISS) and their X-Force penetration testing team.
Jim has also developed and provided training courses on several security products, including being a primary author of the CheckPoint Software Software CCSA/CCSE/CCSI training program, as well as creating and delivering numerous client-focused training programs and events.
