By Ilse Funkhouser, CPO, Careerspan

It starts innocently enough. Companies are rapidly deploying Large Language Model (LLM) tools across their enterprises, promising increased productivity and efficiency. The pitch is compelling: automate the mundane, enhance creativity, and free employees to focus on higher-value work. In 2025, we'll likely see AI assistance integrated into every major enterprise software suite, from email clients to project management tools.

What makes this transition particularly insidious is how well-prepared we are to accept it. Employees are already deeply comfortable with AI systems, having built trust through consumer platforms like ChatGPT, Claude, and even Character.ai and Snapchat's "My AI." These systems are designed to feel safe and intimate, mirroring human conversation patterns while providing genuine utility. They're always available, never judgmental, and promise to remember or forget at our command. This psychological conditioning makes the leap to trusting employer-provided AI tools nearly automatic.

But this widespread adoption of AI tools represents an unfathomable Trojan horse.

While public discourse focuses on AI's potential to eliminate entry-level positions, this perspective misses a far more insidious threat. These corporate-managed AI systems aren't just helping employees work - they're learning, step by step, exactly how employees perform their jobs. Every prompt, every conversation, every interaction, every workflow becomes training data for systems that will eventually replicate those very functions.

What's particularly alarming is how quickly professional standards are eroding. Doctors, lawyers, teachers, and civil servants are already using AI tools, often against policy, sharing sensitive information about patients, clients, and citizens. Major corporations like Coca-Cola are already using AI-generated content trained on artists' work without consent, facing minimal backlash. Others push ethical boundaries with AI with impunity, whether it's training on copyrighted content or harvesting user data. This corporate risk appetite, combined with public apathy toward AI mishaps, creates a perfect storm: companies will continue pushing boundaries until there are actual ramifications, while professionals increasingly trust these tools with sensitive data simply because they're convenient.

As someone who builds these systems, I see their immense potential. But I also see how we're building habits that will be nearly impossible to break once entrenched. In 2026, we'll likely see the emergence of "agentic meshes" - AI systems that can simulate entire workflows by combining learned behaviors from multiple employees. Microsoft's "TinyTroupe" may seem like a novelty right now, but is a harbinger. Consider Windows 11's "Recall" feature, which captures screenshots every five seconds. In a managed enterprise environment, this kind of monitoring, combined with AI interaction data, creates an unprecedented detailed record of how work gets done.

The obvious solution would be comprehensive global data privacy regulations and enforcement with strong worker rights. But in a world where TikTok's ownership raises national security concerns and international data flow agreements constantly shift, meaningful reform seems unlikely. Remember when Google's motto was "Don't be evil"? Remember when OpenAI was a nonprofit dedicated to safe AI development? The speed at which market pressures erode ethical principles now that AI is accessible is staggering.

The corporate calculus is simple: AI systems don't need to perform jobs perfectly, or even as well as the worker they are replacing; they just need to be good enough. One human operator could potentially manage multiple AI systems performing tasks that once required dozens of knowledge workers, still making key corrections where the AI was fallible.

By 2026, we'll see the first wave of AI systems capable of performing complex white-collar jobs with minimal human oversight. These systems won't just be pattern-matching engines; they'll be sophisticated simulacra of human workers, trained on years of captured workplace behaviors and interactions. Forty hours a week for 48 weeks multiplied by only 100 employees with the same role is over 192,000 hours of training data in a single year. This is not just feasible, but coming.

Employees will soon find themselves in an impossible position: use AI tools and risk teaching them to replace you, or refuse and face immediate obsolescence. We're not just sharing individual data points - we're sharing contextual, interlinked information about our work processes, decision-making patterns, and institutional knowledge. This data, aggregated across platforms and time, creates detailed operational profiles that make previous corporate surveillance look primitive.

The solution isn't to avoid AI - that ship has sailed. But we need to approach these tools with the same caution we'd use when sharing secrets with a stranger who's broadcasting on a loudspeaker. Because in many ways, that's exactly what we're doing. We need better frameworks for AI privacy and worker data rights. Most importantly, we need to recognize that our instincts about privacy and trust were shaped by human interaction, and they're failing us in this new paradigm.

The next time your company introduces a new AI productivity tool, remember: you're not just using it - you're teaching it. And somewhere in a corporate strategy room, executives are counting on that very fact. The question isn't whether AI will transform the workplace - it's whether we'll have any say in how that transformation unfolds.

The time for that conversation isn't in five years when these systems are deployed - it's now. There are already companies trying to chart a more ethical path forward. At CareerSpan, we're working to ensure that our users retain access to and some level of decision-making ownership about their data, even when their employer pays for the service. This means that as their careers evolve, their accumulated knowledge and insights doesn't disappear behind a corporate wall.

It's a small step, but an important one: explicitly acknowledging that our user's recollection of past job experiences has value and that this value belongs to the individual, not their employer. While this alone won't solve the broader challenges of workplace AI surveillance, it establishes a precedent that respects worker autonomy and data rights. We need more companies to follow suit, putting user privacy and data ownership at the forefront of AI development, rather than treating it as an afterthought.

##

ABOUT THE AUTHOR

As CPO of Careerspan, Ilse Funkhouser brings over a decade of expertise in data science and AI-driven product development, having previously led technical development as a cofounder that resulted in a $13M Series A round. Combining her Northwestern mathematics background with a Master's in Data Science from the University of Wisconsin, she specializes in building sophisticated AI systems that uplift rather than replace human potential. Driven by a mission to make career development more accessible and personalized, Ilse architected Careerspan's proprietary multi-agent AI framework that delivers genuine, human-centered career coaching at scale.