Categories
DTQ Events

Report: Shadow AI and the Human Cost of Uncontrolled AI Adoption

Categories
DTQ Events

Report: Shadow AI and the Human Cost of Uncontrolled AI Adoption

Data Trust Quotient, a strategic platform and community of thought leaders working at the intersection of data protection, cybersecurity, and data governance, on May 27, 2026, convened a virtual session. The discussion, moderated by Commander Aditya Varma (Retd), brought together five leaders from cybersecurity, enterprise AI, operational resilience, compliance, and critical infrastructure to confront one of the most urgent and under-governed risks in enterprise technology today: Shadow AI — the quiet, well-intentioned, and deeply dangerous adoption of AI tools outside organizational oversight.

Speaker Profiles

Commander Aditya Varma (Retd) — Moderator, Leader Public Sector Security, Cisco (India & SAARC)

The moderator brought two decades of military service and deep experience in public sector cybersecurity to the panel. Drawing on his background at Cisco, where he leads public sector security for India and SAARC, he guided the conversation with sharp operational framing — connecting shadow AI governance to cybersecurity fundamentals like zero trust, observability, and the “security is everybody’s responsibility” doctrine. He closed the main discussion with a crisp four-point synthesis that captured the session’s collective message.

Shivendra Singh Yadav — CTO, NVIDIA Ecosystem, HCL Tech

With a focus on AI transformation, generative AI, and scalable enterprise architecture, this speaker offered a practitioner’s view of what shadow AI looks like from inside a large technology organisation. He coined the memorable phrase “competence camouflage” to describe employees using public LLMs to produce polished outputs without disclosing their AI use — a psychological response to performance pressure, not malicious intent. He also outlined practical architectural responses including API monitoring tools, enterprise-licensed frontier model access, and the concept of AI factories — on-premise AI infrastructure that can reduce both risk and token costs simultaneously.

Sandeep Patel — Independent Cybersecurity & Compliance Consultant

With twenty years of experience across cybersecurity readiness, global operations, and digital transformation, this speaker focused on the governance and regulatory dimensions of shadow AI. He highlighted the particular vulnerability of mid-market and small organisations, which lack both the budget and the personnel to establish governance structures. He raised pointed concerns about India’s regulatory readiness, noting that the Digital Personal Data Protection (DPDP) Act’s enforcement deadline is still being treated with complacency by many organisations. He also made the session’s most forward-looking educational argument: that AI accountability must become part of school curricula, not just corporate training.

Sagar S — Principal Business Continuity & Operational Resilience Consultant, Cohesity

Drawing on extensive experience in operational risk, cyber disruption, and resilience — including frontline work during the 2017 NotPetya cyberattack — this speaker brought a resilience-first lens to the shadow AI problem. He argued that accountability for AI usage cannot sit only at leadership level; it must be distributed to every individual using the tools. He noted that many organisations are knowingly accepting AI governance risk in the short term in exchange for productivity gains, with a plan to govern later — a posture he treated with cautious concern.

Gaurav Ranade — CTO, Technocentric Advisory

With over 27 years of experience across cybersecurity, telecom, and digital transformation, this speaker offered the session’s most technically grounded and systemically wide perspective. He argued that shadow AI is not only an employee behaviour problem but an infrastructure problem — AI tools embedded in enterprise systems may themselves be passing data to unknown third parties or state actors. He drew a sharp parallel between the current shadow AI situation and the BYOD (Bring Your Own Device) crisis of years past, and warned that no organisation has yet built a truly integrated architecture combining data center design, security framework, and AI governance.

Key Insights from the Discussion

1. Shadow AI Is Not an IT Problem — It Is a Human and Leadership Problem

The session’s opening framing was clear and deliberate: shadow AI does not enter organisations because employees are bad actors. It enters because they are trying to work faster, look smarter, and stay competitive — and the organisation has not given them a sanctioned way to do so.

The host’s reference to the Samsung incident was the clearest illustration. Engineers pasting source code into ChatGPT were not acting irresponsibly by their own logic. They were solving an immediate problem. The failure was upstream — no governance structure had anticipated the behaviour, and no sanctioned alternative had been provided.

The moderator summarised the root cause plainly: shadow AI is caused by unmet enterprise demand for speed, intelligence, and productivity. Governance must therefore enable, not merely restrict.

“Shadow AI isn’t just a security problem caused by bad actors. It’s a human problem created via good intentions.” — The Host

2. Competence Camouflage: The Psychological Driver Nobody Talks About

One of the session’s most striking concepts came from the enterprise AI leader at HCL Tech: “competence camouflage.” Employees across seniority levels — managers, team leads, individual contributors — face performance pressure that creates a psychological incentive to use AI tools secretly. When the organisation has not yet mandated or provided AI access, employees turn to public LLMs to produce more polished presentations, better-structured emails, and refined reports.

The tell-tale sign: when output quality suddenly spikes uniformly across a team, and enterprise AI utilisation logs show near-zero usage, the gap reveals where the work is actually being done.

His observation about the consequences went further: employees unknowingly training public LLMs with proprietary organisational data means that structured reports, internal analyses, and strategic frameworks are effectively becoming freely accessible to anyone querying the same tools. The data flows out not through any malicious exfiltration, but through the normal act of trying to do a better job.

3. Banning AI Is a Failed Strategy — Bring It Inside Instead

All five speakers converged on a consistent and emphatic position: organisations that respond to shadow AI by banning tools or threatening employees are making the problem worse, not better.

The enterprise AI leader noted that employees will simply pay for a personal subscription — $10 or $20 a month — and continue using the tools outside any line of visibility. The cybersecurity and compliance consultant confirmed that threats of disciplinary action drove more usage underground, not less. The result: the organisation has neither visibility nor control.

The solution proposed was consistent across the panel — channelise rather than restrict. Bring frontier models into the enterprise environment under appropriate guardrails. Offer enterprise-licensed access. Give employees a sanctioned alternative that is better than what they would access privately. As one speaker framed it: if employees are using a free Gemini subscription and you offer them a $20 Gemini Pro subscription under enterprise terms, no one refuses.

“The faster you bring all these tools into your enterprise purview, the better it is — rather than refraining people from using it.” — Enterprise AI Leader, HCL Tech

4. Mid-Market and SME Organisations Face a Disproportionate Risk

While large enterprises have gatekeepers, audit functions, and dedicated security teams, the cybersecurity and compliance consultant identified small and medium organisations as the sector most exposed to shadow AI damage — and least equipped to respond.

These organisations view AI productivity tools as a business benefit, not a governance challenge. They lack the budget to deploy monitoring infrastructure. They often have no designated person evaluating which AI tools are safe for use. And when a breach occurs, the impact on customer confidence and operational integrity can be existential.

The broader India-specific point raised was equally significant: with DPDP enforcement deadlines still being treated as flexible and AI adoption accelerating rapidly, a large portion of the economy is building on a governance foundation that does not yet exist.

5. Digital Sovereignty Is the Deeper, Less-Discussed Risk

The enterprise AI leader reframed shadow AI as a sovereignty problem, not just a security problem. Sovereignty, he argued, means three things: your data, your infrastructure, and your trusted people. In the current shadow AI landscape, none of those three conditions is being met.

When an employee submits organisational data to a public LLM hosted in another country, the data is not theirs anymore. The infrastructure is not theirs. And the model is being trained — unknowingly — by every user who submits data to it, including competitors, analysts, and adversaries doing the same.

The CTO at Technocentric Advisory expanded this to critical infrastructure: AI tools embedded in defence, government, and public sector environments may themselves be transmitting data to unknown external parties or state actors. This is not a behavioural risk — it is an architectural risk. And it is one that no governance framework in India has yet addressed at the systemic level.

6. Governance Needs Architecture and Telemetry, Not Just Policy

A consistent thread running through the technical answers was that policy documents cannot solve a shadow AI problem. The enterprise AI leader was direct: by the time a policy has been written, circulated, and acknowledged, employees have already adopted three new tools that the policy does not cover.

What organisations need instead is observability — end-to-end visibility across the technology stack, from the API calls being made to the data egressing through employee devices. Tools cited during the session included Microsoft Purview, Varonis, AWS Bedrock Guardrails, and NVIDIA’s guardrails framework.

The moderator added a key structural point: the CICD pipeline needs to be monitored from model onboarding through to deployment, with stress testing at each stage. The conversation also flagged AI agents as the next observability frontier — autonomous systems that act on behalf of users, with their own API calls, data access, and decision-making, represent an exponential expansion of the attack surface. An ungoverned AI agent with access to financial systems or communication channels is not a hypothetical risk; it is an imminent operational reality.

7. Accountability Must Be Distributed, Not Delegated Upward

The operational resilience consultant made a point that echoed the moderator’s military background: accountability for AI usage cannot sit only at the CISO level, the CTO level, or any single function. It must exist at every layer — the individual contributor, the team lead, the business unit head, and the board.

The moderator reinforced this with a principle from naval service: security is everybody’s responsibility. If someone sees unsafe AI usage in their team, the correct response is not to wait for a governance committee to convene. It is to intervene.

The enterprise AI leader framed this behaviourally: accountability is not achieved through policy mandates but through behavioural design. Making safe AI tools more attractive than unsafe ones, building enterprise guardrails into tools people already want to use, and measuring shadow AI usage through indirect means — like blog writing contests that reveal whether employees are drawing on enterprise tools or external LLMs — are the kinds of creative accountability mechanisms that actually work.

8. The Insider Threat Has Been Permanently Redefined

The session closed with audience questions that crystallised one final insight: the boundary between cyber risk and human risk has dissolved.

The cybersecurity and compliance consultant noted that physical security controls — no phones in server rooms, paper-based data handling — are now entirely irrelevant. Every browser, every application, every AI assistant running on every device is a potential exfiltration point. The risk now lives in every click, every prompt, every query an employee submits without fully understanding its downstream consequences.

The CTO at Technocentric Advisory was unambiguous: shadow AI will not go away. It is not a phase. It is an enduring structural condition of modern enterprise, just as insider threats have always existed. The goal is not to eliminate it; it is to mature the organisation’s ability to see it, contain it, and respond when it surfaces.

“Shadow AI will remain in future forever.” — CTO, Technocentric Advisory

Conclusion

The session closed with the moderator drawing together four dimensions that every enterprise leader must now hold simultaneously: shadow AI creates invisible operational exposure; it challenges trust, sovereignty, and organisational control; it requires architecture and telemetry, not just policy; and it directly affects customer confidence, privacy, and accountability.

The answer, the panel agreed, is not fear-led restriction. It is responsible enablement — giving employees safe AI pathways, making usage visible, classifying data rigorously, governing the tools in the environment, holding vendors accountable, and keeping humans responsible for every consequential decision.

The human, as the moderator concluded, must stay in the loop.

Categories
Events

Report: Who Owns AI Accountability? Security, Legal, Compliance, or the Boardroom?

Categories
Events

Report: Who Owns AI Accountability? Security, Legal, Compliance, or the Boardroom?

Open Innovator, on May 21, 2026, hosted a virtual session that brought together four senior leaders across cybersecurity, technology, finance, and compliance to answer one of the defining questions of the AI era: When AI fails inside an enterprise, who picks up the phone? The discussion was moderated by Agrima Sharma and co-hosted by Ananya Gulati.

As it is known, Open Innovator is a thought leadership platform that convenes cross-functional leaders from technology, security, legal, compliance, and the C-suite to tackle the most pressing challenges at the intersection of innovation and accountability. Through live panel discussions, recorded sessions, and community-driven conversations, OI creates a space where practitioners speak plainly about what governance, risk, and responsible deployment really look like on the ground.

Speaker Profiles

Josh Scarpino — Cybersecurity & AI Governance Leader

Josh Scarpino brought a cybersecurity-first lens to AI accountability. He referenced the ARISE framework, which advocates unifying governance across ethics, legal, security, and AI oversight functions into a single operational model. He drew parallels between AI governance failures and longstanding cybersecurity lapses, arguing that organisations are measuring the wrong things — treating governance as a documentation exercise when it must be a demonstrable, measurable practice.

Will Lassalle — CTO & CISO

Will Lassalle spoke from the dual perspective of a technology and security executive, arguing that poorly engineered AI solutions — not just poor governance — are at the root of failures like the Rite Aid case. He emphasised the importance of AI operating committees, controlled deployment, and accountability at the C-suite level. He pushed back firmly against placing sole responsibility on the CISO, calling it both unfair and structurally flawed.

Olivia Phillips — Cybersecurity & Compliance Leader

Olivia Phillips brought the lens of structured, military-grade accountability to the discussion. Drawing on her government background, she advocated for explicit ownership at every layer of the enterprise — from the code level to the board — with clear structures that eliminate ambiguity when something goes wrong. She raised an important point about AI as an insider threat once deployed, requiring ongoing monitoring, re-evaluation, and access governance.

JC Spierer — Finance, Investment & AI Strategy Advisor

JC Spierer introduced the often-overlooked role of finance and investment committees in AI governance, coining the term “prosumer paradox” to describe how business users across organisations — including board members — are adopting AI tools informally, outside of IT oversight. He used BlackRock as an example of an organisation that successfully aligns risk with reward at scale, and raised thought-provoking questions about how accountability for agentic AI systems can be enforced.

Key Insights from the Discussion

1. The Rite Aid Case: A Leadership Failure, Not a Technology Failure

The session opened with the story of Rite Aid Pharmacy — a Fortune 200 company that installed facial recognition cameras in hundreds of stores, built the system using tens of thousands of low-quality images, and deployed it without rigorous testing. The result: innocent customers were flagged as shoplifters, followed through stores, searched, and in some cases had police called on them.

The key insight from the panel: this happened not because the technology was exotic or the company was reckless, but because no one in the leadership pipeline asked who owned the decision. Engineers assumed legal reviewed it. Legal assumed security had audited it. Security assumed compliance signed off. Compliance assumed the board had authorised it. No one had.

2. Accountability Is a Board-Level Obligation — But Responsibility Is Shared

All four speakers converged on a nuanced view: ultimate accountability must sit at the board or CEO level, but every function — engineering, security, legal, compliance, product — carries responsibility for its part of the pipeline.

The cybersecurity governance leader made the analogy to cybersecurity: just as “security is everybody’s responsibility” is the accepted norm for protecting against phishing and human error, so too must AI risk be owned across functions. But when it comes to technology deployed at organisational scale, there must be a distinct, senior-level accountability holder — not a committee that diffuses blame.

3. The CISO Is Being Unfairly Scapegoated

A recurring theme was the industry’s troubling tendency to land all AI accountability on the CISO. Speakers agreed this is both structurally wrong and operationally dangerous.

The cybersecurity and compliance leader noted that the CISO has historically been the “scapegoat” in security failures, and AI is following the same pattern. The CTO & CISO referenced peers who now joke that CISO stands for “Career Is Soon Over” — a reflection of unrealistic expectations placed on a single executive.

The panel’s consensus: the CISO is well-positioned to manage security risk and compliance best practices, but should not be the sole owner of AI governance. A cross-functional AI Operating Committee or AI Governance Committee, with representation from all business units and accountability at the C-suite level, is the right structure.

4. Governance Must Be Operational, Not Just Documented

The cybersecurity governance leader challenged the common enterprise approach of treating AI governance as a documentation problem — policies, frameworks, audit checklists. His argument: documentation governs human behaviour, but autonomous systems behave differently.

When an AI model drifts from its original parameters, or when a deployment decision was made based on policies that have since become outdated, point-in-time audits will not catch the issue. Governance must be continuous, measurable, and tied to demonstrable system behaviour.

A recent statistic cited during the session: 78% of organisations cannot confidently submit an independent AI governance audit within 90 days. That means roughly 4 out of 5 companies do not fully know what they have built and deployed.

5. The Prosumer Paradox: AI Is Already Inside the Boardroom

The finance and AI strategy advisor introduced one of the session’s most distinctive concepts: the prosumer paradox. Half the people in any boardroom are likely already using AI tools — on their laptops, on their phones — without formal IT oversight. These prosumers are not doing anything malicious; they are simply trying to be productive. But they are taking on risk the organisation has not accounted for on its balance sheet.

His point: the finance and investment committee is often the first to know about AI adoption at scale, because at some point, money must be allocated or approved. Bringing this committee into AI governance structures earlier is an underutilised lever.

6. Speed vs. Safety: The Hot Take Debate

The panel debated a pointed hot take: “Companies that move fast on AI and skip governance will win by 2028. The cautious ones will be acquired or irrelevant.”

The responses reflected the complexity of the real landscape:

  • Finance & AI Strategy Advisor (nuanced yes/no): If you move fast and move right, you will win. But velocity without direction leads to crashes, not victories.
  • Cybersecurity & AI Governance Leader (disagrees): Recent legal precedents — including a judge ruling that a venture capital firm could be held liable for advising a portfolio company to cut cybersecurity budgets — signal a coming shift. Organisations that ignore foundational governance will become uninvestable.
  • CTO & CISO (it depends): The jury is out. If everyone rushes in without governance, the most cautious organisations may end up being the only ones still standing.
  • Cybersecurity & Compliance Leader (history repeats itself): The COVID-era remote work rush created BYOD governance failures that took years to resolve. AI is following the same arc. Governance cannot chase deployment; it must run alongside it.

The panel’s collective conclusion: you can build boldly and govern well at the same time. The two are not in opposition.

7. Agentic AI Raises Accountability Questions No One Has Answered Yet

The finance and AI strategy advisor raised the session’s most forward-looking concern: agentic AI — systems that not only execute tasks but train themselves and exercise a degree of independent agency — creates accountability structures that existing governance models are not equipped to handle.

If an agentic AI goes awry, with good intention but bad outcomes, how do you hold it accountable in any meaningful sense? How do you assign consequences? The panel acknowledged there are theoretical answers — including proxy accountability assigned to the human responsible for the system — but noted that no enterprise governance framework has operationalised this yet.

The cybersecurity governance leader added a technical concern: a shared knowledge layer across agentic systems — often proposed as a governance solution — also creates a single, high-value attack vector. If compromised, it could bias an entire agentic workflow.

Conclusion

The session closed with the moderator drawing together the central thread: AI does not fail because technology is broken. It fails because no one in the room raises their hand and says, “That’s my responsibility.”

The Rite Aid case was not an outlier. It was a preview. Across industries, organisations are deploying AI systems with unclear ownership, untested assumptions, and governance frameworks that exist on paper but not in practice.

The panel’s unified message to every leader in attendance: go back to your organisation tomorrow and find the person who is supposed to raise that hand. If you cannot name them, that is not a technology problem. That is your problem. A Part 2 of this conversation is planned, given the depth of interest and the volume of questions that could not be addressed in the session.


This report is based on the recorded panel discussion hosted by Open Innovator on May 21, 2026. All insights are attributed to the respective speakers.