GDPR compliance AI — Can your model undo the damage?

Uber and Twitch make the stakes concrete: control personal data, empower human reviewers, trace model…

The short version

GDPR compliance for AI requires data control, meaningful human review, traceability, and workable deletion or remediation.
Uber received an approximately €825 million fine after software disabled drivers without prior human assessment.
Teams must build stop, trace, override, and repair capabilities into products instead of relying on policies or vendors.

Uber’s software cut drivers off from income before human review. Twitch enabled AI training by default and left users to find the off switch. Welcome to AI privacy in 2026, where models can reason but the undo button remains in beta. For AI systems, GDPR compliance means controlling inputs, explaining consequential automation, honoring data-subject rights, and giving humans real power over major decisions. Policies matter. But if my team cannot stop data entering a model, trace it, or reverse a machine-led decision, the immaculate privacy binder is corporate fan fiction.

Sasha Malysheva posed a different question:

we think it's the wrong question. here's the one we pose: why does a frontier model have to generate your answer at all, when the only thing you need from it is the reasoning?

Monique Verdier, deputy chair of the Dutch Data Protection Authority, captured the human cost in its August 21 Uber announcement. My Dutch runs on optimism and Google Translate, but the original deserves to run intact:

Uber heeft ernstige overtredingen begaan. Chauffeurs werden zonder pardon op non-actief gezet. Waarmee zij dus van het ene op het andere moment geen inkomen meer via Uber hadden. Dat is verboden. Een computer mag niet zelfstandig besluiten nemen die grote gevolgen voor jou hebben. Hier had eerst een mens naar moeten kijken.

Monique Verdier, deputy chair of the Autoriteit Persoonsgegevens, said:

Uber has committed serious infringements. Drivers were deactivated without pardon. From one moment to the next, they no longer had any income through Uber. That's forbidden. A computer should not make decisions on its own that have major consequences for you. These decisions should have been looked at first by a human being.

In plain English: software made a life-changing call, nobody checked first, and drivers suddenly lost access to work.

That is my test. Can I stop the system and find the data? Can the affected person challenge the outcome? Can my team repair the damage? If any answer is “our vendor handles that,” I keep asking questions.

Control the data before the model sees it

GDPR compliance for AI starts with controlling the personal data a system collects, receives, infers, stores, or uses to influence people. The GDPR follows the data; the AI Act imposes separate duties based on the system and its intended purpose. One product can trigger both.

Debating whether a feature is “really AI” is founder catnip: technical, impressive over drinks, and often irrelevant. Prompts, uploaded CVs, account records, voices, inferred profiles, audit logs, and consequential outputs may relate to identifiable people. A machine-failure model using only equipment readings may engage the AI Act without personal data. A customer spreadsheet can trigger the GDPR without a molecule of machine learning. Recruitment software can trigger both by processing applicant information through AI.

Classification has two tracks. First, I map who determines why and how personal data is processed, identifying the likely GDPR controller or processor. Then I record who provides or deploys the AI system, its purpose, and any downstream modifications. A recruitment vendor may provide a system while processing applicant data under a customer’s instructions; the customer may deploy it while controlling that data. Rebranding a tool or substantially changing its purpose can alter its AI Act position without changing its GDPR role. One inventory can support both exercises, but the conclusions need separate labels. Brussels has enough spaghetti; I need not add mine.

Control must follow information everywhere. I approve tools for defined tasks and specify permitted data. I remove identity whenever the job works without it. Sensitive uploads can be blocked; names and account numbers stripped before text reaches a model. Vendor training reuse gets disabled wherever possible. Retention rules cover prompts, retrieval indexes, logs, and outputs. Employees also need an approved product easier than the forbidden browser tab, because convenience body-slams PDF policies every time. If reduced or pseudonymous data works, I send less.

I learned this at a Pasadena kitchen counter, cursor hovering over a chatbot, customer-support export ready to paste. I wanted a quick summary before bed. The file contained email addresses made invisible by hours of staring at columns, and convenience made the bad idea feel normal. I build this stuff, yet my privacy instincts still lost a fistfight with sleep deprivation and a text box.

Vendor diligence belongs in product design, not a questionnaire completed three days before launch. I want direct answers on whether customer inputs become training material, plus usable retention and deletion controls. I ask about subprocessors, transfer locations, and what changes with model updates. The workflow needs logs, known limitations, and instructions matching the product I ship. A data-processing agreement can document responsibility. It cannot reach through the screen and unpaste a customer database.

Twitch shows why defaults matter. In its August 20 warning, the Dutch authority said Twitch’s generative-AI training setting was enabled by default and could cover streams, images, chats, names, voices, and glimpses inside people’s rooms. Amazon could use that material for AI training unless users actively switched it off. Verdier warned that faces and voices incorporated into models may be extremely difficult to remove later. This was regulatory advice about Twitch’s setting, not a final infringement ruling. Nobody knows whether Amazon can fully remove one person’s training data—or its effects inside a model—after training.

Hand closing a webcam privacy shutter beside a monitor, symbolizing GDPR compliance, AI data collection, and European privacy.

Image alt text: Four control points for GDPR compliance in AI systems: stop, trace, override, and delete or remediate.

A human reviewer needs power and context

Meaningful human control is required when software alone makes decisions with legal or similarly significant consequences, unless a valid GDPR exception applies with required safeguards. Reviewers need the context and authority to disagree. Clicking “approve” between two Slack notifications does not count.

I test the consequence before the software’s sophistication. Losing work, being denied credit, getting screened out of employment, or having an account frozen can cross the Article 22 threshold. A basic scoring rule can cause the same damage as a neural network with an expensive French name. Ordinary recommendations usually fall below that threshold, though context can quickly raise the stakes.

A defensible workflow has several links; skip one and the chain breaks. The system receives data and produces a score or recommendation. Before any significant effect, a named reviewer receives relevant evidence and known model limitations. That person can pause, request information, reject the outcome, or escalate. The system records the model version and relevant inputs, linking them to the recommendation and final decision. A separate route lets affected people contest results, explain their position, and obtain genuine reconsideration. Workloads must remain realistic: someone facing a thousand alerts before lunch becomes a rubber stamp in business casual. Human involvement matters only if it can change the outcome.

The Dutch regulator’s Uber case gives this mechanism a painfully expensive anchor. According to its August announcement, Uber software flagged suspected fraud or low customer ratings, then temporarily or permanently disabled driver accounts without prior human assessment. Those actions removed access to paid work, so the authority concluded that the automated decisions had serious consequences and breached GDPR rules governing software-only decisions.

The regulator imposed a fine of about €825 million. The GDPR permits penalties up to 4% of worldwide annual turnover, and the authority estimated Uber’s global turnover at about €45 billion. It also found that drivers lacked adequate information about the automated process.

Uber deserves a fair hearing. The company says the regulator examined historical policies no longer used and that its current process includes human review, safeguards, and an appeal opportunity. Uber has announced an objection, while early outside analysis noted that the full fining decision was not yet public. Without it, outsiders cannot properly inspect the legal reasoning or penalty calculation. The conduct occurred from 2018 through 2022, and the authority says Uber has ended the infringements. We do not yet know how the objection will end.

Uber’s defense also contains an important distinction. An appeal can correct a bad outcome afterward, but cannot replace human assessment before someone loses access to income. Product teams need both stages. Stapling an appeals form onto the workflow after a regulator calls is not product design; it is panic with a submit button.

The case also shows why united European enforcement matters. Reports from 171 French drivers reached the Ligue des droits de l’Homme, which filed a complaint with the French privacy regulator, CNIL. Because Uber’s European headquarters is in the Netherlands, the Dutch authority investigated through the GDPR’s one-stop-shop system. The workers crossed a border; their rights followed.

That is the EU working as intended.

Model genealogy will become compliance infrastructure

Deleting a database row is easy. Removing personal data from modified, fine-tuned, or combined model families is the engineering problem Europe must drag into the open.

Open-weight models create family trees. A developer downloads a base model, fine-tunes it with another dataset, combines it with a second model, and publishes the result. Someone else repeats the process. If an ancestor memorized personal data, related models may retain it or reproduce its effects, though lineage alone cannot prove a record was memorized. Investigators must connect each dataset to its training run and resulting model version, then map downstream deployments and derivatives. Rights requests can follow that genealogy instead of dying in support inboxes. Nobody knows how often memorized personal data survives in each descendant, so traceability is evidence—not magic.

A proper investigation starts with identity verification and obvious source records. I would then search prompt stores, retrieval indexes, output archives, caches, and fine-tuning material. Each match identifies model versions and deployed systems that may need examination. Ordinary stored data can often be deleted or corrected directly. Suspected memorization may require technical testing, vendor help, filtering, model replacement, or retraining. The final response should record what was searched and removed, then explain remaining uncertainty in normal language. “The embeddings team is looking into it” is not an explanation.

France’s CNIL is already building useful machinery. On August 26, it announced an updated Genmod, a demonstrator using public Hugging Face metadata to trace open-weight models’ ancestors and descendants. Its graph database can be rebuilt weekly as derivatives appear. After optimization, a genealogy search with no depth limit averaged about 20 seconds; CNIL published no earlier duration for comparison.

Genmod cannot certify GDPR compliance or prove memorization. Public metadata may be incomplete, while customers using closed APIs need equivalent lineage information from vendors. Still, it shows model families can be searched at practical speed and examined when regulators or individuals ask how access, erasure, and other GDPR rights apply across related models. We do not yet know how often regulators will use this evidence.

The European Commission is moving on the adjacent AI Act front. Servola reported on August 29 that Executive Vice-President Henna Virkkunen said the EU AI Office had “formally sent requests for information to a number of providers of general-purpose AI models, based in different regions of the world.” Those requests concern AI Act obligations; national data-protection authorities continue enforcing the GDPR.

Europe now needs to connect this machinery. Model lineage, human intervention, deletion, and remediation should become shared EU technical standards—not national patchworks navigable only by companies with giant legal departments. A coordinated European market can build compliance infrastructure startups can use while giving homegrown AI companies one serious, demanding market in which to scale.

I am unapologetically federalist about this. European AI champions will not emerge from twenty-seven versions of the same paperwork. They need common infrastructure, strong EU institutions, and rules strict enough to earn trust but concrete enough to implement.

By 2029, enterprise buyers will demand model genealogy with the casual aggression they now reserve for security questionnaires. Any founder unable to answer “where did this person’s data go?” will learn that Europe did not ban the technology.

It made the undo button part of the product.

Frequently asked questions

What does GDPR compliance for AI require?

GDPR compliance for AI requires control over personal data inputs, clear handling of consequential automated decisions, meaningful human review, traceable model and data lineage, and workable deletion or remediation processes. Organizations must also let affected people contest outcomes and obtain genuine reconsideration.

Does GDPR require human review of AI decisions?

Meaningful human control is required when software alone makes decisions with legal or similarly significant consequences, unless a valid GDPR exception applies with required safeguards. The reviewer must receive relevant evidence, understand limitations, and have authority to pause, reject, request information, or escalate the outcome.

Can personal data be deleted from AI models?

Personal data can often be deleted or corrected in source records, prompt stores, retrieval indexes, archives, caches, and fine-tuning material. Suspected model memorization may instead require testing, vendor assistance, filtering, model replacement, or retraining. The response should document what was searched, what was removed, and what uncertainty remains.