Validate the workflow, not only the model. Write down the intended use and rate its risk to patient safety, product quality and data integrity. Then test the whole chain on independent past cases, including the person who reviews the output. Keep generative AI out of critical GMP decisions while Annex 22 is a draft, and log every output and every human change.
We are not your QA unit. We build the workflow so your QA unit can test it. This post is the method we recommend, checked against what EMA, FDA and ISPE have published as of 10 November 2026. It is general information, not regulatory advice.
What is the current rule for AI in GMP?
Nothing new is final. On 10 November 2026, EudraLex Volume 4 still lists Annex 11 in its January 2011 version and no Annex 22. Drafts of a new Annex 22, a revised Annex 11 and a revised Chapter 4 went out for consultation from 7 July to 7 October 2025. Until the final texts are published, the 2011 Annex 11 is the text you are inspected against.
The draft Annex 22 covers AI models in critical applications “with direct impact on patient safety, product quality or data integrity”. It says generative AI and large language models “should not be used in critical GMP applications.” In non-critical uses, it says qualified and trained staff should always be responsible for checking that the outputs suit the intended use.
That line may move. EMA’s workshop report, published on 2 October 2026, says the drafting group “has therefore discussed the scope of Annex 22 to allow dynamic or adaptive models, probabilistic models, and Generative AI/LLMs in GMP applications”, provided they are backed by “a fully documented, robust, risk-based control strategy”. The next sentence reads: “However, concerns remain about the elements needed for such a control strategy.” That is a record of discussion, not a decision.
On timing, EMA’s inspectors’ 2026 to 2028 work plan targets Q4 2026 for handing a final Annex 22 text to the Commission. Minutes of the HMA-EMA Network Data Steering Group from 16 June 2026 list “GMP Annex 22 (workshop in July, publication early 2027)”. Neither is a date of application. For the full scope and history, see what Annex 22 says, and its status.
Model, system or workflow: what are you validating?
The workflow. In January 2026, EMA and FDA published ten guiding principles for AI across the medicine lifecycle, including manufacturing. Principle 8 says risk-based performance assessments “evaluate the complete system including human-AI interactions”.
So we draw the validation boundary around five things: the input, the model step, the automatic checks, the human decision and the record. A model with a strong benchmark can still fail inside a workflow where reviewers approve drafts without reading them. And a vendor’s test of its model says nothing about your documents, your prompts or your reviewers.
The draft Annex 22 already thinks this way about change. Section 10.1 puts “the system it is implemented in, and the whole process it is automating or assisting” under change control, not only the model.
A seven-step method for validating one AI workflow
The draft Annex 22 is written for static models in critical use. For generative AI in non-critical use, the draft says its principles “may be considered where applicable.” We apply them anyway, because they are the clearest written statement yet of what EU inspectors expect from an AI model. Each step below names the section it borrows from.
1. Write the intended use
The process owner writes it, not the vendor. Section 3.1 of the draft asks for the task, the input data “and all common and rare variations”, and known limits. A process expert approves it before acceptance testing starts.
A hypothetical example: “Draft a first summary of a deviation file for the investigator, with a link to every source record. Out of scope: severity classification and closure.” Two sentences like these decide most of what follows.
2. Make the criticality call
Ask one question. Does the output decide or record something that affects patient safety, product quality or data integrity? If yes, the current draft keeps generative AI out. If no, write down why in a few sentences, and have QA sign it.
The workshop report gives you words for that reasoning. It lists decision consequence, model influence, uncertainty, complexity and detectability as factors to weigh. A summary that an investigator checks against linked records has low model influence. The same summary pasted straight into the deviation record does not.
3. Set acceptance criteria before you test
Section 4.2 asks for acceptance criteria that are documented and approved before acceptance testing. Section 4.3 adds that they “should be at least as high as the performance of the process it replaces.” You can only meet that if you know how well the manual process performs today, so take a baseline with real past cases first.
For a summary task, criteria might read: every statement traces to a linked source record, and no deviation-relevant fact is missing. Write them down before anyone sees a test result.
4. Test on independent past cases
Sections 5 and 6 of the draft deal with test data. It should cover the full range of real inputs, including rare ones. It should not be used during development. People who have seen it should not train the model. Where that is impossible, they work in a pair with a colleague who has not seen it.
For an LLM workflow, development means prompts and retrieval. Keep the test cases away from whoever writes the prompts. Use real past cases with known answers. The draft says generating test data with generative AI “is not recommended”. We also suggest running each case more than once, because the same prompt can give a different answer.
5. Test the human step too
Regulators asked about this step directly. At the EMA workshop, regulators asked “how to monitor the performance of the human performing oversight; how to manage automation bias”. The answers named training, qualification, knowledge testing, audit trails, “recording of AI outputs and human modifications”, and monitoring of human-AI team performance as possible evidence.
We suggest planting known errors in some test outputs and recording whether reviewers catch them. That gives you a number for the human step, not only for the model. Section 3.3 of the draft says operator training and consistent performance “should be monitored like any other manual process.” More on designing the approval point.
6. Collect supplier evidence
Buying the model does not buy you out of validation. The revised Annex 11 draft says relying on a vendor “does not change the requirements” and that the regulated user “remains fully responsible” (section 7.1).
The workshop report is blunter about AI vendors. On page 9 it records: “If a third-party model or Application Programming Interface (API) cannot provide necessary outputs, rationale, logs or evidence needed for control, the manufacturer cannot measure and justify the system adequately.” Ask your vendor three things in writing: can we pin the model version, will we see release notes before a change, and what logs do we get. Our checklist on agent permissions and logs covers the rest.
7. Put every change under change control
A new model version is a change. So is a new prompt, a new retrieval source or a new tool the AI can call. Section 10.1 says any change to the model, the system or the process “should be documented and evaluated to determine if the model needs to be retested.”
Write the retest triggers into the validation plan before go-live. Then watch the workflow in use: how often reviewers edit the output, how often they reject it, and what gets escalated. Fold those numbers into the periodic review.
Where GAMP, CSA and the EMA/FDA principles fit
None of these replaces the others. This is how we read them side by side.
Framework | Scope | What it adds | Status (10 Nov 2026) |
|---|---|---|---|
Draft Annex 22 (EU and PIC/S) | AI models in critical GMP applications | Intended use, acceptance criteria, independent test data, change control | Consultation draft, July 2025 |
Draft revised Annex 11 | All computerised systems in GMP | Requirements for software-as-a-service, supplier responsibility, audit trails | Consultation draft; the 2011 version applies |
EMA/FDA guiding principles | AI across the medicine lifecycle, including manufacturing | Ten principles; principle 8 assesses the complete system | Published 14 January 2026 |
EMA reflection paper | AI in the medicinal product lifecycle | EMA’s current thinking on AI use, as background | Adopted; first published 30 September 2024 |
ISPE GAMP Guide: Artificial Intelligence | AI-enabled computerised systems in GxP | Lifecycle approach and the duties of regulated companies and suppliers; used with GAMP 5 (Second Edition) | Published July 2025 (paid guide) |
FDA Computer Software Assurance | Software in medical device production and quality systems (21 CFR 820) | Risk-based assurance; names AI/ML tools; credits work done by suppliers | Final guidance, issued 3 February 2026 |
One caution on CSA. It is FDA device guidance under 21 CFR 820, not drug CGMP guidance. The CSA guidance replaced the September 2025 version, and it says its risk framework can apply to “artificial intelligence/machine learning tools”. It also lets manufacturers count validation activities done by “developers, suppliers, cloud service providers”. The thinking travels well. The rule does not apply to a drug site.
What can you use generative AI for today?
Non-critical work, each task with a named reviewer who has the sources in front of them:
Drafting an SOP revision that then goes through document control.
Summarising a deviation file for the investigator, with a link to every source record.
Searching SOPs and guidelines, with a citation to the exact section.
Our Annex 22 task table pairs each of these with the critical task next to it that stays off limits. The FDA put the review duty plainly in a warning letter in April 2026: “If you use AI as an aid in document creation, you must review the AI generated documents to ensure they were accurate and actually compliant with CGMP.”
What will an inspector ask to see?
A file per AI workflow. An industry speaker at the EMA workshop called this an “AI qualification dossier”. We would put nine things in it:
The approved intended use.
The criticality rationale, signed by QA.
Acceptance criteria and the test plan, approved before testing.
The test set, the results and any deviations from the plan.
Reviewer training records and the results of the planted-error test.
The audit trail: each AI output next to the approved version, with who changed what.
The supplier assessment and contract terms.
The change log, with each retest decision and its reason.
Monitoring data and the latest periodic review.
FDA has already put the human step in writing. In Warning Letter 320-26-58 of 2 April 2026 to a drug manufacturer, FDA wrote that “any output or recommendations from an AI agent must be reviewed and cleared by an authorized human representative of your firm’s QU.”
FAQ
Can generative AI be used in GMP?
Yes, for non-critical tasks, with a qualified person responsible for each output. Under the July 2025 draft of Annex 22, generative AI and large language models should not be used in critical GMP applications, meaning those with direct impact on patient safety, product quality or data integrity.
When will Annex 22 be final?
No date has been announced. EMA’s inspectors’ work plan targets Q4 2026 for handing a final text to the European Commission, and minutes of an HMA-EMA network meeting in June 2026 say publication early 2027. The final text will set its own date of application.
Is the Annex 11 revision in force?
No. The revised Annex 11 is a consultation draft, and the consultation closed on 7 October 2025. The January 2011 version of Annex 11 still applies.
How do you validate LLM output?
Test the workflow on real past cases with known answers, kept apart from whoever writes the prompts. Set the acceptance criteria before testing, run cases more than once, and test the human reviewer too. Validation shows the workflow is fit for its intended use. It does not make LLM output deterministic.
Does human review replace validation?
No. When a regulator at the EMA workshop asked whether human-in-the-loop could ever justify reduced validation, the question was deferred, and an industry speaker answered that it is risk-based rather than a categorical yes or no. The report’s key messages also say oversight “cannot compensate for inadequate validation.”
Where to start
Pick one non-critical task in QA or documentation. Write its intended use in two sentences, make the criticality call with QA, and pull a set of past cases with known answers. That is enough to start a validation plan this month, before the final Annex 22 text lands.
Book a free workflow audit if you have one QA or documentation task you would like AI to draft. Bring the intended use in two sentences. In the 20-minute call we will sketch the workflow and the test plan your QA team would need to sign off.
Sources
European Commission, EudraLex Volume 4: Good Manufacturing Practice guidelines, accessed 10 November 2026.
European Commission, Stakeholders’ consultation on EudraLex Volume 4: Chapter 4, Annex 11 and new Annex 22, 7 July to 7 October 2025.
European Commission, Draft Annex 22: Artificial Intelligence, consultation version, July 2025.
European Commission, Draft revised Annex 11: Computerised Systems, consultation version, July 2025, section 7.1.
EMA, Report: Multistakeholder workshop on expert contributions to artificial intelligence guidance development (Annex 22), EMA/156789/2026, workshop of 30 June 2026, published 2 October 2026 (event page), pages 3, 6, 7, 8 and 9.
EMA, 3-year work plan for the GMDP Inspectors Working Group, 2026 to 2028.
EMA, Final minutes, HMA-EMA joint Network Data Steering Group meeting, 16 June 2026, EMA/143308/2026, 22 June 2026.
EMA, EMA and FDA set common principles for AI in medicine development, 14 January 2026, and the Guiding principles of good AI practice in drug development.
EMA, Reflection paper on the use of AI in the medicinal product lifecycle, EMA/CHMP/CVMP/83833/2023, first published 30 September 2024.
ISPE, GAMP Guide: Artificial Intelligence, July 2025.
FDA, Computer Software Assurance for Production and Quality Management System Software, final guidance, issued 3 February 2026, pages 5 and 6.
FDA, Warning Letter 320-26-58, Purolea Cosmetics Lab, 2 April 2026.