← Blog

"AI-Enabled Medical Device Threat Model Checklist: FDA-Specific Attack Surfaces"

June 3, 2026

Most threat models for connected medical devices cover the same ground: network interfaces, authentication, firmware integrity, data transmission, and software update mechanisms. That coverage is necessary. For an AI-enabled device, it is not sufficient.

FDA's January 2025 draft guidance on AI-enabled devices added security requirements that sit outside the standard device threat modeling framework. The guidance specifically calls out data integrity in the inference pipeline and the connection between security and performance monitoring as areas requiring dedicated attention. I covered what the guidance requires in my breakdown of the full document.

The AI-specific attack categories, including data poisoning, adversarial inputs, model integrity attacks, and inference pipeline manipulation, are not surfaces that STRIDE and similar methodologies were designed to surface. This checklist covers them. It aligns with FDA's January 2025 draft guidance and the NIST AI Risk Management Framework (AI RMF), which FDA references as a complementary resource.

This list covers threat modeling for AI-enabled devices in FDA-regulated contexts. It assumes you already have a working threat model for the underlying connected device. The items below are additive: AI-specific surfaces on top of standard device security.

1. Training Data Provenance

Have you documented the sources of all training data, including the institutions, collection equipment, and collection timeframes?
Is there documented data provenance for your training data that would allow you to demonstrate to FDA that data integrity was maintained between collection and use?
If training data was sourced from third parties, do you have contracts and attestations covering data integrity?
Have you assessed the risk that training data was unrepresentative in ways that would affect model performance in the deployed population?
Have you modeled data poisoning risk: the possibility that training data was intentionally corrupted to cause the model to learn incorrect patterns or produce targeted incorrect outputs?

Training data is part of the mechanism of action for an AI-enabled device. FDA evaluates it accordingly. A threat model that ignores the provenance, integrity, and poisoning risk of training data is incomplete. The NIST AI RMF's GOVERN and MAP functions address data provenance and supply chain risk at this level. If your team is using the AI RMF, the training data controls should already be mapped there and can be referenced directly in the submission.

2. Development Environment and Data Sequestration

Is test data sequestered from all development processes, with documented access controls?
Can you demonstrate that no model developer had access to test data prior to final evaluation?
Are there controls preventing post-hoc tuning after the test dataset was fixed?
Is the model training infrastructure access-controlled and logged?

Data leakage between development and test sets is both a technical threat and a regulatory problem. The security controls around your development environment need to be documented and defensible.

3. Inference Pipeline Integrity

Have you identified every hop in the data pipeline from input acquisition to model inference?
Is each hop in the pipeline treated as a security boundary, with integrity validation on data entering and exiting?
Do you have controls to detect corrupted, manipulated, or degraded inputs before they reach the model?
Have you modeled adversarial input attacks: subtle, crafted perturbations to input data that cause the model to produce incorrect outputs while appearing normal to a clinician?
Have you modeled the consequence of a subtle input corruption that produces plausible but incorrect model output?
Are there alerting or logging mechanisms that would surface unexpected changes in input data characteristics?

This is the surface most teams underestimate. For any device where model output informs a clinical decision, corrupted or manipulated input that produces a plausible but incorrect output is a patient safety event. That is what the threat model has to account for.

4. Model Integrity and Update Pathway

Is the deployed model stored in a way that prevents unauthorized modification?
Are model files verified against a known-good signature or hash at load time?
Is the model update pathway authenticated and integrity-checked end to end?
Have you modeled the risk of a compromised model update and what an adversary could do by replacing your model with a modified version?
If your device supports a Predetermined Change Control Plan (PCCP), have you modeled the security risks of the automated update pathway specifically?

A model that can be replaced by an adversary is a device that can be made to produce any output the adversary wants. The update pathway deserves the same threat modeling attention as firmware update mechanisms. NIST SP 800-218A (Secure Software Development Framework for AI) addresses model integrity verification and deployment pipeline security at the implementation level. If your team is using it, those controls map directly to this section of the FDA submission.

5. Output Handling and Clinical Decision Interface

Have you modeled what a clinician would do if the model produced a plausible but incorrect output?
Is there any logging or audit trail of model outputs that would allow post-hoc review?
Are there rate limits or anomaly detection on model output that would surface unexpected output patterns?
If model output is integrated into an automated clinical workflow, have you modeled the consequences of adversarially manipulated output in that workflow?

The clinical interface is where a model security event becomes a patient safety event. The threat model needs to trace that path explicitly.

6. Performance Monitoring as a Security Signal

Does your performance monitoring plan include the ability to distinguish between drift caused by population changes and drift caused by upstream data pipeline anomalies?
Are there alerting thresholds on model performance metrics that would trigger a security investigation, not just a product investigation?
If performance degrades at a specific site or in a specific patient subgroup, do you have a process to determine whether that degradation is a security event?
Are performance monitoring logs protected against tampering?

FDA's January 2025 guidance explicitly links cybersecurity and performance monitoring. A security event affecting input data integrity will look like performance degradation. A monitoring system that can only detect performance changes, without being able to investigate their cause, will miss security events.

7. Third-Party Model Components and Dependencies

If your device uses a third-party foundation model, pretrained weights, or external inference service, have you modeled the supply chain risk of those dependencies?
Do you have a process for tracking security advisories from third-party AI component vendors?
Is your SBOM inclusive of AI model components and training libraries, not just software packages?
If a third-party model component is updated by the vendor, do you have a process to evaluate the security implications before deploying the update to your device?

AI-enabled devices often have a deeper and less-visible dependency tree than traditional software devices. Foundation models, pretrained embeddings, and cloud inference APIs all represent supply chain attack surfaces that need to be in your threat model. The NIST AI RMF's MEASURE function covers third-party AI component risk assessment and is a natural fit for this section of your submission documentation.

Frequently Asked Questions

Does FDA require a separate threat model for the AI system, or does it extend the existing device threat model?

FDA does not require a separate document, but it expects your threat model to explicitly cover the AI-specific attack surfaces: training data provenance, inference pipeline integrity, model update pathway, and performance monitoring as a security signal. If your existing threat model doesn't address those surfaces, adding them as a dedicated section is cleaner than retrofitting them into a framework built for traditional device components. The reviewer will be looking for evidence that these surfaces were considered, not checking for a specific document structure.

Does the NIST AI RMF replace or supplement STRIDE for AI-enabled device threat modeling?

It supplements. STRIDE is still a valid methodology for the traditional device attack surface. The NIST AI RMF adds a governance and risk management layer specific to AI systems, covering areas like bias, transparency, and model drift that STRIDE doesn't address. FDA's January 2025 guidance references the AI RMF as a complementary resource, not a replacement for the threat modeling your SPDF already requires. Use both.

At what stage of development should AI-specific threat modeling happen?

The same answer as for traditional device threat modeling: as early as possible. FDA's February 2026 cybersecurity guidance is explicit that threat modeling should happen "throughout the design process," not at submission time. For AI-specific surfaces, this matters more, not less. Decisions about training data sources, inference architecture, and model update pathways made during development will determine what your threat model can honestly say. A threat model built after those decisions are locked in can only document what you built. It cannot improve it.

Using This Checklist

This is a starting point, not a complete threat model. The items above define the surfaces to cover. Working through them in the context of your specific device architecture will surface the risks that matter for your product.

The FDA guidance that prompted this list, the January 2025 draft on AI-enabled devices, is still in draft form. But reviewers are using it now, and the security requirements it sets out reflect where the agency's thinking has landed. Building a threat model that covers these surfaces now means your submission documents what you actually built, not what you retrofitted under deadline pressure.

If you want to work through the AI-specific threat modeling for your device, book a 30-minute call. I have worked through security architecture and threat modeling requirements on FDA-regulated devices deployed to live patients. Bring your architecture and we will work through it.