Threat Modeling AI Systems
Map users, prompts, retrieval sources, model endpoints, tools, plugins and downstream actions as separate trust boundaries.
CYBERINTEL HUB / TOPIC MODULE
AI Security
Secure AI applications and LLM workflows while using AI responsibly for defensive operations.
Explore AI Security
Topic-focused references and direct links. This page stays focused on ai security rather than repeating the main portal.
AI security covers both protecting AI applications and using AI safely inside security operations. The focus is on threat modeling, data protection, model/application risks and human validation.
Application risks
LLM applications can introduce risks around prompt injection, sensitive data exposure, insecure tool use, excessive agency and unsafe output handling.
Defensive AI
AI can assist with triage, summarization, enrichment and hunting, but high-impact decisions should retain appropriate human validation.
Governance
Define approved use cases, data boundaries, logging, access controls, testing and accountability before integrating AI into security workflows.
This module presents the technical concepts, operational considerations and defensive engineering practices relevant to the topic.
LLM Threat Modeling
Model trust boundaries between users, prompts, retrieval systems, plugins/tools, models and downstream applications. Assess prompt injection, data leakage, insecure tool invocation and excessive agency.
Data Security
Define what data may enter prompts, retrieval indexes, telemetry and model-training workflows. Apply classification, minimization, access control and retention requirements.
AI-Assisted SOC
Use models for summarization, enrichment and analyst assistance while validating high-impact conclusions against telemetry and authoritative evidence.
AI Governance
Establish approved use cases, model inventory, evaluation criteria, security testing, logging, human oversight and incident-response procedures before production adoption.
Technical Perspective
Architecture, operational and engineering considerations relevant to this topic.
Prompt Injection Defense
Treat retrieved and user-supplied content as untrusted data. Enforce authorization outside the model and do not let model output directly determine high-impact actions.
Model Supply Chain
Track model provenance, dependencies, inference infrastructure, configuration and update processes. Validate third-party models before production use.
Security Testing
Create adversarial test cases for data leakage, unsafe tool use, instruction manipulation and output handling, then track remediation and regression testing.
LLM Security
Understand application-layer risks, data exposure and model misuse.
Open related CyberIntel section →Prompt Injection
Treat model input as untrusted and apply layered validation and authorization.
Open related CyberIntel section →AI for SOC
Use AI to assist triage and analysis while retaining human validation.
Open related CyberIntel section →AI Governance
Define risk, accountability, data handling and acceptable-use controls.
Open related CyberIntel section →Defensive-use: use security tools and techniques only on systems and environments you are authorized to assess. Use intentionally vulnerable labs or isolated test environments for experimentation.