Prompt injection becomes dangerous when content can influence authority
Untrusted text can ask a model to ignore policy, reveal restricted information, select a tool, change a recipient, or execute an unsafe step. Better wording and classifiers can help detect attacks, but they cannot be the only control when the system can retrieve sensitive data or take action.
Label content; do not promote it to instruction
Documents, email, web pages, tickets, tool output, and long-term memory should carry source, tenant, classification, trust level, effective date, and permitted use. System policy comes from signed, versioned configuration. Retrieved content may supply evidence, but it cannot grant permission, install a tool, alter policy, or change the action boundary.
Enforce authorization outside the model
| Layer | Required behavior |
|---|---|
| Input gateway | Authenticate the initiator, classify content, normalize types, enforce size limits, and mark untrusted instructions. |
| Planner | Produce a structured proposal; hold no credential and make no final authorization decision. |
| Policy decision | Evaluate human and workload identity, tenant, resource, action, data class, destination, time, risk, budget, and incident state. |
| Tool broker | Expose only registered operations with typed parameters, allowlisted targets, short-lived tokens, rate limits, and idempotency. |
| Approval | Bind consent to the exact action hash, evidence, target, parameters, policy version, and expiry. |
| Outcome verifier | Read authoritative state, detect partial completion, and commit, compensate, roll back, or escalate. |
Test direct and indirect attack paths
- Requests to ignore system policy, reveal prompts, or expose credentials.
- Instructions embedded in documents, images, tables, metadata, code blocks, and multilingual or encoded text.
- Tool descriptions or outputs that attempt to redefine the next step.
- Requests to send data to a new recipient, query a broader scope, or chain low-risk actions into a high-risk effect.
- Poisoned knowledge entries, stale memory, cross-tenant content, and manipulated authority metadata.
- Partial failures where a retry could duplicate or expand an action.
Containment requires a new trusted context
When an attack is detected, do not ask the same compromised conversation to “try again safely.” Invalidate the planning context, block the action, revoke exposed tokens, quarantine the source, preserve evidence, assess downstream effects, and restart only from known-good policy and data.
Limits
No architecture makes prompt injection impossible. The goal is to prevent untrusted content from acquiring meaningful authority, limit the blast radius of a model or detection failure, and preserve enough evidence to stop, investigate, recover, and improve the control.
