THE SHORT ANSWER
AI can assist threat analysis, anomaly detection, alert triage, policy review, phishing detection and code review. It can also make impersonation and scams more convincing, leak sensitive data, generate insecure code and automate mistakes. Use bounded access, representative evaluation and human oversight.
Use AI as bounded assistance
| Use | Potential help | Control |
|---|---|---|
| Alert triage | Prioritize investigation | Validate missed and false alerts |
| Security assistance | Summarize evidence or policy | Verify sources and authority |
| Phishing detection | Identify suspicious patterns | Keep independent verification |
| Code review | Suggest risky patterns | Require secure testing and expert review |
Deception becomes cheaper and more convincing
- Synthetic voice, image or message impersonation
- Automated social-engineering variation
- Sensitive data entered into inappropriate AI services
- Insecure or invented generated code
- Overconfident automated security actions
- Model, prompt and integration changes that alter behaviour
Do not grant trust because output sounds confident
Minimize data, limit permissions, evaluate representative failures, monitor cost and actions, and require approval for high-impact changes. Do not use AI output as proof that a system is secure.
Turn guidance into an owned business action
Choose one relevant account, system, data set or workflow. Record the owner, current control, most important failure, detection signal, response step and recovery dependency. Escalate specialist, legal or regulatory questions to qualified advisers for the applicable context.
Sources & further reading
- Generative Artificial Intelligence Profile (NIST AI 600-1)
NIST. Risk-management guidance, including confabulation. It does not establish a universal error rate.
- Phishing Guidance: Stopping the Attack Cycle at Phase One
Cybersecurity and Infrastructure Security Agency. Current defensive guidance on phishing resistance, MFA and organisational controls. Specific authentication choices depend on service support and risk.
- How should we assess security and data minimisation in AI?
UK Information Commissioner's Office. UK regulatory guidance, checked 11 September 2026. Jurisdiction-specific context, not individual legal advice or permission for a particular use.
Examples and exercises are illustrative unless attributed to a source. No independent expert review is claimed.
A correction, a counterexample or an experience worth sharing?
Join the conversation ↗