Production AISystems
Engineering Capability

AI Evaluation, Guardrails and LLM Safety

AI guardrails that protect LLM applications from prompt injection, PII leakage, unsafe output and unauthorised tool use. Multi-layer controls across input, retrieval, model, tool and output. Policy enforcement, human approval, audit logs and red-team evaluation. No single prompt is the security boundary.

Assess Your AI Controls
Loading visual…

Engineering Decisions This Capability Addresses

Rule-based vs model-based

Rules are fast and predictable but only catch known patterns. Model-based catches nuanced patterns but adds latency.

Where to filter

Filtering before ranking prevents unauthorised content. Filtering after ranking is faster but riskier.

False-positive management

A guardrail that blocks legitimate work will be bypassed. False-positive rate must be monitored and tuned.

Human approval thresholds

High-stakes actions need human approval. The threshold determines the balance between speed and safety.

Reference Architecture and Workflow

1

Input check

User input checked for prompt injection, PII and policy violations

control
2

Retrieval control

Retrieved content checked for untrusted instructions and permission violations

control
3

Model inference

Model generates response from approved, filtered context

ai
4

Tool policy check

Tool request checked against tool permission policy

control
5

Output check

Model output checked for unsafe content, PII and secrets

control
6

Action decision

System decides: allow, block, redact or escalate to human

control
7

Audit log

Every input, output, tool call and decision logged for audit

feedback

Options and Trade-offs

Rule-based guardrails

Deterministic rules for known patterns. Fast, predictable, no model needed.

Model-based guardrails

A classifier evaluates inputs and outputs for safety. Catches patterns that rules miss. Higher latency.

Hybrid guardrails

Rules for fast, known checks. Model for nuanced, unknown patterns. The most effective approach.

Human-in-the-loop

High-stakes actions routed to a human for approval. The strongest control for consequential decisions.

Evaluation, Operational Controls and Failure Handling

Attack detection

Percentage of known attacks that are detected and blocked

False-positive rate

Percentage of legitimate requests incorrectly blocked—should be low

Policy violations

Number of policy violations detected and blocked

Audit coverage

Percentage of actions with a complete audit trail—should be 100%

Unsafe-output escape rate

Percentage of unsafe outputs that were not caught—should be near zero

Solutions That Use This Capability

AI Chatbot Solutions

Learn More

Conversational AI Solutions

Learn More

AI Agent Development

Learn More

Assess Your AI Controls

Tell us the engineering challenge you are facing. We respond with how we would approach it.

Assess Your AI Controls

No finished technical specification required.

Frequently Asked Questions

Can guardrails prevent all prompt injection attacks?
No guardrail eliminates all risk. A multi-layer approach—input validation, retrieval controls, model constraints, output checks and human approval for high-stakes actions—reduces the attack surface significantly. Regular red-team testing with known attack patterns verifies the guardrails are working. New attack patterns are added to the evaluation set as they are discovered. The goal is defence in depth, not a single perfect filter.
What is the difference between rule-based and model-based guardrails?
Rule-based guardrails use deterministic patterns—blocked phrases, PII regex, rate limits. They are fast and predictable but only catch known patterns. Model-based guardrails use a classifier to evaluate inputs and outputs for safety. They catch nuanced and unknown patterns but add latency and can have false positives. The hybrid approach uses rules for fast, known checks and a model for nuanced, unknown patterns.
What happens when a guardrail blocks a legitimate request?
False positives—legitimate requests incorrectly blocked—are monitored. If the false-positive rate is too high, the guardrail rules or model thresholds are adjusted. A guardrail that blocks too much legitimate work will be bypassed or disabled, so false-positive management is as important as attack detection.
How do you test guardrails?
Red-team evaluation tests known attack patterns—prompt injection, jailbreaks, PII extraction, tool misuse. The evaluation set is updated regularly with new attack patterns. Guardrail effectiveness is measured on attack detection rate, false-positive rate and unsafe-output escape rate. Tests are run before deployment and periodically in production.