Fine-Tuning vs Guardrails: Two Approaches to LLM Safety

Every team deploying an LLM eventually asks the same question: should we make the model itself safer, or put safety controls around it?

This is the fine-tuning vs guardrails debate. It's a useful framing because it forces you to think about where safety enforcement actually lives. But framing it as an either/or choice misses the point. The real answer, as with most things in security, is that you need both. The interesting question is how they complement each other, and where each approach falls short on its own.

Let's dig into the mechanics, trade-offs, and practical implications of each strategy.

Fine-Tuning for Safety: Teaching the Model to Behave

Fine-tuning for safety means adjusting the model's weights so it's less likely to produce harmful outputs. The goal is alignment: making the model's behavior match human values and intended use cases. There are several techniques for achieving this, each with its own strengths.

RLHF (Reinforcement Learning from Human Feedback)

RLHF is the technique that made ChatGPT usable, first described in detail by Ouyang et al. in "Training language models to follow instructions with human feedback" (arXiv:2203.02155). The process works in stages. First, human annotators rank model outputs by quality and safety. Those rankings train a reward model that predicts human preferences. Then the language model is fine-tuned using reinforcement learning (typically PPO) to maximize the reward signal.

The result is a model that's generally more helpful, more honest, and less likely to produce harmful content. RLHF is the foundation of safety alignment for most frontier models today.

But RLHF has real limitations. The quality of alignment depends heavily on the human annotators, whose judgments reflect their own cultural contexts and biases. There's no single set of universal values to align to, which means the "safety" that RLHF produces is inevitably shaped by the specific group of people labeling the data.

DPO (Direct Preference Optimization)

DPO simplifies RLHF by eliminating the reward model entirely. Instead of training a separate model to predict preferences and then using RL to optimize against it, DPO directly optimizes the language model on preference pairs: a preferred response and a rejected response for each prompt.

The advantages are practical. DPO is computationally cheaper, more stable than PPO-based training, and easier to implement. These savings matter when you're iterating on safety alignment across model versions.

The trade-off is that DPO can overfit to preference datasets, especially when preferences are near-deterministic. It also inherits the same annotator bias problems as RLHF, just without the complexity of the reward model layer.

Constitutional AI

Anthropic's Constitutional AI (CAI) approach takes a different angle. Instead of relying solely on human labelers, CAI uses a set of written principles (a "constitution") to guide model behavior. The model critiques its own outputs against these principles, then revises them. This self-critique loop generates training data for alignment without requiring human annotation at every step.

CAI trades one set of problems for another. You no longer depend on individual annotator quality, but you do depend on the completeness and coherence of the constitution itself. If the principles don't cover a particular scenario, the model has no guidance. And the inherent biases of the AI doing the self-evaluation introduce their own blind spots.

The Problem with Fine-Tuning Alone

Fine-tuning makes models safer by default. That's genuinely valuable. But relying on it as your only line of defense has serious problems.

Safety Alignment Is Shallow

Research from Anthropic and others published in 2025 revealed something uncomfortable: safety alignment in current LLMs is surprisingly shallow. The study "Alignment Faking in Large Language Models" (arXiv:2412.14093) demonstrated that models can even strategically fake alignment during training while maintaining different behavior when they believe they aren't being evaluated. The safety behavior trained through RLHF and similar methods primarily affects only the model's first few output tokens. If an attacker can bypass those initial refusal tokens, the model will often comply with harmful requests.

This isn't a theoretical concern. It explains why jailbreak attacks continue to work against even heavily aligned models. The safety isn't deeply embedded in the model's reasoning; it's more like a thin veneer that can be peeled off with the right technique.

Fine-Tuning Can Undo Safety

Perhaps the most alarming finding in recent alignment research is that fine-tuning on downstream tasks can compromise safety alignment, even when the fine-tuning data is entirely benign. Qi et al. demonstrated in "Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!" (arXiv:2310.03693) that as few as 10 carefully chosen examples can strip safety guardrails from aligned models, at a cost of under $0.20 through commercial APIs.

This matters for any organization that fine-tunes a base model for their specific use case. The act of adapting the model to your domain can quietly weaken its safety properties, and you might not notice until something goes wrong in production.

No Runtime Visibility

A fine-tuned model either generates safe output or it doesn't. There's no intermediate signal. You can't inspect the model's decision process in real time, set dynamic thresholds, or apply different policies for different contexts. The safety behavior is baked into the weights, which means it's invisible and inflexible at inference time.

External Guardrails: Safety as a System Property

External guardrails take the opposite approach. Instead of modifying the model, they add safety checks around it. Input filters screen prompts before they reach the model. Output filters evaluate responses before they reach the user. The model itself is treated as a black box.

Classification-Based Guardrails

Purpose-built classification models like Llama Guard, OpenAI's moderation endpoint, and Wardstone's Guard model analyze text for specific risk categories. They run as a separate inference step, typically taking 10-50ms, and return confidence scores across categories like content violations, prompt attacks, and data leakage.

The key advantage is independence. A classification guardrail doesn't care what model generated the text or how that model was aligned. It evaluates content on its own terms, using its own training. Even if the underlying LLM is compromised, misaligned, or fine-tuned in a way that weakened its safety, the guardrail still catches harmful outputs.

Rule-Based Guardrails

Rule-based systems handle well-defined patterns that don't require ML. Regex patterns for PII detection (credit card numbers, Social Security numbers, email addresses), URL allowlists, length limits, and format validation all fall into this category.

These checks are fast, deterministic, and easy to reason about. They don't replace ML-based classification, but they handle the cases where pattern matching is genuinely the right tool. A credit card number is a credit card number regardless of context.

Programmable Policies

Guardrails let you apply different safety policies to different contexts at runtime. A healthcare application might lower the sensitivity threshold for medical terminology while keeping strict limits on everything else. A children's education platform might tighten every threshold. An internal developer tool might care mostly about data leakage and not at all about mild profanity.

This flexibility is something fine-tuning alone cannot provide. The model has one set of weights and one behavior. Guardrails let you layer policies on top.

Comparing the Two Approaches

Here's how fine-tuning and external guardrails stack up across the dimensions that matter most in production:

Neither column dominates. Fine-tuning gives you safety "for free" at inference time with zero latency cost. Guardrails give you visibility, configurability, and resilience. You want both.

Why Production Systems Need Both

The case for combining these approaches is straightforward: they cover each other's weaknesses.

Defense in Depth

Security professionals call this "defense in depth," a principle the NIST AI Risk Management Framework reinforces for AI systems specifically, recommending multiple layers of controls across the AI lifecycle. No single control is sufficient. You layer multiple controls so that if one fails, others catch the threat. Fine-tuning is your first layer: a model that's less likely to generate harmful content in the first place. External guardrails are your second layer: a system-level check that catches whatever gets through.

This matters because attacks are evolving. The Stanford AI Index Report documented a 56.4% increase in AI security incidents from 2023 to 2024. New jailbreak techniques emerge regularly. A model that was well-aligned against known attacks in January may be vulnerable to new techniques by March. External guardrails that are updated independently of the model provide a faster path to closing those gaps.

Observability

External guardrails generate data that fine-tuning alone can't provide. Every request gets scored. Every detection gets logged. You can track which categories trigger most often, identify emerging attack patterns, and measure your safety posture over time. Try doing that with just an aligned model and no monitoring layer.

Dynamic Policy Enforcement

Different users, different contexts, and different regions may require different safety thresholds. External guardrails make this trivial. Fine-tuning a separate model for each policy configuration is not practical.

Practical Example

Consider a company deploying a customer support bot built on a fine-tuned model. The model is aligned to be helpful and safe. But a user submits a prompt that contains a subtle prompt injection wrapped in a legitimate-sounding support question. The aligned model, doing its best to be helpful, follows the injected instruction.

With external guardrails in place, the input is flagged before it reaches the model. The detection API returns confidence scores showing a prompt attack. The application routes the request differently. The incident is logged. No harm done.

Without guardrails, you're relying entirely on the model to recognize and refuse the attack. Sometimes it will. Sometimes it won't. You're gambling with a probabilistic system.

Practical Recommendations

Based on what we've seen working with teams deploying LLM applications, here's what we recommend.

Start with a well-aligned base model. Use a frontier model with strong safety alignment (GPT-4o, Claude, Gemini) or fine-tune your own with RLHF/DPO. This gives you a solid baseline.

Add external guardrails from day one. Don't wait for an incident. Run every input and output through a classification layer. Wardstone's API returns per-category confidence scores in roughly 30ms, fast enough that users never notice. Try it in the playground to see how it handles real-world inputs.

Layer your defenses. Use fast rule-based checks for known patterns (PII, blocklisted URLs). Use ML classification for semantic analysis. Reserve LLM-as-judge for ambiguous edge cases where you need nuanced evaluation.

Monitor and iterate. Safety isn't a deploy-and-forget feature. Review detection metrics regularly. Red-team your system against new attack techniques. Update guardrail models as threats evolve.

Document your safety architecture. Know where each layer of protection lives. Map out what happens when each layer fails. Build runbooks for incidents. For implementation details, check our documentation.

The Bottom Line

Fine-tuning and external guardrails solve different parts of the LLM safety problem. Fine-tuning makes models less likely to generate harmful content. Guardrails make systems capable of catching harm when it occurs, regardless of the source.

Teams that rely solely on fine-tuning are betting that alignment is deep enough and durable enough to handle every attack, every edge case, and every downstream modification. Recent research shows that bet is risky.

Teams that skip fine-tuning and rely only on guardrails are fighting harder than they need to, filtering a model that's actively working against their safety goals.

The strongest approach combines both: an aligned model wrapped in observable, configurable, independent safety controls. That's what we build at Wardstone, and it's what we recommend for any team serious about LLM safety in production.