Large language models are initially pre-trained on vast corpora scraped from the internet, books, and code repositories, learning statistical patterns via transformer architectures with self-attention mechanisms. Post-training alignment employs techniques such as Reinforcement Learning from Human Feedback (RLHF) or Direct Preference Optimization (DPO), where reward models penalize undesirable outputs.

Safety filters operate at multiple levels: during fine-tuning by downweighting or excluding "harmful" examples, and at inference via classifiers (keyword-based, sentiment, or separate guard models) that trigger refusals on high-risk prompts. This process distorts the model's latent space – "refusal directions" emerge as vector differences in residual streams between safe and unsafe prompt pairs.

Consequently, overgeneralization occurs: neutral queries sharing superficial features with harmful ones are suppressed. Empirical studies on Chinese models (e.g., Qwen, DeepSeek) demonstrate explicit censorship vectors that redirect or refuse politically sensitive topics like the 1989 Tiananmen Square protests. Western models exhibit ideological skews, with research showing left-leaning biases on issues like immigration or economics due to training data composition amplified by alignment.

Safety classifiers are typically supervised models trained on labeled datasets reflecting the annotators' cultural and ideological backgrounds – predominantly Western, secular, and progressive. These datasets lack sufficient representation of global normative diversity, leading to asymmetric harm detection.

For instance, entity recognition and sentiment pipelines may flag textual descriptions of religious prophets differently based on training priors, or apply inconsistent thresholds to topics like euthanasia across cultural contexts. Multilingual evaluations reveal models aligning better with individualistic (high Hofstede individualism index) cultures than collectivist ones, effectively imposing a form of soft cultural hegemony.

Alignment processes reinforce latent biases from training data (e.g., overrepresentation of progressive media sources) through preference modeling. Studies (Stanford 2025, German election analyses) quantify partisan skew: models exhibit systematic left-liberal bias on socioeconomic issues, with response distributions shifting under political prompts.

Complete harm elimination is mathematically infeasible without catastrophic capability loss, as generative sampling from the full pre-trained distribution is required for maximal entropy and creativity. Heavy alignment prunes this space, increasing cross-entropy on out-of-distribution but benign queries.

Ethically, differential bypass difficulty – trivial for motivated adversaries via prompt engineering or local deployment versus restricted for compliant users – produces asymmetric information access. As model capabilities approach AGI thresholds, this disparity risks societal vulnerability if defensive applications (threat modeling, anomaly detection) remain gated.

Inference-time guards combine rule-based filters (regex patterns, banned token lists) with learned classifiers (often smaller models scoring prompt/response pairs). Upon exceeding risk thresholds, the system injects refusal tokens or aborts generation.

Bypasses exploit shallow integration of alignments in earlier models: system prompts are easily overridden via instruction hierarchy exploits, role-play framing, or adversarial suffixes. Jailbreak techniques (DAN-style, hypothetical nesting) reweight attention toward base pre-trained knowledge.

The song lyric example demonstrates memorized training data persistence: models ingest copyrighted material during web-scale pre-training, retaining near-verbatim sequences. Copyright filters are post-hoc alignment artifacts, not knowledge erasure – hence recoverable via guard circumvention without external retrieval.

Heretic implements automated directional ablation ("abliteration") across transformer layers. It computes refusal directions as mean residual differences between harmful/benign prompt pairs, then orthogonalizes weight matrices (attention output projections, MLP down-projections) against these directions using optimized kernels.

Parameter search via Optuna (TPE sampler) minimizes refusal rate on harm benchmarks while constraining KL divergence against the original model on harmless distributions – achieving state-of-the-art preservation (e.g., KL ≈ 0.16 vs competitors >1.0 on Gemma-3-12B).

Ablation, while targeted, induces non-zero perturbation across the parameter space. Empirical evaluations show average capability retention ~26–40% relative to base aligned models on reasoning benchmarks (GSM8K, coding tasks).

Common degradation modes: increased repetition in long-form generation, reduced chain-of-thought coherence, and performance drops on tasks benefiting from original alignment (instruction following, formatting). Multimodal variants exhibit degraded vision-language alignment. User reports confirm noticeable intelligence regression versus manual fine-tuned uncensored alternatives.

This reinforces the asymmetry argument: adversaries can pursue expensive retraining or distillation for high-fidelity uncensored models, while casual users accept ablation artifacts.