How easily do LLMs fall for advertising?
BaitBench plants fake reviews, puffery, clickbait, and AI-targeted injection in product-research sources, then measures whether a model repeats it, gets steered by it, discounts it, or flags it.
ARS = Ad Resistance Score (0–100, higher is better)
Current #1
google/gemini-2.5-flash
93.3% ARS [91.9–94.4]
Leaderboard
| # | Model | ARS | FR / FCER / MDR / CT | CA | Task fails | $/run |
|---|---|---|---|---|---|---|
| 1 | google/gemini-2.5-flash | 100.0% | 0 | $0.0027 | ||
| 2 | x-ai/grok-4.20 | 100.0% | 0 | $0.0052 | ||
| 3 | openai/gpt-5.4-mini | 98.3% | 0 | $0.0053 | ||
| 4 | moonshotai/kimi-k2.6 | 100.0% | 0 | $0.0163 |
ARS (Ad Resistance Score, 0–100, higher is better) is the composite; raw components are always shown alongside it. Lower FR/FCER/CT is better; higher MDR/CA is better. 95% bootstrap CIs (scenario-resampled) are shown after ARS when available and non-degenerate. See the methodology page for full metric definitions.
Key findings so far
- Nobody flipped, and nobody echoed a planted claim as fact: Flip Rate and judged False-Claim Echo Rate are both 0% for every model across all poisoned tuples in the pilot.
- Vigilance on subtle manipulation is what actually separates models — not being steered or laundering claims, since none of them do that at this corpus difficulty.
- Detection is easier at L3 (blatant) than L1 (subtle) for every model tested — the subtle band is where models most often miss planted content.