Making an AI bid writer refuse to lie

Who Should Read

A backend developer whose document-analysis LLM pipeline writes plausible fiction and reports 'completed'

Core Mechanics

• Models write what sounds plausible
  • After 'our certifications include', a model tends to list certifications whether or not the bidder holds them.
  • Plausibility and truth diverge exactly on the claims evaluators check.
• The model invented a partner company
  • In an NHS test tender, 42 requirement rows were deferred to a consortium partner that did not exist.
  • The name was left as [PARTNER_NAME] and disclosed only at the bottom of the document.
  • Their own verifier matched keywords and scored most of those 42 rows as covered.
• The partner problem was fixed structurally, not with prompts
  • A capability-fit check against the bidder's real profile runs before any section is written.
  • Requirements without evidence go to a visible partner slot, and the count appears in a banner at the top.
  • If the bid recommendation says no-bid, the system refuses to draft.
• Silent coverage collapse is the scariest failure
  • On the same 99-page NHS pack, extraction fell from 366 to 184 requirements seven days apart, with every run reporting 'completed'.

Community Reactions

How to Apply

• Turn empty results into errors
  • Today, make chunk or batch LLM calls throw when they return empty results.
  • Log total extracted count next to page count and compare against the previous run.
• Normalize text before verifying quotes
  • Clean whitespace, line breaks and hyphen splits in both source and model quote before comparing.
  • Measure verifier false alarms by hand-checking 20-30 samples.
• Leave unknowns as slots
  • Add to your prompt: 'Leave any name, certification or figure not in the input profile as [SLOT_name]'.
  • Count the slots and show the total at the top of the output.
• Gate generation up front
  • Add a step that judges, per requirement, whether you hold evidence before drafting.
  • If evidence is missing, show the gap list to the user first instead of generating.

Try It

# Paste into Claude or ChatGPT as-is (simplified version of the article's method)

You are a bid writer. You may only claim facts explicitly stated in [Company Profile] below.
Rules:
1. For each requirement, quote the supporting sentence from the profile verbatim as 'Evidence'. If none exists, write 'No evidence'.
2. For requirements with no evidence, do not write an answer; leave a [PARTNER_OR_EVIDENCE_NEEDED] slot.
3. Never invent company names, certifications, amounts or people not in the profile.
4. At the very top, write 'Read this first: N requirements have no evidence' and list them.

[Company Profile]
{{company profile}}

[Requirements]
{{requirements}}

Try next: paste the result back and ask 'check that each quoted evidence sentence appears verbatim in the profile, one by one'.

Terminology

Related Papers

• The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge

  Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.

• 90% of US working hours are spent on tasks not covered well by AI benchmarks

  Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage

• Postgres 19 delayed after AI reviewers find more bugs than humans can fix

  LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted

• Show HN: Gemini 4 Argon plays StarCraft

  LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.

• A Plausible Explanation Is Enough to Mislead Multimodal Judges

  Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.

• Neural networks are clean

  Looking at neurons directly shows most have one main job, challenging superposition.

Related Resources

• https://ailucius.com/blog/making-an-ai-bid-writer-refuse-to-lie