Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests | AI Paper Digest

TL;DR Highlight

Replacing buggy code with an LLM-written docstring in test prompts yielded more bug-finding unit tests

Who Should Read

A backend or QA developer who auto-generates unit tests with LLMs but finds they miss bugs

Core Mechanics

Buggy code makes LLMs trust the bug
- The LLM treats wrong behavior in the code as intended and writes tests that assert it.
- The paper calls these 'misguided tests': they pass on the buggy version and fail on the fixed one.
Bug-finding tests also drop
- With buggy input, effective tests (fail on buggy, pass on fixed) average 2.98%.
- With fixed code input it is 8.51%, nearly three times higher.
Remove the code, keep only the docstring
- The LLM derives a docstring (specification) from the code, and the test prompt drops the original implementation.
- Adding the docstring next to the code was far less effective.
No code and no docstring is unreliable
- Dropping both reduces misguided tests but reduces effective tests too.
- Without behavioral info, many tests match neither version.
Have the model audit for bugs before writing the docstring

Evidence

Misguided tests were 3.84% with buggy input
- Measured on 318 Defects4J focal methods, averaged over 13 model configurations.
- With fixed code input it was 0.46% (16.46 tests on average) versus 137.69 for buggy input.
Effective tests rose with docstring-only input
- Average effective tests went from 104.15 (code only) to 186.77 (docstring only).
- Misguided tests fell from 137.69 to 113.00.
The Advanced Docstring Prompt did better
- Against the Base Docstring Prompt, average effective tests rose from 186.77 to 230.23.
- Misguided tests fell from 113.00 to 87.38.
The gap held in 3-round feedback refinement
- Average effective tests grew by 35.69 with docstring input versus 17.62 with buggy code.
- Misguided tests grew by 18.08 with docstring input versus 31.77 with buggy code.

How to Apply

Remove the implementation from the test prompt
- Split your test-generation pipeline into two LLM calls today.
- First generate a docstring from the code with only the signature and surrounding class context.
- Then use that docstring in place of the method body in the test-generation prompt.
Add a bug-audit instruction to docstring generation
- Ask the model to analyze logic mistakes and missing null checks, boundary checks, and error handling first.
- Have it write the docstring for the corrected behavior with those fixes applied.
Force a three-part output for base models
- For non-reasoning models, require 'issues found, fixes required, final docstring' in that order.
- Reasoning models were fine with just the analysis instruction.
Spot-check the generated docstrings
- Skim them so hallucinated behavior does not get locked into tests.
- Have a human add specs for functions that need domain knowledge.

Try It

copy & run

# Paste into Claude/ChatGPT (simplified version of the paper's method: step 1, docstring generation)

You are a professional Java developer. Please help identify the intention and functionality of the method detailed below:

{{method code and surrounding class info}}

Please provide a formal docstring that identifies the functionality and intended specification of the given method. To do this, first perform a critical analysis from the perspective of an expert software engineer auditing for quality and correctness. Identify (1) Logical Mistakes and (2) Robustness Omissions (input validation, null checks, boundary conditions, error handling, sanitization).

Your output should come in three parts.
Part 1: Critical Analysis. If no issues are found, state: "No issues found."
Part 2: Fixes Required. If none, state: "No fixes required."
Part 3: Final Specification Docstring describing the corrected behavior with all fixes applied. Avoid directly quoting the method code, as it might be buggy. Output it in a fenced java block wrapped between /** and */.

# Step 2: in a new conversation, pass only the Part 3 docstring and ask:

You are a professional who writes Java test methods. Details:

{{signature, constructors, fields and other context}}
{{Part 3 docstring}}

Please write some unit tests in Java 11 and Junit 4 maximizing branch and line coverage. Output in Markdown, no explanations.

Next to try: generate tests for the same function from the original code too, and compare which catches more bugs.

Terminology

misguided testA test that passes on the buggy code but fails on the fixed code. It is like a stamp of approval on the bug.

effective testA test that fails on the buggy code and passes on the fixed code. It is a test that actually catches the bug.

focal methodThe method under test. In this paper it is the method that contains the bug.

Defects4JA benchmark dataset pairing real bugs in open-source Java projects with their fixed versions.

sequence scoreThe average per-token log probability a model assigns to an output. It shows how much the model prefers that output.

specification-based testingA black-box approach that builds tests from intended behavior rather than from the implementation.

Related Papers

Related Resources

Original Abstract (Expand)

While Large Language Models (LLMs) show great promise for automating unit test generation, recent studies suggest that the quality of generated tests can be negatively impacted when models are prompted with buggy code. This paper presents a new metric to quantitatively measure the "misguidance effect," a phenomenon where buggy code steers LLMs toward generating tests that validate its erroneous behavior rather than expose it. Our analysis reveals that prompting LLMs with buggy code has a severe, twofold impact: it significantly increases "misguided tests" that assert incorrect behavior while simultaneously suppressing the generation of effective, bug-finding tests. We further corroborate this effect from a model-internal perspective, showing that buggy code skews LLMs' preference toward tests that assert the same erroneous behavior. To counter this, we introduce and validate a specification-based unit test generation paradigm that replaces the code under test in the prompt with an LLM-generated specification docstring. Our results show that this paradigm effectively reduces misguided tests while substantially increasing effective tests, improves multi-round, feedback-driven test generation pipelines, and remains applicable to both buggy and bug-free code. Overall, these results suggest that specification-based prompting is a promising strategy for mitigating misguidance from buggy code in LLM-generated unit tests.