When to clarify a request and when to show an AI prototype

Missing information and preferences that emerge from an example

When commissioning an internal article, a person can answer questions about its deadline and destination. How much technical detail to keep may become clear only after reading a draft. Treating both as the same ambiguity can lead to endless questions or unnecessary guesses.

Clarification can recover missing requirements. ClarifyGPT detects ambiguity in code-generation requirements, asks questions, and revises the requirement using the answers. In a ten-person evaluation, GPT-4's Pass@1 on MBPP-sanitized rose from 70.96% to 80.80%. Pass@1 measures whether the first generated program passes the tests; these numbers do not predict the gain on arbitrary development work. ClarifyGPT

For preferences a person cannot yet articulate, visible alternatives can provide something to judge. We recommend choosing questions or prototypes according to that distinction. A format conversion with fixed inputs and correct outputs does not inherently need iterative prototypes.

What studies of intermediate results actually show

AI Chains studied linked processing steps whose intermediate results and operations people could edit. Its 20-person study reported better task outcomes, transparency, and controllability. Users could investigate the stage where an error appeared. This was not a test showing that presenting a complete first draft improves every kind of work. AI Chains

A text-to-image study combined clarification questions with an editable graph of the agent's interpretation. At least 90% of participants found the approach helpful. That finding alone does not isolate the graph's contribution from the questions' contribution. Proactive Agents for Multi-Turn Text-to-Image Generation

Our design inference is to make misunderstanding discoverable while the relevant stage remains editable. The useful intermediate object depends on the task: an article outline, objects and layout for an image, or tests expressing expected code behavior.

A hypothetical revision of an article outline

Consider a request for an article on AI adoption that nontechnical readers can follow. The agent returns an outline beginning with product names. Without reading a full draft, the requester can identify the problem: the article should first explain which job it helps improve.

The next instruction might be: “Open with a salesperson updating records after a meeting. Keep the product comparison, but explain the comparison criteria before naming products.” The agent revises only the outline and opening. Once the connection between the example and comparison is clear, the requester can commission the full text.

This exchange establishes an explanatory order. It does not verify product capabilities or deployment statistics; those still need source checks. Dialogue and correctness can overlap, as when clarification establishes expected program behavior. Nevertheless, a person's satisfaction cannot substitute for factual verification.

Preserve the agreed brief during revision

Repeated editing can carry an early mistaken assumption forward. A Microsoft Research study compared six generation tasks with instructions supplied at once or spread across turns. Performance fell by 39% on average in the multi-turn condition. Analysis of more than 200,000 simulated conversations found reliance on early assumptions. This compares ways of supplying requirements; it does not establish a uniform penalty for every longer conversation. LLMs Get Lost In Multi-Turn Conversation

After agreeing on an outline, restate the reader, central question, and retained example in a short brief. Preserve the previous version and inspect whether unrelated claims changed. If feedback is not being applied, reconsider the requested change or acceptance criterion instead of merely adding more turns.

Know what will end the collaboration

For an article, finish by checking its fit to the reader and question, the evidence for its claims, and the conditions attached to numbers. Satisfaction with the presentation is not sufficient for publication, while continued conversation without a remaining improvement serves no purpose.

A prototype connects an unarticulated preference to something editable. Ask about conditions the requester can already specify; show intermediate work where inspection is needed to decide. Combine those decisions with source checks or tests so the next revision and the remaining verification are clear.