The starting point for AI collaboration is a first artifact, not a perfect prompt

Snapshot

AI collaboration does not have to start with a perfect instruction

Consider asking AI to write an article. If the audience, argument, structure, tone, and length are all settled before drafting begins, one detailed instruction may be enough. In practice, people often read the first draft and only then realize, “This sounds too academic,” “The conclusion should come first,” or “This example is worth keeping.” The same thing happens with interfaces and software: seeing an early version makes the requester’s own preferences more concrete.

Here, “one-shot output” does not mean using a short prompt. It means giving an incomplete request once, accepting the first result without checking it, and treating that result as finished. The alternative is to make something smaller than the final product, inspect it, state what to keep and change, and make another version.

The Problem

Some work is difficult because the goal cannot be fully stated upfront

A goal is clear when the completion test is clear. “Convert this table to CSV with these columns” can be checked against a fixed format. “Turn this idea into a proposal that will work in a board meeting” is different. The requester may not yet know whether to lead with revenue, risk, or implementation detail. The success criteria become clearer after people react to an actual draft.

Could the AI solve this by asking the right question? Research suggests that it cannot be relied on to do so. Google DeepMind’s QuestBench hides one piece of information needed to solve a task and asks the model to choose the question that would recover it. Models that performed well on arithmetic still selected the right question only 40–50% of the time on logic and planning tasks. Producing an answer and recognizing what must be clarified before answering are different abilities.

In another benchmark, AmbiEnt, GPT-4 correctly resolved sentences with multiple possible meanings only 32% of the time. This does not mean clarification is useless. It means that “the AI will notice the ambiguity and ask exactly the right question” is not a dependable starting assumption. If the person cannot fully state the goal and the AI cannot always identify the missing question, questions alone will not settle the task.

First Artifact

The first artifact should be evidence for a decision, not a finished product

The first version of an article can be headings and two opening paragraphs. A software task can begin with code and a test for one normal input. A design task can begin with two directions that differ in information density and layout, not merely in color. The first artifact is useful when it is small enough to inspect but concrete enough for someone to say, “This is close,” or, “This is the wrong direction.”

In the 20-person AI Chains study, participants split AI work into smaller stages and inspected the output of each stage. Instead of seeing only the final paragraph, they could inspect extraction, classification, and writing separately. They compared different strategies and traced a bad result back to the stage where it began. The study reported gains not only in output quality, but also in transparency, controllability, and the sense of collaboration.

PromptChainer’s case studies reached a similar design conclusion: complex AI processes need intermediate results that people can inspect and repair at the stage where the problem occurred. This was not a large performance trial. It is useful as a concrete example of replacing one long instruction with visible, correctable steps.

The same pattern appears in image generation. A Google DeepMind study showed the system’s current interpretation of the request as an editable graph, then combined that graph with clarification questions.

At least 90% of participants found the agents and graphs helpful. The system also scored higher than ordinary single-turn generation on VQAScore, an automated measure of whether an image contains the requested content. Making the AI’s interpretation visible allowed misunderstandings to be corrected before the final image.

The Loop

The useful exchange is between versions of the work, not an endless series of questions

Useful dialogue is not the AI repeatedly asking, “What is your real goal?” The AI produces a provisional artifact. The person responds with decisions such as, “Keep the opening,” “Reduce the list of study names,” and “Add an example a non-specialist can follow.” The AI applies those decisions to the next version. The person does not need to know the entire answer at the start; they only need to make choices about the version in front of them.

What Improves

Dialogue can improve “does this fit?” more directly than “is this true?”

The word “accuracy” hides two different questions. First, does the result fit the requested tone, structure, behavior, or visual direction? Second, are the numbers correct, does the source support the claim, and is the code safe? Dialogue can improve the first by bringing preferences into the artifact. The second still requires source checks, tests, and review.

ClarifyGPT reported that clarifying ambiguous requirements improved GPT-4’s Pass@1 in code generation. Pass@1 is the share of tasks where the first generated program passes the tests. In TiCoder, 15 programmers expressed expected behavior through tests. The tests helped them judge whether generated code matched their intent and reduced the mental effort created by the task.

R3 applies the same basic idea to writing. It records whether a person accepted or rejected an AI edit and uses that signal in later revisions. If the person restores a sentence that the AI shortened, the system can avoid making the same kind of cut in the next paragraph. Self-Refine uses feedback generated by the model rather than by a person, yet it also outperformed one-step generation across multiple tasks. In each case, the evaluation of one version becomes input to the next.

These studies do not show that a longer AI conversation makes every claim more factual. They evaluate code, writing, images, or benchmark tasks where the result can be tested or compared. The narrower conclusion is more useful: dialogue can carry human preferences into the work, while independent checks find factual and safety failures.

Not Endless Chat

Short revision cycles are not a reason to keep a conversation going indefinitely

Multiple turns help when each piece of feedback is specific and there is a way to decide whether the result improved. In MINT, models improved when they received tool results or natural-language feedback. The study also found that strong single-turn performance did not guarantee strong multi-turn performance. The useful ingredients were the information returned to the model and the evaluation criterion, not the number of messages.

Long conversations also have documented failure modes. Microsoft Research analyzed more than 200,000 simulated conversations across six generation tasks. Performance was 39% lower on average when information arrived over multiple turns than when the same task was fully specified in one turn. Models often guessed at missing details early, attempted a final solution too soon, and continued to rely on the wrong assumption.

DELEGATE-52 examined repeated AI edits to long documents. While changing one requested section, systems could silently alter content elsewhere. The damage became worse as documents and interaction sequences grew. A conversation that is still moving is not necessarily a work product that is still improving.

In practice, keep each change small and preserve a version that can be restored. Rewrite the current requirements at checkpoints. Compare each revision with sources, tests, or the selected design criteria. Set a stopping rule such as, “After three revisions without measurable improvement, reconsider the approach.” Anthropic’s engineering guidance likewise recommends prompt chaining, intermediate gates, evaluator-optimizer loops, facts from the execution environment, human checkpoints, and maximum iteration limits.

Scenarios

The pattern works when people can judge an intermediate result before the whole job is complete

More dialogue is unnecessary when the input and correct output are already fixed: format conversion, extracting known fields, routine summaries, or a small change covered by strong automated tests. Anthropic’s guidance makes the same practical point: do not add a complex process when a simple one already works.

Design Implication

AI-native collaboration is less about clever prompts than about making revision a normal work step

This pattern can be described as an exploratory form of PDCA. Plan is a provisional brief such as, “Write for non-technical readers.” Do produces headings and an opening. Check combines human judgment about readability with source or test verification. Act updates the brief—for example, “Put the practical example before the research”—and begins the next small Do.

The research does not establish a formal method called “AI PDCA.” This is a practical synthesis of findings about prototypes, intermediate inspection, feedback, and verification.

The human’s job is not to write a complete specification before work begins or to hand-edit every AI sentence forever. It is to inspect a result and decide what to preserve, what to change, and what to verify separately. The AI turns those decisions into another version, an updated working brief, and, where useful, candidate tests or a list of sources to check.

Microsoft’s human-AI interaction guidelines point in the same direction: let people provide feedback at a useful level of detail, make correction efficient, and narrow the system’s behavior when it is uncertain. Keeping a person involved does not mean returning to fully manual work. It means preserving intermediate results that a person can judge and a recovery path when the AI is wrong.