Skip to content
finalstate.aiThe 70% Problem: Fast to Prototype, Hard to Finish
121Atlas

121THE DEMO THAT DAZZLES

The 70% Problem: Fast to Prototype, Hard to Finish

A fast prototype can leave production work that requires different checks and expertise. Addy Osmani's 70/30 is a field heuristic, not a measured share of a job.

A working demo leads to unresolved recovery and access tickets.In an illustrative app, the happy path passes. Production tickets for recovery and access remain open; the demo check does not clear them.DemoProductionRecoveryAccess
Illustrative app and production tickets. Their number and size do not measure a share of the work.

THE SHAPE, NOT A MEASURE

Early progress can conceal the cost of finishing.

An explicitly illustrative progress curve rises quickly and then advances unevenly without reaching a completion boundary.An uncalibrated curve shows apparent progress against iteration. It has no percentage ticks or measured task areas. The unfinished endpoint preserves uncertainty about the remaining work.IllustrativeIterationApparent progress
Osmani's 70/30 is a diagnostic heuristic. It estimates neither automated work nor staffing shares.

Osmani's December 2024 field guide describes rapid prototype progress followed by debugging and production work, especially for non-engineers. The schematic curve has no percentage scale. Generating a first pass and checking its reliability may fall on different sides of the jagged frontier.

THE EAGER JUNIOR

AI is like having a very eager junior developer on your team. They can write code quickly, but they need constant supervision and correction.

Addy Osmani, “The 70% Problem,” December 4, 2024

The metaphor comes from Osmani's first-person field guide, not a controlled study; its value is diagnostic, not statistical.

THE MEASURED SPEEDUP

Among completers, the Copilot group took less time.

Fixed bars on one zero-based minute scale show mean completion times of 71.17 minutes for the Copilot group and 160.89 minutes for control.The two bars share one zero-based scale and remain fixed throughout playback. Means condition on completing the task: 35 developers in each group. The 55.8 percent reported time reduction and its interval appear in the adjacent copy, separately from completion rates.Minutes06012018071.17Copilot160.89Control
One JavaScript HTTP-server task, 2022. Of 95 randomized professional developers, 35/45 assigned Copilot and 35/50 controls completed: about 78% versus 70%. The success-rate difference was not statistically significant.

Peng et al., 2023, Results and Figure 6 report 55.8% less completion time, with a 95% confidence interval of 21–89% and a time-comparison t-test p=.0017. Timing runs from repository creation to the first commit passing all 12 tests. Noncompleters are excluded from these means; this is not an all-participant time estimate or a maintenance study.

A timed assignment leaves wider delivery work unmeasured.

The experiment measures a specified task among those who completed it. The field heuristic asks what remains outside that test. Neither establishes that 70% of every job is automated.

WHY THE FINISH RESISTS

A repair can invalidate a neighboring check.

A parser patch fixes a missing-input test while a dependent zero-input test changes from passing to failing; a context check stays open.The two inputs share one parser. After the patch, the missing-input test passes. Rechecking the zero input reveals a failure along the shared dependency. A separate context check remains unresolved. This is an illustrative dependency, with no measured defect rate.∅0ParserMissingZeroContext
Illustrative cases and dependency. A passing local test does not certify the whole application.

In an illustrative app, a patch handles a missing input but breaks the handling of zero. Both tests touch the same parser. Other obligations may need production context to judge. Selecting a relevant verifier and paying for review are different problems; metacognitive demand concerns the operator's effort to monitor and direct the work, not proof of their ability to do it.

THE FALSE ECONOMY

A failed handoff can leave the obligation unassigned.

Two possible handoffs: an accepted transfer keeps a production ticket with an owner; a rejected transfer followed by removal of the original seat leaves its ticket unassigned.The upper route moves the ticket to a replacement who accepts responsibility, allowing the original seat to retire. The lower route first fails its handoff, then removes the original seat. Only after those separate events does the unresolved ticket become unassigned.Original seatReplacementAcceptedUnassigned

When a capable replacement accepts the work and can investigate failures, the original role can change or go. If a trial handoff fails and the team removes the specialist anyway, the obligation loses its owner. Automation alone does not cause that vacancy. Lost practice can also erode diagnostic capacity.

COLLAPSE TASKS, NOT OWNERSHIP

Transfer responsibility before removing capacity.

  1. 01Use a relevant check tested on representative cases before routing repeatable work to automation.
  2. 02Name the person or team responsible for investigating failures and recovering the service.
  3. 03Rehearse diagnosis without automation, and review whether that person or team can still carry the obligation.

A role can change when responsibility transfers successfully. If no capable owner accepts costly exceptions, retain a named owner while redesigning the work: the last-mile obligation.

A production ticket branches through a tested check to automated output and through an exception route to an owner who receives a diagnostic drill.The upper route passes a tested check before output moves onward. The lower route takes exceptions to a person responsible for investigation and recovery. A drill returns a fault to that person for diagnosis; showing the drill does not assert a successful result.CheckOutputOwnerDrill

WHO OWNS THE FINISH

Assign the finish before cutting capacity.

  • Record the work remaining after the happy path passes.
  • Name readiness evidence and when it needs review.
  • Assign failures and maintenance to a capable owner.

Use the 70% problem to inspect delivery obligations. If the readiness record is incomplete, establish what is missing and who will resolve it with the verification owner.

Read the transcript

01 · THE DEMO THAT DAZZLES

Consider an illustrative app. A founder describes it in the morning. By lunch, the happy path works. The screen looks finished, and the old estimate suddenly feels excessive. Then someone tries an input the demo never exercised. Recovery and access tickets remain open. A repair can affect another part of the app, and somebody has to understand enough of the system to maintain it. The prototype demonstrated one working path. Deciding whether to put it into production requires evidence about the work still outstanding.

02 · THE SHAPE, NOT A MEASURE

Addy Osmani used seventy percent to describe a wall he had observed, especially among non-engineers using AI coding tools. A first pass arrives quickly. Further iterations make uneven progress through debugging and production concerns. The number is a field heuristic. There is no measured seventy percent area that belongs to automation, or thirty percent share of a salary that must remain. Its useful question is how the cost of progress changes as the work moves from a persuasive prototype toward reliable use. That cost must be established for the particular project.

03 · THE EAGER JUNIOR

Osmani published his field guide on December fourth, 2024. His comparison was direct. AI is like having a very eager junior developer on your team. They can write code quickly, but they need constant supervision and correction. This is a practitioner metaphor. It draws attention to the expertise needed to inspect, debug, and maintain generated code. It does not measure how much supervision every tool or task will require.

04 · THE MEASURED SPEEDUP

A separate experiment provides a narrower measurement. In 2022, Peng and colleagues randomly assigned ninety-five professional developers to a Copilot group or control. Each had one JavaScript HTTP-server task, checked by twelve tests. Thirty-five developers in each group completed it. Among those completers, mean time was seventy-one point one seven minutes with Copilot and one hundred sixty point eight nine minutes in control. The paper reports a fifty-five point eight percent reduction in completion time, with a ninety-five percent confidence interval of twenty-one to eighty-nine percent. People who did not finish are excluded from those means. The completion-rate difference was not statistically significant. These outcomes do not establish production readiness or long-run maintenance costs.

05 · TWO KINDS OF EVIDENCE

A timed assignment and a field observation answer different questions. The experiment measures one task among people who completed it. Its time averages leave out noncompleters, whose numbers differed between groups. Random assignment alone does not remove that selection concern. The heuristic asks about work outside such a test. Combining them does not yield a percentage of every job that can be automated. To use either for a delivery decision, identify what was checked and what the proposed software still needs.

06 · WHY THE FINISH RESISTS

Return to the illustrative app. Zero and a missing value need different handling. Both pass through the same parser. A patch fixes the missing-value case. The developer reruns the dependent zero-value test, and it now fails. The local improvement has invalidated another check. A further review needs production context and remains open. These are examples of different obligations, not a measured defect rate. Finding the right test is one task; doing the review and interpreting its limits take time as well. Monitoring the model's work demands attention, but that demand does not establish that the operator has the knowledge to resolve every failure.

07 · Advertisement · Bubble AI App Builder

Some ideas do not need another document before they become testable. With Bubble AI, you describe the app you want, and Bubble creates a working starting point: interface, data, and logic you can inspect. From there, you refine visually, connect AI models and services, and turn the first version into something real enough to use.

08 · THE FALSE ECONOMY

Routine generation may move to a tool while maintenance and failures still need an owner. Consider two possible handoffs. A capable replacement team accepts the obligation and demonstrates that it can investigate failures. The original seat can then change or go. In the other case, a trial handoff fails because the replacement cannot diagnose the fault. If the team removes the original specialist anyway, the production ticket is now unassigned. The failed transfer and staffing decision create that vacancy. They are separate from automating the routine output. A proposed saving needs to include the cost of making the transfer work.

09 · COLLAPSE TASKS, NOT OWNERSHIP

Split the work before changing the seat. Test a relevant check on representative cases. Once an output passes that check, route it onward. Send costly exceptions to a named person, team, or on-call specialist responsible for investigation and recovery. Then rehearse an unaided diagnosis with a planted fault. The drill gives the team evidence about retained capacity; scheduling it alone proves nothing. Use its result to decide what practice or support the owner needs. Responsibility can transfer successfully, and the organization can change with it. The remaining obligation needs a capable owner through that change.

10 · WHO OWNS THE FINISH

Use the seventy percent problem to inspect the finish. Record the work remaining after the happy path passes, the evidence needed for readiness, and a date for reviewing that evidence. Assign failures and maintenance to someone able to carry them. An open field calls for a decision about how to resolve it. The prototype may justify further development while leaving it short of production readiness. Establish that distinction before cutting the capacity that would finish and maintain the work.

01 / 10 · THE DEMO THAT DAZZLES0:00 / 6:19