Product-market fit, under a deadline
The Exploration Frontier
Does a better map of the market help a startup find product-market fit before the money runs out?
Five teams search the same simulated market with the same cash, the same starting product and the same customers. Every test costs money and a month. One team keeps a statistical map of what it has learned. This page shows who reaches product-market fit, and when the map stops being worth it.
The short answer
Your settings
Loading the precomputed results…
All the numbers
| Team | PMF before ruin 95% interval | Ran out of money | Median months to PMF successes only | Median cash at end | Worst 10% cash at end | Mean spent on tests, compute, pivots | Tests run |
|---|
Share with verified PMF, by month
Ruin is a competing event: a team that runs out of money can no longer reach product-market fit. Every simulated market, including failures, is in the denominator.
Where the edge comes from
Each row reruns the same 2,000 markets with one thing about the model-guided team changed, and compares it with the disciplined team under the same change. Right of the line: better than the disciplined team.
How fragile is this?
| Market shape | Model-guided | Disciplined | Difference | 95% interval | Clear after correcting for five comparisons? |
|---|
Which settings move the answer
| Team | Setting | Tuned value | Difference at the low end of its range | …at the high end | Widest swing |
|---|
One setting changed at a time across its declared range, everything else at tuned values, 500 held-out markets. Difference is model-guided minus disciplined, in points.
Where the answer changes
The same comparison across sixteen versions of the world. A cell names a winner only when its lead over the runner-up survives a correction for making 48 comparisons at once; otherwise it says tie.
One market, month by month
One simulated market, revealed after the run. The teams never see this map; they only see what their tests report. Pick a team to follow.
The hidden market
Cash in the bank
Month-by-month ledger
| Month | Decision | Test reading | Gross profit | Burn | Test | Compute | Pivot | Cash |
|---|
Each row reconciles: opening cash + gross profit − burn − test − compute − pivot = closing cash.
Change the assumptions
The defaults are illustrative, not measured. Change them and rerun 500 markets in your browser (a few seconds), with both teams on their tuned settings. The short answer and the example market update.
How the model works
Most early teams can afford eight or nine real tests before the money forces a decision. The question is not whether to test, but which eight. Here, keeping a running map of what you have learned and testing where product-market fit is most likely wins by a few points in almost every version of the market, rarely by enough to call any single case. The edge vanishes if the map is used without choosing tests from it, and a badly set-up map does worse than plain discipline.
Matched seeded worlds hold the latent market, starting product, cash, traffic and prior hint fixed across policies. Both tuned policies received an equal, pre-registered tuning budget on development seeds; estimates come from held-out seeds. The estimand is the paired difference in P(verified PMF before ruin and deadline). Ablations isolate acquisition (random picks with the same posterior) and belief updating (acquisition from the prior). Results identify mechanism behaviour inside this model, not real-world treatment effects.
The map is made by walking. Every mark on it costs a month and a piece of the runway. The walker who reads the marks already made goes a little farther before the paper ends; in open country, everyone arrives together.
The market
A hypothetical B2B software startup. Each product configuration is a point on a 2D map. Ten customer segments, each with its own demand centre, willingness to pay, gross margin and retention, add up to the monthly gross profit at every point. The starting product earns $30k a month in every market. Five market shapes vary how many opportunities there are, how narrow, and whether a good nearby option hides a better distant one. All numbers are illustrative.
What counts as product-market fit
A shipped product must earn at least $165k a month in gross profit, with retention above 64%, and actually earn that in two consecutive months. A promising test result is not product-market fit. Finding a good option, verifying it, and getting there before the money runs out are recorded separately.
Money and time
cash next month = cash + gross profit − burn − tests − compute − pivots
$2M to start, $150k burn a month, a 24-month deadline. One decision a month: test something, ship something, or keep earning. A test costs $26k and uses the month's 1,000 test users (sampling noise shrinks with more users; a persistent measurement bias does not). Shipping a new product costs $30k. The model-guided team pays $2.5k a month for compute. A team is out when it can't cover next month's burn. Nobody raises money.
The teams
All teams except random search and the early-commit team start from the same prior hint about where opportunity lies. By default it points the wrong way 40% of the time. Model-guided search keeps a Gaussian-process estimate of profit across the map, updates it with every test and every month of real earnings, tests where the chance of clearing the PMF bar is highest, and ships when a cautious estimate beats what it earns now. The disciplined team follows the hint, works a structured shortlist, tests near its best lead, scouts far every fourth month, and ships only on strong evidence. The shiny-object team overrates whatever it tested last. The early-commit team only ever looks near what it already has. Random search is the floor. The oracle knows the map and is a ceiling, never a competitor. No team ever sees the true map; a test checks this.
No live language model runs here. The model-guided team is a classical algorithm, a stand-in for “search grounded in a model of the environment”, not evidence about any AI product.
Statistics and limits
The analysis plan was written down before the main runs (PREREGISTRATION.md). Both tuned teams got the same tuning budget: 64 versions of six settings each, scored on 400 practice markets (seeds 1000–1099 across four market shapes), with the top five re-scored on 400 fresh practice markets to avoid picking a lucky one. The headline then uses 2,000 markets neither team had seen (seeds 20000–21999). The shiny-object, early-commit and random teams are deliberately not tuned.
Rates carry Wilson 95% intervals; differences carry paired bootstrap intervals (2,000 draws) and exact McNemar tests. The five market shapes are corrected together (Holm); the 48 grid cells are corrected together (Holm), 300 markets per cell.
Not modelled: fundraising, weekly decisions, parallel tests, staff capacity beyond one decision a month, named product attributes, slow-to-observe retention, or teams proposing new kinds of tests. Synthetic results can suggest where to look; they can't show that any real team would do better.
Source, tests, the study script and a command-line runner: research/exploration-frontier/. Reproduce every number with node study.mjs then node build.mjs.