Blog · bart.consulting

Your team was on time. Your roadmap wasn't.

Tue, 20 Oct 2026 00:00:00 GMT

Your team shipped almost everything on time this quarter. The roadmap still slipped.

So you call the client. The team did good work. One piece took longer than expected. The rest is nearly there. All true, and the client hears only one thing: the date moved.

Being right about the usual month doesn't make you right about the deadline.

For my last article, I showed that a single task has no finish date, only a range of likely ones. This time I asked the question that matters for a roadmap. If you use your team's own record to say how much it will finish next month, how often are you right?

I took three software teams with public records, between four and seven years each, and did what a careful lead would do at the start of a month. From what a team had finished in the month just gone, I forecast what it would finish in the month ahead. I made that forecast at every Monday in the records, treating each one as the start of a month, more than 800 times across the three teams.

A real year: MongoDB's Evergreen team in 2018, each four weeks of finished work set against the four weeks before. Six of thirteen months fell short of it.

Each forecast gave me two numbers. A safe number: what the team would finish at least, 85 times in 100. And a typical number, the usual month: what it would finish at least, 50 times in 100. In one month for one of the teams, the safe number was 44 tasks and the typical one 55. Higher certainty buys a smaller promise.

One team's forecast for its next month, drawn as a hundred ways the month could go: at least 55 tasks half the time, at least 44 tasks 85 times in 100.

Then I checked every forecast against what the teams actually finished. The typical number was right. The teams beat it in half of their months, as they should. The safe number was not. A promise I would have bet on 85 times in 100 held three times in four, and when it broke, the team finished about 15 percent less than promised, up to almost 40 percent less in the smallest team.

One month in four, your client would still be waiting.

Once you've had to explain a moved date, you start padding. Your developer says August. Your lead says September. You tell the client October, because you've been here before.

When you turned August into October, how much confidence did you buy? A fixed margin can't tell you, because it knows nothing about how your team actually works.

You don't need to guess. Even a forecast that is too sure of itself knows what a fixed margin never can: how much a month can swing, and how often. Set that against the work still ahead and you can see how likely you are to keep the whole promise before you make it.

The honest version of October is October plus how sure you are you'll hit it. October at seven in ten is a different decision from October at nine in ten. Which one you need depends on what hangs on it: an internal target, a launch, a commitment to a client.

In an example forecast, the developer's August, the lead's September and the client's October hold 3, 5 and 7 times in 10. A date that needs to hold 9 times in 10 lands in November.

Your client needs a date they can plan around. If the chance of hitting October at seven in ten isn't good enough, agree on a later date or a smaller first version before they make commitments of their own. That conversation happens while they can still adjust, not when you're calling to explain a promise that already broke.

Before you promise the next date, ask your lead what chance the team has of hitting it.

I work out the chance behind your next roadmap date from the tasks your team has already finished, before you promise it. No workshop, no new estimate.

See the chance behind your roadmap

The three teams are the Vera C. Rubin Observatory's Data Management software, MongoDB's Evergreen system and Appcelerator Studio, from public records collected in the TAWOS dataset, the same projects as in Finish dates don't exist. Odds do. At every Monday of their records, between four and seven years per team, I forecast the next four weeks of finished work from the four weeks before, drawing those days at random, and compared the forecast with what happened: 807 forecasts in all. The safe number is the one the forecast puts 85 percent of outcomes at or above. The typical number is the one it puts half at or above. The example pair is one of MongoDB's months. The safe number held in 76, 74 and 71 percent of forecasts for the three teams, and when it broke, half the misses were larger than 13, 16 and 39 percent of the promise. The typical number was beaten in 53, 53 and 52 percent.

Forecasting from finished software work is described by Daniel Vacanti and Colleen Johnson. The numerical findings here come from my own analysis.

The data and methods behind "Finish dates don't exist. Odds do."

Wed, 07 Oct 2026 00:00:00 GMT

Abstract

A reasonable estimate can still be followed by a very late finish. In "Finish dates don't exist. Odds do.", I examined how widely completion times varied when software work shared the same recorded estimate and followed the same route through a project's workflow. This article explains the data and methods behind that comparison.

I used historical records from TAWOS, a public dataset of Jira issues from open-source software projects, to study Rubin Observatory Data Management, MongoDB's Evergreen system, and Appcelerator Studio. For each project, I reconstructed the progression from recorded start to completion using status histories, separated delivered work from abandoned work, and built comparison groups around the project's own workflow. Completion times were measured and compared separately within each project.

The spread remained substantial. Among 880 one-point Stories following the observatory project's standard route, half finished within about three days, while roughly five in every hundred took more than four weeks. The two company-operated projects showed the same pattern. In every group of at least a hundred issues, across all three projects, the late finish took at least three and a half times as long as the typical one. For anyone using estimates to set deadlines, the gap matters. A typical duration tells you where the middle was. The later finishes show how much risk that single number leaves out.

The question this analysis addressed

A missed date often leads to a request for better planning. Development is asked to estimate the work again, add the estimates together and choose a new deadline, usually with some extra margin. Sometimes the deadline comes first, and the estimates decide how much work should fit before it. In both versions, the estimates carry the weight of the deadline.

I wanted to examine the relationship that calculation depends on: how much does a task's estimate tell us about its time to completion?

For the main comparison, I selected planned development work, leaving bugs out, and compared completed work with the same estimate within each project. I then narrowed the comparison to work that passed through the same stages in the same order.

I measured elapsed calendar time from the recorded start of development activities to completion. This includes waiting and review as well as development work. A deadline has to account for the whole interval.

The dataset and the records available

TAWOS and the version used here

I used TAWOS, a dataset of public Jira records from open-source software projects. Version 1.1 contains 458,232 issues from 39 projects, including recorded estimates and histories of status changes. Those histories let me reconstruct how work moved from start to completion. The records used here run up to 23 October 2020. [1, 6, 7]

TAWOS was assembled from projects with evidence of Agile practice and at least 200 issues with story-point estimates. The dataset paper describes version 1.0, which also included five Talendforge projects later removed when their tracker ceased to be public. Before the analysis, I audited the imported records against that paper. The data-quality findings from that audit are explained below. [1, 6]

Three projects, and why these three

I selected three projects with enough estimated, completed work to support substantial comparison groups and workflows I could interpret. Together, they cover scientific software and two tools run by software companies, with different ways of organizing the work. [6, 8-11]

Rubin Observatory Data Management (DM) builds the software that turns the observatory's sky images into science data. Its published workflow provided a clear starting point for interpreting the records. [9]
MongoDB Evergreen (EVG) is the system MongoDB uses to test software changes across platforms. [10]
Appcelerator Studio (TISTUD) was a tool for building mobile apps. [11]

I interpreted each project's records separately. The project-specific sections below explain which work entered the comparisons and which periods were used. [7, 8]

How the analysis is put together

I kept the imported records unchanged and stored my reconstructions separately. The analysis follows three steps:

Configuration files define what each project's statuses and resolutions mean, and which work and routes enter the comparison.
The builder turns the recorded status changes into a timeline and a route for each issue.
The study script selects the comparison groups and calculates their completion-time summaries. [7, 8]

Jira supplies the status-change timestamps; my reconstruction rules determine the start and completion events. I set each project's status and route mappings before running the completion-time study. The following sections explain those rules and the corrections needed to make the records usable.

What the source papers established

The closest prior study compared developers' estimates, recorded as story points, with the time issues spent in progress. The relationship was usually weak or moderate, and issues with the same estimate showed substantial variation in duration. Some took much longer than others, even after the researchers removed very short durations and statistical outliers. [2]

Two other papers tested methods for predicting the story-point estimate an issue would receive. In both studies, a prediction using the middle value of earlier estimates was competitive with more elaborate methods. Predicting an assigned estimate and predicting a finish date answer different questions. [3-5]

Alongside the TAWOS dataset paper, these studies informed the data audit and the choice of time measure. [1-4] My comparison focuses on the full interval from development start to completion, including waiting and review, and groups work by both estimate and workflow route. It retains the long durations because those late finishes matter for deadlines. [7, 8]

Why the supplied duration columns were insufficient

TAWOS includes calculated duration fields, but I needed the elapsed time from development start to completion. The audit showed that the supplied fields did not consistently measure that interval. [6]

Resolution time often included the backlog. For 83 percent of resolved issues with a recorded move into In Progress, the value matched the time from ticket creation to resolution. That includes time before development starts, although the dataset paper describes the field as starting when work begins. [1, 6]
Time in progress depended on the workflow. The supplied value was zero for 76 percent of resolved issues. Almost all of those tickets still had status changes; the zero often reflected a mismatch between the field's status definition and the project's workflow. Requiring a positive value would have selected work according to the status names used. [6]

I therefore reconstructed start and completion from each project's status history, interpreting its stages separately. The next section explains those rules. [8]

What the recorded estimate could establish

The supplied estimation date repeated the ticket's creation time, sometimes one hour earlier. It could not establish whether an estimate had been made before development began. [6]

For the stable-estimate comparisons, I selected estimates from 1 to 100 and required the revision flag (Story_Point_Changed_After_Estimation) to be unset. Some issues had several estimate entries in their history despite an unset flag, so I describe this restriction as "no revision recorded by the flag." [1, 2, 6, 7]

Timestamp precision

The timestamps showed seasonal one-hour discrepancies consistent with local clock changes, despite the documentation describing them as UTC. I applied no timezone correction and report durations in elapsed calendar days. An interval crossing a daylight-saving change can therefore carry a one-hour discrepancy. This matters most for groups where half the work finished in less than a day, and for completion times close to the whole-day boundaries used in the charts. [1, 6, 8]

Reconstructing an issue's recorded lifecycle

I reconstructed each Jira issue's timeline from its dated status changes to identify the start, completion and route used in the comparison. [8]

Start and completion

Start: first entry into a status marking work as under way.
Completion: first entry into the final uninterrupted sequence of terminal states, which mark work as ended.

For example, an issue that moves from In Progress to Resolved and later to Closed finishes at the move to Resolved. The later closure adds no time. If it reopens before finishing again, I use that later completion and include all the time in between. A move out of a terminal state and back within 24 hours, with no active spell in between, is an administrative round trip, not a reopen: it does not interrupt the terminal run, so the issue is dated by its first close. [8]

I call the interval from recorded start to final completion cycle time. It includes development, review and waiting, plus time spent in earlier terminal states before a reopening. The interval from ticket creation to the first active state is the pre-start wait and sits outside cycle time.

For the study, I selected delivered issues with a recorded resolution date and a measurable start. Open work and work with no observed active state were outside the completion-time comparison. [7, 8]

Cycle time runs from the first entry into an active status to the first entry into the final run of terminal statuses. Time before a reopen counts; the pre-start wait and a later administrative closure do not. Status names from Appcelerator Studio; the same rules apply in all three projects.

Classifying the stages

Status names differ between projects, so I mapped them separately to six classes. Each continuous period in an active status is an active spell. [8]

active — work marked as under way, such as In Progress.
queue — waiting, including backlog states, blocks and waits after review.
review — undergoing review or waiting for a reviewer.
done — the status label marks the work as delivered.
abandoned — the status label marks the work as not delivered.
terminal — neutral closing states, such as Resolved or Closed; the recorded resolution determines delivery.

A status such as DM's Reviewed means review has finished, so I classed it as queue. An issue in the neutral terminal class is treated as open without a recorded resolution date. An unmapped resolution on a resolved issue in that class stops the build. [8]

Within a cycle, I summed time in active, review and queue states separately. Earlier time in terminal states before a reopen is stored as prior terminal dwell. These intervals add up to cycle time; the study reports the total elapsed interval. [8]

Main flow and standard route

A route is the sequence of status classes an issue passed through, with consecutive occurrences of the same class collapsed into one step. For example, queue → active → review → queue → done means waiting, work, review, another wait and delivery. [8]

Main flow: the routes I accepted for comparison because they ended in delivery and passed through the stages I treated as necessary in that workflow. Rework, requeue and reopen loops remain eligible.
Standard route: the most common eligible route in the project. Within that route, grouping by estimate holds both the estimate and stage sequence constant. [7, 8]

For DM, the handbook's statuses Todo, In Progress, In Review, Reviewed and Done map onto the standard route. The guide defines one point as an idealised half day of work. I use that current documentation as context; the historical workflow decisions come from the project investigation. [8, 9]

Handling history inconsistencies

I ordered each issue's status changes by timestamp, with changelog ID breaking ties. The first genuine status change's origin status applies from ticket creation. Each later interval uses the destination status of the change that opened it. Without genuine changes, I use the current recorded status from creation. [8]

Entries such as Closed → Closed are ignored because they record no change of state. If consecutive events disagree about a state, I retain the label that opened the interval, flag it as disputed and include the interval in the analysis. [8]

I revised the completion boundary and the rule for unchanged status labels after examining the outcomes, added the round-trip rule after examining the reopens, and applied all three rules to all three projects. The Appcelerator Studio section explains what each rule changed. [8]

The project-specific decisions

The same comparison needed different selection rules in each project.

DM: Stories with recorded work and review

I used the full recorded period, from 2014 to October 2020. DM's Stories included code changes alongside operations, reporting and accounting work, so the issue type alone was not enough to select the process I wanted to compare. I required both an active spell and a recorded review before delivery. [7-9]

The review requirement excludes the large queue → active → done route, which included work reviewed on GitHub without entering Jira's review stage. It also excludes queue → active → queue → done, including issues that moved through In Progress → Reviewed → Done: Reviewed is grouped as queue. [8]

Epics and Milestones were excluded because their timelines aggregate other issues. I kept Technical tasks separate because missing parent links prevented a check for overlapping parent and child timelines. Bugs and other execution types appear in separate checks. [8]

I split the results at 1 October 2019, after a workflow change removed the direct move from review to Done. Era 0 is before that boundary; era 1 starts there. I report both the full period and the eras separately, assigning each issue by its reconstructed completion date. The route queue → active → review → done stays eligible in the main flow in both eras; the standard route includes a wait after review. [7, 8]

EVG: separating delivery from cancellation

I selected Task, Improvement and New Feature. I checked that Task was being used for standalone engineering before including it; the result files also show the comparison without Task. Bugs were examined separately. [8]

Both active work and review were required. I grouped In Progress, Debugging and Investigating as active, and In Code Review as review. Closed contained delivered, cancelled and duplicate issues, so the recorded resolution determines delivery. Deployed is grouped as queue because issues normally moved on from it to closure. The endpoint measures completion of the recorded workflow. [8]

The comparison uses era 2, from 5 February 2018 onward. This is the period after Closed replaced Resolved as the normal ending status. [7, 8]

TISTUD: separating completion from later closure

I selected Story, Improvement and New Feature over the full period, from March 2011 to October 2020. Review was optional in this workflow, so I required active work and delivery. Bugs and Technical tasks were examined separately. [7, 8]

The history contained 2,219 Closed → Closed entries and later administrative moves from Resolved to Closed. An earlier rule ended the cycle at the last terminal event. Under that rule, the time by which about 95 percent had finished was near two and a half years. Using the first entry into the final uninterrupted terminal run removed that administrative delay; genuine reopens still extend the measured lifecycle. [7, 8]

I also ignored repeated-label entries so they could not create extra intervals or route steps. I applied the rules for completion and repeated labels to all three projects. Ignoring repeated-label entries left DM and EVG's cohort results unchanged to two decimal places. [7, 8]

The third rule covers administrative round trips. Most of what the history recorded as reopens was a move through Closed → Reopened → Closed within hours, with no work between; one administrator reopened and re-closed 133 issues on the evening of 19 June 2013. EVG showed the same pattern: 106 issues first closed weeks earlier were re-closed out of In Code Review within one hour on 1 June 2020. Such a move, out of a terminal state and back within 24 hours with no active spell in between, is absorbed and the issue is dated by its first close; a genuine reopen still extends the lifecycle. The rule catches 749 of TISTUD's 1,061 delivered issues with a reopen and 220 of EVG's 422. It applies to all three projects; it left the standard-route cohorts unchanged and is reflected in the main-flow figures reported below. [7, 8]

I retained the project's issue-type labels, even where a Story's title looked like a bug report. No era split was adopted: after repeated-label entries were ignored, the mapped workflow retained the same stage structure. [8]

From project records to comparison cohorts

Selecting the comparison groups

I built the comparison groups in three steps:

Keep delivered issues with a recorded resolution date and a measurable cycle time of zero or more minutes. For EVG, use only era 2.
Apply the project's selected issue types and main-flow routes.
Keep estimates from 1 to 100, including fractions within that range, with no revision recorded by the flag. Then select the standard route and group by exact estimate. [6, 7]

The table starts with delivered issues across all types. DM's starting count already requires a measurable cycle; EVG's and TISTUD's do not. Every later row applies that requirement. [7, 8]

Selection step	DM	EVG, from 2018	TISTUD
Delivered issues before type and route selection	14,127	4,200	3,433
Selected types on a main-flow route, with measurable cycle time	5,997	1,629	906
Same, with estimates from 1 to 100 and the revision flag unset	3,778	1,428	775
Same, on the standard route	3,144	1,030	414

Table 1. Counts as each project's population was narrowed, using the issue types selected in the project-specific sections above. [7, 8]

The largest reduction came before filtering estimates, when I applied the requirements for issue types, routes and a measurable cycle. Those requirements excluded DM's skip-review route, EVG issues without a recorded work start or review, and TISTUD issues that closed without a recorded start. [8]

The standard route was different in each project:

DM: queue → active → review → queue → done
EVG: queue → active → review → terminal
TISTUD: queue → active → terminal

Within each standard route, I grouped issues by their exact estimate. DM's groups contain only Stories. EVG and TISTUD combine their selected issue types, so the estimate and route match within a group while the type can differ. [7]

How the published counts fit together

The route figure in the original article covers 14,210 DM Stories with estimates from 1 to 100, including work that was delivered, abandoned or still open at the cutoff. The completion-time comparison here uses 880 delivered one-point DM Stories with a measurable cycle, no revision recorded by the flag, and the standard route. The 880 are among the 14,210, but the counts came from separate queries for different comparisons. [7, 12]

DM's 14,210 estimated Stories followed 176 routes after status labels were grouped into stages, with 4,122 on the standard route. The original route figure reports 3,853 following the handbook's exact status sequence, but the query behind that stricter count was not retained. [12]

How to read the completion times

I used percentiles to compare typical and late finishes:

P50 (the median): the time by which about half the group had finished.
P85: the time by which about 85 percent had finished.
P95: the time by which about 95 percent had finished.

The ratio P95/P50 expresses the late completion point as a multiple of the median. A ratio of 3.5 means that point was three and a half times the median. [7]

I calculated percentiles from unrounded durations and report them in elapsed calendar days, including nights and weekends. Durations and ratios are shown to two decimals, with ratios calculated before rounding. Headline comparisons use groups of at least 100 issues. [7]

Results and checks

DM: variation remained within point groups

Among DM's 3,778 stable-estimate main-flow Stories, the median was 8.15 days and P95 was 88.13 days, a ratio of 10.81. Restricting to the standard route left 3,144 Stories, with a median of 7.17 days, a P95 of 71.34 days, and a ratio of 9.96. Holding the point value constant as well, the 880 one-point Stories have a ratio of 10.18. Matching the route and then the estimate moved the centre of the distribution but barely touched its shape. [7]

Completion times for 880 delivered one-point DM Stories on the standard route, with the revision flag unset, over the full study period. Finishes beyond day 30 are grouped together in the chart and remain in the percentile calculations. [12]

Holding the recorded point value constant produces the following groups with at least 100 observations.

DM recorded points	Issues	P50, days	P85, days	P95, days	P95/P50
1	880	2.74	12.87	27.87	10.18
2	749	6.02	23.05	46.88	7.79
3	264	9.98	33.93	75.52	7.57
4	451	9.08	32.90	60.03	6.61
5	134	13.01	42.14	102.05	7.84
6	218	15.25	43.01	81.01	5.31
8	144	23.04	54.27	88.35	3.84
10	134	25.87	72.10	117.18	4.53

Table 2. DM standard-route point groups, pooled eras. All groups use Story, the stable-estimate restriction, delivered measurable cycles, and the standard route. Ratios use unrounded values. [7]

The central duration rises with the estimate, except that 4-point Stories had a slightly lower median than 3-point ones. The 95th percentile remains several times the median in every group. The shared estimate and route did not make observed durations tightly concentrated.

The time by which about 95 percent had finished, as a multiple of each group's median. Each block is one median: for one-point Stories a block is 2.7 days, and the 95 percent point lies just over ten blocks out. Delivered DM Stories on the standard route with the revision flag unset, full study period; every group has at least 100 issues.

Workflow-era checks

The obvious objection to the pooled DM result is that it straddles a workflow migration. I therefore split the results at the October 2019 boundary. On the standard route, era 0 holds 1,981 Stories running from a median of 8.84 days to a P95 of 78.41 days, a ratio of 8.87; era 1 holds 1,163 running from 5.25 to 57.63 days, a ratio of 10.98. For the one-point standard-route cohort, era 0 holds 478 issues running from 3.07 to 37.91 days, a ratio of 12.34, and era 1 holds 402 running from 2.02 to 20.09 days, a ratio of 9.94. The spread appears in both eras, not only in the pooled figure. [7]

The era comparison does not show that the migration caused any change in completion times. Era assignment follows the endpoint, and other conditions could have changed. The check addresses a narrower concern: substantial spread remains when the eras are examined separately.

EVG and TISTUD under their own rules

The company-operated projects use their own type populations and standard routes. Their point scales are not a shared unit of work across projects.

Project and recorded points	Issues	P50, days	P95, days	P95/P50
EVG, 1	391	1.18	13.91	11.75
EVG, 2	459	2.97	15.11	5.08
EVG, 4	145	5.18	19.36	3.74
TISTUD, 5	155	0.24	4.35	17.84
TISTUD, 8	126	1.76	7.56	4.31

Table 3. Standard-route point groups with at least 100 observations. EVG uses its study era and Task + Improvement + New Feature; TISTUD uses Story + Improvement + New Feature. Separate project results, not pooled estimates. [7]

EVG's 1,428 stable-estimate main-flow issues had a median of 3.80 days and a P95 of 30.00 days. One objection is that the Task type might be driving the result. The sensitivity check excluding Task kept 786 Improvement and New Feature issues, with a median of 4.20 days and a P95 of 33.64 days. The spread was present without Task. [7]

TISTUD's main flow contained 775 stable-estimate issues, with a median of 1.06 days and a P95 of 35.27 days. Its standard-route cohort contained 414 issues, with a median of 0.81 days and a P95 of 7.30 days. The two summaries differ because the main flow also admits routes the standard route excludes: routes through the optional review stage, rework loops and reopens. The 93 stable-estimated issues on the reopen route queue → active → terminal → queue → terminal run from a median of 0.22 days to a P95 of 58.57 days. [7]

The five-point TISTUD cohort has a sub-day median. Its large ratio is sensitive to a small denominator, and timestamp precision matters more at that scale. The eight-point cohort gives the clearer day-scale comparison: a median of 1.76 days and a P95 of 7.56 days.

Across all reported comparisons with at least 100 issues, the lowest P95/P50 was 3.53, for DM's eight-point Stories in era 0. [7]

What the results mean for planning

Across three projects, matching the estimate and workflow route still left a substantial gap between typical and late finishes. Waiting, review and rework are part of the elapsed interval a deadline has to account for. [7, 8]

I compared completed issues using recorded estimates and the routes each issue eventually took. I did not test how reliably the historical percentiles predict future completion times. [7, 8]

For planning, a typical duration is a useful reference. Historical late finishes show the risk that reference leaves out. Asking how likely work is to finish by the date you need makes that risk part of the decision.

References

TAWOS is public. My project mappings, scripts and result files are unpublished working materials, described in references 6–8 and 12.

Tawosi, V., Al-Subaihin, A., Moussa, R., and Sarro, F. (2022). A Versatile Dataset of Agile Open Source Software Projects. MSR 2022, doi:10.1145/3524842.3528029. Dataset provenance and field descriptions, especially §§2.1 and 2.5. The data and its README live in the TAWOS repository; this analysis uses the version 1.1 database download, doi:10.5522/04/21308124.
Tawosi, V., Moussa, R., and Sarro, F. (2022). On the Relationship Between Story Points and Development Effort in Agile Open-Source Software. ESEM 2022. Time definitions in §2.2, sample and filters in §3.3, results in §4.
Tawosi, V., Moussa, R., and Sarro, F. (2022). Agile Effort Estimation: Have We Solved the Problem Yet? Insights From A Replication Study. IEEE Transactions on Software Engineering accepted manuscript, arXiv version 2, 17 December 2022. Experimental results and replication discussion.
Tawosi, V., Al-Subaihin, A., and Sarro, F. (2022). Investigating the Effectiveness of Clustering for Story Point Estimation. SANER 2022. Dataset and chronological split in §IV-B, comparisons in §V.
Choetkiertikul, M., Dam, H. K., Tran, T., Pham, T., Ghose, A., and Menzies, T. (2019). A Deep Learning Model for Estimating Story Points. IEEE Transactions on Software Engineering 45(7). The estimator that reference 3 replicates.
TAWOS import and data-quality investigation, 2026. Unpublished working notes by the author: version reconciliation against the dataset paper, the resolution-time and estimation-date column discrepancies, the timestamp offsets, missing fields, and known data limitations. Investigation records, not a peer-reviewed publication.
Completion-time study, 2026. Unpublished analysis by the author: a method note, a manifest recording the 23 October 2020 cutoff, the study script, per-project result files for DM, EVG, and TISTUD, and findings notes for each project recording interpretation and earlier checks. Tables 1, 2, and 3 are taken from the result files.
Workflow reconstruction, 2026. Unpublished analysis by the author: a method note, the builder, and for each of DM, EVG, and TISTUD the adopted status, type, resolution, route, and era configurations with the investigation and validation records behind them.
LSST Data Management, the Rubin Observatory's software project under its former name, the Large Synoptic Survey Telescope. DM Development Workflow and Data Management Project Management Guide, DMTN-020, both accessed 8 September 2026. Current documentation supplies project context, including the five workflow steps and the idealised half-day story point; the historical workflow interpretations used here come from the investigations in reference 8.
MongoDB (27 July 2016). Evergreen Continuous Integration: Why We Reinvented the Wheel, accessed 8 September 2026. Context for the EVG project.
Axway. Changes to Application Development Services, accessed 8 September 2026. Context for Appcelerator Studio and its discontinuation after the dataset's cutoff.
Figure materials for the original article, 2026. Unpublished, held with the analysis: the completion-day data and its extraction, the bar-figure script, the route figure, the estimate-range figure, and the full listing of the 176 routes taken by the 14,210 estimated Stories.

Nobody used it. So we built more of it.

Thu, 17 Sep 2026 00:00:00 GMT

You build a feature because a customer asks for it. Months later, almost nobody uses it, and the conversation turns to whether it should stay. The usage chart makes removing it look like the obvious decision.

But low usage can hide two different problems: people try the feature and leave, or they never get far enough to try it. The total doesn't tell you which, yet it is often the number deciding what gets cut.

I was ready to write off a feature on that basis myself. Instead, we spent about a third of the development time we had left for the year building more of it. What changed my mind was looking at where we lost people.

A phone showing the old travel-time map around Ghent, next to two numbers that did not fit together: of all visitors, a very small share ever found it, and of those who typed in a place, 97 percent finished all four steps.

The reading everyone in the room shares

For four years I have led product at Spotto, a Belgian real estate marketplace. Somewhere in it lived a search that worked differently from the rest, a unique thing in Belgium. Instead of picking a postcode, you told it where you work and how you travel, and it showed you every home within thirty minutes by bike of that address. Or, if you're like me, you enter your favorite café and make sure it's at a bike-friendly distance. That's our travel-time search.

The travel-time search sat in its own corner of the site, a room with its own door, and a very small percentage of our visitors ever found it.

The reading of a number like that is a straight line. Nobody uses it, so nobody wants it, so stop spending on it. Delete it in the next cleanup or, if someone feels generous, redesign it and hope.

What stopped me was a second number.

The number underneath

The travel-time search opened with an introduction: a headline, a paragraph of explanation and a Start button. Most people stopped there. Some pressed Start but still left the address box empty.

Entering a place was where the work began. You then had to choose how you travel, set a maximum travel time and pick a type of home. Four steps before you saw a single result.

Yet of the people who typed in a place between June and mid-September this year, 97 percent finished all four steps.

Effort is the signal. When people push through four steps to get one specific answer, they are telling you they want that answer, and they are paying for it in patience.

That 97 percent is where low usage split in two for me. Low usage can mean people try the thing and leave. Then you need to find out what made them stop before you send more people into it.

Or it can mean people never reach the thing, and the ones who start for real finish it. Then you have a reason to test a better way in before you write the feature off.

In your product the first kind is the accounts that open the feature once and never again. The second kind is the feature several clicks deep that the few accounts who know about it use every week.

On a usage chart those two look identical. They tell you to investigate different problems.

Two features with the same small number of people finishing. In one, many people reach it and leave. In the other, almost nobody reaches it and 97 percent of those who start finish.

I have cut a feature on the straight-line reading before. We once had a home dossier that showed you your home and all freely available information about it. I cut it on the usage number alone. I never asked how many people had reached it, and I still don't know which of the two problems it had.

Low usage is not a verdict. Find where you lose people.

The fixes we had already tried

Once you decide to keep a feature like this, the instinct is to polish it. Shorten the steps. Rewrite the copy. Run an A/B test on the button.

We had been polishing for years.

Six weeks after the site launched in 2023, our own review called the travel-time search not intuitive enough. We reworked it and released it again at the end of 2024. In 2025 it got its own homepage, a button on the property pages and a campaign on social media. This year it got its own ad on our results pages.

We had improved the tool and advertised its door. After both, the search still reached almost nobody.

Most of the people who did reach the search were already on Spotto. We sent them to a separate screen, and most people who landed on it never got past its first page. So even the door people did find led away from what they were doing.

Shortening the form would improve the part almost nobody abandoned once they had started. Our biggest loss came before that.

The homepage, the button, the campaign and the ad all said some version of "search by travel time". Each advertised another way to search. None put the travel time on the homes people were already browsing.

What the people who found it were worth

The finish rate gave me a reason to challenge deletion. To judge whether more investment made sense, I compared the people who had used the travel-time search with everyone else.

Among registered house-hunters who had saved a search, the ones who saved a travel-time search looked at two and a half times as many homes. They kept more favourites, stayed longer and came back more often, weeks and months later. The same pattern of engagement and retention showed up in three separate sets of records, on every measure I had. They contacted estate agents at about the same rate as other people who saved a search.

People who saved a travel-time search, compared with everyone else who saved a search, on eight measures: they looked at two and a half times as many homes, kept about twice as many favourites and came back more often.

People who found a hidden tool and saved a search were already committed house-hunters. Was the tool helping them, or were they simply more committed to begin with? To get closer to an answer, I compared similar users. Travel-time users still looked at more homes, but the difference in return visits was less certain.

So we built more of it

We didn't polish it and we didn't cut it. We removed the door.

Travel time no longer lives only in its own corner of Spotto. We changed it from a separate search people run into something they have: their own saved places. Every home they look at now shows the travel time to those places.

Spotto's travel-time search on a phone, before and after: an intro screen on a page of its own, against a listing that shows the travel minutes, with the line '3 min naar Werk' magnified.

You add your places, work, school, the sports club, and every listing on the site shows a line with how long it takes you to get there. If you haven't added a place yet, the listing shows the line anyway, with a question mark where the minutes should be. Tapping it is the way in.

The question mark on the listing matters. You are looking at a home, and the missing answer is how long it would take you to get to work from there. The reason to use the feature appears inside the decision you are already making.

The feature works without an account and stays on your device. The account is offered as the way to keep your places everywhere, on all devices.

Four steps on a phone: every listing asks with a question mark, one short form, every listing answers with the minutes, and the map shows your reach.

Why something you keep, and not just a better-placed search? A saved workplace makes the next visit useful immediately: new listings already show whether the commute fits. We wanted to give people a reason to return, with a search that already knew what mattered to them.

What will tell us we were wrong

We wanted to know whether wider use of the feature would translate into more homes viewed and more return visits across the site. We wrote down how we would judge those changes before we saw any results.

More people adding places is not enough. The investment was meant to change how people use the site and whether they return. If adoption grows without those outcomes improving, we have not made the case for spending more on it.

What to do this week

Before you write off a feature for low usage, follow people through it. How many reach it, and how many of those who begin actually finish? That tells you where to look next.

If people start and leave, find out where and why before sending more people in.

If few people reach the feature but those who start finish, test the way in before you throw the feature away. Look for the moment in the existing product where someone needs its answer. Before you build, write down what outcome would justify the investment. More people finding the feature is only the beginning.

If you have a feature like this and want to talk through what the two numbers say, I'm happy to spend 30 minutes on it. No pitch. Just an honest look at what you're dealing with. Book a call here.

I spent almost 10 years on the boundary between deciding what to build and actually shipping it, as a product and delivery lead. Now I run a consulting practice around that, and I write about the patterns every week.

The figures come from Spotto's own analytics between May and September 2026 and its platform records for the year before, rounded. The comparisons set people who found a hidden tool against everyone else. They say what those people did, not what the tool did to them. Visitor figures are undercounts because of cookie consent.

Finish dates don't exist. Odds do.

Wed, 02 Sep 2026 00:00:00 GMT

I checked 14,210 tasks. Then I tried to prove myself wrong.

A software feature does not have a finish date. It has a range of possible ones, and a chance attached to each.

I know because I spent my summer evenings reconstructing all 14,210 tasks from one of the best-documented software projects I could find, which is hard to explain at parties.

I started because of a question I keep hearing in the companies I work with, usually from sales, sometimes from marketing, always at high volume: why is the development team so slow? A date slips. A deal wobbles. Development is asked for a new estimate, and this time, please, one we can actually rely on.

Underneath that request sits an assumption: every feature has one correct duration, and a good-enough team can find it.

It can't. Not because the team isn't good enough. Because the duration isn't there to be found.

The cleanest records I could find

High in the Chilean Andes stands the Vera C. Rubin Observatory. Its software teams spent more than a decade building the systems that turn a nightly flood of sky images into science. The work had federal funding, formal controls, external scrutiny, and more documentation than most companies will ever produce. Because it is science, the project also left unusually complete public records.

Its handbook described a simple five-step process: plan the work, build it, review it, approve it, done. I reconstructed six years of its records and checked how the work actually moved.

Only 3,853 of 14,210 tasks, or 27 percent, followed those five steps in the prescribed order. The others strayed, most of them onto one of 175 other routes. If we cannot even know which route the work will take, an exact arrival time is already suspect.

The routes taken by 14,210 tasks. The official route accounts for 27 percent when followed step by step.

But that comparison is unfair. These were thousands of different tasks. Some were inevitably larger or stranger than others.

So I made it stricter.

The estimate was the same. The time wasn't.

I looked only at the same kind of development work. Then only at tasks the developers had given exactly the same estimate. Then only at tasks whose estimate was never changed. Finally, only at tasks that followed the handbook's five steps in order. That left 880 tasks that looked about as comparable as real software work gets.

More than a third of those 880 tasks were finished within a day, and half within three days. But fourteen in every hundred were still unfinished after thirteen days. Five in every hundred were still unfinished after twenty-eight.

Same kind of work. Same estimate. Same process. Radically different finish times.

Of every hundred comparable observatory tasks, how many finished on each day: 36 within a day, half by day 3, fourteen still going at day 13, five at day 28.

There was an obvious objection. This was scientific software built for an observatory. Maybe ordinary commercial software behaves differently. So I repeated the analysis on two projects run by commercial software companies: the system MongoDB uses to test software changes, and Appcelerator Studio, a tool for building mobile apps. Their teams estimated differently and followed different processes. The exact numbers changed.

The pattern did not.

Even when I compared work with the same estimate that followed the same route from start to finish, the slower tasks still took several times longer than the typical one.

Larger estimates generally did take longer, so the developers were not picking numbers out of a hat. The estimate contained useful information.

It just wasn't a finish date.

In the observatory data, for recorded sizes from 1 to 10, the halfway point generally moves later but the range to 95 percent finished remains wide.

How a reasonable estimate becomes an unreasonable promise

Now scale that up. A feature on the roadmap is a hundred tasks like these, estimated one by one, added together, divided across the team. Twelve weeks, say. The result feels mathematical. It isn't a forecast.

A feature is not finished when its typical task is finished. It is finished when its last necessary task is finished. The last tasks are where the range becomes painful. In the observatory data, five in every hundred were still open after twenty-eight days, on work that usually took three.

That is how a hundred individually reasonable estimates produce one deeply unreliable date. The twelve-week date may be possible. The calculation simply hasn't told us how likely it is.

One date hides the decision

This is where development dates go wrong. Development gives its best estimate. Product puts it on the roadmap. Marketing plans a launch. Sales tells a client. Finance starts expecting the revenue.

At every step, the uncertainty falls away.

What began as "probably around October" becomes "October" and eventually "promised for October." When October passes, development is called slow. It wasn't. The team was asked to be certain about something that was never certain, and then blamed when the odds came due.

One date never removed the uncertainty. It only hid it.

The better question is not "When will it be finished?"

It is "What is the chance it will be finished by the date we need?"

Development can show that chance. The business must decide whether it is good enough.

A date used for internal planning can carry more risk than a date written into a client contract. A launch can move. A regulatory deadline cannot. Sometimes the right answer is a later date. Sometimes it is less scope. Sometimes the opportunity is worth the gamble. That is not a decision development should quietly make on behalf of the company.

Development should forecast. The business should commit.

Put the risk back into the conversation

I can hear the objection: if development only ever gives a chance, nobody can be held to anything anymore. The opposite is true. A forecast can be checked against what eventually happens. If dates described as highly likely keep failing, either the forecast is bad or the way the team works has changed, and both are visible. That is better accountability than arguing afterwards about who said what in a planning meeting.

Your current roadmap already contains risk. Your client promise already has a chance of being broken. Your launch date already has a chance of slipping.

You just cannot see it yet.

"Why is development so slow?" usually has a better answer than "the developers." Someone turned a probability into a promise, and the promise broke.

If you want the chance behind a date you're about to promise, I compute it from the records your team already keeps. No workshop, no new method, no new estimate.

See the chance behind your deadline

This analysis uses public records from three software projects: the Vera C. Rubin Observatory's Data Management software, MongoDB's Evergreen continuous-integration system, and Appcelerator Studio. The 14,210-task example and figures come from the observatory project. I repeated the completion-time comparison separately on two projects run by commercial software companies after reconstructing how each task moved from start to finish and separating finished tasks from cancelled ones. The conclusion held across all three. These are historical outcomes, not guaranteed future results.

The founder is the bottleneck. (Yes, you.)

Fri, 17 Apr 2026 00:00:00 GMT

If you founded a software company and now spend more time in Jira and meetings than in the product and with your clients, this one is for you.

If you work at a company you didn't start, skip it. This isn't your post.

Being the bottleneck in your own company feels exactly like being indispensable.

You're in every meeting because you're needed. Copied on every decision because you matter. Asked because your team respects your judgment.

All of that is also true of a bottleneck.

A bottleneck is, by definition, the most critical node in the system. That's what makes it the bottleneck. And for the system to improve, you have to deliberately make yourself less critical, which isn't a leadership skill anyone teaches.

What's actually happening

In every small tech company I've been part of, missed deadlines get blamed on the usual suspects: wrong hires, bad estimates, scope creep, unclear priorities.

All of those are real. None of them are the root cause.

The root cause is upstream. The founder is the single point of failure in the delivery system. Every priority call, every "what should I work on next," every status question routes through them. The team isn't broken. It's working exactly as designed.

And the design isn't your fault. It grew out of the early days, when you genuinely did need to be in every conversation. That pattern was correct at five people. At twenty-five, it just never got replaced, and now it's the default for how the company runs.

Every delivery consultant you've hired to fix this has been solving the wrong variable, because the variable is upstream of process. No amount of better standups fixes a routing problem.

The 5-minute audit

Six questions. Answer them from evidence, not from instinct.

1. The Slack test. Open your team's main channel. Scroll your last 20 DMs. How many end with someone waiting on your decision or your input?

2. The Friday question. This Friday, ask the person most likely to run things if you stepped back: "If I dropped dead tomorrow, what's the first thing that breaks?" Their face, in the second before they answer, is the real data.

3. The vacation test. Think about the last full week you took off. When you came back, was there a pile of decisions waiting for you? Or had the team made them without you?

4. The client-email test. When a client emails "when will X be ready?", does it come to you first? Or does someone on your team feel entitled to answer without forwarding?

5. The calendar test. Look at your last 10 working days. How many hours were in meetings where you were the decision the room was waiting on, versus meetings where you were just the context?

6. The "blocked" test. Without asking anyone, can you name the three things currently blocked on your team and who's unblocking them? If yes, you're running delivery. If no, nobody is.

If three or more of these produce uncomfortable answers, the rest of this post is for you.

If you want the data version of this audit, not gut feel but actual probabilities, that's what a delivery risk scan does: it reads the work your team has already finished and works out the chance of each deadline landing.

Not every founder-hour is a bottleneck-hour

Before we talk about the fix, a distinction worth making. There are two kinds of founder-involvement, and they get conflated all the time.

Leadership-involvement is deliberate. You're in the meeting because you're choosing direction, shaping culture, or making a call only you can make. That's the job.

Bottleneck-involvement is reactive. You're in the conversation because the question is in front of you, and nobody else feels empowered to answer. That's the problem.

Most founders can't tell which is which in real time. A simple test: at the end of each conversation, ask yourself. Did I move something that couldn't have moved without me? Or did I just clear a queue?

If the honest answer is "cleared a queue" most days, you have your diagnosis.

Why the obvious fix makes things worse

The most common attempt I've seen to solve this is to go dark.

Turn off notifications. Set "schedule time with me for decisions." Announce to the team: "I need deep work, don't ping me unless it's urgent."

It feels like leadership. Protecting focus. Forcing delegation.

It isn't. Here's what actually happens.

Questions don't disappear. They pile up somewhere else, or they die. A new hire has a question, doesn't want to bother you, guesses instead. A senior engineer hits an ambiguous call, can't reach you, sits on it for a day. The team stops asking "is this important enough to interrupt him," learns that the answer is almost never yes, and stops escalating.

The bottleneck doesn't move. It becomes invisible. Things just don't happen, and nobody can quite say why.

Going dark doesn't fix the routing. It breaks it silently.

What it looks like on the other side

Founders who build companies that outlast them eventually reach a version of this.

Your team still checks in with you often. You want them to. The difference is how they check in. They don't arrive with raw questions. They arrive with a thesis. "We're missing this piece of information. I think the answer is X. Do you agree?" You say yes, or no, or "have you considered Y." Thirty seconds later, they go execute. Something moved.

Over time, this is also how they learn to think. Every time they bring you a prepared thesis instead of a raw question, they're practicing the judgment you'd apply yourself. That's how your best people grow into people you can trust with real calls.

You spend your week on product, strategy, clients, and the hires that will shape the next two years. Not on routing.

This takes longer than you want. The first months feel worse, because every time your team brings you a raw question you have to resist answering and coach the pattern instead. It doesn't flip overnight. I've been working on this pattern in my own teams for years. People still drift back to raw questions when things get stressful. The difference isn't that it becomes automatic. The difference is how fast they correct.

What actually works

Fixing the bottleneck isn't about you being less available. It's about the system being able to route around you, and about shaping how it routes to you when it does.

Three changes, in order:

One person, not you, decides what the team works on next when priorities collide. This is a sequencing call, not a strategy call. Most founders conflate the two and own both. Only own the strategy one.
The answer to "what's the status of X" never comes from you. If it does today, it's because you're the only one who knows, which is the problem in miniature.
Shape how your team brings you problems. You want them to check in. You just don't want raw questions. Teach them the pattern: here's what I'm missing, here's what I think the answer is, do you agree? You say yes, no, or "consider X." Thirty seconds, they go execute. Over months, this is how they learn to make the call themselves.

None of these require new hires. All of them require you to stop doing something you currently do, which is harder than hiring anyone.

What to do this week

Start with the Friday question.

Pick the person most likely to run things if you stepped back. Ask them: "If I dropped dead tomorrow, what's the first thing that breaks?" Don't coach the answer. Don't rebut. Write down what they say, word for word.

That's your roadmap for the next 90 days, in priority order.

If you ran the audit and want to talk through what came up, I'm happy to spend 30 minutes on it. No pitch. Just an honest look at what you're dealing with. Book a call here.

I spent almost 10 years inside this problem as a product and delivery lead. Now I'm building a consulting practice around fixing it. I'm writing about the patterns as I go. If you want to follow along come back next week!