Predicted versus delivered
A model is only worth trusting if it has been wrong in public. Below: the two delivered programmes the effort model is calibrated on, and the fitment model's test results, with the misses left in. Clients are anonymised; the numbers are not.
A SEA IT-services programme, multi-module
The model reproduces this programme at 6–8 FTE and 30–40 weeks, with 8,500–11,500 hours against a 7,500-hour plan that expanded by 1,600–2,300 hours once requirements were clarified. That expansion is why the model carries a clarification allowance rather than pretending the RFP is the scope.
A SEA telco programme, TSM-led
The model's first calibration pass came in 16% under the delivered hours on this one; the telco factor was raised and the regression band is now 12,000–17,000 hours over 38–48 weeks at an offshore-led rate tier. The miss is kept in the test so the model can't drift back.
20 cases, 7 easy, 7 medium, 6 hard
The band is what a sales lead reads; it is right every time. The exact feature name is what a delivery lead reads; the model gets most of them and misses a third of the strict checks, usually by naming a neighbouring feature. Those are listed per case in the repository and drive the next prompt change. Each run costs about $0.13 and 80 seconds per case.
A pool of members' own deals, predicted versus actual. The workspace saves every run with the model version it used; when members log what a deal actually took, the pool's median error will be published here and each member's record stays their own. Until then, two deals and a test set are the whole proof, and this page says so.