Ultra-Scale Software Cost Estimation
Sprint-level effort prediction as a temporal modelling problem
Python · PyTorch · XGBoost · BiLSTM · Transformers · SHAP · torch.compile
Most software cost estimation treats story points as a static average. This project treats estimation as what it actually is — a temporal problem, where a team's velocity drifts, scope creeps, and failed sprints carry information forward.
Data quality as a first-class stage
Four sources — the ISBSG and PROMISE project datasets, NASA effort data, and a custom synthetic Agile sprint simulator modelling team learning curves, velocity drift, scope creep and sprint failures — are pulled through a schema harmoniser into one canonical form. Then comes a seven-stage quality pipeline: missing-data imputation, outlier detection by IQR and Isolation Forest, Agile-consistency checks, Kalman-filter temporal smoothing, feature selection, label QA, and SHA-based data versioning with a full audit trail.
Four architectures are then compared on identical data: an XGBoost and Random Forest baseline, a deep BiLSTM with attention, a large Transformer encoder, and a CNN-Transformer hybrid. Training is engineered for an NVIDIA H200 SXM with BF16 mixed precision, gradient accumulation, OneCycleLR with warmup, gradient clipping, curriculum learning and early stopping. Evaluation reports MAE, RMSE, MAPE and R², with attention heatmaps, SHAP values and an ablation study for interpretation.