Predicting CI/CD Workflow Outcomes Using Time-Aware Feature Engineering and Gradient Boosting Models

Document Type : Original Article

Authors

1 Bozorgmehr University of Qaenat

2 Bozorgmehr University of Technology

Abstract
The widespread adoption of Continuous Integration and Continuous Delivery (CI/CD) pipelines, particularly through platforms such as GitHub Actions, has introduced new challenges in ensuring pipeline reliability and predictability. Unexpected workflow failures can disrupt release cycles and increase operational costs. This paper presents a time-aware, data-driven approach for predicting CI/CD workflow execution outcomes. Historical failure patterns are extracted using time-aware cumulative aggregation. Feature engineering is performed incrementally, including Bayesian smoothing to stabilize sparse statistics, Last-K features (k = 5, 10) to capture short-term behavior, and interaction features to model nonlinear dependencies between entities such as repositories and workflow paths, forming the core feature-set (Base_Feature_Set). Additionally, textual features are extracted from workflow logs using regular expressions, capturing error presence, frequency, and message patterns, resulting in an enriched feature-set (Log-Enriched_Feature_Set). Using the large-scale GHALogs dataset, comprising over 513,000 workflow executions from approximately 25,000 repositories, we evaluate five models (Logistic Regression, Random Forest, HistGradientBoosting, XGBoost and CatBoost) under a consistent chronological split for both feature sets. Experimental results show that time-aware structured features alone yield moderate discriminative power (AUC 0.603–0.646). Adding log-derived features, which become available only after execution, raises the AUC of the final CatBoost model to 0.843. Because the log features describe the run being predicted, this improvement reflects post-execution failure-detection performance rather than pre-execution forecasting.

Keywords

Subjects

Volume 3, Issue 2 - Serial Number 2
January 2026
Pages 67-76

  • Receive Date 17 April 2026
  • Revise Date 16 October 2026
  • Accept Date 18 October 2026
  • First Publish Date 18 October 2026
  • Publish Date 01 January 2026