Uncovering Community and Project Insights Through Data Driven Methods

### Overview In this presentation, Karan and Oindrila Chatterjee, data scientists within the Emerging Technologies group at Red Hat, discuss leveraging data-driven workflows and machine learning to analyze GitHub repositories and enhance open-source community health. Under their project titled AI for CI (Artificial Intelligence for Continuous Integration), the team demonstrates how extracting operational and development metrics from repositories like `openshift/origin` and `thoth-station` can optimize software development processes, resource allocation, and project management.

### Key Topics - **Project and Community Metrics**: Measuring velocity, member churn, triage duration, and contribution bottlenecks to help open-source program offices (OSPOs) and development leads evaluate project success. - **The Operate First Initiative**: Promoting an open-source community cloud to address the lack of transparent SRE and cloud operations knowledge, running key workloads like Project Open Data Hub to create reusable open operations datasets. - **AI for CI Toolkit**: An open-source collection of scripts, notebooks, pipelines, and dashboards capturing data from CI/CD systems and GitHub repositories. - **Time-to-Merge Prediction**: Designing an end-to-end machine learning pipeline that predicts how long open pull requests (PRs) will take to merge, providing workload visibility to maintainers and contributors.

### Technical Details - **Data Ingestion and Storage**: Pull request and issue data are collected using `mi` (a PyPI package developed under Project Thoth). The extracted data is staged in S3 storage and queried using the Trino database engine, with exploratory analysis carried out in Jupyter Notebooks. - **Visualization**: SQL tables generated in Trino feed into Apache Superset dashboards to track KPIs like PR size distributions, commit counts, top contributors, and time-to-close metrics. - **Machine Learning Formulation**: The team framed the time-to-merge estimation as a multiclass classification task (with bins such as 0–3 hours, 3–6 hours, or 15–24 hours) rather than regression. Feature engineering incorporated PR size, lines of code added, types of files edited (e.g., config vs. README vs. main code), number of changed files, and timestamp context (e.g., weekday vs. weekend). - **Model Evaluation and Serving**: Comparing SVMs, decision trees, gradient boosted trees, and random forests, the team selected a Random Forest classifier. Models were packaged into a container using Source-to-Image (S2I) on top of a Seldon base image, deployed as an HTTP inference endpoint via Seldon Deployment YAML configurations. - **Automation and Orchestration**: Pipelines connecting data collection, cleaning, feature transformation, training, and S3 artifact publishing were modeled as DAGs using Elyra in JupyterLab and executed across Kubeflow or Airflow instances. A GitHub bot (developed with teammate Harshad) queries the Seldon endpoint upon PR opening and posts predictions directly onto GitHub PRs as labels or comments.

### Takeaways Data-driven methods and AI-assisted tooling can convert standard repository activity into actionable intelligence for open-source teams. All artifacts—including scripts, notebooks, Elyra pipelines, Superset dashboards, and public endpoints—are deployed and accessible via the Operate First cloud. Next steps for the project include generalizing the GitHub bot across arbitrary repositories, gathering live user feedback, and refining the underlying predictive models.