Systems vs. Security — Longitudinal Comparison
A side-by-side longitudinal comparison of artifact evaluation across systems and security conferences tracked in ReproDB (2017-2026).
High-Level Summary
| Systems | Security | |
|---|---|---|
| Total Artifacts | 861 | 2170 |
| Conferences | 7 (ATC, CAIS, EUROSYS, FAST, OSDI, SC, SOSP) | 8 (ACSAC, CHES, NDSS, PETS, SP, SYSTEX, USENIXSEC, WOOT) |
| AE Committee Members | 2368 | 2223 |
| Unique AE Members | 1248 | 1402 |
Artifact Growth Over Time
Total evaluated artifacts per year, split by area. Security adopted AE earlier (2017) than systems (2019) and grew faster, though much of the gap reflects venue expansion rather than per-venue increases.
Badge & Participation Rates
Percentage of artifacts receiving each badge type (top row) and badge rates as a fraction of all accepted papers (bottom row). Open-science mandates — such as USENIX Security’s 2025 policy — are the strongest lever, more than doubling participation in a single year.
Systems — % of AE Artifacts
Security — % of AE Artifacts
Available, Functional & Reproduced: Systems vs. Security
Conference Timeline Coverage
Artifact counts by conference and year. Darker cells indicate more artifacts evaluated that year.
Top Institutions by Area
Top-10 institutions by combined score (artifact creation + AE service) for each area. Institutions specialize: some lean towards artifact creation (producers), others towards evaluation service (consumers), and a few maintain balanced profiles. The two communities have largely distinct institutional ecosystems.
Each bubble is an institution. X = artifact score, Y = AE service score, size = combined score. Color indicates the systems/security balance. Only institutions with combined score ≥ 50 are shown.