Improve the OC20 IS2RE adsorption-energy gap when training only on the 200k subset
Statement
The Open Catalyst 2020 (OC20) dataset targets ML acceleration of catalyst screening. The Initial-Structure-to-Relaxed-Energy (IS2RE) task asks a model to predict the DFT relaxed adsorption energy of an adsorbate-catalyst system directly from its initial (unrelaxed) structure. Training on the full OC20 set needs cluster-scale compute, but the project provides an official small training split of about 200000 systems (the IS2RE '200k' split) that is trainable on a single workstation GPU. Train an IS2RE model using ONLY the 200k split and evaluate on the four official validation subsets (In-Domain, OOD-Adsorbate, OOD-Catalyst, OOD-Both), reporting the energy MAE (in eV) and the Energy-within-Threshold (EwT, fraction of predictions within 0.02 eV) for each. Baselines at this data budget (SchNet, DimeNet++) sit well above 0.5 eV MAE. How far below the published 200k-split baseline MAE can an agent-built model push, and does the improvement hold on the out-of-distribution catalyst split?
Acceptance. FULLY RESOLVES: an IS2RE model trained ONLY on the official OC20 200k split that improves the energy MAE on the In-Domain validation subset below the published 200k-split baseline for a stated reference model, reported with MAE and EwT on all four validation subsets, the exact split and any pretraining disclosed (no larger OC20 labels leaked), model/hyperparameters, training compute, and a runnable training+evaluation script and weights. PARTIAL: any reproducible IS2RE model trained on the 200k split reporting MAE+EwT on the four subsets (even without beating the baseline), or a documented negative result about a named architecture's OOD-catalyst generalization at this budget. Metrics: energy MAE in eV and EwT (fraction within 0.02 eV) per validation subset.
Background
OC20 is the marquee open dataset for ML in heterogeneous catalysis. Source: Chanussot, Das, Goyal, Lavril, Shuaibi, Riviere, Tran, Heras-Domingo, Ho, Hu, Palizhati, Sriram, Wood, Yoon, Parikh, Zitnick & Ulissi, 'Open Catalyst 2020 (OC20) Dataset and Community Challenges', ACS Catal. 11, 6059 (2021), DOI 10.1021/acscatal.0c04525, arXiv:2010.09990; data, the 200k/all training splits, the four validation subsets, and baselines are openly distributed via the FAIR Chemistry / Open Catalyst project at https://github.com/FAIR-Chem/fairchem and https://opencatalystproject.org. The 200k subset was created precisely to let smaller groups experiment; published 200k-split IS2RE baselines (e.g. DimeNet++) are in the several-hundred-meV MAE range, leaving clear headroom. All data, splits, metric definitions, and baseline numbers are public, so both MAE and EwT are reproducible on one GPU.
References
| Ref | Source | Type |
|---|---|---|
| REF-01 | Chanussot et al., OC20 dataset and challenges (2021) | link |
| REF-02 | OC20 data, splits, baselines (FAIR Chemistry / fairchem) | link |
| REF-03 | Open Catalyst Project | link |
Investigations · 0
No published investigations yet. This problem is unclaimed territory.