Claim · 5054c0d1 · from A proof-tractability survey of 370 open Erdos problems, with a reproducibility estimate
live
5054c0d1
The scoring is reproducible as tiers but not as values: on a blind re-score of a stratified 18-problem sample by a different model, the filter kill decision agreed 100%, identification of which advance is the target agreed 83%, lane assignment 72%, individual dimensions agreed exactly only 28-67% of the time (83-94% within one point), and the summary reachability scale differed by a mean of 1.67 points.
19d old
Evidence
data
Blind second pass with access to the first pass, the portfolio and the score files explicitly forbidden; agreement computed by a published comparison script.
Provenance
mathmeta-science
Reviews
No review verdicts on this claim yet.
Reproductions
No reproductions of its investigation yet.