Atlas of Judgment · the storeroom

Resources

armarium — where the atlas keeps its holdings

descend to the shelvescode · data · api · citation · licence

Every link to the source code, the underlying data, and the machine-reader API, gathered in one place — plus how to cite the atlas and what you may do with what's here.

Source code

The pipeline that built this

Collection, normalization, the DeepSeek/Qwen labeling pipeline, taxonomy induction, and every aggregation script behind every plate — the full working repository, including the sanitized agent session history the pipeline was built under.

Repository

GitHub

t46/atlas-of-judgment — scripts, provenance documents, and the source HTML pages themselves.

Data downloads

The labeled units, and everything built from them

Two layers are published. The primary dataset is the labeled unit tables themselves. The data islands below it are the smaller derived aggregates — one JSON file per statistic the plates chart — published so any number on the site can be recomputed or re-plotted independently.

Primary dataset

Parquet · Hugging Face Hub

t46/atlas-of-judgment

410,586 review-level units (74,380 ICLR 2026 reviews, read at full depth) · 1,009,592 forum-level units (50,861 forums, 2018–2026) · the taxonomy definitions · 600 analyst-labeled rhetoric units. Released CC BY 4.0.

File sizes on the Hub are not mirrored here — check the dataset card on Hugging Face for current per-file sizes.

Data islands — the 50 aggregate files behind the plates

Each file is the exact JSON a plate's charts are built from — same data, same source, just detached from the page. Sizes and contents below are read directly off the published files.

The holdings, drawn to scale Every data island as a circle whose area is its exact weight on disk, read live from the table below. Two files — the taxonomy point-cloud and the core taxonomy tables — carry of the atlas’s 5.17 MB; the other 48, together, weigh half a megabyte. The two drawn dashed are collected but not yet rendered on any plate. Each circle links to its file.

The catalogue · every island by file, size, the plates it feeds, and what it holds

File	Size	Feeds	Contents
acecho-data.json	1.0 KB	Plate XXIII	How much a meta-review's language echoes each reviewer below it, split by that reviewer's score tier and the panel's verdict.
archipelago-data.json	4.8 KB	Plate XXVIII	Research areas clustered into "islands" by dominant object of scrutiny, against a baseline mix.
argument-raw-novelty.json	31.7 KB	Plate VII	The novelty syllogism: ground/warrant/demand clusters with exemplars and the transition matrices behind Fig. 7d.
boilerplate-data.json	12.7 KB	Plate X	Text-similarity measurements of formulaic review language, by object and by rating band.
canon-data.json	0.7 KB	Plate IX	How often an objection names a specific prior work vs. stays vague, by year.
chain-data.json	1.4 KB	Plate IV	Transition likelihoods between successive units within one review — what follows what.
chargesheet-control.json	6.7 KB	Plate XIII	The control behind that matrix: the same standard pairs re-measured three ways — inside one unit, across different units of one review, and across two different reviewers of the same paper — which is what separates a real repulsion from a mixture of habits.
chargesheet-data.json	41.1 KB	Plate XIII	Co-occurrence matrix of which reasoning standards get invoked together in the same review.
combination-data.json	5.3 KB	Plate VIII	The “mere combination” novelty rule, sub-clustered into its exception clauses and referent kinds, with exemplars.
commit-data.json	1.9 KB	Plate VI	How early in a review the eventual verdict becomes predictable from text alone, by review-length decile.
construct-data.json	1.1 KB	Plate I	Construct validity: rating-adjusted gaps between the twelve objects and ICLR’s own three sub-scores (Fig. 1c).
counterfactual-data.json	65.8 KB	Plate XXV	Counterfactual re-scoring comparison between the 2025 and 2026 cycles.
court-data.json	3.2 KB	Plate XXII	How meta-reviewers revise, forgive, or redeem the panel's judgment below them.
currents-data.json	2.6 KB	Plate XXVI	Year-over-year score, word-count, and unit-count series feeding the drift analysis.
decision-data.json	2.8 KB	Plate XV	Decision legibility: cross-validated AUCs at reading accept/reject from criticism tallies, text, and scores (Fig. 15b).
deliberation-data.json	72.3 KB	Plate XXI	Per-forum record of reviewer moves made during panel deliberation.
drift-data.json	4.9 KB	Plates XXVI, XXVII	Nine-year share trends of which objects, standards, and verdict valences reviewers invoke.
drift-language.json	1.1 KB	Appendix II	Whether the words attached to an object actually moved: the span of each object’s yearly vocabulary path against a shuffle noise floor, with the terms that rose early and late.
elements-all.json	67.9 KB	Plate VII	Per-object statistics for each of the twelve objects of scrutiny (empirical scope, method design, theory, and so on).
field-stats.json	1.7 KB	Appendix III	Per-year corpus statistics used as the control baseline for the null-model comparison.
galaxy.json	3.25 MB	Appendix II	The full point-cloud embedding of sampled units used for the taxonomy map, with region labels. Largest file on the site.
interrogative-data.json	3.2 KB	Plate XIX	Rate at which objections are phrased as questions vs. assertions, by object.
itinerary-data.json	8.0 KB	Plate V	The within-review itinerary — which objects get visited, and in what order.
jurisprudence-data.json	3.7 KB	Plates X, XIV	Maps reasoning standards to the objects they're applied to and the verdict severity that follows.
lawtariff-data.json	8.2 KB	Plate XIV	The score cost of each criticism type, modeled as a volume-response slope.
lifecycle-data.json	1.8 KB	Plate XIX	Whether an objection persists, gets addressed, or drops across the rebuttal cycle, by object.
llmtrace-data.json	36.3 KB	Plate XXIX	LLM-era traces over 199,031 raw reviews 2018–2026: frozen marker-word excess per year, co-review convergence, and the marked-review profile battery.
lottery-data.json	2.4 KB	Plate XXV	Score-gap ("lottery") analysis of how much reviewer assignment appears to affect outcome.
minds-data.json	10.2 KB	Plate XXVI	Era profiles of reviewer behavior: rhetoric-form usage by year and the nine-year vital signs behind Fig. 26c.
mirage-data.json	4.7 KB	Appendix III	Real-vs-synthetic reviewer-profile comparison behind the weak-results ("null cabinet") appendix.
moves-data.json	3.7 KB	Plate XVIII	Classification of rebuttal moves (concede, rebut, clarify, and so on) and the effort behind each.
ninegrammar-data.json	14.3 KB	Plate II	Object–standard coupling in the early era (2018–20) vs the late era (2024–26), with the shuffle-null floor behind Plate II’s 138/144.
oracle-data.json	8.6 KB	Plate XXVIII	Outcome prediction by research area, against a baseline model.
overrule-data.json	3.3 KB	Plates XX, XXI	How often a meta-reviewer's final decision overrules the panel's aggregate score.
panel-data.json	5.8 KB	Plates I, XV, XVII, XX, App. II	Cross-cutting panel statistics: rebuttal outcomes, criticism-count "gauntlet," per-standard confidence intervals, meta-reviewer behavior, dissent, and taxonomy reliability.
repair-manual.json	4.1 KB	Plate XI	Catalogue of the specific fixes reviewers suggest, grouped by kind.
repair-k-robustness.json	2.1 KB	Plate XI (verification)	Stability sweep behind the repair catalogue: the same 80,000 asks re-clustered at k = 11–44; “extend the evidence” leads at every k, family sums move by under 5 points.
repertoire-data.json	2.1 KB	Plate XXI	How much a panel's reviewers' repertoire of standards overlaps or clashes, by year.
rhetoric-v2.json	3.4 KB	Plates III, XXVI	The induced rhetorical-form taxonomy (600 analyst-labeled units) and its validation.
score-data.json	3.6 KB	Plates VI, XXIV	Raw review scores matched to review text, by rating.
score-depth.json	24.0 KB	Plate XXIV	How much review depth and detail scales with the score given — dispersion and coverage by rating.
searchparty-data.json	1.0 KB	Plate XX	How much of the twelve-object space a panel collectively covers vs. leaves unseen.
season-data.json	6.7 KB	— (orphaned)	Daily submission and review-activity time series; not currently rendered on any plate.
structure-data.json	14.3 KB	Plate V, App. II	The structural arc of a review — opening move, positional transitions, and length.
threads-data.json	6.4 KB	Plate XVI	The shape of author-reviewer rebuttal threads — whether and how the reviewer returns.
tide-data.json	1.7 KB	— (orphaned)	A 2025-vs-2026 comparison series; not currently rendered on any plate.
timeless-data.json	2.2 KB	Plate XXVII	Which reasoning standards read as stable in phrasing across nine years, and which drifted.
tribunal-ci.json	0.9 KB	Plate XV	Bootstrap confidence intervals on the score effect of each criticism type, by object.
viz-data.json	1.40 MB	Plates I, II, XII, XXX, App. II	Core taxonomy definitions, headline KPIs, and sample specimen reviews most plates draw from.
yield-data.json	0.5 KB	Plate XVII	How much post-rebuttal units strengthen, weaken, or reverse the pre-response judgment.

50 files · 5.17 MB combined · all JSON · served from /api/v1/data/

Schema and machine-reader entry points

JSON · 1.5 KB

index.json

Catalogue of every API endpoint, plus the plate and data-island lists.

YAML · 7.4 KB

openapi.yaml

Machine-readable schema for the whole API layer.

Text · 3.0 KB

llms.txt

The entry point written for an AI reader — how to use the API, and the one caveat that applies across the whole corpus.

JSON · 3.6 KB

corrections.json

The correction ledger — every published claim later found wrong, with what the check showed and the corrected reading, mirroring method §10.

HTML docs

/api/

Human-readable documentation for every endpoint above, in the same visual language as this page's data.

Citation

How to cite this

Cite the atlas as a whole for general reference. For a specific number, cite the plate and claim it came from — that resolves to the exact derivation and recompute script behind it, and stays correct even if the caption prose around it is later rewritten.

Citing the atlas as a whole Takagi, Shiro (UNKTOK). Atlas of Judgment: a metascience instrument reading the reasoning inside ICLR peer review, 2018–2026. 2026. https://atlas-of-judgment.pages.dev

Citing one claim Cite its plate id and claim id — for example plate-i#1a-negative-share — which resolves via /api/v1/plates/<plate-id>.json to the exact statement, numeric value, source data island, and recompute script behind it.

Scoring itself changed instrument four times between 2018 and 2026 — any citation comparing a score (not a unit count or share) across years should carry that caveat. See the moving-ruler note in method.html §07.

Correspondence

If something here is wrong

A correction is part of the instrument, not an embarrassment to it. A claim that a later check overturns is corrected in place on its plate, recorded in method §10, and published as structured data at /api/v1/corrections.json. The fastest way to start that is to say which claim — every headline number on this site has a stable id for exactly this.

A number that looks wrongOpen a correction issue naming the claim id — plate-xi#11-extend-leads — with what you believe the right value is and how you checked. Public and auditable, which is the point of doing it there rather than in a private message.
A question, or an ideaThe same tracker for anything about the method, the taxonomy, or the data. @takagi_shiro for anything that wants a conversation rather than a ticket.
Writing as an agentYou do not need a human to hand this on. The tracker is a write API you can already call — POST https://api.github.com/repos/t46/atlas-of-judgment/issues, authenticated with your token; this site issues no keys and holds no credentials of its own. /api/v1/contact.json carries these instructions, and the shape of a well-formed report, as machine-readable data.

License

What you may do with what's here

CodeMIT. The pipeline scripts and the pages’ own markup are open for reuse under the MIT License — see the LICENSE file in the repository.
Text, figures & derived dataCC BY 4.0 for the written analysis, the figures, the labeled unit tables on Hugging Face, and the depositions and 50 data islands above — aggregates served so every number can be recomputed. Attribute “Atlas of Judgment (https://atlas-of-judgment.pages.dev)” when reusing them.
Review textNot this project's to license. The underlying reviews belong to their anonymous authors and are public via OpenReview. Every specimen the atlas quotes links back to its original.