arxiv-archive-subcategory
hierarchical classification · science publishing · 200 items · primary question type choice. Source gfissore/arxiv-abstracts-2021 (CC0-1.0); labels are outcome labels, never an LLM judge. Download these samples (JSON).
| System | DecisionScore | Accuracy (top pick) | Answered |
|---|---|---|---|
| Jev | 76.8 | 88.5% | 200 / 200 |
| System One Scorer | 70.6 | 81.5% | 200 / 200 |
| decider-2b | 70.1 | 78.0% | 200 / 200 |
| JevFish | 69.3 | 78.5% | 200 / 200 |
| OpenJev | 65.4 | 86.5% | 200 / 200 |
| NanoJev | 53.7 | 29.5% | 200 / 200 |
data-and-operations:arxiv-34d4b5d025fe · 166 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"title": "New examples of Calabi-Yau threefolds and genus zero surfaces",
"abstract": "We classify the subgroups of the automorphism group of the product of 4 projective lines admitting an invariant anticanonical smooth divisor on which the action is free. As a first application, we describe new examples of Calabi-Yau 3-folds with small Hodge numbers. In particular, the Picard number is 1 and the number of moduli is 5. Furthermore, the fundamental group is non-trivial. We also construct a new family of minimal surfaces of general type with geometric genus zero, K^2=3 and fundamental group of order 16. We show that this family dominates an irreducible component of dimension 4 of the moduli space of the surfaces of general type."
}Question archive · choice · primary (ranked)
Read state.title and state.abstract. Which arXiv archive (top-level subject group) is this paper's primary category in?
11 options
astro_ph— Astrophysicscond_mat— Condensed matter physicscs— Computer scienceecon— Economicseess— Electrical engineering and systems sciencemath— Mathematicsnlin— Nonlinear sciencesphysics— Physics (general physics archive: optics, fluids, plasma, atomic, chemical, applied, ...)q_bio— Quantitative biologyq_fin— Quantitative financestat— Statistics
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: math
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | math | correct | 100.0% | 1.000 |
|
| System One Scorer | math | correct | 71.3% | 0.954 |
|
| decider-2b | math | correct | 99.0% | 1.000 |
|
| JevFish | math | correct | 96.7% | 0.999 |
|
| OpenJev | math | correct | 99.9% | 1.000 |
|
| NanoJev | stat | wrong | 33.5% | 0.683 |
|
Question subcategory · choice
Read state.title and state.abstract. Within the paper's true archive, which subcategory is its primary category? The options are that archive's subcategories.
30 options
math_ac— math.acmath_ag— math.agmath_ap— math.apmath_at— math.atmath_ca— math.camath_co— math.comath_ct— math.ctmath_cv— math.cvmath_dg— math.dgmath_ds— math.dsmath_fa— math.famath_gm— math.gmmath_gn— math.gnmath_gr— math.grmath_gt— math.gtmath_ho— math.homath_kt— math.ktmath_lo— math.lomath_mg— math.mgmath_na— math.namath_nt— math.ntmath_oa— math.oamath_oc— math.ocmath_pr— math.prmath_qa— math.qamath_ra— math.ramath_rt— math.rtmath_sg— math.sgmath_sp— math.spmath_st— math.st
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: math_ag
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | math_ag | correct | 96.0% | 0.999 |
|
| System One Scorer | math_ag | correct | 13.8% | 0.605 |
|
| decider-2b | math_ac | wrong | 12.3% | 0.546 |
|
| JevFish | math_ac | wrong | 5.1% | 0.505 |
|
| OpenJev | math_ca | wrong | 1.1% | 0.424 |
|
| NanoJev | math_qa | wrong | 3.5% | 0.517 |
|
data-and-operations:arxiv-623ca84587da · 84 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"title": "On the existence of certain weak Fano threefolds of Picard number two",
"abstract": "This article settles the question of existence of smooth weak Fano threefolds of Picard number two with small anti-canonical map and previously classified numerical invariants obtained by blowing up certain curves on smooth Fano threefolds of Picard number 1 with the exception of 12 numerical cases."
}Question archive · choice · primary (ranked)
Read state.title and state.abstract. Which arXiv archive (top-level subject group) is this paper's primary category in?
11 options
astro_ph— Astrophysicscond_mat— Condensed matter physicscs— Computer scienceecon— Economicseess— Electrical engineering and systems sciencemath— Mathematicsnlin— Nonlinear sciencesphysics— Physics (general physics archive: optics, fluids, plasma, atomic, chemical, applied, ...)q_bio— Quantitative biologyq_fin— Quantitative financestat— Statistics
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: math
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | math | correct | 100.0% | 1.000 |
|
| System One Scorer | math | correct | 64.8% | 0.929 |
|
| decider-2b | math | correct | 99.3% | 1.000 |
|
| JevFish | math | correct | 96.2% | 0.999 |
|
| OpenJev | math | correct | 99.9% | 1.000 |
|
| NanoJev | math | correct | 47.5% | 0.804 |
|
Question subcategory · choice
Read state.title and state.abstract. Within the paper's true archive, which subcategory is its primary category? The options are that archive's subcategories.
30 options
math_ac— math.acmath_ag— math.agmath_ap— math.apmath_at— math.atmath_ca— math.camath_co— math.comath_ct— math.ctmath_cv— math.cvmath_dg— math.dgmath_ds— math.dsmath_fa— math.famath_gm— math.gmmath_gn— math.gnmath_gr— math.grmath_gt— math.gtmath_ho— math.homath_kt— math.ktmath_lo— math.lomath_mg— math.mgmath_na— math.namath_nt— math.ntmath_oa— math.oamath_oc— math.ocmath_pr— math.prmath_qa— math.qamath_ra— math.ramath_rt— math.rtmath_sg— math.sgmath_sp— math.spmath_st— math.st
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: math_ag
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | math_ag | correct | 98.0% | 1.000 |
|
| System One Scorer | math_mg | wrong | 10.5% | 0.573 |
|
| decider-2b | math_ac | wrong | 14.3% | 0.552 |
|
| JevFish | math_ac | wrong | 6.4% | 0.517 |
|
| OpenJev | math_st | wrong | 1.2% | 0.430 |
|
| NanoJev | math_qa | wrong | 3.4% | 0.515 |
|
data-and-operations:arxiv-896e849c9b4e · 191 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"title": "Stochastic theory of protein synthesis and polysome: ribosome profile on a single mRNA transcript",
"abstract": "The process of polymerizing a protein by a ribosome, using a messenger RNA (mRNA) as the corresponding template, is called {\\it translation}. Ribosome may be regarded as a molecular motor for which the mRNA template serves also as the track. Often several ribosomes may translate the same (mRNA) simultaneously. The ribosomes bound simultaneously to a single mRNA transcript are the members of a polyribosome (or, simply, {\\it polysome}). Experimentally measured {\\it polysome profile} gives the distribution of polysome {\\it sizes}. Recently a breakthrough in determining the instantaneous {\\it positions} of the ribosomes on a given mRNA track has been achieved and the technique is called {\\it ribosome profiling} \\cite{ingolia10,guo10}."
}Question archive · choice · primary (ranked)
Read state.title and state.abstract. Which arXiv archive (top-level subject group) is this paper's primary category in?
11 options
astro_ph— Astrophysicscond_mat— Condensed matter physicscs— Computer scienceecon— Economicseess— Electrical engineering and systems sciencemath— Mathematicsnlin— Nonlinear sciencesphysics— Physics (general physics archive: optics, fluids, plasma, atomic, chemical, applied, ...)q_bio— Quantitative biologyq_fin— Quantitative financestat— Statistics
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: physics
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | q_bio | wrong | 2.0% | 0.049 |
|
| System One Scorer | q_bio | wrong | 1.7% | 0.217 |
|
| decider-2b | q_bio | wrong | 1.4% | 0.123 |
|
| JevFish | q_bio | wrong | 1.9% | 0.182 |
|
| OpenJev | q_bio | wrong | 27.7% | 0.548 |
|
| NanoJev | stat | wrong | 7.5% | 0.298 |
|
Question subcategory · choice
Read state.title and state.abstract. Within the paper's true archive, which subcategory is its primary category? The options are that archive's subcategories.
21 options
physics_acc_ph— physics.acc.phphysics_ao_ph— physics.ao.phphysics_atm_clus— physics.atm.clusphysics_atom_ph— physics.atom.phphysics_bio_ph— physics.bio.phphysics_chem_ph— physics.chem.phphysics_class_ph— physics.class.phphysics_comp_ph— physics.comp.phphysics_data_an— physics.data.anphysics_ed_ph— physics.ed.phphysics_flu_dyn— physics.flu.dynphysics_gen_ph— physics.gen.phphysics_geo_ph— physics.geo.phphysics_hist_ph— physics.hist.phphysics_ins_det— physics.ins.detphysics_med_ph— physics.med.phphysics_optics— physics.opticsphysics_plasm_ph— physics.plasm.phphysics_pop_ph— physics.pop.phphysics_soc_ph— physics.soc.phphysics_space_ph— physics.space.ph
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: physics_bio_ph
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | physics_bio_ph | correct | 99.0% | 1.000 |
|
| System One Scorer | physics_chem_ph | wrong | 6.2% | 0.522 |
|
| decider-2b | physics_bio_ph | correct | 72.6% | 0.952 |
|
| JevFish | physics_bio_ph | correct | 71.7% | 0.955 |
|
| OpenJev | physics_bio_ph | correct | 41.9% | 0.790 |
|
| NanoJev | physics_data_an | wrong | 5.1% | 0.525 |
|
data-and-operations:arxiv-8d6972262680 · 112 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"title": "On the lack of dispersion for a class of magnetic Dirac flows",
"abstract": "We show that global Strichartz estimates for magnetic Dirac operators generally fails, if the potentials do not decay fast enough at infinity. In order to prove this, we construct some explicit examples of homogeneous magnetic potentials with less than Coulomb decay, i. e. with homogeneity-degree more than -1, such that the magnetic field points to a fixed direction, which does not depend on $x\\in \\mathbb{R}^3$."
}Question archive · choice · primary (ranked)
Read state.title and state.abstract. Which arXiv archive (top-level subject group) is this paper's primary category in?
11 options
astro_ph— Astrophysicscond_mat— Condensed matter physicscs— Computer scienceecon— Economicseess— Electrical engineering and systems sciencemath— Mathematicsnlin— Nonlinear sciencesphysics— Physics (general physics archive: optics, fluids, plasma, atomic, chemical, applied, ...)q_bio— Quantitative biologyq_fin— Quantitative financestat— Statistics
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: math
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | math | correct | 95.0% | 0.998 |
|
| System One Scorer | math | correct | 51.1% | 0.865 |
|
| decider-2b | math | correct | 72.2% | 0.939 |
|
| JevFish | math | correct | 56.8% | 0.843 |
|
| OpenJev | math | correct | 77.2% | 0.952 |
|
| NanoJev | stat | wrong | 28.4% | 0.542 |
|
Question subcategory · choice
Read state.title and state.abstract. Within the paper's true archive, which subcategory is its primary category? The options are that archive's subcategories.
30 options
math_ac— math.acmath_ag— math.agmath_ap— math.apmath_at— math.atmath_ca— math.camath_co— math.comath_ct— math.ctmath_cv— math.cvmath_dg— math.dgmath_ds— math.dsmath_fa— math.famath_gm— math.gmmath_gn— math.gnmath_gr— math.grmath_gt— math.gtmath_ho— math.homath_kt— math.ktmath_lo— math.lomath_mg— math.mgmath_na— math.namath_nt— math.ntmath_oa— math.oamath_oc— math.ocmath_pr— math.prmath_qa— math.qamath_ra— math.ramath_rt— math.rtmath_sg— math.sgmath_sp— math.spmath_st— math.st
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: math_ap
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | math_fa | wrong | 10.0% | 0.515 |
|
| System One Scorer | math_ap | correct | 9.5% | 0.570 |
|
| decider-2b | math_ac | wrong | 5.8% | 0.494 |
|
| JevFish | math_ac | wrong | 4.6% | 0.500 |
|
| OpenJev | math_st | wrong | 3.0% | 0.456 |
|
| NanoJev | math_mg | wrong | 2.5% | 0.506 |
|
data-and-operations:arxiv-b87c1c2f175e · 189 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"title": "Tanaka's equation on the circle and stochastic flows",
"abstract": "We define a Tanaka's equation on an oriented graph with two edges and two vertices. This graph will be embedded in the unit circle. Extending this equation to flows of kernels, we show that the laws of the flows of kernels $K$ solution of Tanaka's equation can be classified by pairs of probability measures $(m^+,m^-)$ on $[0,1]$, with mean 1/2. What happens at the first vertex is governed by $m^+$, and at the second by $m^-$. For each vertex $P$, we construct a sequence of stopping times along which the image of the whole circle by $K$ is reduced to $P$. We also prove that the supports of these flows contains a finite number of points, and that except for some particular cases this number of points can be arbitrarily large."
}Question archive · choice · primary (ranked)
Read state.title and state.abstract. Which arXiv archive (top-level subject group) is this paper's primary category in?
11 options
astro_ph— Astrophysicscond_mat— Condensed matter physicscs— Computer scienceecon— Economicseess— Electrical engineering and systems sciencemath— Mathematicsnlin— Nonlinear sciencesphysics— Physics (general physics archive: optics, fluids, plasma, atomic, chemical, applied, ...)q_bio— Quantitative biologyq_fin— Quantitative financestat— Statistics
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: math
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | math | correct | 99.0% | 1.000 |
|
| System One Scorer | math | correct | 36.6% | 0.766 |
|
| decider-2b | math | correct | 61.7% | 0.881 |
|
| JevFish | math | correct | 59.5% | 0.867 |
|
| OpenJev | math | correct | 99.6% | 1.000 |
|
| NanoJev | math | correct | 50.8% | 0.842 |
|
Question subcategory · choice
Read state.title and state.abstract. Within the paper's true archive, which subcategory is its primary category? The options are that archive's subcategories.
30 options
math_ac— math.acmath_ag— math.agmath_ap— math.apmath_at— math.atmath_ca— math.camath_co— math.comath_ct— math.ctmath_cv— math.cvmath_dg— math.dgmath_ds— math.dsmath_fa— math.famath_gm— math.gmmath_gn— math.gnmath_gr— math.grmath_gt— math.gtmath_ho— math.homath_kt— math.ktmath_lo— math.lomath_mg— math.mgmath_na— math.namath_nt— math.ntmath_oa— math.oamath_oc— math.ocmath_pr— math.prmath_qa— math.qamath_ra— math.ramath_rt— math.rtmath_sg— math.sgmath_sp— math.spmath_st— math.st
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: math_pr
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | math_st | wrong | 2.0% | 0.376 |
|
| System One Scorer | math_st | wrong | 7.8% | 0.554 |
|
| decider-2b | math_ac | wrong | 2.5% | 0.459 |
|
| JevFish | math_st | wrong | 2.8% | 0.482 |
|
| OpenJev | math_st | wrong | 0.2% | 0.351 |
|
| NanoJev | math_qa | wrong | 3.1% | 0.513 |
|
data-and-operations:arxiv-baf88e3ff5a7 · 115 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"title": "Asymptotics of the $s$-perimeter as $s\\searrow 0$",
"abstract": "We deal with the asymptotic behavior of the $s$-perimeter of a set $E$ inside a domain $\\Omega$ as $s\\searrow0$. We prove necessary and sufficient conditions for the existence of such limit, by also providing an explicit formulation in terms of the Lebesgue measure of $E$ and $\\Omega$. Moreover, we construct examples of sets for which the limit does not exist."
}Question archive · choice · primary (ranked)
Read state.title and state.abstract. Which arXiv archive (top-level subject group) is this paper's primary category in?
11 options
astro_ph— Astrophysicscond_mat— Condensed matter physicscs— Computer scienceecon— Economicseess— Electrical engineering and systems sciencemath— Mathematicsnlin— Nonlinear sciencesphysics— Physics (general physics archive: optics, fluids, plasma, atomic, chemical, applied, ...)q_bio— Quantitative biologyq_fin— Quantitative financestat— Statistics
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: math
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | math | correct | 100.0% | 1.000 |
|
| System One Scorer | math | correct | 55.2% | 0.888 |
|
| decider-2b | math | correct | 88.1% | 0.989 |
|
| JevFish | math | correct | 83.3% | 0.979 |
|
| OpenJev | math | correct | 99.9% | 1.000 |
|
| NanoJev | math | correct | 52.9% | 0.856 |
|
Question subcategory · choice
Read state.title and state.abstract. Within the paper's true archive, which subcategory is its primary category? The options are that archive's subcategories.
30 options
math_ac— math.acmath_ag— math.agmath_ap— math.apmath_at— math.atmath_ca— math.camath_co— math.comath_ct— math.ctmath_cv— math.cvmath_dg— math.dgmath_ds— math.dsmath_fa— math.famath_gm— math.gmmath_gn— math.gnmath_gr— math.grmath_gt— math.gtmath_ho— math.homath_kt— math.ktmath_lo— math.lomath_mg— math.mgmath_na— math.namath_nt— math.ntmath_oa— math.oamath_oc— math.ocmath_pr— math.prmath_qa— math.qamath_ra— math.ramath_rt— math.rtmath_sg— math.sgmath_sp— math.spmath_st— math.st
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: math_ap
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | math_mg | wrong | 2.0% | 0.374 |
|
| System One Scorer | math_ag | wrong | 3.0% | 0.499 |
|
| decider-2b | math_ac | wrong | 6.4% | 0.517 |
|
| JevFish | math_ac | wrong | 5.0% | 0.507 |
|
| OpenJev | math_ct | wrong | 2.8% | 0.432 |
|
| NanoJev | math_qa | wrong | 3.1% | 0.513 |
|