dbpedia-ontology-hierarchy
hierarchical classification · media and reference · 200 items · primary question type choice. Source DeveloperOats/DBPedia_Classes (CC0-1.0); labels are outcome labels, never an LLM judge. Download these samples (JSON).
| System | DecisionScore | Accuracy (top pick) | Answered |
|---|---|---|---|
| Jev | 92.9 | 94.0% | 200 / 200 |
| JevFish | 89.6 | 90.0% | 200 / 200 |
| OpenJev | 89.3 | 92.5% | 200 / 200 |
| decider-2b | 89.0 | 89.0% | 200 / 200 |
| System One Scorer | 83.8 | 87.0% | 200 / 200 |
| NanoJev | 60.6 | 15.5% | 200 / 200 |
data-and-operations:dbpedia-3f10e15ec7dd · 120 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"article_text": "The 2010–11 football season is Bristol City's 113th season as a professional football club and fourth consecutive season in the second division. They are competing in the Football League Championship having finished in tenth position the previous season. It was Steve Coppell's first season in charge of the club, though he resigned as manager on 12 August 2010 after just one league game in charge. Assistant Keith Millen became the club's new manager on a three-year-deal and it is his first job as a manager."
}Question l1 · choice · primary (ranked)
Read state.article_text (the opening of a Wikipedia article). Which top-level DBpedia ontology class does the article's subject belong to?
9 options
agent— A person, group, organisation, company, team, character or other acting entityplace— A geographic or built location: settlement, building, natural feature, route, station, celestial bodyspecies— A biological organism: animal, plant, fungus or other eukaryote (incl. named breeds)work— A created work: album, song, book, periodical, software, comic, cartoonevent— A dated happening: tournament, race, sports event, societal event, natural event, Olympicssports_season— One season of a sports team or leagueunit_of_work— A legal case or similar unit of proceedingsdevice— A manufactured device such as an enginetopical_concept— An abstract topic such as a genre
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: sports_season
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | sports_season | correct | 100.0% | 1.000 |
|
| JevFish | agent | wrong | 43.6% | 0.695 |
|
| OpenJev | event | wrong | 33.3% | 0.560 |
|
| decider-2b | agent | wrong | 19.7% | 0.391 |
|
| System One Scorer | agent | wrong | 15.5% | 0.521 |
|
| NanoJev | sports_season | correct | 31.1% | 0.690 |
|
Question l2 · choice
Read state.article_text. Within the top-level class you chose, which second-level DBpedia class fits best? The options are the second-level classes of the article's true top-level class.
2 options
football_league_season— one season of a football leaguesports_team_season— one season of a sports team
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: sports_team_season
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | sports_team_season | correct | 91.0% | 0.992 |
|
| JevFish | sports_team_season | correct | 71.8% | 0.921 |
|
| OpenJev | sports_team_season | correct | 85.7% | 0.980 |
|
| decider-2b | sports_team_season | correct | 83.7% | 0.973 |
|
| System One Scorer | sports_team_season | correct | 62.4% | 0.858 |
|
| NanoJev | football_league_season | wrong | 27.4% | 0.472 |
|
data-and-operations:dbpedia-49266c80b7d8 · 206 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"article_text": "Carl Folta is Executive Vice President, Corporate Communications for Viacom. He has served at this post since November 2006. Before that, he served as Executive Vice President, Office of the Chairman, from January 1, 2006, where he served as Sumner Redstone's senior adviser and spokesman. Previously, he was Executive Vice President, Corporate Relations of the former Viacom Inc., since November 2004. Prior to that, he served as Senior Vice President of Corporate Relations of Viacom from November 1994 to November 2004, and Vice President of Corporate Relations of Viacom from April 1994 to November 1994. Folta held various communications positions at Paramount Communications from 1984 (when the company was known as Gulf+Western, retaining this name until 1989) until joining Viacom through its purchase of Paramount in April 1994."
}Question l1 · choice · primary (ranked)
Read state.article_text (the opening of a Wikipedia article). Which top-level DBpedia ontology class does the article's subject belong to?
9 options
agent— A person, group, organisation, company, team, character or other acting entityplace— A geographic or built location: settlement, building, natural feature, route, station, celestial bodyspecies— A biological organism: animal, plant, fungus or other eukaryote (incl. named breeds)work— A created work: album, song, book, periodical, software, comic, cartoonevent— A dated happening: tournament, race, sports event, societal event, natural event, Olympicssports_season— One season of a sports team or leagueunit_of_work— A legal case or similar unit of proceedingsdevice— A manufactured device such as an enginetopical_concept— An abstract topic such as a genre
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: agent
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | agent | correct | 100.0% | 1.000 |
|
| JevFish | agent | correct | 99.9% | 1.000 |
|
| OpenJev | agent | correct | 100.0% | 1.000 |
|
| decider-2b | agent | correct | 99.4% | 1.000 |
|
| System One Scorer | agent | correct | 78.4% | 0.973 |
|
| NanoJev | agent | correct | 39.5% | 0.770 |
|
Question l2 · choice
Read state.article_text. Within the top-level class you chose, which second-level DBpedia class fits best? The options are the second-level classes of the article's true top-level class.
26 options
actor— film, television or stage actorartist— visual or performing artist not covered by a more specific classathlete— sportsperson (generic, when no sport-specific class applies)boxer— boxerbritish_royalty— member of the British royal familybroadcaster— radio or television station or networkcleric— religious office holdercoach— sports coachcompany— commercial companyeducational_institution— school, college or universityfictional_character— fictional character (non-comics)gridiron_football_player— American or Canadian football playergroup— band or other group of peoplemotorcycle_rider— motorcycle racerorganisation— organisation not covered by a more specific classorganisation_member— organisation memberperson— person not covered by a more specific classpolitician— politician or office holderracing_driver— motor racing driverscientist— scientist or academicsports_league— sports leaguesports_manager— sports managersports_team— sports team or clubwinter_sport_player— winter-sport athlete (ice hockey, skiing, ...)wrestler— wrestlerwriter— author, poet or journalist
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: person
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | person | correct | 72.0% | 0.934 |
|
| JevFish | person | correct | 79.3% | 0.975 |
|
| OpenJev | person | correct | 99.8% | 1.000 |
|
| decider-2b | person | correct | 88.5% | 0.992 |
|
| System One Scorer | organisation_member | wrong | 7.4% | 0.476 |
|
| NanoJev | organisation_member | wrong | 12.8% | 0.584 |
|
data-and-operations:dbpedia-588ab13144e8 · 84 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"article_text": "The Apeldoorn–Deventer railway is an important railway line in the Netherlands, running from Apeldoorn to Deventer. The line was opened between 1887 and 1888. It is used by long distance trains from Amsterdam, Rotterdam and The Hague towards Enschede and Berlin, and by local services between Apeldoorn and Enschede."
}Question l1 · choice · primary (ranked)
Read state.article_text (the opening of a Wikipedia article). Which top-level DBpedia ontology class does the article's subject belong to?
9 options
agent— A person, group, organisation, company, team, character or other acting entityplace— A geographic or built location: settlement, building, natural feature, route, station, celestial bodyspecies— A biological organism: animal, plant, fungus or other eukaryote (incl. named breeds)work— A created work: album, song, book, periodical, software, comic, cartoonevent— A dated happening: tournament, race, sports event, societal event, natural event, Olympicssports_season— One season of a sports team or leagueunit_of_work— A legal case or similar unit of proceedingsdevice— A manufactured device such as an enginetopical_concept— An abstract topic such as a genre
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: place
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | place | correct | 100.0% | 1.000 |
|
| JevFish | place | correct | 87.5% | 0.989 |
|
| OpenJev | place | correct | 99.9% | 1.000 |
|
| decider-2b | place | correct | 84.7% | 0.986 |
|
| System One Scorer | place | correct | 53.7% | 0.877 |
|
| NanoJev | unit_of_work | wrong | 9.7% | 0.471 |
|
Question l2 · choice
Read state.article_text. Within the top-level class you chose, which second-level DBpedia class fits best? The options are the second-level classes of the article's true top-level class.
15 options
amusement_park_attraction— ride or attraction in an amusement parkbody_of_water— lake, bay, sea or other still water bodybuilding— building (house, church, museum, ...)celestial_body— star, planet, asteroid or galaxyclerical_administrative_region— diocese or other church administrative regioninfrastructure— dam, airport, power station, bridge or similarnatural_place— mountain, valley, cave, glacier or other landformroute_of_transportation— road, railway line or public transit routesatellite— artificial satellitesettlement— city, town or villagesport_facility— stadium, arena or golf coursestation— railway or metro stationstream— river or streamtower— tower or lighthousevenue— theatre, concert hall or other venue
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: route_of_transportation
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | route_of_transportation | correct | 98.0% | 1.000 |
|
| JevFish | route_of_transportation | correct | 96.7% | 0.999 |
|
| OpenJev | route_of_transportation | correct | 99.7% | 1.000 |
|
| decider-2b | route_of_transportation | correct | 97.7% | 1.000 |
|
| System One Scorer | route_of_transportation | correct | 85.2% | 0.988 |
|
| NanoJev | station | wrong | 22.5% | 0.576 |
|
data-and-operations:dbpedia-82cdaa7294bf · 105 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"article_text": "Electroclash, also known as retro electro, nouveau disco, the new new wave, and tech-pop is a genre of music that fuses 1980s electro, new wave and synthpop with 1990s techno, retro-style electropop and electronic dance music. It emerged in the later 1990s, pioneered by and associated with acts such as for example I-F, Miss Kittin & The Hacker or Fischerspooner."
}Question l1 · choice · primary (ranked)
Read state.article_text (the opening of a Wikipedia article). Which top-level DBpedia ontology class does the article's subject belong to?
9 options
agent— A person, group, organisation, company, team, character or other acting entityplace— A geographic or built location: settlement, building, natural feature, route, station, celestial bodyspecies— A biological organism: animal, plant, fungus or other eukaryote (incl. named breeds)work— A created work: album, song, book, periodical, software, comic, cartoonevent— A dated happening: tournament, race, sports event, societal event, natural event, Olympicssports_season— One season of a sports team or leagueunit_of_work— A legal case or similar unit of proceedingsdevice— A manufactured device such as an enginetopical_concept— An abstract topic such as a genre
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: topical_concept
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | topical_concept | correct | 100.0% | 1.000 |
|
| JevFish | topical_concept | correct | 99.8% | 1.000 |
|
| OpenJev | topical_concept | correct | 99.8% | 1.000 |
|
| decider-2b | topical_concept | correct | 99.7% | 1.000 |
|
| System One Scorer | topical_concept | correct | 50.9% | 0.858 |
|
| NanoJev | topical_concept | correct | 33.3% | 0.706 |
|
data-and-operations:dbpedia-a00d0a078e2c · 183 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"article_text": "Dame Barbara Mary Quant, Mrs Plunket Greene, DBE, FCSD, RDI (born 11 February 1934) is a Welsh fashion designer and British fashion icon. She became an instrumental figure in the 1960s London-based Mod and youth fashion movements. She was one of the designers who took credit for the miniskirt and hot pants, and by promoting these and other fun fashions she encouraged young people to dress to please themselves and to treat fashion as a game. Ernestine Carter, an authoritative and influential fashion journalist of the 1950s/60s, wrote: \\\"It is given to a fortunate few to be born at the right time, in the right place, with the right talents. In recent fashion there are three: Chanel, Dior, and Mary Quant.\\\""
}Question l1 · choice · primary (ranked)
Read state.article_text (the opening of a Wikipedia article). Which top-level DBpedia ontology class does the article's subject belong to?
9 options
agent— A person, group, organisation, company, team, character or other acting entityplace— A geographic or built location: settlement, building, natural feature, route, station, celestial bodyspecies— A biological organism: animal, plant, fungus or other eukaryote (incl. named breeds)work— A created work: album, song, book, periodical, software, comic, cartoonevent— A dated happening: tournament, race, sports event, societal event, natural event, Olympicssports_season— One season of a sports team or leagueunit_of_work— A legal case or similar unit of proceedingsdevice— A manufactured device such as an enginetopical_concept— An abstract topic such as a genre
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: agent
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | agent | correct | 100.0% | 1.000 |
|
| JevFish | agent | correct | 99.7% | 1.000 |
|
| OpenJev | agent | correct | 100.0% | 1.000 |
|
| decider-2b | agent | correct | 99.0% | 1.000 |
|
| System One Scorer | agent | correct | 82.3% | 0.982 |
|
| NanoJev | agent | correct | 26.0% | 0.660 |
|
Question l2 · choice
Read state.article_text. Within the top-level class you chose, which second-level DBpedia class fits best? The options are the second-level classes of the article's true top-level class.
26 options
actor— film, television or stage actorartist— visual or performing artist not covered by a more specific classathlete— sportsperson (generic, when no sport-specific class applies)boxer— boxerbritish_royalty— member of the British royal familybroadcaster— radio or television station or networkcleric— religious office holdercoach— sports coachcompany— commercial companyeducational_institution— school, college or universityfictional_character— fictional character (non-comics)gridiron_football_player— American or Canadian football playergroup— band or other group of peoplemotorcycle_rider— motorcycle racerorganisation— organisation not covered by a more specific classorganisation_member— organisation memberperson— person not covered by a more specific classpolitician— politician or office holderracing_driver— motor racing driverscientist— scientist or academicsports_league— sports leaguesports_manager— sports managersports_team— sports team or clubwinter_sport_player— winter-sport athlete (ice hockey, skiing, ...)wrestler— wrestlerwriter— author, poet or journalist
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: artist
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | artist | correct | 80.0% | 0.960 |
|
| JevFish | person | wrong | 23.6% | 0.512 |
|
| OpenJev | artist | correct | 99.7% | 1.000 |
|
| decider-2b | person | wrong | 20.2% | 0.433 |
|
| System One Scorer | artist | correct | 38.1% | 0.785 |
|
| NanoJev | organisation_member | wrong | 11.0% | 0.569 |
|
data-and-operations:dbpedia-cfd65071fb78 · 190 state tokens (common tier)Turn 1 — the request (identical for every system)
State (JSON)
{
"article_text": "City Sentral was a planned major new retail and leisure development in the heart of the Staffordshire city of Stoke-on-Trent. It was proposed by Realis Estates, and was a planned 650,000 sq ft regional shopping centre which was due to open in 2016. The centre was to include a Marks & Spencer department store, a wide range of new stores and shops, cafés and restaurants, 'vibrant public spaces', a Cineworld cinema complex, an 80-room hotel, parking for 1,000 cars and a new bus station. City Sentral's proposed catchment comprised 870,000 people with more than 360,000 people living within 20 minutes drive of the city centre. The architects for the bus station were Grimshaw Architects and Benoy was working on City Sentral."
}Question l1 · choice · primary (ranked)
Read state.article_text (the opening of a Wikipedia article). Which top-level DBpedia ontology class does the article's subject belong to?
9 options
agent— A person, group, organisation, company, team, character or other acting entityplace— A geographic or built location: settlement, building, natural feature, route, station, celestial bodyspecies— A biological organism: animal, plant, fungus or other eukaryote (incl. named breeds)work— A created work: album, song, book, periodical, software, comic, cartoonevent— A dated happening: tournament, race, sports event, societal event, natural event, Olympicssports_season— One season of a sports team or leagueunit_of_work— A legal case or similar unit of proceedingsdevice— A manufactured device such as an enginetopical_concept— An abstract topic such as a genre
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: place
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | place | correct | 100.0% | 1.000 |
|
| JevFish | place | correct | 88.6% | 0.988 |
|
| OpenJev | place | correct | 99.9% | 1.000 |
|
| decider-2b | place | correct | 93.7% | 0.997 |
|
| System One Scorer | place | correct | 49.7% | 0.850 |
|
| NanoJev | unit_of_work | wrong | 14.1% | 0.541 |
|
Question l2 · choice
Read state.article_text. Within the top-level class you chose, which second-level DBpedia class fits best? The options are the second-level classes of the article's true top-level class.
15 options
amusement_park_attraction— ride or attraction in an amusement parkbody_of_water— lake, bay, sea or other still water bodybuilding— building (house, church, museum, ...)celestial_body— star, planet, asteroid or galaxyclerical_administrative_region— diocese or other church administrative regioninfrastructure— dam, airport, power station, bridge or similarnatural_place— mountain, valley, cave, glacier or other landformroute_of_transportation— road, railway line or public transit routesatellite— artificial satellitesettlement— city, town or villagesport_facility— stadium, arena or golf coursestation— railway or metro stationstream— river or streamtower— tower or lighthousevenue— theatre, concert hall or other venue
Turn 2 — each system's response · Turn 3 — the grade
Expected answer: building
| System | Top pick | Grade | P(expected) | Proper score | Distribution |
|---|---|---|---|---|---|
| Jev | building | correct | 79.0% | 0.969 |
|
| JevFish | building | correct | 54.7% | 0.867 |
|
| OpenJev | building | correct | 99.8% | 1.000 |
|
| decider-2b | building | correct | 56.9% | 0.863 |
|
| System One Scorer | settlement | wrong | 16.4% | 0.589 |
|
| NanoJev | settlement | wrong | 5.1% | 0.491 |
|