Typed Decision Bench · tasks · Retrieval and knowledge

`casehold-cited-holding`

citation verification · legal · 200 items · primary question type choice. Source coastalcph/lex_glue (CC-BY-4.0); labels are outcome labels, never an LLM judge. Download these samples (JSON).

System	DecisionScore	Accuracy (top pick)	Answered
Jev	87.0	79.5%	200 / 200
JevFish	80.0	74.5%	200 / 200
decider-2b	80.0	70.5%	200 / 200
OpenJev	74.7	66.5%	200 / 200
System One Scorer	69.5	51.5%	200 / 200
NanoJev	61.2	33.8%	198 / 200

Sample 1 of 6 · retrieval-and-knowledge:casehold-1078 · 228 state tokens (common tier)

Turn 1 — the request (identical for every system)

State (JSON)

{
  "citing_context": "to rule on a motion and therefore forgot to rule on a motion to strike. Reply at 2. The court finds this argument speculative at best. In light of the “relative unseemliness of a court’s altering a legal ruling as to the same litigants,” Zdanok, 327 F.2d at 953, the court needs greater justification to re visit Judge Johnson’s ruling than conjecture about whether she forgot something or felt rushed. Because the defendant fails to offer such justification, its motion for reconsiderat 650 F.2d 663, 666 (5th Cir.1981) (finding that the grant of summary judgment for the defendant \"was so inconsistent with the plaintiff's request for leave to amend to state a new claim for relief as implicitly to deny the motion to amend”); Wimberly v. Clark Controller Co., 364 F.2d 225, 227 (6th Cir.1966) (<HOLDING>"
}

Question cited_holding · choice · primary (ranked)

The state holds citing_context, an excerpt of a US court opinion ending at a citation whose parenthetical holding was replaced by <HOLDING>. Which candidate holding is the one the citing sentence relies on for the proposition it states? Pick the holding that supplies exactly that proposition, not merely the same area of law.

5 options

holding_0 — holding that wjhile it is certainly the better practice to specifically rule on all pending motions the determination of a motion need not always be expressed but may be implied by an entiy of an order inconsistent with granting the relief sought
holding_1 — holding death sentence need not be pending in order for this court to engage in review of issues on appeal
holding_2 — holding order granting motion to withdraw plea to be the functional equivalent of an order granting a new trial because like a postverdict mistrial it returns case to posture it had been before trial
holding_3 — holding that the scheduling order merely prescribes the date by which all motions to amend shall be filed the date on the scheduling order does not confine the district courts consideration of the merits of such motions and does not preclude it from finding that an amendment would result in …
holding_4 — holding interpretation may not be inconsistent with regulation

Turn 2 — each system's response · Turn 3 — the grade

Expected answer: holding_0

System	Top pick	Grade	P(expected)	Proper score	Distribution
Jev	`holding_0`	correct	100.0%	1.000	`holding_0` 100.0% `holding_1` 0.0% `holding_3` 0.0% `holding_4` 0.0% `holding_2` 0.0%
JevFish	`holding_0`	correct	67.3%	0.918	`holding_0` 67.3% `holding_3` 23.0% `holding_4` 4.5% `holding_2` 3.8% `holding_1` 1.4%
decider-2b	`holding_0`	correct	72.8%	0.947	`holding_0` 72.8% `holding_3` 16.4% `holding_2` 5.7% `holding_1` 3.0% `holding_4` 2.1%
OpenJev	`holding_0`	correct	99.8%	1.000	`holding_0` 99.8% `holding_3` 0.2% `holding_4` 0.0% `holding_1` 0.0% `holding_2` 0.0%
System One Scorer	`holding_0`	correct	93.0%	0.996	`holding_0` 93.0% `holding_3` 4.2% `holding_2` 1.7% `holding_1` 0.9% `holding_4` 0.2%
NanoJev	`holding_0`	correct	29.2%	0.668	`holding_0` 29.2% `holding_3` 28.4% `holding_2` 25.9% `holding_4` 8.5% `holding_1` 8.0%

Sample 2 of 6 · retrieval-and-knowledge:casehold-1405 · 210 state tokens (common tier)

Turn 1 — the request (identical for every system)

State (JSON)

{
  "citing_context": "Nations with ample notice of its intent to charge her as a habitual offender, and she had a fair opportunity to present any defense to that charge. ¶15. Moreover, at the March 23, 2014 hearing on the State’s motion to amend the indictment to charge her as a habitual offender, Nations’s attorney specifically stated: ‘Tour honor, we don’t have any opposition [to the State’s motion]. It’s either — she’s habitual or she’s not. I think that we had already talked and she is in that status.” After Nations’s attorney’s statement, the trial judge granted the State’s motion. It is difficult to imagine a clearer waiver of the issue. Thus, Nations’s claim of unfair surprise is not only without merit but also procedurally barred. Newberry v. State, 85 So.3d 884, 888 (¶ 8) (Miss.Ct.App.2011) (<HOLDING>"
}

Question cited_holding · choice · primary (ranked)

5 options

holding_0 — holding that a noticebased objection to a motion to amend an indictment to charge the defendant as a habitual offender is waived if the defendant fails to make a specific and contemporaneous objection
holding_1 — holding that an objection was adequate when the judge cut short the objection and the defendant was not afforded the opportunity to explain his objection fully
holding_2 — holding a contemporaneous objection is required to preserve an issue for appellate review
holding_3 — holding that a party fails to preserve an evidentiary issue for appeal not only by failing to make a specific objection but also by making the wrong specific objection
holding_4 — holding that defendant waived his objection to the prosecutors use of her peremptory challenges by failing to make a contemporaneous objection during jury selection

Turn 2 — each system's response · Turn 3 — the grade

Expected answer: holding_0

System	Top pick	Grade	P(expected)	Proper score	Distribution
Jev	`holding_0`	correct	98.0%	1.000	`holding_0` 98.0% `holding_2` 2.0% `holding_3` 0.0% `holding_1` 0.0% `holding_4` 0.0%
JevFish	`holding_0`	correct	92.0%	0.996	`holding_0` 92.0% `holding_3` 3.0% `holding_2` 2.2% `holding_4` 1.8% `holding_1` 1.1%
decider-2b	`holding_0`	correct	97.3%	1.000	`holding_0` 97.3% `holding_2` 0.9% `holding_1` 0.8% `holding_3` 0.8% `holding_4` 0.2%
OpenJev	`holding_0`	correct	99.9%	1.000	`holding_0` 99.9% `holding_1` 0.1% `holding_3` 0.0% `holding_2` 0.0% `holding_4` 0.0%
System One Scorer	`holding_0`	correct	53.6%	0.864	`holding_0` 53.6% `holding_4` 16.0% `holding_1` 11.3% `holding_2` 10.8% `holding_3` 8.3%
NanoJev	`holding_0`	correct	23.3%	0.632	`holding_0` 23.3% `holding_2` 20.5% `holding_1` 19.9% `holding_3` 19.1% `holding_4` 17.2%

Sample 3 of 6 · retrieval-and-knowledge:casehold-1529 · 187 state tokens (common tier)

Turn 1 — the request (identical for every system)

State (JSON)

{
  "citing_context": "of the laws of the forum state and may have the minimum contacts with the forum state sufficient to meet the due process requirements of the exercise of personal jurisdiction by the forum state. In conclusion, we hold that the signing of a guaranty by a nonresident of a debt owed to a New Mexico creditor does not in and of itself constitute a sufficient contact upon which to base in personam jurisdiction over a nonresident. Rather, the circumstances surrounding the signing of such obligations must be closely examined in each case to determine whether the quality and nature of defendant’s contacts with New Mexico justify the assertion of personal jurisdiction over him in an action on the obligation. Here, the Hiatts did nothing more tha on, Inc., 558 F.Supp. 1118, 1123-24 (N.D.Ill.1983) (<HOLDING>"
}

Question cited_holding · choice · primary (ranked)

5 options

holding_0 — holding that a choice of law provision is not sufficient to confer personal jurisdiction over a nonresident defendant
holding_1 — holding that court did not have personal jurisdiction over nonresident defendant because plaintiff failed to show that defendant was assignee of assignor over whom court had personal jurisdiction
holding_2 — holding no personal jurisdiction over nonresident guarantor
holding_3 — holding that nonresident defendants failure to make payments in florida coupled with forum selection clause sufficient for court to exercise personal jurisdiction over nonresident defendant
holding_4 — holding that court did not have personal jurisdiction over nonresident defendant where defendant assignee received patents from assignor over whom court had personal jurisdiction in part because the unilateral activity of those who claim some relationship with a nonresident defendant cannot satisfy …

Turn 2 — each system's response · Turn 3 — the grade

Expected answer: holding_0

System	Top pick	Grade	P(expected)	Proper score	Distribution
Jev	`holding_2`	wrong	23.0%	0.482	`holding_2` 66.0% `holding_0` 23.0% `holding_4` 8.0% `holding_1` 2.0% `holding_3` 1.0%
JevFish	`holding_0`	correct	40.0%	0.739	`holding_0` 40.0% `holding_2` 38.3% `holding_1` 7.5% `holding_4` 7.4% `holding_3` 6.8%
decider-2b	`holding_0`	correct	49.1%	0.826	`holding_0` 49.1% `holding_2` 25.0% `holding_1` 11.6% `holding_3` 10.0% `holding_4` 4.2%
OpenJev	`holding_2`	wrong	0.0%	0.002	`holding_2` 99.9% `holding_0` 0.0% `holding_4` 0.0% `holding_3` 0.0% `holding_1` 0.0%
System One Scorer	`holding_4`	wrong	7.1%	0.402	`holding_4` 43.2% `holding_2` 37.3% `holding_3` 7.2% `holding_0` 7.1% `holding_1` 5.2%
NanoJev	`holding_0`	correct	39.6%	0.766	`holding_0` 39.6% `holding_1` 21.2% `holding_4` 17.2% `holding_3` 15.1% `holding_2` 6.9%

Sample 4 of 6 · retrieval-and-knowledge:casehold-2653 · 209 state tokens (common tier)

Turn 1 — the request (identical for every system)

State (JSON)

{
  "citing_context": "no confidential/fiduciary relationship existed because the parties were adversaries, each represented by their own attorney. On appeal, Virginia contends the law permits the inference of a confidential relationship to the extent Edwin holds property in excess of $690,000 that rightfully belongs to her. We disagree with Virginia’s argument. Texas law recognizes that a relationship on which a constructive trust can be based may be an informal fiduciary relationship that arises from a moral, social, domestic, or purely personal relationship of trust and confidence. Thigpen v. Locke, 363 S.W.2d 247, 253 (Tex. 1962). However, any such fiduciary relationship between Edwin and Virginia terminated when they divorced. See Solares v. Solares, 232 S.W.3d 873, 881 (Tex. App.-Dallas 2007, no pet.) (<HOLDING>"
}

Question cited_holding · choice · primary (ranked)

5 options

holding_0 — holding that a corporate entity must be represented by licensed counsel
holding_1 — holding in a contested divorce where each spouse is independently represented by counsel the fiduciary relationship terminates
holding_2 — holding that the automatic stay terminates completely as to a serially filing spouse but remains in force as to a newly filing spouse
holding_3 — recognizing a defendants constitutional right to be represented by counsel of his own choice
holding_4 — recognizing that the rights accorded by the homestead laws vest independently in each spouse regardless of whether one spouse or both actually owns the fee interest in the homestead

Turn 2 — each system's response · Turn 3 — the grade

Expected answer: holding_1

System	Top pick	Grade	P(expected)	Proper score	Distribution
Jev	`holding_1`	correct	100.0%	1.000	`holding_1` 100.0% `holding_4` 0.0% `holding_0` 0.0% `holding_3` 0.0% `holding_2` 0.0%
JevFish	`holding_1`	correct	95.4%	0.999	`holding_1` 95.4% `holding_3` 1.7% `holding_4` 1.6% `holding_0` 0.8% `holding_2` 0.5%
decider-2b	`holding_1`	correct	94.0%	0.998	`holding_1` 94.0% `holding_3` 2.0% `holding_2` 1.9% `holding_4` 1.2% `holding_0` 0.8%
OpenJev	`holding_1`	correct	100.0%	1.000	`holding_1` 100.0% `holding_3` 0.0% `holding_0` 0.0% `holding_2` 0.0% `holding_4` 0.0%
System One Scorer	`holding_1`	correct	95.6%	0.999	`holding_1` 95.6% `holding_2` 3.0% `holding_3` 0.9% `holding_4` 0.4% `holding_0` 0.2%
NanoJev	`holding_1`	correct	37.3%	0.744	`holding_1` 37.3% `holding_4` 27.7% `holding_2` 12.5% `holding_3` 11.6% `holding_0` 10.9%

Sample 5 of 6 · retrieval-and-knowledge:casehold-3523 · 245 state tokens (common tier)

Turn 1 — the request (identical for every system)

State (JSON)

{
  "citing_context": "Alaska consumer fraud act); Colo.Rev.Stat. Ann. § 6-l-113(2)(a) (the court may award three times the actual damages, or $500, whichever is greater, if it is proven that the defendant engaged in bad faith conduct); Quist v. Specialties Supply Co., Inc., 12 P.3d 863, 867 (Colo.Ct.App.2000) (where the court held plaintiff was precluded under principles of collateral estoppel from challenging the amount of actual damages fixed by the judgment, but holding that collateral estoppel did not preclude plaintiff from pursuing enhanced punitive damages); Ohio Rev. Code Ann. 1345.09(B)(the injured party may seek and the trial court in its discretion may rescind the contract or award treble the actual damages or $200, whichever is greater); Crye v. Smolak, 110 Ohio App.3d 504, 674 N.E. 2d 779 (1996)(<HOLDING>"
}

Question cited_holding · choice · primary (ranked)

5 options

holding_0 — holding that treble damages are appropriate under ohio law when the seller engages in an unconscionable practice or deceptive act
holding_1 — holding treble damages under the civil theft statute are punitive
holding_2 — holding that a plaintiff may receive both treble damages under rico and state law punitive damages for the same course of conduct
holding_3 — recognizing that an award of treble damages for an antitrust violation and punitive damages for common law tortious interference with business relations based on same conduct would be duplicative holding that punitive and treble damages cannot both be awarded and indicating that a special jury …
holding_4 — holding that treble damages under the housing and rent act are remedial in nature

Turn 2 — each system's response · Turn 3 — the grade

Expected answer: holding_0

System	Top pick	Grade	P(expected)	Proper score	Distribution
Jev	`holding_0`	correct	95.0%	0.998	`holding_0` 95.0% `holding_1` 2.0% `holding_4` 1.0% `holding_2` 1.0% `holding_3` 1.0%
JevFish	`holding_2`	wrong	21.0%	0.556	`holding_2` 46.4% `holding_0` 21.0% `holding_3` 20.0% `holding_4` 7.2% `holding_1` 5.4%
decider-2b	`holding_0`	correct	40.7%	0.765	`holding_0` 40.7% `holding_2` 29.1% `holding_3` 14.8% `holding_1` 9.2% `holding_4` 6.2%
OpenJev	`holding_0`	correct	99.9%	1.000	`holding_0` 99.9% `holding_1` 0.0% `holding_3` 0.0% `holding_2` 0.0% `holding_4` 0.0%
System One Scorer	`holding_3`	wrong	4.1%	0.315	`holding_3` 64.7% `holding_1` 12.4% `holding_2` 9.9% `holding_4` 8.9% `holding_0` 4.1%
NanoJev	`holding_2`	wrong	15.8%	0.549	`holding_2` 30.6% `holding_1` 22.7% `holding_3` 17.5% `holding_0` 15.8% `holding_4` 13.5%

Sample 6 of 6 · retrieval-and-knowledge:casehold-885 · 242 state tokens (common tier)

Turn 1 — the request (identical for every system)

State (JSON)

{
  "citing_context": "with Rule 32.6(b)’s pleading requirements. Ala. R.Crim. P. 32.7(d). While the Court of Criminal Appeals refers to all three of these rules in its peitinent order, we focus our analysis on Rule 32.6(b), as Frazier’s failure to provide \"full disclosure of the factual basis” of the relevant claim was the sine qua non of the Court of Criminal Appeals’s dismissal. 7 . It is uncontroversial that \"[i]f the petitioner has failed to exhaust state remedies that are no longer available, that failure is a procedural default which will bar federal habeas relief, unless either the cause and prejudice or the fundamental miscarriage of justice exception is established.” Smith v. Jones, 256 F.3d 1135, 1138 (11th Cir.2001); see also Owen v. Sec’y for Dep't of Corr., 568 F.3d 894, 908 (11th Cir.2006) (<HOLDING>"
}

Question cited_holding · choice · primary (ranked)

5 options

holding_0 — holding that a petitioner may overcome a procedural default by demonstrating either 1 an adequate cause for and actual prejudice arising from the default or 2 that a miscarriage of justice caused by a substantial denial of constitutional rights will occur if the petitioners federal claims are not …
holding_1 — holding that in the case of state procedural default a federal habeas review of the claims is barred unless the prisoner can demonstrate among other things that failure to consider the claims will result in a fundamental miscarriage of justice
holding_2 — holding that trial court must make entry of default prior to entry of default judgment and court may not make entry of default when there is no default in law or in fact
holding_3 — holding that a federal court may excuse a state habeas petitioners procedural default if the petitioner can show cause for the failure to raise the claim and prejudice resulting from such failure
holding_4 — holding that the novelty of a petitioners claim can constitute cause for a procedural default

Turn 2 — each system's response · Turn 3 — the grade

Expected answer: holding_0

System	Top pick	Grade	P(expected)	Proper score	Distribution
Jev	`holding_0`	correct	83.0%	0.975	`holding_0` 83.0% `holding_1` 14.0% `holding_3` 3.0% `holding_4` 0.0% `holding_2` 0.0%
JevFish	`holding_0`	correct	54.0%	0.840	`holding_0` 54.0% `holding_1` 31.2% `holding_3` 10.2% `holding_4` 4.1% `holding_2` 0.6%
decider-2b	`holding_0`	correct	57.0%	0.856	`holding_0` 57.0% `holding_1` 30.5% `holding_3` 10.1% `holding_2` 1.6% `holding_4` 0.8%
OpenJev	`holding_1`	wrong	7.8%	0.159	`holding_1` 91.2% `holding_0` 7.8% `holding_3` 1.0% `holding_2` 0.0% `holding_4` 0.0%
System One Scorer	`holding_0`	correct	97.9%	1.000	`holding_0` 97.9% `holding_1` 1.5% `holding_3` 0.4% `holding_4` 0.1% `holding_2` 0.1%
NanoJev	`holding_3`	wrong	16.0%	0.547	`holding_3` 32.0% `holding_4` 24.3% `holding_0` 16.0% `holding_2` 15.2% `holding_1` 12.4%