Full benchmark source
HumanEval: every task.
164 tasks · complete evaluation split
164 of 164 tasks · humaneval
humaneval-0Python · from typing import List def has_close_elements(numbers: List[float], threshold: float) -> bool: """ Check if in gi…
Original task
from typing import List def has_close_elements(numbers: List[float], threshold: float) -> bool: """ Check if in given list of numbers, are any two numbers closer to each other than given threshold. >>> has_close_elements([1.0, 2.0, 3.0], 0.5) False >>> has_close_elements([1.0, 2.8, 3.0, 4.0, 5.0, 2.0], 0.3) True """
- Source ID
- HumanEval/0
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-1Python · from typing import List def separate_paren_groups(paren_string: str) -> List[str]: """ Input to this function is a…
Original task
from typing import List def separate_paren_groups(paren_string: str) -> List[str]: """ Input to this function is a string containing multiple groups of nested parentheses. Your goal is to separate those group into separate strings and return the list of those. Separate groups are balanced (each open brace is properly closed) and not nested within each other Ignore any spaces in the input string. >>> separate_paren_groups('( ) (( )) (( )( ))') ['()', '(())', '(()())'] """
- Source ID
- HumanEval/1
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-10Python · def is_palindrome(string: str) -> bool: """ Test if given string is a palindrome """ return string == string[:…
Original task
def is_palindrome(string: str) -> bool: """ Test if given string is a palindrome """ return string == string[::-1] def make_palindrome(string: str) -> str: """ Find the shortest palindrome that begins with a supplied string. Algorithm idea is simple: - Find the longest postfix of supplied string that is a palindrome. - Append to the end of the string reverse of a string prefix that comes before the palindromic suffix. >>> make_palindrome('') '' >>> make_palindrome('cat') 'catac' >>> make_palindrome('cata') 'catac' """
- Source ID
- HumanEval/10
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-100Python · def make_a_pile(n): """ Given a positive integer n, you have to make a pile of n levels of stones. The firs…
Original task
def make_a_pile(n): """ Given a positive integer n, you have to make a pile of n levels of stones. The first level has n stones. The number of stones in the next level is: - the next odd number if n is odd. - the next even number if n is even. Return the number of stones in each level in a list, where element at index i represents the number of stones in the level (i+1). Examples: >>> make_a_pile(3) [3, 5, 7] """
- Source ID
- HumanEval/100
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-101Python · def words_string(s): """ You will be given a string of words separated by commas or spaces. Your task is to…
Original task
def words_string(s): """ You will be given a string of words separated by commas or spaces. Your task is to split the string into words and return an array of the words. For example: words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"] words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"] """
- Source ID
- HumanEval/101
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-102Python · def choose_num(x, y): """This function takes two positive numbers x and y and returns the biggest even integer …
Original task
def choose_num(x, y): """This function takes two positive numbers x and y and returns the biggest even integer number that is in the range [x, y] inclusive. If there's no such number, then the function should return -1. For example: choose_num(12, 15) = 14 choose_num(13, 12) = -1 """
- Source ID
- HumanEval/102
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-103Python · def rounded_avg(n, m): """You are given two positive integers n and m, and your task is to compute the average …
Original task
def rounded_avg(n, m): """You are given two positive integers n and m, and your task is to compute the average of the integers from n through m (including n and m). Round the answer to the nearest integer and convert that to binary. If n is greater than m, return -1. Example: rounded_avg(1, 5) => "0b11" rounded_avg(7, 5) => -1 rounded_avg(10, 20) => "0b1111" rounded_avg(20, 33) => "0b11010" """
- Source ID
- HumanEval/103
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-104Python · def unique_digits(x): """Given a list of positive integers x. return a sorted list of all elements that hasn't…
Original task
def unique_digits(x): """Given a list of positive integers x. return a sorted list of all elements that hasn't any even digit. Note: Returned list should be sorted in increasing order. For example: >>> unique_digits([15, 33, 1422, 1]) [1, 15, 33] >>> unique_digits([152, 323, 1422, 10]) [] """
- Source ID
- HumanEval/104
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-105Python · def by_length(arr): """ Given an array of integers, sort the integers that are between 1 and 9 inclusive, r…
Original task
def by_length(arr): """ Given an array of integers, sort the integers that are between 1 and 9 inclusive, reverse the resulting array, and then replace each digit by its corresponding name from "One", "Two", "Three", "Four", "Five", "Six", "Seven", "Eight", "Nine". For example: arr = [2, 1, 1, 4, 5, 8, 2, 3] -> sort arr -> [1, 1, 2, 2, 3, 4, 5, 8] -> reverse arr -> [8, 5, 4, 3, 2, 2, 1, 1] return ["Eight", "Five", "Four", "Three", "Two", "Two", "One", "One"] If the array is empty, return an empty array: arr = [] return [] If the array has any strange number ignore it: arr = [1, -1 , 55] -> sort arr -> [-1, 1, 55] -> reverse arr -> [55, 1, -1] return = ['One'] """
- Source ID
- HumanEval/105
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-106Python · def f(n): """ Implement the function f that takes n as a parameter, and returns a list of size n, such that the…
Original task
def f(n): """ Implement the function f that takes n as a parameter, and returns a list of size n, such that the value of the element at index i is the factorial of i if i is even or the sum of numbers from 1 to i otherwise. i starts from 1. the factorial of i is the multiplication of the numbers from 1 to i (1 * 2 * ... * i). Example: f(5) == [1, 2, 6, 24, 15] """
- Source ID
- HumanEval/106
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-107Python · def even_odd_palindrome(n): """ Given a positive integer n, return a tuple that has the number of even and odd …
Original task
def even_odd_palindrome(n): """ Given a positive integer n, return a tuple that has the number of even and odd integer palindromes that fall within the range(1, n), inclusive. Example 1: Input: 3 Output: (1, 2) Explanation: Integer palindrome are 1, 2, 3. one of them is even, and two of them are odd. Example 2: Input: 12 Output: (4, 6) Explanation: Integer palindrome are 1, 2, 3, 4, 5, 6, 7, 8, 9, 11. four of them are even, and 6 of them are odd. Note: 1. 1 <= n <= 10^3 2. returned tuple has the number of even and odd integer palindromes respectively. """
- Source ID
- HumanEval/107
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-108Python · def count_nums(arr): """ Write a function count_nums which takes an array of integers and returns the numbe…
Original task
def count_nums(arr): """ Write a function count_nums which takes an array of integers and returns the number of elements which has a sum of digits > 0. If a number is negative, then its first signed digit will be negative: e.g. -123 has signed digits -1, 2, and 3. >>> count_nums([]) == 0 >>> count_nums([-1, 11, -11]) == 1 >>> count_nums([1, 1, 2]) == 3 """
- Source ID
- HumanEval/108
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-109Python · def move_one_ball(arr): """We have an array 'arr' of N integers arr[1], arr[2], ..., arr[N].The numbers in the …
Original task
def move_one_ball(arr): """We have an array 'arr' of N integers arr[1], arr[2], ..., arr[N].The numbers in the array will be randomly ordered. Your task is to determine if it is possible to get an array sorted in non-decreasing order by performing the following operation on the given array: You are allowed to perform right shift operation any number of times. One right shift operation means shifting all elements of the array by one position in the right direction. The last element of the array will be moved to the starting position in the array i.e. 0th index. If it is possible to obtain the sorted array by performing the above operation then return True else return False. If the given array is empty then return True. Note: The given list is guaranteed to have unique elements. For Example: move_one_ball([3, 4, 5, 1, 2])==>True Explanation: By performin 2 right shift operations, non-decreasing order can be achieved for the given array. move_one_ball([3, 5, 4, 1, 2])==>False Explanation:It is not possible to get non-decreasing order for the given array by performing any number of right shift operations. """
- Source ID
- HumanEval/109
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-11Python · from typing import List def string_xor(a: str, b: str) -> str: """ Input are two strings a and b consisting only o…
Original task
from typing import List def string_xor(a: str, b: str) -> str: """ Input are two strings a and b consisting only of 1s and 0s. Perform binary XOR on these inputs and return result also as a string. >>> string_xor('010', '110') '100' """
- Source ID
- HumanEval/11
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-110Python · def exchange(lst1, lst2): """In this problem, you will implement a function that takes two lists of numbers, an…
Original task
def exchange(lst1, lst2): """In this problem, you will implement a function that takes two lists of numbers, and determines whether it is possible to perform an exchange of elements between them to make lst1 a list of only even numbers. There is no limit on the number of exchanged elements between lst1 and lst2. If it is possible to exchange elements between the lst1 and lst2 to make all the elements of lst1 to be even, return "YES". Otherwise, return "NO". For example: exchange([1, 2, 3, 4], [1, 2, 3, 4]) => "YES" exchange([1, 2, 3, 4], [1, 5, 3, 4]) => "NO" It is assumed that the input lists will be non-empty. """
- Source ID
- HumanEval/110
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-111Python · def histogram(test): """Given a string representing a space separated lowercase letters, return a dictionary of…
Original task
def histogram(test): """Given a string representing a space separated lowercase letters, return a dictionary of the letter with the most repetition and containing the corresponding count. If several letters have the same occurrence, return all of them. Example: histogram('a b c') == {'a': 1, 'b': 1, 'c': 1} histogram('a b b a') == {'a': 2, 'b': 2} histogram('a b c a b') == {'a': 2, 'b': 2} histogram('b b b b a') == {'b': 4} histogram('') == {} """
- Source ID
- HumanEval/111
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-112Python · def reverse_delete(s,c): """Task We are given two strings s and c, you have to deleted all the characters in s …
Original task
def reverse_delete(s,c): """Task We are given two strings s and c, you have to deleted all the characters in s that are equal to any character in c then check if the result string is palindrome. A string is called palindrome if it reads the same backward as forward. You should return a tuple containing the result string and True/False for the check. Example For s = "abcde", c = "ae", the result should be ('bcd',False) For s = "abcdef", c = "b" the result should be ('acdef',False) For s = "abcdedcba", c = "ab", the result should be ('cdedc',True) """
- Source ID
- HumanEval/112
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-113Python · def odd_count(lst): """Given a list of strings, where each string consists of only digits, return a list. Each …
Original task
def odd_count(lst): """Given a list of strings, where each string consists of only digits, return a list. Each element i of the output should be "the number of odd elements in the string i of the input." where all the i's should be replaced by the number of odd digits in the i'th string of the input. >>> odd_count(['1234567']) ["the number of odd elements 4n the str4ng 4 of the 4nput."] >>> odd_count(['3',"11111111"]) ["the number of odd elements 1n the str1ng 1 of the 1nput.", "the number of odd elements 8n the str8ng 8 of the 8nput."] """
- Source ID
- HumanEval/113
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-114Python · def minSubArraySum(nums): """ Given an array of integers nums, find the minimum sum of any non-empty sub-array …
Original task
def minSubArraySum(nums): """ Given an array of integers nums, find the minimum sum of any non-empty sub-array of nums. Example minSubArraySum([2, 3, 4, 1, 2, 4]) == 1 minSubArraySum([-1, -2, -3]) == -6 """
- Source ID
- HumanEval/114
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-115Python · def max_fill(grid, capacity): import math """ You are given a rectangular grid of wells. Each row represent…
Original task
def max_fill(grid, capacity): import math """ You are given a rectangular grid of wells. Each row represents a single well, and each 1 in a row represents a single unit of water. Each well has a corresponding bucket that can be used to extract water from it, and all buckets have the same capacity. Your task is to use the buckets to empty the wells. Output the number of times you need to lower the buckets. Example 1: Input: grid : [[0,0,1,0], [0,1,0,0], [1,1,1,1]] bucket_capacity : 1 Output: 6 Example 2: Input: grid : [[0,0,1,1], [0,0,0,0], [1,1,1,1], [0,1,1,1]] bucket_capacity : 2 Output: 5 Example 3: Input: grid : [[0,0,0], [0,0,0]] bucket_capacity : 5 Output: 0 Constraints: * all wells have the same length * 1 <= grid.length <= 10^2 * 1 <= grid[:,1].length <= 10^2 * grid[i][j] -> 0 | 1 * 1 <= capacity <= 10 """
- Source ID
- HumanEval/115
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-116Python · def sort_array(arr): """ In this Kata, you have to sort an array of non-negative integers according to numb…
Original task
def sort_array(arr): """ In this Kata, you have to sort an array of non-negative integers according to number of ones in their binary representation in ascending order. For similar number of ones, sort based on decimal value. It must be implemented like this: >>> sort_array([1, 5, 2, 3, 4]) == [1, 2, 3, 4, 5] >>> sort_array([-2, -3, -4, -5, -6]) == [-6, -5, -4, -3, -2] >>> sort_array([1, 0, 2, 3, 4]) [0, 1, 2, 3, 4] """
- Source ID
- HumanEval/116
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-117Python · def select_words(s, n): """Given a string s and a natural number n, you have been tasked to implement a functi…
Original task
def select_words(s, n): """Given a string s and a natural number n, you have been tasked to implement a function that returns a list of all words from string s that contain exactly n consonants, in order these words appear in the string s. If the string s is empty then the function should return an empty list. Note: you may assume the input string contains only letters and spaces. Examples: select_words("Mary had a little lamb", 4) ==> ["little"] select_words("Mary had a little lamb", 3) ==> ["Mary", "lamb"] select_words("simple white space", 2) ==> [] select_words("Hello world", 4) ==> ["world"] select_words("Uncle sam", 3) ==> ["Uncle"] """
- Source ID
- HumanEval/117
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-118Python · def get_closest_vowel(word): """You are given a word. Your task is to find the closest vowel that stands between …
Original task
def get_closest_vowel(word): """You are given a word. Your task is to find the closest vowel that stands between two consonants from the right side of the word (case sensitive). Vowels in the beginning and ending doesn't count. Return empty string if you didn't find any vowel met the above condition. You may assume that the given string contains English letter only. Example: get_closest_vowel("yogurt") ==> "u" get_closest_vowel("FULL") ==> "U" get_closest_vowel("quick") ==> "" get_closest_vowel("ab") ==> "" """
- Source ID
- HumanEval/118
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-119Python · def match_parens(lst): ''' You are given a list of two strings, both strings consist of open parentheses '(…
Original task
def match_parens(lst): ''' You are given a list of two strings, both strings consist of open parentheses '(' or close parentheses ')' only. Your job is to check if it is possible to concatenate the two strings in some order, that the resulting string will be good. A string S is considered to be good if and only if all parentheses in S are balanced. For example: the string '(())()' is good, while the string '())' is not. Return 'Yes' if there's a way to make a good string, and return 'No' otherwise. Examples: match_parens(['()(', ')']) == 'Yes' match_parens([')', ')']) == 'No' '''
- Source ID
- HumanEval/119
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-12Python · from typing import List, Optional def longest(strings: List[str]) -> Optional[str]: """ Out of list of strings, re…
Original task
from typing import List, Optional def longest(strings: List[str]) -> Optional[str]: """ Out of list of strings, return the longest one. Return the first one in case of multiple strings of the same length. Return None in case the input list is empty. >>> longest([]) >>> longest(['a', 'b', 'c']) 'a' >>> longest(['a', 'bb', 'ccc']) 'ccc' """
- Source ID
- HumanEval/12
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-120Python · def maximum(arr, k): """ Given an array arr of integers and a positive integer k, return a sorted list of …
Original task
def maximum(arr, k): """ Given an array arr of integers and a positive integer k, return a sorted list of length k with the maximum k numbers in arr. Example 1: Input: arr = [-3, -4, 5], k = 3 Output: [-4, -3, 5] Example 2: Input: arr = [4, -4, 4], k = 2 Output: [4, 4] Example 3: Input: arr = [-3, 2, 1, 2, -1, -2, 1], k = 1 Output: [2] Note: 1. The length of the array will be in the range of [1, 1000]. 2. The elements in the array will be in the range of [-1000, 1000]. 3. 0 <= k <= len(arr) """
- Source ID
- HumanEval/120
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-121Python · def solution(lst): """Given a non-empty list of integers, return the sum of all of the odd elements that are in eve…
Original task
def solution(lst): """Given a non-empty list of integers, return the sum of all of the odd elements that are in even positions. Examples solution([5, 8, 7, 1]) ==> 12 solution([3, 3, 3, 3, 3]) ==> 9 solution([30, 13, 24, 321]) ==>0 """
- Source ID
- HumanEval/121
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-122Python · def add_elements(arr, k): """ Given a non-empty array of integers arr and an integer k, return the sum of t…
Original task
def add_elements(arr, k): """ Given a non-empty array of integers arr and an integer k, return the sum of the elements with at most two digits from the first k elements of arr. Example: Input: arr = [111,21,3,4000,5,6,7,8,9], k = 4 Output: 24 # sum of 21 + 3 Constraints: 1. 1 <= len(arr) <= 100 2. 1 <= k <= len(arr) """
- Source ID
- HumanEval/122
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-123Python · def get_odd_collatz(n): """ Given a positive integer n, return a sorted list that has the odd numbers in collat…
Original task
def get_odd_collatz(n): """ Given a positive integer n, return a sorted list that has the odd numbers in collatz sequence. The Collatz conjecture is a conjecture in mathematics that concerns a sequence defined as follows: start with any positive integer n. Then each term is obtained from the previous term as follows: if the previous term is even, the next term is one half of the previous term. If the previous term is odd, the next term is 3 times the previous term plus 1. The conjecture is that no matter what value of n, the sequence will always reach 1. Note: 1. Collatz(1) is [1]. 2. returned list sorted in increasing order. For example: get_odd_collatz(5) returns [1, 5] # The collatz sequence for 5 is [5, 16, 8, 4, 2, 1], so the odd numbers are only 1, and 5. """
- Source ID
- HumanEval/123
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-124Python · def valid_date(date): """You have to write a function which validates a given date string and returns True if t…
Original task
def valid_date(date): """You have to write a function which validates a given date string and returns True if the date is valid otherwise False. The date is valid if all of the following rules are satisfied: 1. The date string is not empty. 2. The number of days is not less than 1 or higher than 31 days for months 1,3,5,7,8,10,12. And the number of days is not less than 1 or higher than 30 days for months 4,6,9,11. And, the number of days is not less than 1 or higher than 29 for the month 2. 3. The months should not be less than 1 or higher than 12. 4. The date should be in the format: mm-dd-yyyy for example: valid_date('03-11-2000') => True valid_date('15-01-2012') => False valid_date('04-0-2040') => False valid_date('06-04-2020') => True valid_date('06/04/2020') => False """
- Source ID
- HumanEval/124
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-125Python · def split_words(txt): ''' Given a string of words, return a list of words split on whitespace, if no whitespace…
Original task
def split_words(txt): ''' Given a string of words, return a list of words split on whitespace, if no whitespaces exists in the text you should split on commas ',' if no commas exists you should return the number of lower-case letters with odd order in the alphabet, ord('a') = 0, ord('b') = 1, ... ord('z') = 25 Examples split_words("Hello world!") ➞ ["Hello", "world!"] split_words("Hello,world!") ➞ ["Hello", "world!"] split_words("abcdef") == 3 '''
- Source ID
- HumanEval/125
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-126Python · def is_sorted(lst): ''' Given a list of numbers, return whether or not they are sorted in ascending order. …
Original task
def is_sorted(lst): ''' Given a list of numbers, return whether or not they are sorted in ascending order. If list has more than 1 duplicate of the same number, return False. Assume no negative numbers and only integers. Examples is_sorted([5]) ➞ True is_sorted([1, 2, 3, 4, 5]) ➞ True is_sorted([1, 3, 2, 4, 5]) ➞ False is_sorted([1, 2, 3, 4, 5, 6]) ➞ True is_sorted([1, 2, 3, 4, 5, 6, 7]) ➞ True is_sorted([1, 3, 2, 4, 5, 6, 7]) ➞ False is_sorted([1, 2, 2, 3, 3, 4]) ➞ True is_sorted([1, 2, 2, 2, 3, 4]) ➞ False '''
- Source ID
- HumanEval/126
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-127Python · def intersection(interval1, interval2): """You are given two intervals, where each interval is a pair of intege…
Original task
def intersection(interval1, interval2): """You are given two intervals, where each interval is a pair of integers. For example, interval = (start, end) = (1, 2). The given intervals are closed which means that the interval (start, end) includes both start and end. For each given interval, it is assumed that its start is less or equal its end. Your task is to determine whether the length of intersection of these two intervals is a prime number. Example, the intersection of the intervals (1, 3), (2, 4) is (2, 3) which its length is 1, which not a prime number. If the length of the intersection is a prime number, return "YES", otherwise, return "NO". If the two intervals don't intersect, return "NO". [input/output] samples: intersection((1, 2), (2, 3)) ==> "NO" intersection((-1, 1), (0, 4)) ==> "NO" intersection((-3, -1), (-5, 5)) ==> "YES" """
- Source ID
- HumanEval/127
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-128Python · def prod_signs(arr): """ You are given an array arr of integers and you need to return sum of magnitudes of…
Original task
def prod_signs(arr): """ You are given an array arr of integers and you need to return sum of magnitudes of integers multiplied by product of all signs of each number in the array, represented by 1, -1 or 0. Note: return None for empty arr. Example: >>> prod_signs([1, 2, 2, -4]) == -9 >>> prod_signs([0, 1]) == 0 >>> prod_signs([]) == None """
- Source ID
- HumanEval/128
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-129Python · def minPath(grid, k): """ Given a grid with N rows and N columns (N >= 2) and a positive integer k, each c…
Original task
def minPath(grid, k): """ Given a grid with N rows and N columns (N >= 2) and a positive integer k, each cell of the grid contains a value. Every integer in the range [1, N * N] inclusive appears exactly once on the cells of the grid. You have to find the minimum path of length k in the grid. You can start from any cell, and in each step you can move to any of the neighbor cells, in other words, you can go to cells which share an edge with you current cell. Please note that a path of length k means visiting exactly k cells (not necessarily distinct). You CANNOT go off the grid. A path A (of length k) is considered less than a path B (of length k) if after making the ordered lists of the values on the cells that A and B go through (let's call them lst_A and lst_B), lst_A is lexicographically less than lst_B, in other words, there exist an integer index i (1 <= i <= k) such that lst_A[i] < lst_B[i] and for any j (1 <= j < i) we have lst_A[j] = lst_B[j]. It is guaranteed that the answer is unique. Return an ordered list of the values on the cells that the minimum path go through. Examples: Input: grid = [ [1,2,3], [4,5,6], [7,8,9]], k = 3 Output: [1, 2, 1] Input: grid = [ [5,9,3], [4,1,6], [7,8,2]], k = 1 Output: [1] """
- Source ID
- HumanEval/129
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-13Python · def greatest_common_divisor(a: int, b: int) -> int: """ Return a greatest common divisor of two integers a and b …
Original task
def greatest_common_divisor(a: int, b: int) -> int: """ Return a greatest common divisor of two integers a and b >>> greatest_common_divisor(3, 5) 1 >>> greatest_common_divisor(25, 15) 5 """
- Source ID
- HumanEval/13
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-130Python · def tri(n): """Everyone knows Fibonacci sequence, it was studied deeply by mathematicians in the last couple c…
Original task
def tri(n): """Everyone knows Fibonacci sequence, it was studied deeply by mathematicians in the last couple centuries. However, what people don't know is Tribonacci sequence. Tribonacci sequence is defined by the recurrence: tri(1) = 3 tri(n) = 1 + n / 2, if n is even. tri(n) = tri(n - 1) + tri(n - 2) + tri(n + 1), if n is odd. For example: tri(2) = 1 + (2 / 2) = 2 tri(4) = 3 tri(3) = tri(2) + tri(1) + tri(4) = 2 + 3 + 3 = 8 You are given a non-negative integer number n, you have to a return a list of the first n + 1 numbers of the Tribonacci sequence. Examples: tri(3) = [1, 3, 2, 8] """
- Source ID
- HumanEval/130
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-131Python · def digits(n): """Given a positive integer n, return the product of the odd digits. Return 0 if all digits are …
Original task
def digits(n): """Given a positive integer n, return the product of the odd digits. Return 0 if all digits are even. For example: digits(1) == 1 digits(4) == 0 digits(235) == 15 """
- Source ID
- HumanEval/131
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-132Python · def is_nested(string): ''' Create a function that takes a string as input which contains only square brackets. …
Original task
def is_nested(string): ''' Create a function that takes a string as input which contains only square brackets. The function should return True if and only if there is a valid subsequence of brackets where at least one bracket in the subsequence is nested. is_nested('[[]]') ➞ True is_nested('[]]]]]]][[[[[]') ➞ False is_nested('[][]') ➞ False is_nested('[]') ➞ False is_nested('[[][]]') ➞ True is_nested('[[]][[') ➞ True '''
- Source ID
- HumanEval/132
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-133Python · def sum_squares(lst): """You are given a list of numbers. You need to return the sum of squared numbers in the…
Original task
def sum_squares(lst): """You are given a list of numbers. You need to return the sum of squared numbers in the given list, round each element in the list to the upper int(Ceiling) first. Examples: For lst = [1,2,3] the output should be 14 For lst = [1,4,9] the output should be 98 For lst = [1,3,5,7] the output should be 84 For lst = [1.4,4.2,0] the output should be 29 For lst = [-2.4,1,1] the output should be 6 """
- Source ID
- HumanEval/133
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-134Python · def check_if_last_char_is_a_letter(txt): ''' Create a function that returns True if the last character of a…
Original task
def check_if_last_char_is_a_letter(txt): ''' Create a function that returns True if the last character of a given string is an alphabetical character and is not a part of a word, and False otherwise. Note: "word" is a group of characters separated by space. Examples: check_if_last_char_is_a_letter("apple pie") ➞ False check_if_last_char_is_a_letter("apple pi e") ➞ True check_if_last_char_is_a_letter("apple pi e ") ➞ False check_if_last_char_is_a_letter("") ➞ False '''
- Source ID
- HumanEval/134
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-135Python · def can_arrange(arr): """Create a function which returns the largest index of an element which is not greater t…
Original task
def can_arrange(arr): """Create a function which returns the largest index of an element which is not greater than or equal to the element immediately preceding it. If no such element exists then return -1. The given array will not contain duplicate values. Examples: can_arrange([1,2,4,3,5]) = 3 can_arrange([1,2,3]) = -1 """
- Source ID
- HumanEval/135
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-136Python · def largest_smallest_integers(lst): ''' Create a function that returns a tuple (a, b), where 'a' is the lar…
Original task
def largest_smallest_integers(lst): ''' Create a function that returns a tuple (a, b), where 'a' is the largest of negative integers, and 'b' is the smallest of positive integers in a list. If there is no negative or positive integers, return them as None. Examples: largest_smallest_integers([2, 4, 1, 3, 5, 7]) == (None, 1) largest_smallest_integers([]) == (None, None) largest_smallest_integers([0]) == (None, None) '''
- Source ID
- HumanEval/136
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-137Python · def compare_one(a, b): """ Create a function that takes integers, floats, or strings representing real numb…
Original task
def compare_one(a, b): """ Create a function that takes integers, floats, or strings representing real numbers, and returns the larger variable in its given variable type. Return None if the values are equal. Note: If a real number is represented as a string, the floating point might be . or , compare_one(1, 2.5) ➞ 2.5 compare_one(1, "2,3") ➞ "2,3" compare_one("5,1", "6") ➞ "6" compare_one("1", 1) ➞ None """
- Source ID
- HumanEval/137
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-138Python · def is_equal_to_sum_even(n): """Evaluate whether the given number n can be written as the sum of exactly 4 positive…
Original task
def is_equal_to_sum_even(n): """Evaluate whether the given number n can be written as the sum of exactly 4 positive even numbers Example is_equal_to_sum_even(4) == False is_equal_to_sum_even(6) == False is_equal_to_sum_even(8) == True """
- Source ID
- HumanEval/138
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-139Python · def special_factorial(n): """The Brazilian factorial is defined as: brazilian_factorial(n) = n! * (n-1)! * (n-2…
Original task
def special_factorial(n): """The Brazilian factorial is defined as: brazilian_factorial(n) = n! * (n-1)! * (n-2)! * ... * 1! where n > 0 For example: >>> special_factorial(4) 288 The function will receive an integer as input and should return the special factorial of this integer. """
- Source ID
- HumanEval/139
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-14Python · from typing import List def all_prefixes(string: str) -> List[str]: """ Return list of all prefixes from shortest …
Original task
from typing import List def all_prefixes(string: str) -> List[str]: """ Return list of all prefixes from shortest to longest of the input string >>> all_prefixes('abc') ['a', 'ab', 'abc'] """
- Source ID
- HumanEval/14
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-140Python · def fix_spaces(text): """ Given a string text, replace all spaces in it with underscores, and if a string …
Original task
def fix_spaces(text): """ Given a string text, replace all spaces in it with underscores, and if a string has more than 2 consecutive spaces, then replace all consecutive spaces with - fix_spaces("Example") == "Example" fix_spaces("Example 1") == "Example_1" fix_spaces(" Example 2") == "_Example_2" fix_spaces(" Example 3") == "_Example-3" """
- Source ID
- HumanEval/140
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-141Python · def file_name_check(file_name): """Create a function which takes a string representing a file's name, and returns …
Original task
def file_name_check(file_name): """Create a function which takes a string representing a file's name, and returns 'Yes' if the the file's name is valid, and returns 'No' otherwise. A file's name is considered to be valid if and only if all the following conditions are met: - There should not be more than three digits ('0'-'9') in the file's name. - The file's name contains exactly one dot '.' - The substring before the dot should not be empty, and it starts with a letter from the latin alphapet ('a'-'z' and 'A'-'Z'). - The substring after the dot should be one of these: ['txt', 'exe', 'dll'] Examples: file_name_check("example.txt") # => 'Yes' file_name_check("1example.dll") # => 'No' (the name should start with a latin alphapet letter) """
- Source ID
- HumanEval/141
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-142Python · def sum_squares(lst): """" This function will take a list of integers. For all entries in the list, the funct…
Original task
def sum_squares(lst): """" This function will take a list of integers. For all entries in the list, the function shall square the integer entry if its index is a multiple of 3 and will cube the integer entry if its index is a multiple of 4 and not a multiple of 3. The function will not change the entries in the list whose indexes are not a multiple of 3 or 4. The function shall then return the sum of all entries. Examples: For lst = [1,2,3] the output should be 6 For lst = [] the output should be 0 For lst = [-1,-5,2,-1,-5] the output should be -126 """
- Source ID
- HumanEval/142
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-143Python · def words_in_sentence(sentence): """ You are given a string representing a sentence, the sentence contains …
Original task
def words_in_sentence(sentence): """ You are given a string representing a sentence, the sentence contains some words separated by a space, and you have to return a string that contains the words from the original sentence, whose lengths are prime numbers, the order of the words in the new string should be the same as the original one. Example 1: Input: sentence = "This is a test" Output: "is" Example 2: Input: sentence = "lets go for swimming" Output: "go for" Constraints: * 1 <= len(sentence) <= 100 * sentence contains only letters """
- Source ID
- HumanEval/143
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-144Python · def simplify(x, n): """Your task is to implement a function that will simplify the expression x * n. The functi…
Original task
def simplify(x, n): """Your task is to implement a function that will simplify the expression x * n. The function returns True if x * n evaluates to a whole number and False otherwise. Both x and n, are string representation of a fraction, and have the following format, <numerator>/<denominator> where both numerator and denominator are positive whole numbers. You can assume that x, and n are valid fractions, and do not have zero as denominator. simplify("1/5", "5/1") = True simplify("1/6", "2/1") = False simplify("7/10", "10/2") = False """
- Source ID
- HumanEval/144
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-145Python · def order_by_points(nums): """ Write a function which sorts the given list of integers in ascending order a…
Original task
def order_by_points(nums): """ Write a function which sorts the given list of integers in ascending order according to the sum of their digits. Note: if there are several items with similar sum of their digits, order them based on their index in original list. For example: >>> order_by_points([1, 11, -1, -11, -12]) == [-1, -11, 1, -12, 11] >>> order_by_points([]) == [] """
- Source ID
- HumanEval/145
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-146Python · def specialFilter(nums): """Write a function that takes an array of numbers as input and returns the number of…
Original task
def specialFilter(nums): """Write a function that takes an array of numbers as input and returns the number of elements in the array that are greater than 10 and both first and last digits of a number are odd (1, 3, 5, 7, 9). For example: specialFilter([15, -73, 14, -15]) => 1 specialFilter([33, -2, -3, 45, 21, 109]) => 2 """
- Source ID
- HumanEval/146
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-147Python · def get_max_triples(n): """ You are given a positive integer n. You have to create an integer array a of length…
Original task
def get_max_triples(n): """ You are given a positive integer n. You have to create an integer array a of length n. For each i (1 ≤ i ≤ n), the value of a[i] = i * i - i + 1. Return the number of triples (a[i], a[j], a[k]) of a where i < j < k, and a[i] + a[j] + a[k] is a multiple of 3. Example : Input: n = 5 Output: 1 Explanation: a = [1, 3, 7, 13, 21] The only valid triple is (1, 7, 13). """
- Source ID
- HumanEval/147
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-148Python · def bf(planet1, planet2): ''' There are eight planets in our solar system: the closerst to the Sun is Merc…
Original task
def bf(planet1, planet2): ''' There are eight planets in our solar system: the closerst to the Sun is Mercury, the next one is Venus, then Earth, Mars, Jupiter, Saturn, Uranus, Neptune. Write a function that takes two planet names as strings planet1 and planet2. The function should return a tuple containing all planets whose orbits are located between the orbit of planet1 and the orbit of planet2, sorted by the proximity to the sun. The function should return an empty tuple if planet1 or planet2 are not correct planet names. Examples bf("Jupiter", "Neptune") ==> ("Saturn", "Uranus") bf("Earth", "Mercury") ==> ("Venus") bf("Mercury", "Uranus") ==> ("Venus", "Earth", "Mars", "Jupiter", "Saturn") '''
- Source ID
- HumanEval/148
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-149Python · def sorted_list_sum(lst): """Write a function that accepts a list of strings as a parameter, deletes the string…
Original task
def sorted_list_sum(lst): """Write a function that accepts a list of strings as a parameter, deletes the strings that have odd lengths from it, and returns the resulted list with a sorted order, The list is always a list of strings and never an array of numbers, and it may contain duplicates. The order of the list should be ascending by length of each word, and you should return the list sorted by that rule. If two words have the same length, sort the list alphabetically. The function should return a list of strings in sorted order. You may assume that all words will have the same length. For example: assert list_sort(["aa", "a", "aaa"]) => ["aa"] assert list_sort(["ab", "a", "aaa", "cd"]) => ["ab", "cd"] """
- Source ID
- HumanEval/149
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-15Python · def string_sequence(n: int) -> str: """ Return a string containing space-delimited numbers starting from 0 upto n …
Original task
def string_sequence(n: int) -> str: """ Return a string containing space-delimited numbers starting from 0 upto n inclusive. >>> string_sequence(0) '0' >>> string_sequence(5) '0 1 2 3 4 5' """
- Source ID
- HumanEval/15
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-150Python · def x_or_y(n, x, y): """A simple program which should return the value of x if n is a prime number and should …
Original task
def x_or_y(n, x, y): """A simple program which should return the value of x if n is a prime number and should return the value of y otherwise. Examples: for x_or_y(7, 34, 12) == 34 for x_or_y(15, 8, 5) == 5 """
- Source ID
- HumanEval/150
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-151Python · def double_the_difference(lst): ''' Given a list of numbers, return the sum of squares of the numbers in th…
Original task
def double_the_difference(lst): ''' Given a list of numbers, return the sum of squares of the numbers in the list that are odd. Ignore numbers that are negative or not integers. double_the_difference([1, 3, 2, 0]) == 1 + 9 + 0 + 0 = 10 double_the_difference([-1, -2, 0]) == 0 double_the_difference([9, -2]) == 81 double_the_difference([0]) == 0 If the input list is empty, return 0. '''
- Source ID
- HumanEval/151
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-152Python · def compare(game,guess): """I think we all remember that feeling when the result of some long-awaited event is …
Original task
def compare(game,guess): """I think we all remember that feeling when the result of some long-awaited event is finally known. The feelings and thoughts you have at that moment are definitely worth noting down and comparing. Your task is to determine if a person correctly guessed the results of a number of matches. You are given two arrays of scores and guesses of equal length, where each index shows a match. Return an array of the same length denoting how far off each guess was. If they have guessed correctly, the value is 0, and if not, the value is the absolute difference between the guess and the score. example: compare([1,2,3,4,5,1],[1,2,3,4,2,-2]) -> [0,0,0,0,3,3] compare([0,5,0,0,0,4],[4,1,1,0,0,-2]) -> [4,4,1,0,0,6] """
- Source ID
- HumanEval/152
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-153Python · def Strongest_Extension(class_name, extensions): """You will be given the name of a class (a string) and a list of …
Original task
def Strongest_Extension(class_name, extensions): """You will be given the name of a class (a string) and a list of extensions. The extensions are to be used to load additional classes to the class. The strength of the extension is as follows: Let CAP be the number of the uppercase letters in the extension's name, and let SM be the number of lowercase letters in the extension's name, the strength is given by the fraction CAP - SM. You should find the strongest extension and return a string in this format: ClassName.StrongestExtensionName. If there are two or more extensions with the same strength, you should choose the one that comes first in the list. For example, if you are given "Slices" as the class and a list of the extensions: ['SErviNGSliCes', 'Cheese', 'StuFfed'] then you should return 'Slices.SErviNGSliCes' since 'SErviNGSliCes' is the strongest extension (its strength is -1). Example: for Strongest_Extension('my_class', ['AA', 'Be', 'CC']) == 'my_class.AA' """
- Source ID
- HumanEval/153
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-154Python · def cycpattern_check(a , b): """You are given 2 words. You need to return True if the second word or any of its rot…
Original task
def cycpattern_check(a , b): """You are given 2 words. You need to return True if the second word or any of its rotations is a substring in the first word cycpattern_check("abcd","abd") => False cycpattern_check("hello","ell") => True cycpattern_check("whassup","psus") => False cycpattern_check("abab","baa") => True cycpattern_check("efef","eeff") => False cycpattern_check("himenss","simen") => True """
- Source ID
- HumanEval/154
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-155Python · def even_odd_count(num): """Given an integer. return a tuple that has the number of even and odd digits respectivel…
Original task
def even_odd_count(num): """Given an integer. return a tuple that has the number of even and odd digits respectively. Example: even_odd_count(-12) ==> (1, 1) even_odd_count(123) ==> (1, 2) """
- Source ID
- HumanEval/155
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-156Python · def int_to_mini_roman(number): """ Given a positive integer, obtain its roman numeral equivalent as a string, …
Original task
def int_to_mini_roman(number): """ Given a positive integer, obtain its roman numeral equivalent as a string, and return it in lowercase. Restrictions: 1 <= num <= 1000 Examples: >>> int_to_mini_roman(19) == 'xix' >>> int_to_mini_roman(152) == 'clii' >>> int_to_mini_roman(426) == 'cdxxvi' """
- Source ID
- HumanEval/156
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-157Python · def right_angle_triangle(a, b, c): ''' Given the lengths of the three sides of a triangle. Return True if the t…
Original task
def right_angle_triangle(a, b, c): ''' Given the lengths of the three sides of a triangle. Return True if the three sides form a right-angled triangle, False otherwise. A right-angled triangle is a triangle in which one angle is right angle or 90 degree. Example: right_angle_triangle(3, 4, 5) == True right_angle_triangle(1, 2, 3) == False '''
- Source ID
- HumanEval/157
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-158Python · def find_max(words): """Write a function that accepts a list of strings. The list contains different words. Ret…
Original task
def find_max(words): """Write a function that accepts a list of strings. The list contains different words. Return the word with maximum number of unique characters. If multiple strings have maximum number of unique characters, return the one which comes first in lexicographical order. find_max(["name", "of", "string"]) == "string" find_max(["name", "enam", "game"]) == "enam" find_max(["aaaaaaa", "bb" ,"cc"]) == ""aaaaaaa" """
- Source ID
- HumanEval/158
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-159Python · def eat(number, need, remaining): """ You're a hungry rabbit, and you already have eaten a certain number of ca…
Original task
def eat(number, need, remaining): """ You're a hungry rabbit, and you already have eaten a certain number of carrots, but now you need to eat more carrots to complete the day's meals. you should return an array of [ total number of eaten carrots after your meals, the number of carrots left after your meals ] if there are not enough remaining carrots, you will eat all remaining carrots, but will still be hungry. Example: * eat(5, 6, 10) -> [11, 4] * eat(4, 8, 9) -> [12, 1] * eat(1, 10, 10) -> [11, 0] * eat(2, 11, 5) -> [7, 0] Variables: @number : integer the number of carrots that you have eaten. @need : integer the number of carrots that you need to eat. @remaining : integer the number of remaining carrots thet exist in stock Constrain: * 0 <= number <= 1000 * 0 <= need <= 1000 * 0 <= remaining <= 1000 Have fun :) """
- Source ID
- HumanEval/159
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-16Python · def count_distinct_characters(string: str) -> int: """ Given a string, find out how many distinct characters (rega…
Original task
def count_distinct_characters(string: str) -> int: """ Given a string, find out how many distinct characters (regardless of case) does it consist of >>> count_distinct_characters('xyzXYZ') 3 >>> count_distinct_characters('Jerry') 4 """
- Source ID
- HumanEval/16
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-160Python · def do_algebra(operator, operand): """ Given two lists operator, and operand. The first list has basic algebra …
Original task
def do_algebra(operator, operand): """ Given two lists operator, and operand. The first list has basic algebra operations, and the second list is a list of integers. Use the two given lists to build the algebric expression and return the evaluation of this expression. The basic algebra operations: Addition ( + ) Subtraction ( - ) Multiplication ( * ) Floor division ( // ) Exponentiation ( ** ) Example: operator['+', '*', '-'] array = [2, 3, 4, 5] result = 2 + 3 * 4 - 5 => result = 9 Note: The length of operator list is equal to the length of operand list minus one. Operand is a list of of non-negative integers. Operator list has at least one operator, and operand list has at least two operands. """
- Source ID
- HumanEval/160
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-161Python · def solve(s): """You are given a string s. if s[i] is a letter, reverse its case from lower to upper or vise ve…
Original task
def solve(s): """You are given a string s. if s[i] is a letter, reverse its case from lower to upper or vise versa, otherwise keep it as it is. If the string contains no letters, reverse the string. The function should return the resulted string. Examples solve("1234") = "4321" solve("ab") = "AB" solve("#a@C") = "#A@c" """
- Source ID
- HumanEval/161
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-162Python · def string_to_md5(text): """ Given a string 'text', return its md5 hash equivalent string. If 'text' is an …
Original task
def string_to_md5(text): """ Given a string 'text', return its md5 hash equivalent string. If 'text' is an empty string, return None. >>> string_to_md5('Hello world') == '3e25960a79dbc69b674cd4ec67a72c62' """
- Source ID
- HumanEval/162
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-163Python · def generate_integers(a, b): """ Given two positive integers a and b, return the even digits between a and …
Original task
def generate_integers(a, b): """ Given two positive integers a and b, return the even digits between a and b, in ascending order. For example: generate_integers(2, 8) => [2, 4, 6, 8] generate_integers(8, 2) => [2, 4, 6, 8] generate_integers(10, 14) => [] """
- Source ID
- HumanEval/163
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-17Python · from typing import List def parse_music(music_string: str) -> List[int]: """ Input to this function is a string re…
Original task
from typing import List def parse_music(music_string: str) -> List[int]: """ Input to this function is a string representing musical notes in a special ASCII format. Your task is to parse this string and return list of integers corresponding to how many beats does each not last. Here is a legend: 'o' - whole note, lasts four beats 'o|' - half note, lasts two beats '.|' - quater note, lasts one beat >>> parse_music('o o| .| o| o| .| .| .| .| o o') [4, 2, 1, 2, 2, 1, 1, 1, 1, 4, 4] """
- Source ID
- HumanEval/17
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-18Python · def how_many_times(string: str, substring: str) -> int: """ Find how many times a given substring can be found in …
Original task
def how_many_times(string: str, substring: str) -> int: """ Find how many times a given substring can be found in the original string. Count overlaping cases. >>> how_many_times('', 'a') 0 >>> how_many_times('aaa', 'a') 3 >>> how_many_times('aaaa', 'aa') 3 """
- Source ID
- HumanEval/18
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-19Python · from typing import List def sort_numbers(numbers: str) -> str: """ Input is a space-delimited string of numberals …
Original task
from typing import List def sort_numbers(numbers: str) -> str: """ Input is a space-delimited string of numberals from 'zero' to 'nine'. Valid choices are 'zero', 'one', 'two', 'three', 'four', 'five', 'six', 'seven', 'eight' and 'nine'. Return the string with numbers sorted from smallest to largest >>> sort_numbers('three one five') 'one three five' """
- Source ID
- HumanEval/19
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-2Python · def truncate_number(number: float) -> float: """ Given a positive floating point number, it can be decomposed into…
Original task
def truncate_number(number: float) -> float: """ Given a positive floating point number, it can be decomposed into and integer part (largest integer smaller than given number) and decimals (leftover part always smaller than 1). Return the decimal part of the number. >>> truncate_number(3.5) 0.5 """
- Source ID
- HumanEval/2
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-20Python · from typing import List, Tuple def find_closest_elements(numbers: List[float]) -> Tuple[float, float]: """ From a …
Original task
from typing import List, Tuple def find_closest_elements(numbers: List[float]) -> Tuple[float, float]: """ From a supplied list of numbers (of length at least two) select and return two that are the closest to each other and return them in order (smaller number, larger number). >>> find_closest_elements([1.0, 2.0, 3.0, 4.0, 5.0, 2.2]) (2.0, 2.2) >>> find_closest_elements([1.0, 2.0, 3.0, 4.0, 5.0, 2.0]) (2.0, 2.0) """
- Source ID
- HumanEval/20
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-21Python · from typing import List def rescale_to_unit(numbers: List[float]) -> List[float]: """ Given list of numbers (of at…
Original task
from typing import List def rescale_to_unit(numbers: List[float]) -> List[float]: """ Given list of numbers (of at least two elements), apply a linear transform to that list, such that the smallest number will become 0 and the largest will become 1 >>> rescale_to_unit([1.0, 2.0, 3.0, 4.0, 5.0]) [0.0, 0.25, 0.5, 0.75, 1.0] """
- Source ID
- HumanEval/21
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-22Python · from typing import List, Any def filter_integers(values: List[Any]) -> List[int]: """ Filter given list of any pyt…
Original task
from typing import List, Any def filter_integers(values: List[Any]) -> List[int]: """ Filter given list of any python values only for integers >>> filter_integers(['a', 3.14, 5]) [5] >>> filter_integers([1, 2, 3, 'abc', {}, []]) [1, 2, 3] """
- Source ID
- HumanEval/22
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-23Python · def strlen(string: str) -> int: """ Return length of given string >>> strlen('') 0 >>> strlen('abc') …
Original task
def strlen(string: str) -> int: """ Return length of given string >>> strlen('') 0 >>> strlen('abc') 3 """
- Source ID
- HumanEval/23
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-24Python · def largest_divisor(n: int) -> int: """ For a given number n, find the largest number that divides n evenly, small…
Original task
def largest_divisor(n: int) -> int: """ For a given number n, find the largest number that divides n evenly, smaller than n >>> largest_divisor(15) 5 """
- Source ID
- HumanEval/24
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-25Python · from typing import List def factorize(n: int) -> List[int]: """ Return list of prime factors of given integer in t…
Original task
from typing import List def factorize(n: int) -> List[int]: """ Return list of prime factors of given integer in the order from smallest to largest. Each of the factors should be listed number of times corresponding to how many times it appeares in factorization. Input number should be equal to the product of all factors >>> factorize(8) [2, 2, 2] >>> factorize(25) [5, 5] >>> factorize(70) [2, 5, 7] """
- Source ID
- HumanEval/25
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-26Python · from typing import List def remove_duplicates(numbers: List[int]) -> List[int]: """ From a list of integers, remov…
Original task
from typing import List def remove_duplicates(numbers: List[int]) -> List[int]: """ From a list of integers, remove all elements that occur more than once. Keep order of elements left the same as in the input. >>> remove_duplicates([1, 2, 3, 2, 4]) [1, 3, 4] """
- Source ID
- HumanEval/26
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-27Python · def flip_case(string: str) -> str: """ For a given string, flip lowercase characters to uppercase and uppercase to…
Original task
def flip_case(string: str) -> str: """ For a given string, flip lowercase characters to uppercase and uppercase to lowercase. >>> flip_case('Hello') 'hELLO' """
- Source ID
- HumanEval/27
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-28Python · from typing import List def concatenate(strings: List[str]) -> str: """ Concatenate list of strings into a single …
Original task
from typing import List def concatenate(strings: List[str]) -> str: """ Concatenate list of strings into a single string >>> concatenate([]) '' >>> concatenate(['a', 'b', 'c']) 'abc' """
- Source ID
- HumanEval/28
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-29Python · from typing import List def filter_by_prefix(strings: List[str], prefix: str) -> List[str]: """ Filter an input li…
Original task
from typing import List def filter_by_prefix(strings: List[str], prefix: str) -> List[str]: """ Filter an input list of strings only for ones that start with a given prefix. >>> filter_by_prefix([], 'a') [] >>> filter_by_prefix(['abc', 'bcd', 'cde', 'array'], 'a') ['abc', 'array'] """
- Source ID
- HumanEval/29
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-3Python · from typing import List def below_zero(operations: List[int]) -> bool: """ You're given a list of deposit and with…
Original task
from typing import List def below_zero(operations: List[int]) -> bool: """ You're given a list of deposit and withdrawal operations on a bank account that starts with zero balance. Your task is to detect if at any point the balance of account fallls below zero, and at that point function should return True. Otherwise it should return False. >>> below_zero([1, 2, 3]) False >>> below_zero([1, 2, -4, 5]) True """
- Source ID
- HumanEval/3
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-30Python · def get_positive(l: list): """Return only positive numbers in the list. >>> get_positive([-1, 2, -4, 5, 6]) …
Original task
def get_positive(l: list): """Return only positive numbers in the list. >>> get_positive([-1, 2, -4, 5, 6]) [2, 5, 6] >>> get_positive([5, 3, -5, 2, -3, 3, 9, 0, 123, 1, -10]) [5, 3, 2, 3, 9, 123, 1] """
- Source ID
- HumanEval/30
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-31Python · def is_prime(n): """Return true if a given number is prime, and false otherwise. >>> is_prime(6) False …
Original task
def is_prime(n): """Return true if a given number is prime, and false otherwise. >>> is_prime(6) False >>> is_prime(101) True >>> is_prime(11) True >>> is_prime(13441) True >>> is_prime(61) True >>> is_prime(4) False >>> is_prime(1) False """
- Source ID
- HumanEval/31
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-32Python · import math def poly(xs: list, x: float): """ Evaluates polynomial with coefficients xs at point x. return…
Original task
import math def poly(xs: list, x: float): """ Evaluates polynomial with coefficients xs at point x. return xs[0] + xs[1] * x + xs[1] * x^2 + .... xs[n] * x^n """ return sum([coeff * math.pow(x, i) for i, coeff in enumerate(xs)]) def find_zero(xs: list): """ xs are coefficients of a polynomial. find_zero find x such that poly(x) = 0. find_zero returns only only zero point, even if there are many. Moreover, find_zero only takes list xs having even number of coefficients and largest non zero coefficient as it guarantees a solution. >>> round(find_zero([1, 2]), 2) # f(x) = 1 + 2x -0.5 >>> round(find_zero([-6, 11, -6, 1]), 2) # (x - 1) * (x - 2) * (x - 3) = -6 + 11x - 6x^2 + x^3 1.0 """
- Source ID
- HumanEval/32
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-33Python · def sort_third(l: list): """This function takes a list l and returns a list l' such that l' is identical to l …
Original task
def sort_third(l: list): """This function takes a list l and returns a list l' such that l' is identical to l in the indicies that are not divisible by three, while its values at the indicies that are divisible by three are equal to the values of the corresponding indicies of l, but sorted. >>> sort_third([1, 2, 3]) [1, 2, 3] >>> sort_third([5, 6, 3, 4, 8, 9, 2]) [2, 6, 3, 4, 8, 9, 5] """
- Source ID
- HumanEval/33
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-34Python · def unique(l: list): """Return sorted unique elements in a list >>> unique([5, 3, 5, 2, 3, 3, 9, 0, 123]) …
Original task
def unique(l: list): """Return sorted unique elements in a list >>> unique([5, 3, 5, 2, 3, 3, 9, 0, 123]) [0, 2, 3, 5, 9, 123] """
- Source ID
- HumanEval/34
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-35Python · def max_element(l: list): """Return maximum element in the list. >>> max_element([1, 2, 3]) 3 >>> max_…
Original task
def max_element(l: list): """Return maximum element in the list. >>> max_element([1, 2, 3]) 3 >>> max_element([5, 3, -5, 2, -3, 3, 9, 0, 123, 1, -10]) 123 """
- Source ID
- HumanEval/35
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-36Python · def fizz_buzz(n: int): """Return the number of times the digit 7 appears in integers less than n which are divisib…
Original task
def fizz_buzz(n: int): """Return the number of times the digit 7 appears in integers less than n which are divisible by 11 or 13. >>> fizz_buzz(50) 0 >>> fizz_buzz(78) 2 >>> fizz_buzz(79) 3 """
- Source ID
- HumanEval/36
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-37Python · def sort_even(l: list): """This function takes a list l and returns a list l' such that l' is identical to l i…
Original task
def sort_even(l: list): """This function takes a list l and returns a list l' such that l' is identical to l in the odd indicies, while its values at the even indicies are equal to the values of the even indicies of l, but sorted. >>> sort_even([1, 2, 3]) [1, 2, 3] >>> sort_even([5, 6, 3, 4]) [3, 6, 5, 4] """
- Source ID
- HumanEval/37
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-38Python · def encode_cyclic(s: str): """ returns encoded string by cycling groups of three characters. """ # spl…
Original task
def encode_cyclic(s: str): """ returns encoded string by cycling groups of three characters. """ # split string to groups. Each of length 3. groups = [s[(3 * i):min((3 * i + 3), len(s))] for i in range((len(s) + 2) // 3)] # cycle elements in each group. Unless group has fewer elements than 3. groups = [(group[1:] + group[0]) if len(group) == 3 else group for group in groups] return "".join(groups) def decode_cyclic(s: str): """ takes as input string encoded with encode_cyclic function. Returns decoded string. """
- Source ID
- HumanEval/38
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-39Python · def prime_fib(n: int): """ prime_fib returns n-th number that is a Fibonacci number and it's also prime. >…
Original task
def prime_fib(n: int): """ prime_fib returns n-th number that is a Fibonacci number and it's also prime. >>> prime_fib(1) 2 >>> prime_fib(2) 3 >>> prime_fib(3) 5 >>> prime_fib(4) 13 >>> prime_fib(5) 89 """
- Source ID
- HumanEval/39
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-4Python · from typing import List def mean_absolute_deviation(numbers: List[float]) -> float: """ For a given list of input …
Original task
from typing import List def mean_absolute_deviation(numbers: List[float]) -> float: """ For a given list of input numbers, calculate Mean Absolute Deviation around the mean of this dataset. Mean Absolute Deviation is the average absolute difference between each element and a centerpoint (mean in this case): MAD = average | x - x_mean | >>> mean_absolute_deviation([1.0, 2.0, 3.0, 4.0]) 1.0 """
- Source ID
- HumanEval/4
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-40Python · def triples_sum_to_zero(l: list): """ triples_sum_to_zero takes a list of integers as an input. it returns…
Original task
def triples_sum_to_zero(l: list): """ triples_sum_to_zero takes a list of integers as an input. it returns True if there are three distinct elements in the list that sum to zero, and False otherwise. >>> triples_sum_to_zero([1, 3, 5, 0]) False >>> triples_sum_to_zero([1, 3, -2, 1]) True >>> triples_sum_to_zero([1, 2, 3, 7]) False >>> triples_sum_to_zero([2, 4, -5, 3, 9, 7]) True >>> triples_sum_to_zero([1]) False """
- Source ID
- HumanEval/40
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-41Python · def car_race_collision(n: int): """ Imagine a road that's a perfectly straight infinitely long line. n car…
Original task
def car_race_collision(n: int): """ Imagine a road that's a perfectly straight infinitely long line. n cars are driving left to right; simultaneously, a different set of n cars are driving right to left. The two sets of cars start out being very far from each other. All cars move in the same speed. Two cars are said to collide when a car that's moving left to right hits a car that's moving right to left. However, the cars are infinitely sturdy and strong; as a result, they continue moving in their trajectory as if they did not collide. This function outputs the number of such collisions. """
- Source ID
- HumanEval/41
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-42Python · def incr_list(l: list): """Return list with elements incremented by 1. >>> incr_list([1, 2, 3]) [2, 3, 4] …
Original task
def incr_list(l: list): """Return list with elements incremented by 1. >>> incr_list([1, 2, 3]) [2, 3, 4] >>> incr_list([5, 3, 5, 2, 3, 3, 9, 0, 123]) [6, 4, 6, 3, 4, 4, 10, 1, 124] """
- Source ID
- HumanEval/42
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-43Python · def pairs_sum_to_zero(l): """ pairs_sum_to_zero takes a list of integers as an input. it returns True if t…
Original task
def pairs_sum_to_zero(l): """ pairs_sum_to_zero takes a list of integers as an input. it returns True if there are two distinct elements in the list that sum to zero, and False otherwise. >>> pairs_sum_to_zero([1, 3, 5, 0]) False >>> pairs_sum_to_zero([1, 3, -2, 1]) False >>> pairs_sum_to_zero([1, 2, 3, 7]) False >>> pairs_sum_to_zero([2, 4, -5, 3, 5, 7]) True >>> pairs_sum_to_zero([1]) False """
- Source ID
- HumanEval/43
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-44Python · def change_base(x: int, base: int): """Change numerical base of input number x to base. return string represen…
Original task
def change_base(x: int, base: int): """Change numerical base of input number x to base. return string representation after the conversion. base numbers are less than 10. >>> change_base(8, 3) '22' >>> change_base(8, 2) '1000' >>> change_base(7, 2) '111' """
- Source ID
- HumanEval/44
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-45Python · def triangle_area(a, h): """Given length of a side and high return area for a triangle. >>> triangle_area(5, 3…
Original task
def triangle_area(a, h): """Given length of a side and high return area for a triangle. >>> triangle_area(5, 3) 7.5 """
- Source ID
- HumanEval/45
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-46Python · def fib4(n: int): """The Fib4 number sequence is a sequence similar to the Fibbonacci sequnece that's defined as f…
Original task
def fib4(n: int): """The Fib4 number sequence is a sequence similar to the Fibbonacci sequnece that's defined as follows: fib4(0) -> 0 fib4(1) -> 0 fib4(2) -> 2 fib4(3) -> 0 fib4(n) -> fib4(n-1) + fib4(n-2) + fib4(n-3) + fib4(n-4). Please write a function to efficiently compute the n-th element of the fib4 number sequence. Do not use recursion. >>> fib4(5) 4 >>> fib4(6) 8 >>> fib4(7) 14 """
- Source ID
- HumanEval/46
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-47Python · def median(l: list): """Return median of elements in the list l. >>> median([3, 1, 2, 4, 5]) 3 >>> med…
Original task
def median(l: list): """Return median of elements in the list l. >>> median([3, 1, 2, 4, 5]) 3 >>> median([-10, 4, 6, 1000, 10, 20]) 15.0 """
- Source ID
- HumanEval/47
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-48Python · def is_palindrome(text: str): """ Checks if given string is a palindrome >>> is_palindrome('') True …
Original task
def is_palindrome(text: str): """ Checks if given string is a palindrome >>> is_palindrome('') True >>> is_palindrome('aba') True >>> is_palindrome('aaaaa') True >>> is_palindrome('zbcd') False """
- Source ID
- HumanEval/48
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-49Python · def modp(n: int, p: int): """Return 2^n modulo p (be aware of numerics). >>> modp(3, 5) 3 >>> modp(110…
Original task
def modp(n: int, p: int): """Return 2^n modulo p (be aware of numerics). >>> modp(3, 5) 3 >>> modp(1101, 101) 2 >>> modp(0, 101) 1 >>> modp(3, 11) 8 >>> modp(100, 101) 1 """
- Source ID
- HumanEval/49
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-5Python · from typing import List def intersperse(numbers: List[int], delimeter: int) -> List[int]: """ Insert a number 'del…
Original task
from typing import List def intersperse(numbers: List[int], delimeter: int) -> List[int]: """ Insert a number 'delimeter' between every two consecutive elements of input list `numbers' >>> intersperse([], 4) [] >>> intersperse([1, 2, 3], 4) [1, 4, 2, 4, 3] """
- Source ID
- HumanEval/5
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-50Python · def encode_shift(s: str): """ returns encoded string by shifting every character by 5 in the alphabet. """…
Original task
def encode_shift(s: str): """ returns encoded string by shifting every character by 5 in the alphabet. """ return "".join([chr(((ord(ch) + 5 - ord("a")) % 26) + ord("a")) for ch in s]) def decode_shift(s: str): """ takes as input string encoded with encode_shift function. Returns decoded string. """
- Source ID
- HumanEval/50
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-51Python · def remove_vowels(text): """ remove_vowels is a function that takes string and returns string without vowels. …
Original task
def remove_vowels(text): """ remove_vowels is a function that takes string and returns string without vowels. >>> remove_vowels('') '' >>> remove_vowels("abcdef\nghijklm") 'bcdf\nghjklm' >>> remove_vowels('abcdef') 'bcdf' >>> remove_vowels('aaaaa') '' >>> remove_vowels('aaBAA') 'B' >>> remove_vowels('zbcd') 'zbcd' """
- Source ID
- HumanEval/51
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-52Python · def below_threshold(l: list, t: int): """Return True if all numbers in the list l are below threshold t. >>> b…
Original task
def below_threshold(l: list, t: int): """Return True if all numbers in the list l are below threshold t. >>> below_threshold([1, 2, 4, 10], 100) True >>> below_threshold([1, 20, 4, 10], 5) False """
- Source ID
- HumanEval/52
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-53Python · def add(x: int, y: int): """Add two numbers x and y >>> add(2, 3) 5 >>> add(5, 7) 12 """
Original task
def add(x: int, y: int): """Add two numbers x and y >>> add(2, 3) 5 >>> add(5, 7) 12 """
- Source ID
- HumanEval/53
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-54Python · def same_chars(s0: str, s1: str): """ Check if two words have the same characters. >>> same_chars('eabcdzz…
Original task
def same_chars(s0: str, s1: str): """ Check if two words have the same characters. >>> same_chars('eabcdzzzz', 'dddzzzzzzzddeddabc') True >>> same_chars('abcd', 'dddddddabc') True >>> same_chars('dddddddabc', 'abcd') True >>> same_chars('eabcd', 'dddddddabc') False >>> same_chars('abcd', 'dddddddabce') False >>> same_chars('eabcdzzzz', 'dddzzzzzzzddddabc') False """
- Source ID
- HumanEval/54
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-55Python · def fib(n: int): """Return n-th Fibonacci number. >>> fib(10) 55 >>> fib(1) 1 >>> fib(8) 2…
Original task
def fib(n: int): """Return n-th Fibonacci number. >>> fib(10) 55 >>> fib(1) 1 >>> fib(8) 21 """
- Source ID
- HumanEval/55
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-56Python · def correct_bracketing(brackets: str): """ brackets is a string of "<" and ">". return True if every opening b…
Original task
def correct_bracketing(brackets: str): """ brackets is a string of "<" and ">". return True if every opening bracket has a corresponding closing bracket. >>> correct_bracketing("<") False >>> correct_bracketing("<>") True >>> correct_bracketing("<<><>>") True >>> correct_bracketing("><<>") False """
- Source ID
- HumanEval/56
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-57Python · def monotonic(l: list): """Return True is list elements are monotonically increasing or decreasing. >>> monoto…
Original task
def monotonic(l: list): """Return True is list elements are monotonically increasing or decreasing. >>> monotonic([1, 2, 4, 20]) True >>> monotonic([1, 20, 4, 10]) False >>> monotonic([4, 1, 0, -10]) True """
- Source ID
- HumanEval/57
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-58Python · def common(l1: list, l2: list): """Return sorted unique common elements for two lists. >>> common([1, 4, 3, 34…
Original task
def common(l1: list, l2: list): """Return sorted unique common elements for two lists. >>> common([1, 4, 3, 34, 653, 2, 5], [5, 7, 1, 5, 9, 653, 121]) [1, 5, 653] >>> common([5, 3, 2, 8], [3, 2]) [2, 3] """
- Source ID
- HumanEval/58
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-59Python · def largest_prime_factor(n: int): """Return the largest prime factor of n. Assume n > 1 and is not a prime. >>…
Original task
def largest_prime_factor(n: int): """Return the largest prime factor of n. Assume n > 1 and is not a prime. >>> largest_prime_factor(13195) 29 >>> largest_prime_factor(2048) 2 """
- Source ID
- HumanEval/59
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-6Python · from typing import List def parse_nested_parens(paren_string: str) -> List[int]: """ Input to this function is a s…
Original task
from typing import List def parse_nested_parens(paren_string: str) -> List[int]: """ Input to this function is a string represented multiple groups for nested parentheses separated by spaces. For each of the group, output the deepest level of nesting of parentheses. E.g. (()()) has maximum two levels of nesting while ((())) has three. >>> parse_nested_parens('(()()) ((())) () ((())()())') [2, 3, 1, 3] """
- Source ID
- HumanEval/6
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-60Python · def sum_to_n(n: int): """sum_to_n is a function that sums numbers from 1 to n. >>> sum_to_n(30) 465 >>…
Original task
def sum_to_n(n: int): """sum_to_n is a function that sums numbers from 1 to n. >>> sum_to_n(30) 465 >>> sum_to_n(100) 5050 >>> sum_to_n(5) 15 >>> sum_to_n(10) 55 >>> sum_to_n(1) 1 """
- Source ID
- HumanEval/60
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-61Python · def correct_bracketing(brackets: str): """ brackets is a string of "(" and ")". return True if every opening b…
Original task
def correct_bracketing(brackets: str): """ brackets is a string of "(" and ")". return True if every opening bracket has a corresponding closing bracket. >>> correct_bracketing("(") False >>> correct_bracketing("()") True >>> correct_bracketing("(()())") True >>> correct_bracketing(")(()") False """
- Source ID
- HumanEval/61
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-62Python · def derivative(xs: list): """ xs represent coefficients of a polynomial. xs[0] + xs[1] * x + xs[2] * x^2 + ...…
Original task
def derivative(xs: list): """ xs represent coefficients of a polynomial. xs[0] + xs[1] * x + xs[2] * x^2 + .... Return derivative of this polynomial in the same form. >>> derivative([3, 1, 2, 4, 5]) [1, 4, 12, 20] >>> derivative([1, 2, 3]) [2, 6] """
- Source ID
- HumanEval/62
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-63Python · def fibfib(n: int): """The FibFib number sequence is a sequence similar to the Fibbonacci sequnece that's defined …
Original task
def fibfib(n: int): """The FibFib number sequence is a sequence similar to the Fibbonacci sequnece that's defined as follows: fibfib(0) == 0 fibfib(1) == 0 fibfib(2) == 1 fibfib(n) == fibfib(n-1) + fibfib(n-2) + fibfib(n-3). Please write a function to efficiently compute the n-th element of the fibfib number sequence. >>> fibfib(1) 0 >>> fibfib(5) 4 >>> fibfib(8) 24 """
- Source ID
- HumanEval/63
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-64Python · FIX = """ Add more test cases. """ def vowels_count(s): """Write a function vowels_count which takes a string repr…
Original task
FIX = """ Add more test cases. """ def vowels_count(s): """Write a function vowels_count which takes a string representing a word as input and returns the number of vowels in the string. Vowels in this case are 'a', 'e', 'i', 'o', 'u'. Here, 'y' is also a vowel, but only when it is at the end of the given word. Example: >>> vowels_count("abcde") 2 >>> vowels_count("ACEDY") 3 """
- Source ID
- HumanEval/64
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-65Python · def circular_shift(x, shift): """Circular shift the digits of the integer x, shift the digits right by shift an…
Original task
def circular_shift(x, shift): """Circular shift the digits of the integer x, shift the digits right by shift and return the result as a string. If shift > number of digits, return digits reversed. >>> circular_shift(12, 1) "21" >>> circular_shift(12, 2) "12" """
- Source ID
- HumanEval/65
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-66Python · def digitSum(s): """Task Write a function that takes a string as input and returns the sum of the upper charact…
Original task
def digitSum(s): """Task Write a function that takes a string as input and returns the sum of the upper characters only' ASCII codes. Examples: digitSum("") => 0 digitSum("abAB") => 131 digitSum("abcCd") => 67 digitSum("helloE") => 69 digitSum("woArBld") => 131 digitSum("aAaaaXa") => 153 """
- Source ID
- HumanEval/66
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-67Python · def fruit_distribution(s,n): """ In this task, you will be given a string that represents a number of apples an…
Original task
def fruit_distribution(s,n): """ In this task, you will be given a string that represents a number of apples and oranges that are distributed in a basket of fruit this basket contains apples, oranges, and mango fruits. Given the string that represents the total number of the oranges and apples and an integer that represent the total number of the fruits in the basket return the number of the mango fruits in the basket. for examble: fruit_distribution("5 apples and 6 oranges", 19) ->19 - 5 - 6 = 8 fruit_distribution("0 apples and 1 oranges",3) -> 3 - 0 - 1 = 2 fruit_distribution("2 apples and 3 oranges", 100) -> 100 - 2 - 3 = 95 fruit_distribution("100 apples and 1 oranges",120) -> 120 - 100 - 1 = 19 """
- Source ID
- HumanEval/67
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-68Python · def pluck(arr): """ "Given an array representing a branch of a tree that has non-negative integer nodes you…
Original task
def pluck(arr): """ "Given an array representing a branch of a tree that has non-negative integer nodes your task is to pluck one of the nodes and return it. The plucked node should be the node with the smallest even value. If multiple nodes with the same smallest even value are found return the node that has smallest index. The plucked node should be returned in a list, [ smalest_value, its index ], If there are no even values or the given array is empty, return []. Example 1: Input: [4,2,3] Output: [2, 1] Explanation: 2 has the smallest even value, and 2 has the smallest index. Example 2: Input: [1,2,3] Output: [2, 1] Explanation: 2 has the smallest even value, and 2 has the smallest index. Example 3: Input: [] Output: [] Example 4: Input: [5, 0, 3, 0, 4, 2] Output: [0, 1] Explanation: 0 is the smallest value, but there are two zeros, so we will choose the first zero, which has the smallest index. Constraints: * 1 <= nodes.length <= 10000 * 0 <= node.value """
- Source ID
- HumanEval/68
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-69Python · def search(lst): ''' You are given a non-empty list of positive integers. Return the greatest integer that is g…
Original task
def search(lst): ''' You are given a non-empty list of positive integers. Return the greatest integer that is greater than zero, and has a frequency greater than or equal to the value of the integer itself. The frequency of an integer is the number of times it appears in the list. If no such a value exist, return -1. Examples: search([4, 1, 2, 2, 3, 1]) == 2 search([1, 2, 2, 3, 3, 3, 4, 4, 4]) == 3 search([5, 5, 4, 4, 4]) == -1 '''
- Source ID
- HumanEval/69
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-7Python · from typing import List def filter_by_substring(strings: List[str], substring: str) -> List[str]: """ Filter an in…
Original task
from typing import List def filter_by_substring(strings: List[str], substring: str) -> List[str]: """ Filter an input list of strings only for ones that contain given substring >>> filter_by_substring([], 'a') [] >>> filter_by_substring(['abc', 'bacd', 'cde', 'array'], 'a') ['abc', 'bacd', 'array'] """
- Source ID
- HumanEval/7
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-70Python · def strange_sort_list(lst): ''' Given list of integers, return list in strange order. Strange sorting, is w…
Original task
def strange_sort_list(lst): ''' Given list of integers, return list in strange order. Strange sorting, is when you start with the minimum value, then maximum of the remaining integers, then minimum and so on. Examples: strange_sort_list([1, 2, 3, 4]) == [1, 4, 2, 3] strange_sort_list([5, 5, 5, 5]) == [5, 5, 5, 5] strange_sort_list([]) == [] '''
- Source ID
- HumanEval/70
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-71Python · def triangle_area(a, b, c): ''' Given the lengths of the three sides of a triangle. Return the area of the …
Original task
def triangle_area(a, b, c): ''' Given the lengths of the three sides of a triangle. Return the area of the triangle rounded to 2 decimal points if the three sides form a valid triangle. Otherwise return -1 Three sides make a valid triangle when the sum of any two sides is greater than the third side. Example: triangle_area(3, 4, 5) == 6.00 triangle_area(1, 2, 10) == -1 '''
- Source ID
- HumanEval/71
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-72Python · def will_it_fly(q,w): ''' Write a function that returns True if the object q will fly, and False otherwise. …
Original task
def will_it_fly(q,w): ''' Write a function that returns True if the object q will fly, and False otherwise. The object q will fly if it's balanced (it is a palindromic list) and the sum of its elements is less than or equal the maximum possible weight w. Example: will_it_fly([1, 2], 5) ➞ False # 1+2 is less than the maximum possible weight, but it's unbalanced. will_it_fly([3, 2, 3], 1) ➞ False # it's balanced, but 3+2+3 is more than the maximum possible weight. will_it_fly([3, 2, 3], 9) ➞ True # 3+2+3 is less than the maximum possible weight, and it's balanced. will_it_fly([3], 5) ➞ True # 3 is less than the maximum possible weight, and it's balanced. '''
- Source ID
- HumanEval/72
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-73Python · def smallest_change(arr): """ Given an array arr of integers, find the minimum number of elements that need…
Original task
def smallest_change(arr): """ Given an array arr of integers, find the minimum number of elements that need to be changed to make the array palindromic. A palindromic array is an array that is read the same backwards and forwards. In one change, you can change one element to any other element. For example: smallest_change([1,2,3,5,4,7,9,6]) == 4 smallest_change([1, 2, 3, 4, 3, 2, 2]) == 1 smallest_change([1, 2, 3, 2, 1]) == 0 """
- Source ID
- HumanEval/73
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-74Python · def total_match(lst1, lst2): ''' Write a function that accepts two lists of strings and returns the list that h…
Original task
def total_match(lst1, lst2): ''' Write a function that accepts two lists of strings and returns the list that has total number of chars in the all strings of the list less than the other list. if the two lists have the same number of chars, return the first list. Examples total_match([], []) ➞ [] total_match(['hi', 'admin'], ['hI', 'Hi']) ➞ ['hI', 'Hi'] total_match(['hi', 'admin'], ['hi', 'hi', 'admin', 'project']) ➞ ['hi', 'admin'] total_match(['hi', 'admin'], ['hI', 'hi', 'hi']) ➞ ['hI', 'hi', 'hi'] total_match(['4'], ['1', '2', '3', '4', '5']) ➞ ['4'] '''
- Source ID
- HumanEval/74
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-75Python · def is_multiply_prime(a): """Write a function that returns true if the given number is the multiplication of 3 prim…
Original task
def is_multiply_prime(a): """Write a function that returns true if the given number is the multiplication of 3 prime numbers and false otherwise. Knowing that (a) is less then 100. Example: is_multiply_prime(30) == True 30 = 2 * 3 * 5 """
- Source ID
- HumanEval/75
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-76Python · def is_simple_power(x, n): """Your task is to write a function that returns true if a number x is a simple powe…
Original task
def is_simple_power(x, n): """Your task is to write a function that returns true if a number x is a simple power of n and false in other cases. x is a simple power of n if n**int=x For example: is_simple_power(1, 4) => true is_simple_power(2, 2) => true is_simple_power(8, 2) => true is_simple_power(3, 2) => false is_simple_power(3, 1) => false is_simple_power(5, 3) => false """
- Source ID
- HumanEval/76
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-77Python · def iscube(a): ''' Write a function that takes an integer a and returns True if this ingeger is a cube of …
Original task
def iscube(a): ''' Write a function that takes an integer a and returns True if this ingeger is a cube of some integer number. Note: you may assume the input is always valid. Examples: iscube(1) ==> True iscube(2) ==> False iscube(-1) ==> True iscube(64) ==> True iscube(0) ==> True iscube(180) ==> False '''
- Source ID
- HumanEval/77
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-78Python · def hex_key(num): """You have been tasked to write a function that receives a hexadecimal number as a string a…
Original task
def hex_key(num): """You have been tasked to write a function that receives a hexadecimal number as a string and counts the number of hexadecimal digits that are primes (prime number, or a prime, is a natural number greater than 1 that is not a product of two smaller natural numbers). Hexadecimal digits are 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F. Prime numbers are 2, 3, 5, 7, 11, 13, 17,... So you have to determine a number of the following digits: 2, 3, 5, 7, B (=decimal 11), D (=decimal 13). Note: you may assume the input is always correct or empty string, and symbols A,B,C,D,E,F are always uppercase. Examples: For num = "AB" the output should be 1. For num = "1077E" the output should be 2. For num = "ABED1A33" the output should be 4. For num = "123456789ABCDEF0" the output should be 6. For num = "2020" the output should be 2. """
- Source ID
- HumanEval/78
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-79Python · def decimal_to_binary(decimal): """You will be given a number in decimal form and your task is to convert it to …
Original task
def decimal_to_binary(decimal): """You will be given a number in decimal form and your task is to convert it to binary format. The function should return a string, with each character representing a binary number. Each character in the string will be '0' or '1'. There will be an extra couple of characters 'db' at the beginning and at the end of the string. The extra characters are there to help with the format. Examples: decimal_to_binary(15) # returns "db1111db" decimal_to_binary(32) # returns "db100000db" """
- Source ID
- HumanEval/79
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-8Python · from typing import List, Tuple def sum_product(numbers: List[int]) -> Tuple[int, int]: """ For a given list of int…
Original task
from typing import List, Tuple def sum_product(numbers: List[int]) -> Tuple[int, int]: """ For a given list of integers, return a tuple consisting of a sum and a product of all the integers in a list. Empty sum should be equal to 0 and empty product should be equal to 1. >>> sum_product([]) (0, 1) >>> sum_product([1, 2, 3, 4]) (10, 24) """
- Source ID
- HumanEval/8
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-80Python · def is_happy(s): """You are given a string s. Your task is to check if the string is happy or not. A string…
Original task
def is_happy(s): """You are given a string s. Your task is to check if the string is happy or not. A string is happy if its length is at least 3 and every 3 consecutive letters are distinct For example: is_happy(a) => False is_happy(aa) => False is_happy(abcd) => True is_happy(aabb) => False is_happy(adb) => True is_happy(xyy) => False """
- Source ID
- HumanEval/80
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-81Python · def numerical_letter_grade(grades): """It is the last week of the semester and the teacher has to give the grades …
Original task
def numerical_letter_grade(grades): """It is the last week of the semester and the teacher has to give the grades to students. The teacher has been making her own algorithm for grading. The only problem is, she has lost the code she used for grading. She has given you a list of GPAs for some students and you have to write a function that can output a list of letter grades using the following table: GPA | Letter grade 4.0 A+ > 3.7 A > 3.3 A- > 3.0 B+ > 2.7 B > 2.3 B- > 2.0 C+ > 1.7 C > 1.3 C- > 1.0 D+ > 0.7 D > 0.0 D- 0.0 E Example: grade_equation([4.0, 3, 1.7, 2, 3.5]) ==> ['A+', 'B', 'C-', 'C', 'A-'] """
- Source ID
- HumanEval/81
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-82Python · def prime_length(string): """Write a function that takes a string and returns True if the string length is a pr…
Original task
def prime_length(string): """Write a function that takes a string and returns True if the string length is a prime number or False otherwise Examples prime_length('Hello') == True prime_length('abcdcba') == True prime_length('kittens') == True prime_length('orange') == False """
- Source ID
- HumanEval/82
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-83Python · def starts_one_ends(n): """ Given a positive integer n, return the count of the numbers of n-digit positive…
Original task
def starts_one_ends(n): """ Given a positive integer n, return the count of the numbers of n-digit positive integers that start or end with 1. """
- Source ID
- HumanEval/83
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-84Python · def solve(N): """Given a positive integer N, return the total sum of its digits in binary. Example …
Original task
def solve(N): """Given a positive integer N, return the total sum of its digits in binary. Example For N = 1000, the sum of digits will be 1 the output should be "1". For N = 150, the sum of digits will be 6 the output should be "110". For N = 147, the sum of digits will be 12 the output should be "1100". Variables: @N integer Constraints: 0 ≤ N ≤ 10000. Output: a string of binary number """
- Source ID
- HumanEval/84
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-85Python · def add(lst): """Given a non-empty list of integers lst. add the even elements that are at odd indices.. Exam…
Original task
def add(lst): """Given a non-empty list of integers lst. add the even elements that are at odd indices.. Examples: add([4, 2, 6, 7]) ==> 2 """
- Source ID
- HumanEval/85
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-86Python · def anti_shuffle(s): """ Write a function that takes a string and returns an ordered version of it. Ordered…
Original task
def anti_shuffle(s): """ Write a function that takes a string and returns an ordered version of it. Ordered version of string, is a string where all words (separated by space) are replaced by a new word where all the characters arranged in ascending order based on ascii value. Note: You should keep the order of words and blank spaces in the sentence. For example: anti_shuffle('Hi') returns 'Hi' anti_shuffle('hello') returns 'ehllo' anti_shuffle('Hello World!!!') returns 'Hello !!!Wdlor' """
- Source ID
- HumanEval/86
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-87Python · def get_row(lst, x): """ You are given a 2 dimensional data, as a nested lists, which is similar to matrix,…
Original task
def get_row(lst, x): """ You are given a 2 dimensional data, as a nested lists, which is similar to matrix, however, unlike matrices, each row may contain a different number of columns. Given lst, and integer x, find integers x in the list, and return list of tuples, [(x1, y1), (x2, y2) ...] such that each tuple is a coordinate - (row, columns), starting with 0. Sort coordinates initially by rows in ascending order. Also, sort coordinates of the row by columns in descending order. Examples: get_row([ [1,2,3,4,5,6], [1,2,3,4,1,6], [1,2,3,4,5,1] ], 1) == [(0, 0), (1, 4), (1, 0), (2, 5), (2, 0)] get_row([], 1) == [] get_row([[], [1], [1, 2, 3]], 3) == [(2, 2)] """
- Source ID
- HumanEval/87
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-88Python · def sort_array(array): """ Given an array of non-negative integers, return a copy of the given array after sort…
Original task
def sort_array(array): """ Given an array of non-negative integers, return a copy of the given array after sorting, you will sort the given array in ascending order if the sum( first index value, last index value) is odd, or sort it in descending order if the sum( first index value, last index value) is even. Note: * don't change the given array. Examples: * sort_array([]) => [] * sort_array([5]) => [5] * sort_array([2, 4, 3, 0, 1, 5]) => [0, 1, 2, 3, 4, 5] * sort_array([2, 4, 3, 0, 1, 5, 6]) => [6, 5, 4, 3, 2, 1, 0] """
- Source ID
- HumanEval/88
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-89Python · def encrypt(s): """Create a function encrypt that takes a string as an argument and returns a string encrypted …
Original task
def encrypt(s): """Create a function encrypt that takes a string as an argument and returns a string encrypted with the alphabet being rotated. The alphabet should be rotated in a manner such that the letters shift down by two multiplied to two places. For example: encrypt('hi') returns 'lm' encrypt('asdfghjkl') returns 'ewhjklnop' encrypt('gf') returns 'kj' encrypt('et') returns 'ix' """
- Source ID
- HumanEval/89
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-9Python · from typing import List, Tuple def rolling_max(numbers: List[int]) -> List[int]: """ From a given list of integers…
Original task
from typing import List, Tuple def rolling_max(numbers: List[int]) -> List[int]: """ From a given list of integers, generate a list of rolling maximum element found until given moment in the sequence. >>> rolling_max([1, 2, 3, 2, 3, 4, 2]) [1, 2, 3, 3, 3, 4, 4] """
- Source ID
- HumanEval/9
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-90Python · def next_smallest(lst): """ You are given a list of integers. Write a function next_smallest() that returns…
Original task
def next_smallest(lst): """ You are given a list of integers. Write a function next_smallest() that returns the 2nd smallest element of the list. Return None if there is no such element. next_smallest([1, 2, 3, 4, 5]) == 2 next_smallest([5, 1, 4, 3, 2]) == 2 next_smallest([]) == None next_smallest([1, 1]) == None """
- Source ID
- HumanEval/90
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-91Python · def is_bored(S): """ You'll be given a string of words, and your task is to count the number of boredoms. A…
Original task
def is_bored(S): """ You'll be given a string of words, and your task is to count the number of boredoms. A boredom is a sentence that starts with the word "I". Sentences are delimited by '.', '?' or '!'. For example: >>> is_bored("Hello world") 0 >>> is_bored("The sky is blue. The sun is shining. I love this weather") 1 """
- Source ID
- HumanEval/91
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-92Python · def any_int(x, y, z): ''' Create a function that takes 3 numbers. Returns true if one of the numbers is equ…
Original task
def any_int(x, y, z): ''' Create a function that takes 3 numbers. Returns true if one of the numbers is equal to the sum of the other two, and all numbers are integers. Returns false in any other cases. Examples any_int(5, 2, 7) ➞ True any_int(3, 2, 2) ➞ False any_int(3, -2, 1) ➞ True any_int(3.6, -2.2, 2) ➞ False '''
- Source ID
- HumanEval/92
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-93Python · def encode(message): """ Write a function that takes a message, and encodes in such a way that it swaps ca…
Original task
def encode(message): """ Write a function that takes a message, and encodes in such a way that it swaps case of all letters, replaces all vowels in the message with the letter that appears 2 places ahead of that vowel in the english alphabet. Assume only letters. Examples: >>> encode('test') 'TGST' >>> encode('This is a message') 'tHKS KS C MGSSCGG' """
- Source ID
- HumanEval/93
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-94Python · def skjkasdkd(lst): """You are given a list of integers. You need to find the largest prime value and return t…
Original task
def skjkasdkd(lst): """You are given a list of integers. You need to find the largest prime value and return the sum of its digits. Examples: For lst = [0,3,2,1,3,5,7,4,5,5,5,2,181,32,4,32,3,2,32,324,4,3] the output should be 10 For lst = [1,0,1,8,2,4597,2,1,3,40,1,2,1,2,4,2,5,1] the output should be 25 For lst = [1,3,1,32,5107,34,83278,109,163,23,2323,32,30,1,9,3] the output should be 13 For lst = [0,724,32,71,99,32,6,0,5,91,83,0,5,6] the output should be 11 For lst = [0,81,12,3,1,21] the output should be 3 For lst = [0,8,1,2,1,7] the output should be 7 """
- Source ID
- HumanEval/94
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-95Python · def check_dict_case(dict): """ Given a dictionary, return True if all keys are strings in lower case or al…
Original task
def check_dict_case(dict): """ Given a dictionary, return True if all keys are strings in lower case or all keys are strings in upper case, else return False. The function should return False is the given dictionary is empty. Examples: check_dict_case({"a":"apple", "b":"banana"}) should return True. check_dict_case({"a":"apple", "A":"banana", "B":"banana"}) should return False. check_dict_case({"a":"apple", 8:"banana", "a":"apple"}) should return False. check_dict_case({"Name":"John", "Age":"36", "City":"Houston"}) should return False. check_dict_case({"STATE":"NC", "ZIP":"12345" }) should return True. """
- Source ID
- HumanEval/95
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-96Python · def count_up_to(n): """Implement a function that takes an non-negative integer and returns an array of the first n …
Original task
def count_up_to(n): """Implement a function that takes an non-negative integer and returns an array of the first n integers that are prime numbers and less than n. for example: count_up_to(5) => [2,3] count_up_to(11) => [2,3,5,7] count_up_to(0) => [] count_up_to(20) => [2,3,5,7,11,13,17,19] count_up_to(1) => [] count_up_to(18) => [2,3,5,7,11,13,17] """
- Source ID
- HumanEval/96
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-97Python · def multiply(a, b): """Complete the function that takes two integers and returns the product of their unit dig…
Original task
def multiply(a, b): """Complete the function that takes two integers and returns the product of their unit digits. Assume the input is always valid. Examples: multiply(148, 412) should return 16. multiply(19, 28) should return 72. multiply(2020, 1851) should return 0. multiply(14,-15) should return 20. """
- Source ID
- HumanEval/97
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-98Python · def count_upper(s): """ Given a string s, count the number of uppercase vowels in even indices. For ex…
Original task
def count_upper(s): """ Given a string s, count the number of uppercase vowels in even indices. For example: count_upper('aBCdEf') returns 1 count_upper('abcdefg') returns 0 count_upper('dBBE') returns 0 """
- Source ID
- HumanEval/98
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →humaneval-99Python · def closest_integer(value): ''' Create a function that takes a value (string) representing a number and ret…
Original task
def closest_integer(value): ''' Create a function that takes a value (string) representing a number and returns the closest integer to it. If the number is equidistant from two integers, round it away from zero. Examples >>> closest_integer("10") 10 >>> closest_integer("15.3") 15 Note: Rounding away from zero means that if the given number is equidistant from two integers, the one you should return is the one that is the farthest from zero. For example closest_integer("14.5") should return 15 and closest_integer("-14.5") should return -15. '''
- Source ID
- HumanEval/99
- Category
- Python
Execution and trajectory
This is a source task definition. Model responses, tool calls, screenshots, costs and grader criteria belong to a measured run. A task’s presence in this roster does not indicate that it has been attempted or passed.
Inspect published run evidence →