ACL2024

NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Queries

Shudan Zhang, Hanlin Zhao, Xiao Liu, Qinkai Zheng, Zehan Qi, Xiaotao Gu, Yuxiao Dong, Jie Tang

摘要

Case of HumanEval Case of NaturalCodeBench def has_close_elements(numbers: List[float], threshold: float) -> bool: """ Check if in given list of numbers, are any two numbers closer to each other than given threshold. """ Hello, please write a Python function for me. The function should read a markdown file, add numbering like x.y.z... to the titles of each level, and then return the modified string. Please note not to write into the original file. def add_section_numbering(markdown_file): """ markdown_file is the path to the markdown file. Return modified markdown file content string """ * SZ, HZ, XL, and QZ contributed equally.