Guangtai Liang

CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models

2024

Code generation models based on the pre-training and fine-tuning paradigm have been increasingly attempted by both academia and industry, resulting in well-known industrial models such as Codex, CodeGen, and PanGu-Coder. To evaluate the effectiveness of …

On the Evaluation of Large Language Models in Unit Test Generation

2024 · arXiv (Cornell University)

Unit testing is an essential activity in software development for verifying the correctness of software components. However, manually writing unit tests is challenging and time-consuming. The emergence of Large Language Models (LLMs) offers a new …