Dezhi Ran

CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models

2024

Code generation models based on the pre-training and fine-tuning paradigm have been increasingly attempted by both academia and industry, resulting in well-known industrial models such as Codex, CodeGen, and PanGu-Coder. To evaluate the effectiveness of …