COMBA-PROMPT: Comprehensive Benchmark and Augmentation for Verilog Generation Leveraging Dual-LLM Prompting

The rapid advancement of Large Language Models (LLMs) has shown significant potential in RTL design, but challenges remain, particularly with naive few-shot prompting methods in both RTL generation and refinement. To address these, we introduce COMBA-PROMPT, a novel LLM-powered framework that integrates structured prompt customization with post-processing techniques to improve Verilog generation. We propose a customized XML-based prompt format that allows flexibility in defining complex sequential and combinational logic, enhancing accuracy across various design complexities. For postgeneration refinement, we introduce Topmost Exception Debugging, a debugging method that prioritizes critical exceptions, reducing errors in syntax and functionality. We also introduce the Fix Rate metric, which measures success in resolving syntax and testbench failures. Using the RTLLM-guided workflow with the GPT-4o Mini model, we achieve a syntax pass rate of 100% and a functional pass rate of 90%, with fix rates of 94.70% and 88.97%, respectively. Our benchmarks, Fix Rate and Statistics of Exceptions, provide insights for future fine-tuning and dataset development. The framework is available as open-source at: https://anonymous.4open.science/r/COMBA-PROMPT.