CoT-Decoding: Complex Reasoning via Chain-of-Thought Decoding

Complex reasoning is a challenging task. In recent years, Chain-of-Thought (CoT)-based reasoning methods have become a research hotspot. However, existing methods faced two limitations: first, they lacked fine-grained analysis at the step level of the reasoning chain; second, they typically employed greedy decoding strategies, which often led to local optima, thereby limiting the reasoning performance. To address these issues, we proposed the CoT-Decoding framework, which consists of two models: faithful reasoning step generation and reasoning chain decoding. In the first model, we designed a relevance determination module based on in-context learning and small Long Short-Term Memory (sLSTM). Subsequently, a large language model (LLM) was used to decompose complex questions into sub-questions, and their relevance was determined. Relevant evidence was then retrieved through cross-referencing. In the second model, we combined in-context learning with a Siamese network to design a logic step scoring module. We also developed a reasoning chain decoding module based on logic step scoring and contrastive search strategies. This model comprehensively considered model probability, contrastive scores, and logical coherence, selecting the highest-scoring reasoning steps from the generated faithful reasoning steps to decode the final reasoning chain. CoT-Decoding framework outperformed existing baseline models across five benchmarks: HOTPOTQA, 2WIKIMQA, MUSIQUE, FERMI, and STRATEGYQA, with F1 scores of 62.1, 66.7, 31.0, 41.7, and 82.2, respectively. Even in scenarios with lightweight LLMs, CoT-Decoding demonstrated outstanding performance, showcasing its potential for application in resource-constrained environments.