Reducing Hallucinations in LLM-Generated Code via Semantic Triangulation
Large language models (LLMs) can generate executable code from natural language descriptions, but the resulting programs frequently contain bugs due to hallucinations. In the absence of formal specifications, existing approaches attempt to assess correctness using LLM-generated proxies such as tests or auto-formalized specifications. However, these proxies are produced by the same imperfect models and thus often corroborate rather than catch errors, especially when the model exhibits correlated errors. We introduce semantic triangulation, a theory-grounded framework that decorrelates model errors by transforming the original problem into a dissociative variant—one likely requiring a fundamentally different algorithm—and checks consistency between independently sampled solutions to both problems. We identify theoretical requirements for this framework, and we prove that under a formal model of LLM hallucinations, these properties confer higher confidence in program correctness. We instantiate the framework through four concrete triangulation methods based on problem inversion, decomposition, and solution enumeration. Evaluated on LiveCodeBench and CodeElo across GPT-4o, DeepSeek-V3, and Gemini 2.5 Flash, our tool increases the probability of selecting a correct program by 16% over baselines (test generation, metamorphic testing, and auto-formalized specifications) and achieves 7% higher reliability and 7% higher F1 score in selection-or-abstention scenarios, while being the only method that consistently handles inexact problems admitting multiple valid solutions.
Mon 5 OctDisplayed time zone: Pacific Time (US & Canada) change
15:30 - 17:00 | LLMs for Code GenerationOOPSLA / SIGPLAN track at Junior Ballroom 1&2 Chair(s): Grigory Fedyukovich Florida State University | ||
15:30 18mTalk | EditFlow: Benchmarking and Optimizing Code Edit Recommendation Systems via Reconstruction of Developer Flows OOPSLA Chenyan Liu Shanghai Jiao Tong University; National University of Singapore, Yun Lin Shanghai Jiao Tong University, Jiaxin Chang Shanghai Jiao Tong University, Jiawei Liu Shanghai Jiao Tong University, Binhang Qi National University of Singapore, Bo Jiang Bytedance Network Technology, Zhiyong Huang National University of Singapore, Jin Song Dong National University of Singapore DOI | ||
15:48 18mTalk | Reducing Hallucinations in LLM-Generated Code via Semantic Triangulation OOPSLA Yihan Dai Peking University, Sijie Liang Peking University, Haotian Xu Peking University, Peichu Xie Independent, Sergey Mechtaev Peking University DOI | ||
16:06 18mTalk | T-REX: Teaching Large Language Models to Reason with Verbalized Execution Semantics OOPSLA Yan Wang Central University of Finance and Economics, Ling Ding Central University of Finance and Economics, Jiechen Sun Independent, Tien N. Nguyen University of Texas at Dallas, Shaohua Wang Central University of Finance and Economics, Aashish Yadavally University of Central Florida, Xin Xia Zhejiang University, Yanan Zheng Yale University DOI | ||
16:24 18mTalk | InspectCoder: Dynamic Analysis-Driven Self Repair through Interactive LLM-Debugger Collaboration OOPSLA Yunkun Wang Zhejiang University, Yue Zhang Alibaba, Guochang Li Zhejiang University, Chen Zhi Zhejiang University, Binhua Li Alibaba, Fei Huang Alibaba, Yongbin Li Alibaba, Shuiguang Deng Zhejiang University DOI | ||
16:42 18mTalk | TreeCoder: Systematic Exploration and Optimisation of Decoding and Constraints for LLM Code Generation SIGPLAN track Henrijs Princis University of Bristol, Arindam Sharma Imperial College London, Cristina David University of Bristol | ||