T-REX: Teaching Large Language Models to Reason with Verbalized Execution Semantics
Large language models (LLMs) have shown strong performance in static code tasks like code search, summarization, and generation, but remain limited in dynamic code reasoning, which involves inferring how programs behave during execution without actually running them. This limitation stems from LLMs being trained on static code and lacking the necessary runtime context. In this paper, we present T-REX, a novel teacher-student framework for execution prediction that addresses these limitations by grounding LLM training in actual execution and corresponding execution semantics. T-REX uses a large teacher model (Explainer) to generate fine-grained, stepwise natural language rationales explaining how program state transitions from one statement to another during actual execution. These rationales are used to train a smaller student model (Reasoner) to predict next program states, enabling accurate simulation of program behavior with lower computational cost. Our execution-grounded, rationale-driven training aligns with transition-aware execution semantics at the statement level, enhancing prediction accuracy. Our experiments show that T-REX enables Reasoner to outperform much larger GPT-4o and GPT-4o-mini models across multiple dimensions of runtime behavior prediction, while also aiding in static detection of runtime errors as well as in debugging. Finally, we discuss how T-REX can be generalized to static emulation of any dynamic analysis through such a teacher-student distillation, illustrating with the specific case of dynamic program slicing in Python.
Mon 5 OctDisplayed time zone: Pacific Time (US & Canada) change
15:30 - 17:00 | LLMs for Code GenerationOOPSLA / SIGPLAN track at Junior Ballroom 1&2 Chair(s): Grigory Fedyukovich Florida State University | ||
15:30 18mTalk | EditFlow: Benchmarking and Optimizing Code Edit Recommendation Systems via Reconstruction of Developer Flows OOPSLA Chenyan Liu Shanghai Jiao Tong University; National University of Singapore, Yun Lin Shanghai Jiao Tong University, Jiaxin Chang Shanghai Jiao Tong University, Jiawei Liu Shanghai Jiao Tong University, Binhang Qi National University of Singapore, Bo Jiang Bytedance Network Technology, Zhiyong Huang National University of Singapore, Jin Song Dong National University of Singapore DOI | ||
15:48 18mTalk | Reducing Hallucinations in LLM-Generated Code via Semantic Triangulation OOPSLA Yihan Dai Peking University, Sijie Liang Peking University, Haotian Xu Peking University, Peichu Xie Independent, Sergey Mechtaev Peking University DOI | ||
16:06 18mTalk | T-REX: Teaching Large Language Models to Reason with Verbalized Execution Semantics OOPSLA Yan Wang Central University of Finance and Economics, Ling Ding Central University of Finance and Economics, Jiechen Sun Independent, Tien N. Nguyen University of Texas at Dallas, Shaohua Wang Central University of Finance and Economics, Aashish Yadavally University of Central Florida, Xin Xia Zhejiang University, Yanan Zheng Yale University DOI | ||
16:24 18mTalk | InspectCoder: Dynamic Analysis-Driven Self Repair through Interactive LLM-Debugger Collaboration OOPSLA Yunkun Wang Zhejiang University, Yue Zhang Alibaba, Guochang Li Zhejiang University, Chen Zhi Zhejiang University, Binhua Li Alibaba, Fei Huang Alibaba, Yongbin Li Alibaba, Shuiguang Deng Zhejiang University DOI | ||
16:42 18mTalk | TreeCoder: Systematic Exploration and Optimisation of Decoding and Constraints for LLM Code Generation SIGPLAN track Henrijs Princis University of Bristol, Arindam Sharma Imperial College London, Cristina David University of Bristol | ||