Reducing hallucinations in large language models – especially when they generate or modify software – remains a major challenge in artificial intelligence (AI).
Kai Huang, a doctoral student at Professor Chunyang Chen's Chair of Software Engineering & AI at TUM Campus Heilbronn, received the ACM SIGSOFT Distinguished Paper Award at this year's Conference on Automated Software Engineering (ASE 2025) in the South Korean capital Seoul for his approach to using visual information for optimization. The title of his scientific publication is: ”GUIRepair – Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Repair”.
The key feature of Huang's research approach is that “GUIRepair” transforms LLMs from pure text understanders into visually based problem solvers, thereby significantly reducing hallucinations, misinterpretations and invalid repairs. This could be a real game changer. Feedback on the prototype is promising: “GUIRepair achieved the best performance among open-source repair systems, surpassing the strong baselines developed by leading research institutions – our academic prototype can compete at industry-grade levels,” reports Huang.
LLM Learns Human Behavior
The prototype does not work like classic language models, but “incorporates visual understanding into the AI repair process, enabling it identifying the real cause of a problem and choosing a fix that truly works.” In his work, the scientist discovered untapped potential: “The most surprising result was discovering that, with a well-designed sequence of processing steps, an LLM can start behaving almost like a human developer. It learned to read visual cues from the interface, connect them to the underlying code, and choose fixes that match the intended visual outcome.”
This achievement has now been recognized with an award, encouraging Huang to continue on his path: “This award reflects the hard work of our entire team and the meaningful impact of our research. I am particularly grateful to Professor Chunyang Chen, whose extensive expertise in GUI testing and continuous guidance made this work possible and inspired us to push the project forward. GUIRepair is the result of true teamwork and shared dedication.”
The doctoral student has already planned the next steps: “Ultimately, our goal is to build a new generation of vision-aware software engineering tools that empower developers to produce reliable, user-centered software at scale.”
We wish Kai Huang and his team continued success and join him in celebrating his award!
Kai Huang, a doctoral student at Professor Chunyang Chen's Chair of Software Engineering & AI at TUM Campus Heilbronn, received the ACM SIGSOFT Distinguished Paper Award at this year's Conference on Automated Software Engineering (ASE 2025) in the South Korean capital Seoul for his approach to using visual information for optimization. The title of his scientific publication is: ”GUIRepair – Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Repair”.
The key feature of Huang's research approach is that “GUIRepair” transforms LLMs from pure text understanders into visually based problem solvers, thereby significantly reducing hallucinations, misinterpretations and invalid repairs. This could be a real game changer. Feedback on the prototype is promising: “GUIRepair achieved the best performance among open-source repair systems, surpassing the strong baselines developed by leading research institutions – our academic prototype can compete at industry-grade levels,” reports Huang.
LLM Learns Human Behavior
The prototype does not work like classic language models, but “incorporates visual understanding into the AI repair process, enabling it identifying the real cause of a problem and choosing a fix that truly works.” In his work, the scientist discovered untapped potential: “The most surprising result was discovering that, with a well-designed sequence of processing steps, an LLM can start behaving almost like a human developer. It learned to read visual cues from the interface, connect them to the underlying code, and choose fixes that match the intended visual outcome.”
This achievement has now been recognized with an award, encouraging Huang to continue on his path: “This award reflects the hard work of our entire team and the meaningful impact of our research. I am particularly grateful to Professor Chunyang Chen, whose extensive expertise in GUI testing and continuous guidance made this work possible and inspired us to push the project forward. GUIRepair is the result of true teamwork and shared dedication.”
The doctoral student has already planned the next steps: “Ultimately, our goal is to build a new generation of vision-aware software engineering tools that empower developers to produce reliable, user-centered software at scale.”
We wish Kai Huang and his team continued success and join him in celebrating his award!
