BEGIN:VCALENDAR VERSION:2.0 X-WR-CALNAME:EventsCalendar PRODID:-//hacksw/handcal//NONSGML v1.0//EN CALSCALE:GREGORIAN BEGIN:VTIMEZONE TZID:America/New_York LAST-MODIFIED:20240422T053451Z TZURL:https://www.tzurl.org/zoneinfo-outlook/America/New_York X-LIC-LOCATION:America/New_York BEGIN:DAYLIGHT TZNAME:EDT TZOFFSETFROM:-0500 TZOFFSETTO:-0400 DTSTART:19700308T020000 RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU END:DAYLIGHT BEGIN:STANDARD TZNAME:EST TZOFFSETFROM:-0400 TZOFFSETTO:-0500 DTSTART:19701101T020000 RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU END:STANDARD END:VTIMEZONE BEGIN:VEVENT CATEGORIES:College of Engineering,Thesis/Dissertations DESCRIPTION:Advisor: Dr. Jiawei Yuan, Department of Computer and Informatio n Science Committee Members: Dr. Yuchou Chang, Department of Computer and Information Science Dr. Gokhan Kul, Department of Computer and Informatio n Science Dr. Liudong Xing, Department of Electrical & Computer Engineerin g Abstract Given the recent advances in large language models (LLMs) and their remarkable capabilities of natural language understanding and text g eneration, LLMs have increasingly been integrated into robotic systems, en abling robots to understand high-level human instructions, reason about ta sk objectives, and generate code for robot execution. However, enabling LL Ms to reliably understand high-level human instructions and produce execut able robot operations remains challenging. For example, LLMs can misinterp ret human intentions, forget task objectives, hallucinate unavailable robo t functions, generate syntactically invalid code, produce logically incons istent action sequences, or make unsupported assumptions about environment observation.This proposed research first enhances the reliability of LLMs by enabling them to generate valid and feasible task plans and robot oper ational code, identify errors, and recover from failures. First, this rese arch developed GSCE, a structured prompt framework to enhance LLM reasonin g and generate reliable robot operation code. Building on GSCE, this resea rch further enhances reliability by developing a closed-loop framework tha t evaluates the robot behavior and provides feedback for correcting genera ted code in simulation before its deployment on a physical robot. To reduc e the configuration effort and execution time associated with specialized robotic simulators, this research further developed an LLM-driven static t ext-based simulation framework without dynamically executing the code in a physical environment or simulator during corrective code refinement. More over, to achieve reliable task planning and execution on mobile robots tha t host on-board LLMs, this research designed Ro-SLM, a framework that leve rages prior knowledge to teach on-board language models and enable them to perform reliable task planning and execution with performance approaching substantially larger models. Despite this progress, challenges remain as the diversity and complexity of robotic tasks continue to increase. In par ticular, performance may degrade on previously unseen tasks, long-horizon operations, and tasks that require complex reasoning and decision-making. Therefore, to further enhance the reliability of LLM-driven mobile robots, this research proposes to address: 1) problem solving, in which on-board language models will select, reuse, and orchestrate skills to complete tas ks, enabling the robot to solve problems by recombining its learned knowle dge rather than memorizing complete task solutions. This direction aims to enhance the reliability of LLM-driven mobile robots under complex, divers e, and previously unseen tasks; 2) decision making, the on-board language model will review past experiences, anticipate possible future consequence s and failures for the skill that the robot will conduct, and correct the errors before executing the robot operations, which enables the robot to m ake reliable decisions for the given task before executing the robot opera tions. This direction aims to further improve the reliability of LLM-drive n mobile robots for tasks that require long-horizon operations and complex decision-making.For further information, please contact Dr. Jiawei Yuan a t jyuan@umassd.edu\nEvent page: https://www.umassd.edu/events/cms/8-28-26- llm-driven-mobile-robot-task-planning-and-execution.php X-ALT-DESC;FMTTYPE=text/html:

Advisor: Dr. Jiawei Yuan\, Depa rtment of Computer and Information Science

\n

Committee Members:

\ n\n

Abstract

\n

Given the recent advances in large language models (LLMs) and their remarkable capabilities of natural langua ge understanding and text generation\, LLMs have increasingly been integra ted into robotic systems\, enabling robots to understand high-level human instructions\, reason about task objectives\, and generate code for robot execution. However\, enabling LLMs to reliably understand high-level human instructions and produce executable robot operations remains challenging. For example\, LLMs can misinterpret human intentions\, forget task object ives\, hallucinate unavailable robot functions\, generate syntactically in valid code\, produce logically inconsistent action sequences\, or make uns upported assumptions about environment observation.
This proposed res earch first enhances the reliability of LLMs by enabling them to generate valid and feasible task plans and robot operational code\, identify errors \, and recover from failures. First\, this research developed GSCE\, a str uctured prompt framework to enhance LLM reasoning and generate reliable ro bot operation code. Building on GSCE\, this research further enhances reli ability by developing a closed-loop framework that evaluates the robot beh avior and provides feedback for correcting generated code in simulation be fore its deployment on a physical robot. To reduce the configuration effor t and execution time associated with specialized robotic simulators\, this research further developed an LLM-driven static text-based simulation fra mework without dynamically executing the code in a physical environment or simulator during corrective code refinement. Moreover\, to achieve reliab le task planning and execution on mobile robots that host on-board LLMs\, this research designed Ro-SLM\, a framework that leverages prior knowledge to teach on-board language models and enable them to perform reliable tas k planning and execution with performance approaching substantially larger models.

\n

Despite this progress\, challenges remain as the diversit y and complexity of robotic tasks continue to increase. In particular\, pe rformance may degrade on previously unseen tasks\, long-horizon operations \, and tasks that require complex reasoning and decision-making. Therefore \, to further enhance the reliability of LLM-driven mobile robots\, this r esearch proposes to address: 1) problem solving\, in which on-board langua ge models will select\, reuse\, and orchestrate skills to complete tasks\, enabling the robot to solve problems by recombining its learned knowledge rather than

\n

memorizing complete task solutions. This direction ai ms to enhance the reliability of LLM-driven mobile robots under complex\, diverse\, and previously unseen tasks\; 2) decision making\, the on-board language model will review past experiences\, anticipate possible future c onsequences and failures for the skill that the robot will conduct\, and c orrect the errors before executing the robot operations\, which enables th e robot to make reliable decisions for the given task before executing the robot operations. This direction aims to further improve the reliability of LLM-driven mobile robots for tasks that require long-horizon operations and complex decision-making.
For further information\, please contac t Dr. Jiawei Yuan at jyuan@umassd.edu

Event page: https://www.umassd.edu/events/cms/8-28-26-llm-driven-mobil e-robot-task-planning-and-execution.php

DTSTAMP:20260731T155843 DTSTART;TZID=America/New_York:20260828T153000 DTEND;TZID=America/New_York:20260828T163000 LOCATION:Dion 311 SUMMARY;LANGUAGE=en-us:Towards Reliable LLM-driven Mobile Robot Task Planni ng and Execution UID:2c986a6b500173bdfbd398ba57f82318@www.umassd.edu END:VEVENT END:VCALENDAR