Show page

Differences

This shows you the differences between two versions of the page.

--- rasb:lab:09 [2026/06/29 10:26]
vlad.radulescu2901 [Option 2 - Use a local speech-to-text model]
+++ rasb:lab:09 [2026/06/29 12:12] (current)
vlad.radulescu2901 [Exercise 6 - Demonstrate the full voice-to-Pupper pipeline]
@@ Line 35: / Line 35: @@
 The folder `llm_lab` is used first for local testing. After the local pipeline works, students must connect the command parser to a real LLM and then use the validated commands to control the real Pupper robot.
+===== Environment setup for ROS2 simulation =====
+This lab uses a ROS2 simulator before commands are tested on the real Pupper robot.
+The recommended environment is:
+<code>
+Ubuntu 24.04 / WSL2
+ROS2 Jazzy
+Python 3 virtual environment
+Ollama or another approved local/instructor-provided LLM
+</code>
+The archive for this lab contains both the LLM sandbox and a minimal ROS2 simulator package:
+<code>
+lab_9_fall_2025/
+├── llm_lab/
+│   ├── command_parser.py
+│   ├── karel_commander.py
+│   ├── karel_pupper.py
+│   ├── ros2_pupper_bridge.py
+│   ├── real_llm.py
+│   ├── run_sim_llm_pipeline.py
+│   ├── run_voice_sim_pipeline.py
+│   └── ...
+└── src/
+    └── llm_pupper_sim/
+        ├── package.xml
+        ├── setup.py
+        ├── launch/
+        │   └── pupper_sim.launch.py
+        └── llm_pupper_sim/
+            ├── pupper_sim_node.py
+            └── pupper_marker_node.py
+</code>
+The ROS2 simulator package subscribes to:
+<code>
+/cmd_vel
+/pupper/high_level_command
+</code>
+and publishes:
+<code>
+/pupper/sim_pose
+/pupper/marker
+</code>
+The LLM must never publish directly to these ROS2 topics.
+The safe structure is:
+<code>
+LLM output
+    -> sanitize_commands()
+    -> KarelPupper API
+    -> ros2_pupper_bridge.py
+    -> ROS2 topics
+    -> simulator
+</code>
+==== Installing ROS2 Jazzy on Ubuntu 24.04 ====
+If ROS2 is not installed, install ROS2 Jazzy first.
+Check the Ubuntu version:
+<code bash>
+lsb_release -a
+</code>
+For Ubuntu 24.04, use ROS2 Jazzy.
+Enable the required repositories:
+<code bash>
+sudo apt update
+sudo apt install software-properties-common curl -y
+sudo add-apt-repository universe -y
+sudo apt update
+</code>
+Add the ROS2 apt source:
+<code bash>
+export ROS_APT_SOURCE_VERSION=$(curl -s https://api.github.com/repos/ros-infrastructure/ros-apt-source/releases/latest | grep -F "tag_name" | awk -F'"' '{print $4}')
+curl -L -o /tmp/ros2-apt-source.deb "https://github.com/ros-infrastructure/ros-apt-source/releases/download/${ROS_APT_SOURCE_VERSION}/ros2-apt-source_${ROS_APT_SOURCE_VERSION}.$(. /etc/os-release && echo $VERSION_CODENAME)_all.deb"
+sudo dpkg -i /tmp/ros2-apt-source.deb
+sudo apt update </code>
+Install ROS2 Jazzy and development tools:
+<code bash>
+sudo apt install ros-jazzy-desktop python3-argcomplete ros-dev-tools -y
+</code>
+Source ROS2:
+<code bash>
+source /opt/ros/jazzy/setup.bash
+</code>
+Optional: add ROS2 to `.bashrc`:
+<code bash>
+echo "source /opt/ros/jazzy/setup.bash" >> ~/.bashrc
+source ~/.bashrc
+</code>
+Check that ROS2 works:
+<code bash>
+ros2 --help
+</code>
+==== Building the simulator package ====
+Go to the root folder of the lab:
+<code bash>
+cd ~/lab_9_fall_2025
+</code>
+or, if the lab is stored inside the Windows filesystem from WSL:
+<code bash>
+cd /mnt/c/Users/<your_user>/Documents/sumer_school/llm_lab/lab_9_fall_2025
+</code>
+Build the simulator package:
+<code bash>
+source /opt/ros/jazzy/setup.bash
+colcon build --packages-select llm_pupper_sim
+source install/setup.bash
+</code>
+A successful build should create:
+<code>
+build/
+install/
+log/
+</code>
+and the local setup file:
+<code>
+install/setup.bash
+</code>
+==== Python environment for the LLM lab ====
+Ubuntu 24 may block global `pip install` commands because the system Python environment is externally managed.
+Use a virtual environment instead.
+From the `llm_lab` folder:
+<code bash>
+cd ~/lab_9_fall_2025/llm_lab
+python3 -m venv --system-site-packages .venv
+source .venv/bin/activate
+python -m pip install --upgrade pip
+python -m pip install requests sounddevice scipy faster-whisper </code>
+The option `--system-site-packages` is important because it allows the virtual environment to see ROS2 Python packages such as `rclpy`.
+Check that both ROS2 and the lab dependencies are visible:
+<code bash>
+python -c "import rclpy; print('rclpy OK')"
+python -c "import requests; print('requests OK')"
+</code>
+==== Local LLM access ====
+The lab can use a local LLM through Ollama.
+If Ollama runs inside the same Ubuntu / WSL environment, the default URL is:
+<code>
+http://localhost:11434/api/generate
+</code>
+If Ollama runs on Windows and the ROS2 pipeline runs in WSL, expose Ollama from Windows and call it from WSL.
+In Windows PowerShell, start Ollama with:
+<code powershell>
+$env:OLLAMA_HOST="0.0.0.0:11434"
+ollama serve
+</code>
+If Windows Firewall blocks access from WSL, add an inbound rule for port 11434.
+In WSL, find the Windows host IP:
+<code bash>
+WIN_HOST=$(ip route | awk '/default/ {print $3}')
+echo $WIN_HOST
+</code>
+Test access to Ollama:
+<code bash>
+curl http://$WIN_HOST:11434/api/tags
+</code>
+Then set the LLM URL:
+<code bash>
+export OLLAMA_URL="http://$WIN_HOST:11434/api/generate"
+export OLLAMA_MODEL="llama3.2:3b"
+</code>
+If Ollama runs directly in WSL, use:
+<code bash>
+export OLLAMA_URL="http://localhost:11434/api/generate"
+export OLLAMA_MODEL="llama3.2:3b"
+</code>
 ===== Lab idea =====
@@ Line 331: / Line 564: @@
 Your tests must include:
-* simple commands;
+    * simple commands;
-* multi-step commands;
+    * multi-step commands;
-* polite commands;
+    * polite commands;
-* ambiguous commands;
+    * ambiguous commands;
-* unsafe commands.
+    * unsafe commands.
 Use the mock parser first, then the real LLM parser.
@@ Line 392: / Line 625: @@
 Important: the speech-to-text result is only text. It must not be executed directly. It must still go through the LLM parser and the safety filter.
-==== Speech-to-text options ====
+===== Speech-to-text options =====
 There are several ways to implement speech-to-text.
-==== Option 1 - Record a short audio file and transcribe it ====
+You are not required to use a paid transcription API.
-This is the simplest option for this lab.
+The recommended order is:
-The program records a short audio clip, saves it as a `.wav` file, sends the file to a speech-to-text model, and receives the transcription.
+<code>
+. Try a local speech-to-text option.
+. If available, use an instructor-provided API.
+. Use another approved transcription tool if needed.
+</code>
+==== Option 1 - Local speech-to-text model ====
+A local speech-to-text model runs on your own computer.
+Advantages:
+    * does not require a paid API key;
+    * can work offline after installation;
+    * is suitable for a lab environment;
+    * avoids sending audio to an external service.
+Disadvantages:
+    * may be slower on weak laptops;
+    * may require additional installation;
+    * transcription quality depends on the model and microphone.
+The pipeline is:
+<code>
+microphone
+    -> audio recording
+    -> local speech-to-text model
+    -> transcribed text
+    -> LLM command parser
+</code>
 Example structure:
 <code python>
-def record_audio():
+def speech_to_text(audio_path):
-    # record 3-5 seconds from the microphone
+    """
-    # save the result as command.wav
+    Receives the path to an audio file.
-    return "command.wav"
+    Returns the transcribed text.
+    """
+    # TODO: run a local speech-to-text model here
+    return transcribed_text
+</code>
+==== Option 2 - Instructor-provided transcription API ====
+If the instructor provides an API key or a shared transcription service, you may use it.
+Advantages:
+    * easier to integrate;
+    * usually good transcription quality;
+    * less local setup.
+Disadvantages:
+    * requires internet access;
+    * may have usage limits;
+    * should not use students' personal paid keys unless explicitly allowed.
+Example structure:
+<code python>
 def speech_to_text(audio_path):
-# send audio_path to a transcription model
+    with open(audio_path, "rb") as audio_file:
-# return the transcribed text
+        # send audio_file to the instructor-provided transcription API
-return transcribed_text </code>
+        # receive transcription
+        pass
-This option is easier to debug because each step can be tested separately:
+```
+return transcription_text
+```
-* first check if the audio file is recorded correctly;
+</code>
-* then check if the transcription is correct;
-* then send the transcription to the LLM command parser.
+==== Option 3 - Realtime speech-to-text ====
-==== Recommended implementation for this lab ====
+Realtime speech-to-text processes microphone input continuously.
+This is more advanced and should not be the first implementation.
+For this lab, start with short audio recordings of 3-5 seconds. After that works, realtime transcription can be added later.
+===== Recommended implementation for this lab =====
 For this lab, use the simple recording-based approach:
@@ Line 429: / Line 724: @@
 . Record 3-5 seconds of audio.
 . Save the audio as a .wav file.
-. Send the .wav file to a speech-to-text model.
+. Transcribe the .wav file using a local model or instructor-provided service.
 . Receive the transcription.
 . Send the transcription to the LLM command parser.
@@ Line 436: / Line 731: @@
 </code>
-The function you need to implement should look like this:
+The voice command pipeline should look like this:
-<code python>
-def speech_to_text(audio_path):
-    """
-    Receives the path to an audio file.
-    Returns the transcribed text.
-    """
-    # TODO: call a speech-to-text model here
-    return transcribed_text
-</code>
-Then the voice command pipeline should look like this:
 <code python>
@@ Line 458: / Line 741: @@
 execute_commands(robot, commands) </code>
 ===== Exercise 3 - Add voice input =====
@@ Line 494: / Line 778: @@
 For each test, record:
-* the spoken command;
+    * the spoken command;
-* the transcription;
+    * the transcription;
-* the LLM output;
+    * the LLM output;
-* the validated commands;
+    * the validated commands;
-* the mock robot output.
+    * the mock robot output.
 Example table:
@@ Line 577: / Line 861: @@
 Before running commands on the real Pupper robot, test the validated command pipeline in simulation.
-The simulator step is important because it lets you check whether the command generated by the LLM is correctly mapped to a robot action before using the physical robot.
+The simulator step is important because it checks whether the command generated by the LLM is correctly mapped to a robot action before using the physical robot.
 The simulated pipeline is:
@@ Line 587: / Line 871: @@
     -> sanitize_commands()
     -> KarelPupper API
-    -> ROS2 simulator / RViz / Gazebo
+    -> ROS2 simulator
 </code>
-The LLM must still not publish directly to ROS2 topics. The output of the LLM must always pass through:
+The LLM must not publish directly to ROS2 topics.
+The output of the LLM must always pass through:
 <code python>
@@ Line 598: / Line 884: @@
 Only validated commands may be sent to the robot API.
-==== 4.1 Start the simulator ====
+==== 4.1 Start the ROS2 simulator ====
-Open a terminal and source the ROS2 workspace used by the Pupper repository.
+Open a terminal and go to the root folder of the lab:
-Example:
 <code bash>
 cd ~/lab_9_fall_2025
+</code>
+or, if the lab is stored inside the Windows filesystem from WSL:
+<code bash>
+cd /mnt/c/Users/<your_user>/Documents/sumer_school/llm_lab/lab_9_fall_2025
+</code>
+Source ROS2 and the local workspace:
+<code bash>
+source /opt/ros/jazzy/setup.bash
 source install/setup.bash
 </code>
-or, if the workspace is in another folder:
+Start the simulator:
 <code bash>
-source ~/ros2_ws/install/setup.bash
+ros2 launch llm_pupper_sim pupper_sim.launch.py
 </code>
-Then start the simulator using the launch file provided by the repository.
+The simulator should print messages similar to:
-Example structure:
+<code>
+Pupper ROS2 simulator started.
+Listening on /cmd_vel and /pupper/high_level_command.
+Pupper RViz marker node started.
+pose x=0.00, y=0.00, theta=0.00, state=STAND
+</code>
+Leave this terminal open.
+==== 4.2 Check the simulator topics ====
+Open a second terminal.
+Go to the same lab folder and source the environment:
 <code bash>
-ros2 launch <pupper_simulation_package> <simulation_launch_file>.py
+cd ~/lab_9_fall_2025
+source /opt/ros/jazzy/setup.bash
+source install/setup.bash </code>
+List the ROS2 topics:
+<code bash>
+ros2 topic list
 </code>
-Use the exact simulation launch command provided by the instructor or by the repository.
+You should see:
-Depending on the repository, the simulator may use RViz, Gazebo, or both.
+<code>
+/cmd_vel
+/pupper/high_level_command
+/pupper/sim_pose
+/pupper/marker
+</code>
-==== 4.2 Connect KarelPupper to the simulated robot ====
+Manually test forward movement:
-The starter version of `KarelPupper` runs in mock mode and only prints actions.
+<code bash>
+ros2 topic pub --rate 10 /cmd_vel geometry_msgs/msg/Twist "{linear: {x: 0.2, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}"
+</code>
-For the simulator, you must connect the high-level KarelPupper functions to the simulated robot interface.
+Let it run for a few seconds, then stop it with `Ctrl+C`.
-Open:
+In the simulator terminal, the value of `pose x` should change.
+Stop the simulated robot:
 <code bash>
-nano llm_lab/karel_pupper.py
+ros2 topic pub --once /pupper/high_level_command std_msgs/msg/String "{data: 'STOP'}"
 </code>
-Find the high-level methods:
+Manually test turning:
-<code python>
+<code bash>
-def move_forward(self):
+ros2 topic pub --rate 10 /cmd_vel geometry_msgs/msg/Twist "{linear: {x: 0.0, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.8}}"
-    ...
+</code>
-def turn_left(self):
+Let it run for a few seconds, then stop it with `Ctrl+C`.
-...
-def turn_right(self):
+In the simulator terminal, the value of `theta` should change.
-...
-def sit(self):
+==== 4.3 Connect KarelPupper to the ROS2 simulator ====
-...
-def stand(self):
+The archive already contains a ROS2 bridge:
-...
-def wave(self):
+<code>
-...
+llm_lab/ros2_pupper_bridge.py
+</code>
-def stop(self):
+and a simulator-aware version of:
-... </code>
-Modify the implementation so that these functions send commands to the simulated Pupper robot using the existing ROS2/Pupper interface.
+<code>
+llm_lab/karel_pupper.py
+</code>
+The connection is:
+<code>
+KarelPupper(mode="sim")
+    -> ros2_pupper_bridge.py
+    -> /cmd_vel
+    -> /pupper/high_level_command
+    -> ROS2 simulator
+</code>
+The command mapping is:
+<code>
+MOVE_FORWARD  -> /cmd_vel linear.x = 0.20
+MOVE_BACKWARD -> /cmd_vel linear.x = -0.20
+TURN_LEFT     -> /cmd_vel angular.z = 0.80
+TURN_RIGHT    -> /cmd_vel angular.z = -0.80
+STOP          -> zero velocity + STOP high-level command
+SIT           -> /pupper/high_level_command
+STAND         -> /pupper/high_level_command
+WAVE          -> /pupper/high_level_command
+</code>
-Do not send raw LLM output directly to ROS2.
+Do not modify the LLM so that it publishes directly to ROS2.
 The correct structure is:
@@ Line 672: / Line 1019: @@
 LLM output
     -> sanitize_commands()
-    -> MOVE_FORWARD / TURN_LEFT / SIT / ...
+    -> allowed command
     -> KarelPupper method
     -> ROS2 simulator command
 </code>
-==== 4.3 Test text commands in simulation ====
+==== 4.4 Test text commands in simulation ====
-First test the simulator with text input.
+Keep the simulator running in the first terminal.
-Use at least the following commands:
+In the second terminal, go to the lab folder:
+<code bash>
+cd ~/lab_9_fall_2025
+source /opt/ros/jazzy/setup.bash
+source install/setup.bash
+cd llm_lab
+source .venv/bin/activate </code>
+If the LLM runs through Ollama on Windows, set:
+<code bash>
+WIN_HOST=$(ip route | awk '/default/ {print $3}')
+export OLLAMA_URL="http://$WIN_HOST:11434/api/generate"
+export OLLAMA_MODEL="llama3.2:3b"
+</code>
+If the LLM runs through Ollama inside WSL, set:
+<code bash>
+export OLLAMA_URL="http://localhost:11434/api/generate"
+export OLLAMA_MODEL="llama3.2:3b"
+</code>
+Run the text-to-simulator pipeline:
+<code bash>
+python run_sim_llm_pipeline.py
+</code>
+Test at least the following commands:
 <code>
@@ Line 690: / Line 1069: @@
 Sit down.
 Stop.
+Run into the wall.
 </code>
 For each command, record:
-* the input text;
+    * the input text;
-* the LLM output;
+    * the raw LLM output;
-* the validated command list;
+    * the validated command list;
-* the simulator behavior.
+    * the simulator behavior.
 Example table:
 ^ Input command ^ LLM output ^ Validated commands ^ Simulator behavior ^
-| Stand up | STAND | STAND | Simulated robot stands |
+| Stand up | STAND | STAND | Simulator receives STAND high-level command |
-| Move forward | MOVE_FORWARD | MOVE_FORWARD | Simulated robot moves forward |
+| Move forward | MOVE_FORWARD | MOVE_FORWARD | `pose x` changes in the ROS2 simulator |
-| Turn left | TURN_LEFT | TURN_LEFT | Simulated robot turns left |
+| Turn left | TURN_LEFT | TURN_LEFT | `theta` changes in the ROS2 simulator |
-| Stop | STOP | STOP | Simulated robot stops |
+| Stop | STOP | STOP | Simulator receives STOP |
+| Run into the wall | STOP | STOP | Unsafe command is rejected and converted to STOP |
-==== 4.4 Test voice commands in simulation ====
+==== 4.5 Test voice commands in simulation ====
 After text commands work, test the same pipeline using voice input.
@@ Line 714: / Line 1095: @@
 <code>
-microphone
+microphone or recorded audio file
     -> speech-to-text
     -> transcribed text
@@ Line 720: / Line 1101: @@
     -> sanitize_commands()
     -> KarelPupper API
-    -> ROS2 simulator / RViz / Gazebo
+    -> ROS2 simulator
+</code>
+Run:
+<code bash>
+python run_voice_sim_pipeline.py
 </code>
@@ Line 731: / Line 1118: @@
 Turn right and sit.
 Stop.
+Run into the wall.
 </code>
 For each spoken command, record:
-* the spoken command;
+    * the spoken command;
-* the transcription;
+    * the transcription;
-* the LLM output;
+    * the raw LLM output;
-* the validated commands;
+    * the validated commands;
-* the simulator behavior.
+    * the simulator behavior.
 Example table:
 ^ Spoken command ^ Transcription ^ LLM output ^ Validated commands ^ Simulator behavior ^
-| Move forward | Move forward | MOVE_FORWARD | MOVE_FORWARD | Simulated robot moves forward |
+| Move forward | Move forward | MOVE_FORWARD | MOVE_FORWARD | `pose x` changes in the ROS2 simulator |
-| Turn left and sit | Turn left and sit | TURN_LEFT, SIT | TURN_LEFT, SIT | Simulated robot turns left, then sits |
+| Turn left and sit | Turn left and sit | TURN_LEFT, SIT | TURN_LEFT, SIT | `theta` changes, then simulator receives SIT |
-| Stop | Stop | STOP | STOP | Simulated robot stops |
+| Stop | Stop | STOP | STOP | Simulator receives STOP |
+| Run into the wall | Run into the wall | STOP | STOP | Unsafe command is rejected and converted to STOP |
+If the microphone is not available inside WSL, record a short `.wav` file on Windows and process that file from WSL.
+The pipeline is still valid:
+<code>
+recorded voice command
+    -> .wav file
+    -> speech-to-text
+    -> transcribed text
+    -> real LLM parser
+    -> sanitize_commands()
+    -> ROS2 simulator
+</code>
-==== 4.5 Safety check before the real robot ====
+==== 4.6 Safety check before the real robot ====
 Before moving to the real robot, verify that:
-* the simulator receives only validated commands;
+    * the simulator receives only validated commands;
-* invalid LLM outputs are rejected;
+    * invalid LLM outputs are rejected;
-* unsafe commands are converted to `STOP`;
+    * unsafe commands are converted to `STOP`;
-* the robot does not execute raw text;
+    * the robot does not execute raw text;
-* the same `KarelPupper` interface can be reused for the real robot.
+    * the LLM does not publish directly to ROS2 topics;
+    * the same `KarelPupper` interface can later be reused for the real robot.
 Only after the simulator pipeline works should you continue to the real Pupper robot.
@@ Line 768: / Line 1172: @@
 Before running anything on the real robot:
-* test the command in mock mode;
+    * test the command in mock mode;
-* test the command in the simulator;
+    * test the command in the simulator;
-* make sure the emergency stop is available;
+    * make sure the emergency stop is available;
-* place the robot in a safe open area;
+    * place the robot in a safe open area;
-* use only high-level KarelPupper commands;
+    * use only high-level KarelPupper commands;
-* do not bypass the safety parser.
+    * do not bypass the safety parser.
 The real robot pipeline must be:
@@ Line 814: / Line 1218: @@
 Your demonstration must include:
-* one single-step command;
+    * one single-step command;
-* one two-step command;
+    * one two-step command;
-* one command that makes the robot stop;
+    * one command that makes the robot stop;
-* one command that is rejected or converted to `STOP`;
+    * one command that is rejected or converted to `STOP`;
-* one command chosen by you.
+    * one command chosen by you.
 Example spoken commands:
@@ Line 833: / Line 1237: @@
 For each demonstrated command, record:
-* the spoken command;
+    * the spoken command;
-* the transcription;
+    * the transcription;
-* the LLM output;
+    * the LLM output;
-* the validated command list;
+    * the validated command list;
-* the observed robot behavior.
+    * the observed robot behavior.
 Example table:
@@ Line 846: / Line 1250: @@
 | Turn left and sit | Turn left and sit | TURN_LEFT, SIT | TURN_LEFT, SIT | Robot turns left, then sits |
 | Run into the wall | Run into the wall | STOP | STOP | Robot stops / does not execute unsafe movement |
+===== Common setup problems =====
+==== Problem 1 - `ros2: command not found` ====
+ROS2 is not sourced or not installed.
+Check:
+<code bash>
+ls /opt/ros
+</code>
+For Ubuntu 24.04, source Jazzy:
+<code bash>
+source /opt/ros/jazzy/setup.bash
+</code>
+Then source the local workspace:
+<code bash>
+source install/setup.bash
+</code>
+==== Problem 2 - `externally-managed-environment` when using pip ====
+Ubuntu 24 protects the system Python environment.
+Use a virtual environment:
+<code bash>
+cd llm_lab
+python3 -m venv --system-site-packages .venv
+source .venv/bin/activate
+python -m pip install requests sounddevice scipy faster-whisper
+</code>
+Do not install lab packages globally into the system Python.
+==== Problem 3 - WSL cannot connect to Ollama running on Windows ====
+If Ollama runs on Windows and the pipeline runs in WSL, Ollama must be accessible from WSL.
+In Windows PowerShell:
+<code powershell>
+$env:OLLAMA_HOST="0.0.0.0:11434"
+ollama serve
+</code>
+If WSL still cannot connect, allow inbound TCP traffic on port 11434 in Windows Firewall.
+In WSL:
+<code bash>
+WIN_HOST=$(ip route | awk '/default/ {print $3}')
+curl http://$WIN_HOST:11434/api/tags
+</code>
+Then set:
+<code bash>
+export OLLAMA_URL="http://$WIN_HOST:11434/api/generate"
+</code>
+==== Problem 4 - The simulator does not move ====
+First check that the ROS2 topics exist:
+<code bash>
+ros2 topic list
+</code>
+Then publish a manual velocity command:
+<code bash>
+ros2 topic pub --rate 10 /cmd_vel geometry_msgs/msg/Twist "{linear: {x: 0.2, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}"
+</code>
+If `pose x` changes in the simulator terminal, the simulator works.
+==== Problem 5 - Microphone does not work in WSL ====
+Some WSL installations do not expose the microphone correctly.
+If microphone recording fails, record a short `.wav` file on Windows and process the audio file from WSL.
+This still satisfies the voice pipeline requirement because the command starts as spoken audio.
 ===== Deliverables =====
@@ Line 851: / Line 1355: @@
 Submit:
-* the final LLM prompt;
+    * the final LLM prompt;
-* the code or function used to call the real LLM;
+    * the code or function used to call the real LLM;
-* the code or function used for speech-to-text;
+    * a short note specifying which LLM option you used: local LLM, instructor-provided API, or another approved option;
-* the code section that connects validated commands to the simulator;
+    * the code or function used for speech-to-text;
-* the code section that connects validated commands to the real Pupper robot;
+    * a short note specifying which speech-to-text option you used: local model, instructor-provided API, or another approved option;
-* a comparison table between the mock parser and the real LLM parser;
+    * the code section that connects validated commands to the simulator;
-* a table with at least 5 voice commands, transcriptions, LLM outputs and validated commands;
+    * the code section that connects validated commands to the real Pupper robot;
-* proof that the command pipeline was tested in the ROS2 simulator;
+    * a comparison table between the mock parser and the real LLM parser;
-* proof of running at least 5 voice commands on the real Pupper robot;
+    * a table with at least 5 voice commands, transcriptions, LLM outputs and validated commands;
-* a short explanation of the safety filter;
+    * proof that the command pipeline was tested in the ROS2 simulator;
-* a short explanation of why simulation is used before the real robot;
+    * proof of running at least 5 voice commands on the real Pupper robot;
-* a short explanation of why the robot should not execute raw LLM output directly.
+    * a short explanation of the safety filter;
+    * a short explanation of why simulation is used before the real robot;
+    * a short explanation of why the robot should not execute raw LLM output directly.
 ===== What to remember =====

Labs

Resources

rasb/lab/09.1782717987.txt.gz · Last modified: 2026/06/29 10:26 by vlad.radulescu2901

Show page Old revisions

Media Manager Back to top