Differences
This shows you the differences between two versions of the page.
|
rasb:lab:10 [2026/07/11 19:34] atoader Lab 10: full lab content - YOLO/DETR detection, PD ball following, Gemma 4 function calling, open-vocabulary VLM tracking |
rasb:lab:10 [2026/07/11 19:50] (current) atoader |
||
|---|---|---|---|
| Line 4: | Line 4: | ||
| You will build a complete perception-to-action pipeline: the camera image goes through an object detector, the detector output feeds a PD controller, and the PD controller makes Pupper walk towards a ball. After that works, you will put the LLM from [[.:09|Lab 9]] back in the loop so that Pupper decides //what// to follow based on natural language, and finally you will use a vision-language model to follow objects that no classic detector knows about. | You will build a complete perception-to-action pipeline: the camera image goes through an object detector, the detector output feeds a PD controller, and the PD controller makes Pupper walk towards a ball. After that works, you will put the LLM from [[.:09|Lab 9]] back in the loop so that Pupper decides //what// to follow based on natural language, and finally you will use a vision-language model to follow objects that no classic detector knows about. | ||
| - | |||
| - | This lab is based on Lecture 7 ("Computer Vision for Robotics") of Stanford's CS123 "A Hands-On Introduction to Building AI-Enabled Robots", which uses the same Pupper v3 platform: | ||
| - | |||
| - | * [[https://cs123-stanford.readthedocs.io/en/latest/schedule/lectures/fall-25/lec-7.html|CS123 Lecture 7 page]] | ||
| - | * [[https://docs.google.com/presentation/d/1D4PkpB8UiqQQ3B4OK1W82MBIK9-kjJVnK3rV9RYrwSE/edit?usp=sharing|CS123 Lecture 7 slides]] | ||
| The full pipeline you will build: | The full pipeline you will build: | ||
