EggBot: A Vision-Guided Egg-Scrambling Robot
In the spring quarter of my freshman year (2026) at Stanford, I took CS 225A (Experimental Robotics), and the class gave me extraordinary (given that I was just a freshman) hands-on depth in robotics. I worked in a team of four (Olivia Taylor, Vakula Venkatesh, Vaishnavi Thumuganti, and I), and together we built the software for a robot arm that could autonomously cook scrambled eggs, going through the entire process (from cracking and whisking eggs, to scrambling them in the pan) on its own using a vision system instead of relying on exact, pre-determined motions every time.
We ran the project on a Franka Emika arm mounted on a TidyBot base, with a wrist-mounted Intel RealSense RGB-D camera for perception. We used a vision model to find grasp points on eggs, tongs, pans, etc. and converted those points from pixels on the 2D camera feed into world coordinates. We used a combination of op-space and joint-space controllers for the robot's motion, and force control in certain areas where things needed to be handled gently (i.e. when doing pretty much anything with the eggs).
The project turned out to be far more challenging than I originally expected; not only was it a bulkier project than I had ever taken on before, but there were a lot of little things I ended up having to learn on the fly (especially the OpenSai library, which I had no experience with before the class). Having made it through in one piece, however, it also turned out to be a highly rewarding experience. More than just gaining experience with certain libraries, I've also gained a good view of the field and open challenges, particularly those in classical (as opposed to learning-based) methods of robotics. It seems that sometimes learning things on the job is the best way to learn things in the first place...
For anyone looking for more technical detail: my teammate Olivia put together a more technical write-up of the full project, including photos, demo videos, and a breakdown of the cooking pipeline. You can find this and the project GitHub repo linked below.
Links: