<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[Understanding Robots]]></title><description><![CDATA[Exploring how robotics works and how it's changing our world.]]></description><link>https://www.understandingrobots.org</link><image><url>https://substackcdn.com/image/fetch/$s_!VPZP!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F53c5320f-39d8-49f9-bbe8-1e7014a78de6_1280x1280.png</url><title>Understanding Robots</title><link>https://www.understandingrobots.org</link></image><generator>Substack</generator><lastBuildDate>Mon, 16 Nov 2026 10:30:11 GMT</lastBuildDate><atom:link href="https://www.understandingrobots.org/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[Kai Williams]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[understandingrobots@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[understandingrobots@substack.com]]></itunes:email><itunes:name><![CDATA[Kai Williams]]></itunes:name></itunes:owner><itunes:author><![CDATA[Kai Williams]]></itunes:author><googleplay:owner><![CDATA[understandingrobots@substack.com]]></googleplay:owner><googleplay:email><![CDATA[understandingrobots@substack.com]]></googleplay:email><googleplay:author><![CDATA[Kai Williams]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[This startup thinks home robots need a personality]]></title><description><![CDATA[How ex-Pixar animators helped make an annoyingly cute robot.]]></description><link>https://www.understandingrobots.org/p/this-startup-thinks-home-robots-need</link><guid isPermaLink="false">https://www.understandingrobots.org/p/this-startup-thinks-home-robots-need</guid><dc:creator><![CDATA[Kai Williams]]></dc:creator><pubDate>Thu, 05 Nov 2026 23:43:08 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!k24x!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Two weeks ago, I went up to the seventh floor of the New Museum in Manhattan to watch the New York-based startup Tuesday Labs show off its new &#8220;gentleman-in-training,&#8221; Voxel. It was unlike any other robot demo I&#8217;ve ever seen, because Tuesday Labs focused more on the robot&#8217;s personality than its current capabilities.</p><p>&#8220;We will soon have robots in our homes,&#8221; said CEO Rosalind Shinkle. &#8220;The question really quickly becomes not what they will do, but how they will do it.&#8221;</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!k24x!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!k24x!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png 424w, https://substackcdn.com/image/fetch/$s_!k24x!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png 848w, https://substackcdn.com/image/fetch/$s_!k24x!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png 1272w, https://substackcdn.com/image/fetch/$s_!k24x!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!k24x!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png" width="1456" height="970" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:970,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!k24x!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png 424w, https://substackcdn.com/image/fetch/$s_!k24x!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png 848w, https://substackcdn.com/image/fetch/$s_!k24x!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png 1272w, https://substackcdn.com/image/fetch/$s_!k24x!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86876118-3162-4a31-a98b-b3d8a6185861_2048x1365.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Tuesday Labs CEO Rosalind Shinkle with Voxel, the company&#8217;s robot. (Photo courtesy of Tuesday Labs)</figcaption></figure></div><p>Tuesday Labs is betting that making a home robot that people want to buy requires giving that robot a strong character. So the company has hired ex-Pixar animators to flesh out Voxel&#8217;s personality.</p><p>Voxel was the star of the show. As Shinkle was wrapping up her remarks, Voxel ambled around the stage charming the audience. It&#8217;s not a tall robot, standing at maybe three and a half feet. Its playful and exaggerated motions were reminiscent of WALL-E. After the talk was over, Voxel posed for pictures with attendees.</p><p>There&#8217;s no question that Voxel is charming. But it&#8217;s not clear whether customers are going to want to pay $20,000 for Voxel. Because so far, the company hasn&#8217;t demonstrated the robot doing much &#8212; if any &#8212; useful work.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><h1>How Voxel&#8217;s character was made</h1><p>At times, the presentation bordered on the philosophical. The company refers to Voxel with he/him pronouns and passed out a 50-page book to attendees titled &#8220;Etiquette in the Age of Robots,&#8221; which includes whimsical rules for living with a home robot. (Sample suggestion: if you&#8217;d like to please the robot, &#8220;drop a sock in the hallway.&#8221;)</p><p>The company explained how it crafted Voxel&#8217;s personality using tricks derived from both animation and robotics.</p><p>Animation expertise was supplied in part by Alonso Martinez, who worked for 11 years at Pixar. He described the company&#8217;s process. The team started by deciding what the robot cares about deeply: a perfectionist drive to make spaces clean. Once they had that core, Martinez used the shape of the robot to communicate character: curves signal friendliness, while a large forehead gives it a &#8220;youthful feel.&#8221;</p><p>The company&#8217;s head of machine learning, Nikhil Bakshi, said Tuesday Labs used a robotics technique &#8212; reinforcement learning &#8212; to train the robot to behave in a cute way. Most humanoid robots today are trained to walk via large-scale reinforcement learning in simulation, as I described in a <a href="https://www.understandingai.org/p/robot-startups-are-trying-everything">September piece</a> for Understanding AI.</p><p>The goal at many companies is to approximate a human gait, so they train the robot to mimic human reference data. Voxel, however, was trained to mimic reference movements sketched out by on-staff animators &#8212; things like wiggling the arms in a cute way or making eye contact.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!__RB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F51de3cee-3347-4a1c-94d9-9fa76d0d7a9d_2048x1365.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!__RB!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F51de3cee-3347-4a1c-94d9-9fa76d0d7a9d_2048x1365.png 424w, https://substackcdn.com/image/fetch/$s_!__RB!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F51de3cee-3347-4a1c-94d9-9fa76d0d7a9d_2048x1365.png 848w, https://substackcdn.com/image/fetch/$s_!__RB!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F51de3cee-3347-4a1c-94d9-9fa76d0d7a9d_2048x1365.png 1272w, https://substackcdn.com/image/fetch/$s_!__RB!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F51de3cee-3347-4a1c-94d9-9fa76d0d7a9d_2048x1365.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!__RB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F51de3cee-3347-4a1c-94d9-9fa76d0d7a9d_2048x1365.png" width="1456" height="970" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/51de3cee-3347-4a1c-94d9-9fa76d0d7a9d_2048x1365.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:970,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!__RB!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F51de3cee-3347-4a1c-94d9-9fa76d0d7a9d_2048x1365.png 424w, https://substackcdn.com/image/fetch/$s_!__RB!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F51de3cee-3347-4a1c-94d9-9fa76d0d7a9d_2048x1365.png 848w, https://substackcdn.com/image/fetch/$s_!__RB!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F51de3cee-3347-4a1c-94d9-9fa76d0d7a9d_2048x1365.png 1272w, https://substackcdn.com/image/fetch/$s_!__RB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F51de3cee-3347-4a1c-94d9-9fa76d0d7a9d_2048x1365.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">The audience reacts to Voxel. (Photo courtesy of Tuesday Labs)</figcaption></figure></div><p>When Shinkle brought the robot out onto the stage, the crowd audibly &#8220;awwed&#8221; as one might with a puppy. Aside from a slightly annoying whirring sound, Voxel exuded a spunky attitude appropriate for an elementary-schooler-sized robot.</p><h1>Demonstrating capabilities is less clear</h1><p>Shinkle promised that by the time the robot was sent to the first customers, Voxel would be able to do four things:</p><ul><li><p>&#8220;Home reset,&#8221; where the robot returns the living space to some preset standard every day while the person is at work.</p></li></ul><ul><li><p>Help find a lost item by telling the person where it last saw that item.</p></li><li><p>Retrieve a named item.</p></li><li><p>Tend plants &#8212; one of the Voxel promotional videos included it watering a plant.</p></li></ul><p>However, the event barely focused on Voxel&#8217;s capabilities at all. Presenters didn&#8217;t explain the company&#8217;s technical strategy to train Voxel on these tasks. The company is apparently planning to follow standard industry techniques for non-personality aspects of the robot.</p><p>&#8220;The biggest thing we want to do is create a character-driven robot,&#8221; said Paul Vizzio, Tuesday Labs&#8217; hardware lead, in a promotional video. &#8220;The quickest way for us to do that is use the most proven technology at least on the robotics side there is and add our character development on top of that.&#8221;</p><p>The current prototype of Voxel is a modified version of a <a href="https://www.understandingai.org/p/i-spent-4000-on-a-robot-dog-from">Unitree</a> G1 robot &#8212; perhaps the most popular humanoid robot in the world today. And the reinforcement learning approach to teach Voxel to move around is pretty standard for the industry.</p><p>Reaching broad-based capability, however, especially around manipulation in unstructured settings, is still very much an open research question. There&#8217;s the risk that Tuesday Labs builds a robot that&#8217;s charming but useless.</p><p>When I asked Martinez if Tuesday Labs was putting the cart before the horse, he said that character was a long-term bet the company was making. &#8220;Personality is not going to be solved in two years,&#8221; he told me.</p><p>We&#8217;ll see if robotics capabilities can be solved sooner. The company aims to start shipping its first robots to customers in 2028.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p>]]></content:encoded></item><item><title><![CDATA[How Waymo plans to grow its fleet and stay ahead of Tesla]]></title><description><![CDATA[A deal with Toyota will help Waymo address its biggest weakness.]]></description><link>https://www.understandingrobots.org/p/how-waymo-plans-to-grow-its-fleet</link><guid isPermaLink="false">https://www.understandingrobots.org/p/how-waymo-plans-to-grow-its-fleet</guid><dc:creator><![CDATA[Timothy B. Lee]]></dc:creator><pubDate>Fri, 30 Oct 2026 17:39:25 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!ap7S!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Waymo announces partnerships all the time, but this morning&#8217;s announcement of a manufacturing partnership with Toyota seems particularly important. &#8220;Toyota and Waymo will begin initial operations to test and validate an autonomous Toyota bZ4X fleet in Japan in 2027,&#8221; Waymo said in a <a href="https://waymo.com/blog/2026/10/toyota-bz4x-partnership/">blog post</a>.</p><p>After debuting the vehicle in Japan, Waymo plans to expand it to the UK and other markets in the coming years.</p><p>I had never heard of the bZ4X, but it&#8217;s an electric crossover SUV that retails for $37,000. That&#8217;s about half the retail cost of the Jaguar I-Pace, which has been the backbone of Waymo&#8217;s fleet over the last six years. The price is on par with the two vehicles Waymo plans to introduce next: Chinese-made &#8220;Ojai&#8221; vehicles, which are being rolled out now in Texas and California, and a version of Hyundai&#8217;s Ioniq 5, which is expected to begin production in the coming months.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!ap7S!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!ap7S!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg 424w, https://substackcdn.com/image/fetch/$s_!ap7S!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg 848w, https://substackcdn.com/image/fetch/$s_!ap7S!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!ap7S!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!ap7S!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg" width="1456" height="889" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/bf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:889,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:315683,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.understandingrobots.org/i/222818361?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!ap7S!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg 424w, https://substackcdn.com/image/fetch/$s_!ap7S!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg 848w, https://substackcdn.com/image/fetch/$s_!ap7S!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!ap7S!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbf10e22d-6dac-4e3e-bf11-812ae7d2c73a_1562x954.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Toyota plans to manufacture this vehicle for Waymo&#8217;s fleet. (Photo courtesy of Waymo)</figcaption></figure></div><p>Probably the most important sentence in Waymo&#8217;s blog post is the one that says the entire manufacturing process &#8220;will take place under one roof, from assembling the base vehicle to integrating Waymo&#8217;s AV hardware.&#8221;</p><p>That&#8217;s not how Waymo did things in the past. Waymo purchased fully assembled I-Pace vehicles from Jaguar and shipped them to another vendor, Magna. Magna had to take the vehicles apart to add sensors and other hardware. Waymo has never said exactly how much this &#8220;upfitting&#8221; process takes, but a reasonable ballpark estimate is around $50,000. This helps to explain why this generation of vehicles was rumored to cost around $150,000 each &#8212; $70,000 for the base vehicle, $30,000 for the sensors and other extra hardware, and $50,000 for upfitting.</p><p>Waymo&#8217;s current generation of vehicles, dubbed Ojai, uses a cheaper base vehicle made by Chinese company Zeekr. Waymo likely took steps to streamline the upfitting process for Ojai as well. So these vehicles should have been much cheaper than their predecessor. However, Waymo suffered a major setback in 2024 when Joe Biden <a href="https://www.reuters.com/markets/us/biden-sharply-hikes-us-tariffs-billions-chinese-chips-cars-2024-05-14/">slapped 102.5% tariffs</a> on Chinese-made electric vehicles.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><p>These tariffs may have erased much of the cost savings from using the cheaper base vehicle, though it&#8217;s not clear how much. Estimates of the cost of the Ojai vehicle have ranged from <a href="https://www.govtech.com/transportation/in-detroit-waymo-rolls-out-fully-autonomous-vehicle-testing?utm_source=chatgpt.com">$70,000</a> to <a href="https://thenextweb.com/news/waymo-ojai-robotaxi-geely-sixth-generation-fleet-cost?utm_source=chatgpt.com">$125,000</a>.</p><p>In any event, it&#8217;s clear that Waymo needs more cars at lower prices. And that&#8217;s what Waymo&#8217;s recent deals with Hyundai and Toyota are designed to deliver. Completing the entire manufacturing process under one roof, as Toyota plans to do, should avoid the high upfitting costs of Waymo&#8217;s early vehicles. And making vehicles in Japan for use in the Japanese market (and eventually other non-US countries) allows Waymo to entirely sidestep headaches related to US tariffs.</p><p>This is important because I view vehicle supply as the biggest obstacle to Waymo&#8217;s growth and eventual profitability. The company&#8217;s technology now works quite well and Waymo has an excellent safety record. Waymo is operational in 15 markets. Waymo is facing some regulatory headwinds in blue states, but there is still plenty of room to expand service in red states and internationally.</p><p>But it&#8217;s hard to turn a profit with a vehicle that costs $150,000 or even $100,000. And it&#8217;s even harder to scale up a profitable service if you can&#8217;t get vehicles at all. Jaguar ended production of the I-Pace in 2024. Waymo seems to have scooped up a few thousand of these vehicles from Jaguar&#8217;s final few months of production and spent the last 18 months <a href="https://waymo.com/blog/2025/05/scaling-our-fleet-through-us-manufacturing/">adding them</a> to its robotaxi fleet.</p><p>But upfitting an expensive, discontinued vehicle was never going to be a long-term solution. Neither was importing Chinese-made cars subject to a 102.5% tariff in your home market.</p><p>If all goes well, Waymo will eventually have three different vehicle options to choose from &#8212; Zeekr, Hyundai, and Toyota. This will give Waymo some leverage in its negotiations with each supplier. And it provides a bit of an insurance policy: if manufacturing hits a snag at any one of these companies, Waymo can order extra vehicles from the other two.</p><p>But we don&#8217;t know how soon we will reach this point. Waymo says it will &#8220;begin initial operations&#8221; in 2027, but it&#8217;s not clear if this represents the start of volume production or if (as seems more likely) volume production will begin in 2028 or beyond.</p><p>Improving vehicle supply is particularly crucial to Waymo right now because its most formidable rival &#8212; Tesla &#8212; is a vertically integrated car manufacturer. Tesla could face a number of challenges scaling up its Robotaxi service, but manufacturing enough vehicles isn&#8217;t going to be one of them. Indeed, Tesla has added another 400 Cybercabs to its Texas fleet in the month since I last <a href="https://www.understandingrobots.org/p/tesla-is-finally-scaling-up-driverless">wrote about Tesla&#8217;s growing presence</a> in the state. At that pace of growth, Tesla will reach Waymo&#8217;s current fleet size &#8212; around 4,000 vehicles nationally &#8212; some time next year.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!bxBa!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3191b00-79ad-4513-bd9f-a7bba48053aa_1920x1440.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!bxBa!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3191b00-79ad-4513-bd9f-a7bba48053aa_1920x1440.png 424w, https://substackcdn.com/image/fetch/$s_!bxBa!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3191b00-79ad-4513-bd9f-a7bba48053aa_1920x1440.png 848w, https://substackcdn.com/image/fetch/$s_!bxBa!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3191b00-79ad-4513-bd9f-a7bba48053aa_1920x1440.png 1272w, https://substackcdn.com/image/fetch/$s_!bxBa!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3191b00-79ad-4513-bd9f-a7bba48053aa_1920x1440.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!bxBa!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3191b00-79ad-4513-bd9f-a7bba48053aa_1920x1440.png" width="1456" height="1092" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/c3191b00-79ad-4513-bd9f-a7bba48053aa_1920x1440.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1092,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!bxBa!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3191b00-79ad-4513-bd9f-a7bba48053aa_1920x1440.png 424w, https://substackcdn.com/image/fetch/$s_!bxBa!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3191b00-79ad-4513-bd9f-a7bba48053aa_1920x1440.png 848w, https://substackcdn.com/image/fetch/$s_!bxBa!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3191b00-79ad-4513-bd9f-a7bba48053aa_1920x1440.png 1272w, https://substackcdn.com/image/fetch/$s_!bxBa!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3191b00-79ad-4513-bd9f-a7bba48053aa_1920x1440.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Tesla could face a number of other challenges, however. To see examples, we only have to look at the kinds of challenges Waymo faced as it scaled up its own AV services over the last five years. For example, in 2023 Waymo <a href="https://www.understandingai.org/p/do-driverless-cars-have-a-first-responder">had a lot of friction</a> with first responders in San Francisco, who complained that Waymo&#8217;s vehicles would get stuck at crash scenes or drive over fire hoses.</p><p>There have also been a few incidents where many Waymos froze simultaneously during unusual incidents. Last December, <a href="https://www.cnbc.com/2025/12/23/waymo-updating-fleet-after-san-francisco-blackout-to-improve-navigation.html">dozens of Waymos froze</a> during a power outage that knocked out many stoplights. Then on the Fourth of July, unusually severe traffic <a href="https://sfstandard.com/2026/07/05/waymo-sf-gridlock-fourth-of-july-2026/">caused a number of Waymos to stall</a>. Some ran out of power and had to be towed.</p><p>Incidents like these are easy to overlook with a few dozen vehicles, but they become increasingly serious when there are hundreds or even thousands of vehicles in a city. If Tesla handles them clumsily, it could spark a political backlash that forces the company to slow down its expansion.</p><p>Most importantly, Tesla could get into a serious crash. Cruise looked like Waymo&#8217;s most formidable rival until October 2023, when a <a href="https://www.understandingai.org/p/california-suspension-is-an-existential">serious crash</a> &#8212; and a botched post-crash response by the company &#8212; forced Cruise off the road. I&#8217;m not predicting that this will necessarily happen to Tesla, but it&#8217;s certainly one way the current AV race could end badly for the company. If there&#8217;s even a single fatal crash involving a driverless Cybercab where Tesla is plausibly at fault, it could change how elected officials view the technology, even in a conservative state like Texas.</p><p>Still, Waymo can&#8217;t afford to get complacent because Elon Musk is very good at completing difficult logistical tasks quickly. Think, for example, of the <a href="https://x.com/elonmusk/status/1830650370336473253">122-day sprint</a> to build xAI&#8217;s Colossus data center &#8212; far faster than other hyperscalers or frontier labs have been able to manage. If Musk brings that same hustle to the process of manufacturing Cybercabs, building depots, and hiring support staff, Tesla could reach Waymo&#8217;s current scale surprisingly fast.</p><p>Last week, Semafor <a href="https://www.semafor.com/article/10/19/2026/waymo-plans-to-quintuple-its-fleet">reported</a> that Waymo is aiming to grow its fleet from 4,000 vehicles now to 8,000 at the end of 2026 and 20,000 at the end of 2027. Growing fivefold in 15 months would be an impressive accomplishment for most companies. But it might not be enough to stay ahead of Tesla.</p>]]></content:encoded></item><item><title><![CDATA[Why the emergence of robotics APIs is a big deal for the industry]]></title><description><![CDATA[It&#8217;s becoming easier for startups to access the top robot models.]]></description><link>https://www.understandingrobots.org/p/why-the-emergence-of-robotics-apis</link><guid isPermaLink="false">https://www.understandingrobots.org/p/why-the-emergence-of-robotics-apis</guid><dc:creator><![CDATA[Kai Williams]]></dc:creator><pubDate>Fri, 23 Oct 2026 13:18:53 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!hRur!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>On October 13, two leading <a href="https://www.understandingai.org/p/19-robotics-companies-to-watch">robotics startups</a>, Generalist and Physical Intelligence (PI), <a href="https://x.com/GeneralistAI/status/2110007693926928434">each</a> <a href="https://www.pi.website/blog/api-beta">announced</a> that they were opening up waitlists for access to their best robotics models via an application programming interface.</p><p>At first glance, these announcements might seem modest. Companies already could partner with PI and Generalist for access &#8212; several had already gained access to one company or the other prior to these announcements. And Google has had a similar waitlist for its best robotics models for over a year now.</p><p>But I think these two waitlists &#8212; as well as related announcements from <a href="https://www.perceptron.inc/blog/introducing-perceptron-mk1-5">Perceptron</a>, <a href="https://dreamlabs.ai/research/dream-1/">Dream Labs</a> and <a href="https://x.com/chris_j_paxton/status/2112643904155865128">General Intuition</a> in the past month &#8212; mark an important transition in the development of robotics.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><p>Robotics companies often face strong incentives to vertically integrate, managing everything from hardware to software to deployment in-house. This can make a company richer if it succeeds in making a compelling product &#8212; Tesla is a case in point. But if every company vertically integrates, there is a higher barrier to entry for new companies to join the field and the whole industry&#8217;s progress slows.</p><p>If startups can easily access strong robot models from companies like Physical Intelligence and Generalist, however, that might let them focus more closely on other parts of the robotics stack. Meanwhile, the experiences of those other startups can filter back to model makers, which might speed the pace of overall robotics development.</p><p>&#8220;This is a great trend for the space because it&#8217;s going to lead to faster development and faster deployments,&#8221; said <a href="https://stackpad.ai/sam">Sam Padilla</a>, a robotics entrepreneur.</p><p>We&#8217;ve already seen a version of this play out for robotics hardware. In 2024, Unitree released the G1, a cheap humanoid that researchers (and startups) could buy and modify. This dramatically lowered the barrier to entry for humanoid software research, which has in turn led to a boom in development of new methods. Strong demand quickly boosted Unitree&#8217;s profits, allowing it to invest more in further development of its robots.</p><p>Robotics companies are hoping to do the same thing on the software side. While companies have previously open-sourced robot models, they haven&#8217;t done so consistently. Now, PI and Generalist are taking a significant step toward broad commercial availability.</p><h1>What PI and Generalist offer</h1><p>This concept of an application programming interface (API) might be familiar to readers knowledgeable about LLMs. Major AI developers like Anthropic and OpenAI all offer general API access to their best models. In fact, OpenAI&#8217;s first commercial product was an API for GPT-3.</p><p>With LLMs, APIs let anyone send text (possibly along with images or other data) to an AI model and get a response. The customer doesn&#8217;t have access to under-the-hood details, like the exact model weights used to compute the response.</p><p>Robot model APIs work similarly, except that the input and output are slightly different. The input is likely to include photos and other sensor data, whereas the output is likely a list of actions for the robot to follow.</p><p>This may seem like an obvious way to sell access to a model, but it&#8217;s not the only way. Imagine if people could only access OpenAI&#8217;s GPT model through <a href="http://chatgpt.com">chatgpt.com</a>. This would let humans use the model, but it would be tricky for a developer to use GPT to classify a bunch of text because a computer program can&#8217;t easily navigate to the website.</p><p>For robotics, the <a href="http://chatgpt.com">chatgpt.com</a> approach is closer to the default. Companies like Figure have developed AI systems, but so far have only talked about selling the hardware and the software together. While that gives Figure more control over its products, it also means that a different hardware maker can&#8217;t use Figure&#8217;s software.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><h1>Robot APIs are hard</h1><p>This tendency toward bundling software and hardware is partly because it is more difficult to make an effective robot model API than an LLM API.</p><p>Compared to LLMs, general-purpose robot models have a lot more diversity in tasks and a higher standard of accuracy.</p><p>There are a lot of different things to use a robot for. For instance, PI&#8217;s API <a href="https://www.pi.website/blog/api-beta">announcement</a> notes that startups it partners with have used its models to scoop dirt on a construction site, harvest strawberries, portion ingredients in commercial kitchens, fold laundry and help feed cows on a dairy farm. (As well as plenty of other tasks.)</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!hRur!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!hRur!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png 424w, https://substackcdn.com/image/fetch/$s_!hRur!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png 848w, https://substackcdn.com/image/fetch/$s_!hRur!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!hRur!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!hRur!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png" width="1456" height="819" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/e8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:819,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!hRur!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png 424w, https://substackcdn.com/image/fetch/$s_!hRur!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png 848w, https://substackcdn.com/image/fetch/$s_!hRur!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!hRur!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8fa8e70-4b25-4c5e-b22f-a6aabaff1bc2_1920x1080.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">PI models controlling a variety of robots from different startups. (Screenshot from Physical Intelligence promotional video in API announcement)</figcaption></figure></div><p>Those different tasks require different robots. The autonomous wheel loader scooping dirt has very different controls than the pair of robot arms portioning ingredients. (Let alone underwater robots, one of the form factors that Generalist&#8217;s waitlist form asks about.)</p><p>While it&#8217;s true that LLMs also have to deal with a variety of tasks, many more of those tasks are in training datasets &#8212; and there aren&#8217;t different embodiments to worry about. In contrast, the datasets available for many physical tasks (and robot designs) are scant &#8212; or might only exist with the startup trying to solve the task in question.</p><p>So unlike LLMs, it&#8217;s often not enough to provide a general-purpose API that works out of the box. Fine-tuning is also often necessary.</p><p>Physical Intelligence describes its API process as having customers &#8220;upload data, launch fine-tuning on compute and infrastructure we manage, and access fine-tuned models via a hosted gateway to run inference.&#8221; One of the questions its waitlist form asks is how many hours of teleoperation data the prospective customer company has collected for the desired task.</p><p>This week another startup, Dream Labs, offered a completely zero-shot API &#8212; meaning no fine-tuning. However, the startup&#8217;s model only works zero-shot with two-arm fixed robots. For other form factors (or higher performance), it still offers a fine-tuning API if customers want improved performance.</p><p>Generalist doesn&#8217;t mention anything about fine-tuning in its waitlist application, but I wouldn&#8217;t be surprised if it also required fine-tuning to get the most out of its robot models.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a></p><p>The other big challenge that robot models face that LLMs do not is the need for consistently low latency. Because robots work in the physical world, they need to be able to respond very quickly; even a half-second delay could potentially lead to a serious safety incident.</p><p>This has led some to speculate that robot models will need to be run on AI chips that are on the physical robot. For instance, Sunday Robotics <a href="https://x.com/chichengcc/status/2078159388288938298">moved</a> its robotics stack from cloud servers onto its humanoid after its &#8220;model became more reliable than your average home Wi-Fi.&#8221; The company found that network jitters from home Wi-Fi were harming robotic performance too much.</p><p>Yet offering an API service that gives customers access to the model weights is problematic. It&#8217;s <a href="https://www.the-substrate.net/p/having-the-best-robotics-model-isnt">already difficult</a> to secure model weights on robots when the developer also builds the hardware. If another developer is making the hardware, it may be able to easily steal the model weights.</p><p>So companies seem keen to serve APIs over the cloud. In PI&#8217;s announcement, it notes that &#8220;most applications shown above, from Japan to Switzerland to deployments across the United States, ran with cloud-hosted inference through the API.&#8221;</p><p>Yet some model access will have to be local. PI acknowledges in its post that its early API deployments &#8220;highlighted areas for improvement, including network connectivity challenges for some.&#8221; Both PI and Generalist ask if customers need local inference for their models.</p><p>So there are still important technical challenges around security and reliability to work out, and it&#8217;s unclear what the long-term equilibrium will be.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><h1>The democratization of robot intelligence</h1><p>Robot model APIs let more robot companies access strong models.</p><p>We&#8217;ve seen this story play out with hardware. Not all robot bodies are easy to program: when Timothy B. Lee <a href="https://www.understandingai.org/p/i-spent-4000-on-a-robot-dog-from">bought</a> the cheap version of a Unitree robot dog, he could only use the provided controller. He couldn&#8217;t program the robot to do anything else.</p><p>But with hardware, it&#8217;s become increasingly possible to buy versions that allow for custom programming, even for newer robot categories like quadrupeds and humanoids. Unitree offers &#8220;EDU&#8221; versions of its legged robots that can be programmed. These can cost three to four times more &#8212; but are valuable enough that researchers will pay for the privilege.</p><p>This access to good hardware has led to an explosion of software startups like PI. The reverse now might happen with hardware.</p><p>This may eventually lead to the commoditization of robot hardware companies. For instance, Google DeepMind explicitly aims to be the &#8220;<a href="https://www.wired.com/story/google-hires-cto-boston-dynamics-demis-hassabis-android/">Android</a>&#8221; of robotics.</p><p>At the same time, it might be very helpful for Physical Intelligence and Generalist to have a broad range of customers using their models. As PI writes in its blog post announcing the API:</p><blockquote><p>The more our models are used, the more we learn how to improve them: by understanding the potential and limitations of our current models in real-world applications, we can make the next generation of models more broadly applicable, and make tangible progress toward truly general physical intelligence. But the range of real-world tasks is enormous, and understanding the broad range of tasks that robots could do requires making our models available for these tasks.</p></blockquote><p>The diversity might let these companies eventually build a &#8220;data moat&#8221;: the experiences of individual partner companies might lead to a better model for everyone.</p><p>However, it&#8217;s too early to tell whether that bet will pan out. We&#8217;re still in early days for these types of APIs. <a href="https://substack.com/@leosu2026/note/c-365225747">According</a> to Leo Su, who writes a <a href="https://leosu2026.substack.com/">robotics newsletter</a>, PI is not yet charging its partner companies for API access.</p><p>It&#8217;s unclear if PI and Generalist will ever release their API &#8212; even a cloud version &#8212; to the general public. After a year and a half, Google Gemini&#8217;s API still has gated models.</p><p>There are strong incentives against open release for (perceived) leaders like these two companies. Having a more public API gives competitors a lot of information about the model, including the potential to replicate the models by training on their outputs &#8212; a process known as distillation. It can also expose more nuanced strengths and weaknesses about the model, which may not be flattering for a company trying to sell to investors.</p><p>However, if PI and Generalist don&#8217;t open their APIs, they might lose market share to others that do. On Monday, the startup Dream Labs came out of stealth and <a href="https://x.com/dreamlabs_ai/status/2112230668805828801">announced</a> that it was releasing its first model, Dream-1, as a general API that anyone can try.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>It is true that Generalist&#8217;s GEN-1.5 model is capable of some one-shot adaptation, where the robot is able to complete a task after a single demonstration of the task in its context window. However, in the company&#8217;s <a href="https://generalistai.com/blog/gen-1.5">blog post</a>, it noted that the success rates of this adaptation were around 60% by task, too low for deployment. Having five minutes of fine-tuning data was somewhat more consistent, and I imagine that having more data helps to reach a production-level bar of consistency.</p></div></div>]]></content:encoded></item><item><title><![CDATA[Washington is worried about China’s robot dominance]]></title><description><![CDATA[I didn&#8217;t see very many robots.]]></description><link>https://www.understandingrobots.org/p/washington-is-worried-about-chinas</link><guid isPermaLink="false">https://www.understandingrobots.org/p/washington-is-worried-about-chinas</guid><dc:creator><![CDATA[Kai Williams]]></dc:creator><pubDate>Fri, 16 Oct 2026 17:29:20 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!I0Hl!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>On Wednesday, I attended the SCSP AI+ Robotics summit in Washington DC, organized by the <a href="https://en.wikipedia.org/wiki/Special_Competitive_Studies_Project">Special Competitive Studies Project</a>, a non-partisan think tank founded by ex-Google CEO Eric Schmidt in 2021.</p><p>The event&#8217;s motivation could be summarized in one chart:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!9J3U!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F949ee974-eb85-4d7c-ae42-c4cf10b5dc73_1920x1440.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!9J3U!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F949ee974-eb85-4d7c-ae42-c4cf10b5dc73_1920x1440.png 424w, https://substackcdn.com/image/fetch/$s_!9J3U!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F949ee974-eb85-4d7c-ae42-c4cf10b5dc73_1920x1440.png 848w, https://substackcdn.com/image/fetch/$s_!9J3U!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F949ee974-eb85-4d7c-ae42-c4cf10b5dc73_1920x1440.png 1272w, https://substackcdn.com/image/fetch/$s_!9J3U!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F949ee974-eb85-4d7c-ae42-c4cf10b5dc73_1920x1440.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!9J3U!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F949ee974-eb85-4d7c-ae42-c4cf10b5dc73_1920x1440.png" width="1456" height="1092" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/949ee974-eb85-4d7c-ae42-c4cf10b5dc73_1920x1440.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1092,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!9J3U!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F949ee974-eb85-4d7c-ae42-c4cf10b5dc73_1920x1440.png 424w, https://substackcdn.com/image/fetch/$s_!9J3U!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F949ee974-eb85-4d7c-ae42-c4cf10b5dc73_1920x1440.png 848w, https://substackcdn.com/image/fetch/$s_!9J3U!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F949ee974-eb85-4d7c-ae42-c4cf10b5dc73_1920x1440.png 1272w, https://substackcdn.com/image/fetch/$s_!9J3U!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F949ee974-eb85-4d7c-ae42-c4cf10b5dc73_1920x1440.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>In 2025, about 354,000 industrial robots were installed in China, <a href="https://ifr.org/downloads/press_docs/Market_Presentation_WR_Press_Conference_2026.pdf">according</a> to the International Federation of Robotics. This is over nine times as many as in the US, and more than the rest of the world combined. The majority of industrial robots installed in China in 2025 were made by Chinese companies.</p><p>While these numbers only cover industrial robots, the numbers are if anything more lopsided in newer categories of robots. In 2023, Chinese companies made <a href="https://cdn.sanity.io/files/d8lrla4f/staging/2dc719bdfc5bb19502243c3a94fbb4f0522773a8.pdf#page=26">almost 90%</a> of the collaborative robots (<a href="https://en.wikipedia.org/wiki/Cobot">cobots</a>) shipped within the country &#8212; and <a href="https://smartanalyticsglobal.com/global-humanoid-robot-shipments-2026-agibot-unitree/">over 97%</a> of the humanoids shipped across the world in the first half of 2026.</p><p>This has provoked fears that Chinese industry could gain an increasing edge over America in the coming years. As SCSP argued in an <a href="https://www.scsp.ai/wp-content/uploads/2026/10/NSCRAM_Interim-Report.pdf">interim report</a> released the day before the conference, &#8220;Physical AI at scale will define the next industrial era, and the United States is not on a trajectory to lead it.&#8221;</p><p>So the one-day conference assembled tens of speakers across industry, academia, and politics to focus on how the US might respond. Here&#8217;s some of what I learned.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><h1>1. &#8220;Defensive versus offensive measures&#8221;</h1><p>Recently, the most prominent US actions in robotics have been &#8220;defensive,&#8221; said <a href="https://www.linkedin.com/in/jenn-wall-5b69093/">Jenn Wall</a> in the first panel. Wall is the general counsel of Google&#8217;s robotics software company Intrinsic.</p><p>What she means is that US policymakers have been more focused on keeping foreign robots out of the US market than promoting the growth of the US robotics industry. In July, for example, the Federal Communications Commission (FCC) <a href="https://attacksurfaceai.substack.com/p/can-the-fcc-secure-the-us-robotics">added</a> new mobile robots (like quadrupeds and humanoids) made outside the US to its Covered List, meaning that they cannot be imported into the US without an exemption from the Department of Defense. (Existing robots are not affected.) Tariffs make it more expensive to source robot components from China.</p><p>Wall thought that the US needed to find &#8220;the right mix of defensive versus offensive measures.&#8221; The SCSP event focused heavily on the latter.</p><p>This may be in part because many of the companies participating in the event would benefit from &#8220;offensive&#8221; policies like tax incentives or state-funded worker training programs. But it was also an admission of weakness: robotics looks likely to take off in China regardless of US policies.</p><p>The &#8220;race discussion&#8221; has a very different flavor in robotics compared to AI. In AI, the US holds significant leverage over Chinese companies &#8212; especially around <a href="https://www.understandingai.org/p/a-big-lesson-of-my-china-visit-compute">access to compute</a>, so policymakers debate how much to slow Chinese progress. With robotics, in contrast, discussions tend to focus on how to make the US more competitive.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a> </p><h1>2. The flywheel problem</h1><p>In the same panel, Agility Robotics CTO Pras Velagapudi was asked whether the US was too focused on defensive measures. Velagapudi responded, &#8220;the single most important factor in advancing robots is getting robots out there to have practice doing it?&#8221;</p><p>Velagapudi, like many of the other speakers, believes that robotics deployment creates a &#8220;flywheel.&#8221; The more robots a company deploys, the better it can improve its product and lower costs, which in turn leads to more deployments.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-2" href="#footnote-2" target="_self">2</a> </p><p>This flywheel has arguably started for legged robots in China. In June, SemiAnalysis <a href="https://newsletter.semianalysis.com/p/chinas-unitree-will-dominate-global">argued</a> that <a href="https://www.understandingai.org/p/i-spent-4000-on-a-robot-dog-from">Unitree</a> is running exactly this positive feedback loop to create the cheapest quadrupeds and humanoids possible &#8212; and it might eventually dominate the market. Unitree&#8217;s Chinese competitors also have pilot deployments in factories and in stores.</p><p>This flywheel is much less far along in the US. But how to &#8220;grease that flywheel,&#8221; as Velagapudi put it, is still somewhat of an open question.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><h1>3. Jobs, jobs, jobs</h1><p>Beyond deployment, perhaps the biggest focus on Wednesday was on human jobs. Specifically, on whether robots would put humans out of work.</p><p>This is an old concern. Jeff Burnstein, the president of the Association for Advancing Automation (A3), told me that he&#8217;s been asked about it &#8220;all the time, for forty years.&#8221;</p><p>But Burnstein argued that robots and automation increase jobs, rather than removing them. &#8220;We have charts that show whenever robot sales have gone up in America, unemployment has gone down.&#8221; That isn&#8217;t just correlation, he thinks. &#8220;When companies invest in robots, they also win new business and they invest in people.&#8221;</p><p>This argument is a variant of the Jevons paradox, the observation that as production grows more efficient, the total resources used can increase if lower prices unlock a large increase in demand. Most of the speakers agreed that this applied to manufacturing labor, sometimes adding a geopolitical twist: if manufacturing in America gets cheap enough because of robots, US market share could increase dramatically. &#8220;We&#8217;re not competitive today,&#8221; said Standard Bots CEO Evan Beard on a panel on scaling. &#8220;If we get competitive, then we could bring back a lot of manufacturing, and we could have a lot more jobs.&#8221;</p><p>In the AI world, many people speculate about robots getting so good that they completely replace human labor. No one at Wednesday&#8217;s conference seemed to believe this was likely &#8212; or perhaps no one wanted to say so publicly because of the political consequences.</p><h1>4. Make robot technicians cool again</h1><p>Roboticists have a second (standard) argument against the job loss point: the blue-collar labor shortage.</p><p>&#8220;It&#8217;s not like we&#8217;re throwing people out of work. The real issue is we can&#8217;t find people to do some of these jobs, especially the dull, dirty, dangerous ones,&#8221; Burnstein said.</p><p>Several speakers referenced a <a href="https://www.deloitte.com/us/en/insights/industry/manufacturing-industrial-products/supporting-us-manufacturing-growth-amid-workforce-challenges.html">projection</a> that there may be up to 1.9 million unfilled vacancies in manufacturing by 2033.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-3" href="#footnote-3" target="_self">3</a> This was taken as a key motivation to push more robots into manufacturing.</p><p>But the lack of skilled manufacturing labor is mirrored by a lack of skilled robot technicians to deal with the robots in the factories. No one cited specific numbers, but several speakers made a concerted effort to make that work sound cool.</p><p>&#8220;You can go to high school, you can get a two-year community college degree and you can go program a robot and be making like $75,000 a year,&#8221; said FANUC America CEO Mike Cicco. &#8220;We tell that story quite a bit that there&#8217;s a really good life to be had, a blue-collar life to be had.&#8221;</p><p>One thing that may make finding workers easier is that robots are getting much more straightforward. Standard Bots, for instance, on Thursday <a href="https://www.businesswire.com/news/home/20261015248811/en/Standard-Bots-Enables-Any-Manufacturing-Worker-To-Be-A-Robot-Operator-with-New-AI-Platform">released</a> a platform which lets factory employees program robots using an iPad app without needing experience with coding or robots. </p><p>Still, finding enough people to integrate and install robots seems like a major challenge.</p><h1>5. The unclear role of allies</h1><p>One place where the US might find the talent to jumpstart its manufacturing and robotics is from allies. Yet their role in the summit was quite unclear.</p><p>The event explicitly focused on how to increase America&#8217;s industrial capacity, rather than its network of allies. Sen. Ted Budd (R-NC), who co-chaired SCSP&#8217;s report commission, said in an introductory video, &#8220;I&#8217;m focused on a simple question: how do we turn America&#8217;s leadership in artificial intelligence into American industrial strength?&#8221;</p><p>Some panelists did acknowledge the importance of ally support. &#8220;Unless we work with allied nations, it is a big risk to us. If that country wants to prevent America from growing, they can,&#8221; said Cicco, who heads the American subsidiary of a Japanese company. </p><p>But recent American robotics policy has sometimes cut against cooperation. The FCC&#8217;s recent restrictions on robot imports applied to all foreign countries, even allies. As Charlie Mlcek, a legislative assistant for Rep. Bob Latta (R-OH), put it on a panel, it treats &#8220;Canada and China similarly.&#8221; </p><p>While Mlcek said that there are efforts in Congress to exempt NATO allies, there may be structural barriers to too much cooperation. </p><p>Reva Goujon, a director at the Rhodium Group, pointed out to me that one of China&#8217;s biggest advantages in robotics hardware is the concentration of component suppliers into a very small geographic area in Shenzhen. This allows Chinese robotics companies to iterate very quickly &#8212; if they need a new type of part, they can drive over to a supplier, make a request, and have the new part within a couple of hours. </p><p>We won&#8217;t be able to replicate that type of concentrated innovation in the United States if the robotics supply chain is spread across the entire network of allies, however. So American policymakers may want to onshore as much robot component manufacturing as possible &#8212; without alienating allies.</p><h1>6. DC doesn&#8217;t have enough robots</h1><p>Throughout the whole day, I only saw two robots. One was Boston Dynamics&#8217; robot dog Spot, demonstrated on stage by Brendan Schulman, the company&#8217;s vice president of policy and government relations. The other was a modified version of Spot for welding owned by Path Robotics.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!I0Hl!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!I0Hl!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg 424w, https://substackcdn.com/image/fetch/$s_!I0Hl!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg 848w, https://substackcdn.com/image/fetch/$s_!I0Hl!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!I0Hl!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!I0Hl!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg" width="987" height="617" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:617,&quot;width&quot;:987,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!I0Hl!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg 424w, https://substackcdn.com/image/fetch/$s_!I0Hl!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg 848w, https://substackcdn.com/image/fetch/$s_!I0Hl!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!I0Hl!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8be9afb-df7d-4f4c-914e-097ff43360ce_987x617.jpeg 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Brendan Schulman speaks about US robotics policy while the quadruped Spot holds a plushie of &#8220;Bill&#8221; from the Schoolhouse Rock! song &#8220;<a href="https://www.youtube.com/watch?v=SZ8psP4S6BQ">I&#8217;m Just a Bill</a>.&#8221; (Photo by Kai Williams)</figcaption></figure></div><p>Schulman&#8217;s demo was fairly basic by my lights: he had the robot dog walk up some stairs, navigate over some boxes, pick up an object, and do a little dance. Still, a substantial proportion of the audience pulled out their phones to record it. I got the sense that it was the first time that many people had seen a legged robot like Spot.</p><p>It&#8217;s much easier to see robots in action in San Francisco &#8212; or major Chinese cities &#8212; than it is in DC. I wonder if this makes it more difficult for policymakers to understand the current state of the art.</p><h1>7. Robot preemption might be the next policy fight</h1><p>Throughout the day, speakers called for a national robotics strategy. American robotics strategy is arrayed across several federal departments, and there&#8217;s no one individual or institution directly responsible for increasing the amount of American robotics deployment. One of the key takeaways from the SCSP report was that America needs to coordinate its effort better.</p><p>This focus on centralization may lead to a second fight: whether to preempt state and county laws around robotics. &#8220;This is going to be the debate when it comes to physical AI &#8212; what is the balance between state and local versus federal,&#8221; Mlcek said in his panel.</p><p>Preemption has been in the news a lot over the past year and a half as some Republicans have repeatedly <a href="https://www.transformernews.ai/p/another-preemption-failure-shows-ai-industry-losing-battle-regulation-scalise">tried and failed</a> to prevent states from regulating AI on their own. This has polarized some people skeptical of AI away from preemption as a policy tool.</p><p>Yet compared to AI, robotics preemption might have a stronger case. As we&#8217;ve seen <a href="https://www.understandingai.org/p/waymos-next-five-cities-are-all-in?utm_source=publication-search">with autonomous vehicles</a>, there are many existing laws that need to be changed to allow for the broad deployment of robotics. A patchwork of state regulations might significantly slow robotics development and adoption.</p><p>Given the labor concerns though, it may be even more difficult to pass a robotics preemption bill, even if it was less broad than earlier AI safety preemption efforts. We&#8217;ll see how the politics play out over the coming year or two.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>I do find this a bit confusing though. The US has some leverage over Chinese robotics, as Nvidia&#8217;s edge chips are <a href="https://dset.tw/en/research/000500-1/">used </a><em><a href="https://dset.tw/en/research/000500-1/">very</a></em><a href="https://dset.tw/en/research/000500-1/"> widely</a> across the industry, including in China. Would limits on these chips slow down Chinese robot progress? I don&#8217;t think this is necessarily a good idea, but I am a little surprised I didn&#8217;t hear any discussion of it on Wednesday.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-2" href="#footnote-anchor-2" class="footnote-number" contenteditable="false" target="_self">2</a><div class="footnote-content"><p> Some speakers framed this flywheel in terms of data collection. Modern general-purpose robot models require <a href="https://www.understandingai.org/p/robot-startups-are-trying-everything">a lot of physical data</a>, which is currently difficult to obtain. Many hope that deployments will create real-world data at scale, which will improve future iterations of the model. This is a plausible theory &#8212; I&#8217;ve written about it before, but some roboticists disagree. As the researcher Chris Paxton <a href="https://itcanthink.substack.com/p/how-much-is-robot-deployment-data">wrote</a>, &#8220;most robotics deployments are fundamentally <em>not</em> scalable, requiring large, specialized support teams and large amounts of resources to maintain, and the data that these deployments produce is almost certainly less valuable than you think it is.&#8221; However, the broader idea of a flywheel, where early deployments help jumpstart the market, seems much more solid to me.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-3" href="#footnote-anchor-3" class="footnote-number" contenteditable="false" target="_self">3</a><div class="footnote-content"><p>Though Susan Helper, an economist at Case Western Reserve, poked a hole in the number by analogizing it to buying a car. &#8220;I&#8217;d really love to have a Mercedes. But I really would like to pay less than $10,000 for it. If I can&#8217;t find a $10,000 brand new Mercedes, do I have a Mercedes shortage or do I have a willingness to pay shortage?&#8221;</p></div></div>]]></content:encoded></item><item><title><![CDATA[Why testing robot models is even harder than testing LLMs]]></title><description><![CDATA[The real world is complicated.]]></description><link>https://www.understandingrobots.org/p/why-testing-robot-models-is-even</link><guid isPermaLink="false">https://www.understandingrobots.org/p/why-testing-robot-models-is-even</guid><dc:creator><![CDATA[Kai Williams]]></dc:creator><pubDate>Fri, 09 Oct 2026 17:47:02 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!bUVe!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>People frequently ask me how quickly robot capabilities are advancing. The honest answer is I don&#8217;t know &#8212; and I&#8217;m not sure anyone else does either. A big reason is that today&#8217;s best robotics benchmarks are far less informative than benchmarks for conventional frontier models.</p><p>Last week, as I was figuring out whether GPT-6 Astra was good at robotics, I carefully looked into the model&#8217;s performance on the <a href="https://robodojo-benchmark.com/">RoboDojo benchmark</a>. RoboDojo is a benchmark made by a consortium of academics from the University of Hong Kong, UC Berkeley, Tsinghua, and others.</p><p>RoboDojo seemed to indicate that yes, <a href="https://www.understandingrobots.org/p/openais-astra-model-is-shockingly">Astra is good at robotics</a>. The model was (briefly) the <a href="https://robodojo-benchmark.com/report/gpt-6-astra-eval">top-scoring model</a> on the leaderboard, beating out specialized robotics models from companies like Physical Intelligence. Yet when I looked closer at Astra&#8217;s performance, I realized that the story was more complicated than the headline number suggested.</p><p>RoboDojo&#8217;s simulation benchmark is a collection of 42 tasks spread across five categories: generalization, precision, long-horizon, memory and open tasks. Some tasks require dexterous movements and precision, like inserting tubes into a rack.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!bUVe!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!bUVe!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png 424w, https://substackcdn.com/image/fetch/$s_!bUVe!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png 848w, https://substackcdn.com/image/fetch/$s_!bUVe!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png 1272w, https://substackcdn.com/image/fetch/$s_!bUVe!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!bUVe!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png" width="1426" height="817" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:817,&quot;width&quot;:1426,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!bUVe!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png 424w, https://substackcdn.com/image/fetch/$s_!bUVe!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png 848w, https://substackcdn.com/image/fetch/$s_!bUVe!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png 1272w, https://substackcdn.com/image/fetch/$s_!bUVe!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F63f2ff16-53cc-4a5a-9c35-cfee0e30891d_1426x817.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">A simulated robot inserts a test tube into a rack. (Screenshot from RoboDojo&#8217;s <a href="https://robodojo-benchmark.com/doc/sim-tasks/insert-tubes/">task specification</a>)</figcaption></figure></div><p>Other tasks only test basic manipulation, but they require some &#8220;book smarts.&#8221; For instance, another task asks the robot to arrange number tiles from left to right to form the largest possible number.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!beZp!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7c7cfc3a-57fb-471c-955c-c453027c9b08_1437x945.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!beZp!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7c7cfc3a-57fb-471c-955c-c453027c9b08_1437x945.png 424w, https://substackcdn.com/image/fetch/$s_!beZp!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7c7cfc3a-57fb-471c-955c-c453027c9b08_1437x945.png 848w, https://substackcdn.com/image/fetch/$s_!beZp!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7c7cfc3a-57fb-471c-955c-c453027c9b08_1437x945.png 1272w, https://substackcdn.com/image/fetch/$s_!beZp!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7c7cfc3a-57fb-471c-955c-c453027c9b08_1437x945.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!beZp!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7c7cfc3a-57fb-471c-955c-c453027c9b08_1437x945.png" width="1437" height="945" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/7c7cfc3a-57fb-471c-955c-c453027c9b08_1437x945.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:945,&quot;width&quot;:1437,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!beZp!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7c7cfc3a-57fb-471c-955c-c453027c9b08_1437x945.png 424w, https://substackcdn.com/image/fetch/$s_!beZp!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7c7cfc3a-57fb-471c-955c-c453027c9b08_1437x945.png 848w, https://substackcdn.com/image/fetch/$s_!beZp!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7c7cfc3a-57fb-471c-955c-c453027c9b08_1437x945.png 1272w, https://substackcdn.com/image/fetch/$s_!beZp!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7c7cfc3a-57fb-471c-955c-c453027c9b08_1437x945.png 1456w" sizes="100vw"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Astra struggled with the precise manipulation tasks but aced the &#8220;book smarts&#8221; tasks. In comparison, the strongest specialized robot models were better at precision but couldn&#8217;t handle semantic variation. Enough of the 42 tasks tested these semantic skills that Astra ended up having the highest average score of all the models evaluated.</p><p>If the benchmark had included even a couple more tasks Astra was weak at, some other model might have performed better. Astra&#8217;s top ranking depended heavily on how the benchmark was set up.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a></p><p>Astra&#8217;s spiky performance on RoboDojo led me to conclude that it is closer to a &#8220;system two&#8221; model, but this uneven performance also points to a challenge in understanding robots: robustly measuring (and interpreting) the capabilities of generalist robots is incredibly difficult. Robot benchmarking has challenges that LLM benchmarking does not have, and there is far less information comparing different models with each other.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><h1>The real world is complicated</h1><p>The primary reason robotics benchmarking is hard is the same reason robotics is hard: the real world is complicated, unforgiving and expensive to deal with.</p><p>Basically every robot startup that has made a robot has a story about how something completely unexpected happened when its team took that robot into a new setting.</p><p>Sometimes, it&#8217;s a new type of flooring. When Fauna Robotics (now owned by Amazon) was teaching its robot to walk, it could move around on almost all surfaces fine. But one &#8220;extremely high-friction, textured carpet&#8221; tripped the robot up, said the company&#8217;s CTO <a href="https://www.manufacturingdive.com/news/prototype-to-deployment-lessons-learned-fauna-ultra-robotics-standard-bots/823144/">on a panel</a>. &#8220;That motivated us to go back to our office and buy a bunch of carpets.&#8221;</p><p>Other times, it&#8217;s the difficulty of predicting human behavior. At one point, Fetch Robotics (now owned by Skild AI) was <a href="https://realworldrobots.github.io/assets/files/Warehouse_Deployments.pdf#page=3">testing</a> its warehouse picking robot in simulation. The simulation assumed that humans would always go to the nearest robot, but that turned out not to be true. This slight mismatch in behavior &#8220;had a huge impact on the performance of our algorithms.&#8221;</p><p>It can even be a subtle hardware bug: Agility Robotics <a href="https://www.understandingai.org/p/why-humanoid-robots-wont-catch-up#:~:text=The%20CTO%20of%20Agility%20Robotics%2C%20Pras%20Velagapudi%2C%20told%20me%20that%20at%20one%20point%2C%20Agility%20faced%20a%20perplexing%20failure%2E">had issues</a> where some of its older robots would occasionally shut down while crouching. It turned out to be because of a chip that flexed out of position over time.</p><p>The same real-world complexity applies to benchmarks that try to capture a broader range of robot abilities, rather than a single future deployment task. This is true whether the robot is run on real hardware or simulated in a computer.</p><p>For real-world benchmarks, the initial setup can sometimes be straightforward. To test how well a model can stack blocks, all that&#8217;s necessary is a couple of blocks, a table and a robot.</p><p>But running the evaluation itself is difficult and expensive. A human usually needs to reset the task and the robot after every run. Robots can break down or overheat.</p><p>And both the number of robots and the number of humans constrain the number of evaluation runs. While an LLM benchmark with 250 questions can be run quickly in parallel, a robotics benchmark with that many tasks would require either a huge number of robots or a good deal of time to complete the tasks one by one.</p><p>As a result, most robotics benchmarks today consist of only a small number of tasks with dozens of trials per task. The independent evaluator Robocurve currently <a href="https://robocurve.org/">benchmarks</a> models on six robotics tasks, for instance, while RoboDojo&#8217;s real-world evaluation suite has 18.</p><p>Even so, running a couple hundred trials is a substantial effort. When the startup Pantograph <a href="https://pantograph.com/journal/vlm-harness">benchmarked</a> Astra and Fable 5.1 using six robots, it took a day and a half to complete 160 trials, according to Camille Fassett, the company&#8217;s head of operations.</p><p>Simulation benchmarks are much easier to run, but they require a lot of work to set up. One must &#8220;get the friction parameters right, masses, make sure contact is working properly, et cetera,&#8221; <a href="https://itcanthink.substack.com/p/how-do-we-quantify-progress-in-robotics">observed</a> roboticist Chris Paxton in January. &#8220;This only gets harder as I start to scale simulation up.&#8221;<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-2" href="#footnote-2" target="_self">2</a></p><p>There&#8217;s also always the worry that results in simulation don&#8217;t necessarily transfer into the real world, something called the <a href="https://www.understandingai.org/i/214044192/the-sim-to-real-gap">sim-to-real gap</a>. This is especially true for tasks that require a lot of complex manipulation, as that is much more difficult to encode into a computer.</p><p>So most robotics benchmarking today in both the real world and in simulation covers fairly simple tasks, and evaluation suites tend to have tens of tasks, rather than the hundreds or thousands that an LLM benchmark might have.</p><h1>Robo-benchmaxxing</h1><p>That, in turn, makes it difficult to turn individual evaluations into measurements about general robotics progress.</p><p>While LLM benchmarking also has this problem, the major LLM developers all optimize for essentially the same thing: coding and agentic tasks. So model capabilities on benchmarks <a href="https://epoch.ai/gradient-updates/benchmark-scores-general-capability-claudiness">correlate strongly</a> along just one or two dimensions.</p><p>This has let evaluators figure out ways to boil down an AI model&#8217;s performance into one holistic judgment, either of software engineering ability (like the METR <a href="https://www.understandingai.org/p/why-its-getting-harder-to-measure">time-horizon benchmark</a>) or general capability (like the <a href="https://epoch.ai/eci">Epoch Capabilities Index</a>).</p><p>There&#8217;s nothing like that yet for robotics. Self-contained tasks like stacking blocks or folding shirts give some information about how capable a robot model is, but less about the trajectory of the entire field.</p><p>Worse, robotics companies often optimize robots to perform well on specific tasks. This is often innocuous: most real-world deployments today involve robots performing a narrow task over and over. But it means that solid performance on a task doesn&#8217;t tell us much about how well the robot might perform other tasks, even those closely related.</p><p>A crisp example of this dynamic comes from the most recent edition of the <a href="https://en.wikipedia.org/wiki/World_Humanoid_Robot_Games">World Humanoid Robot Games</a>, hosted in Beijing in August. Robot makers competed in a number of events, ranging from traditional athletic activities like soccer to robot-specific competitions like power tool assembly.</p><p>Many of these events featured impressive performances &#8212; such as when a robot broke the human world record in the 100-meter race. But those performances didn&#8217;t necessarily transfer to other, more useful skills. In the 100-meter race, robots didn&#8217;t stop like a human would but instead ran into a barrier. One caught on fire.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!82is!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9f787bc6-4c45-4830-83df-3b09c84a48d8_1581x810.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!82is!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9f787bc6-4c45-4830-83df-3b09c84a48d8_1581x810.png 424w, https://substackcdn.com/image/fetch/$s_!82is!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9f787bc6-4c45-4830-83df-3b09c84a48d8_1581x810.png 848w, https://substackcdn.com/image/fetch/$s_!82is!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9f787bc6-4c45-4830-83df-3b09c84a48d8_1581x810.png 1272w, https://substackcdn.com/image/fetch/$s_!82is!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9f787bc6-4c45-4830-83df-3b09c84a48d8_1581x810.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!82is!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9f787bc6-4c45-4830-83df-3b09c84a48d8_1581x810.png" width="1456" height="746" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/9f787bc6-4c45-4830-83df-3b09c84a48d8_1581x810.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:746,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!82is!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9f787bc6-4c45-4830-83df-3b09c84a48d8_1581x810.png 424w, https://substackcdn.com/image/fetch/$s_!82is!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9f787bc6-4c45-4830-83df-3b09c84a48d8_1581x810.png 848w, https://substackcdn.com/image/fetch/$s_!82is!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9f787bc6-4c45-4830-83df-3b09c84a48d8_1581x810.png 1272w, https://substackcdn.com/image/fetch/$s_!82is!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9f787bc6-4c45-4830-83df-3b09c84a48d8_1581x810.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Humanoid robots lie crumpled immediately after breaking the 100-meter world record at the 2026 World Humanoid Robot Games. (Screenshot from a compilation of clips posted on YouTube by <a href="https://www.youtube.com/watch?v=KyFZuBgPBsg">CNBC-TV18</a>)</figcaption></figure></div><p>That isn&#8217;t to say that competitions and benchmarks like the Robot Games aren&#8217;t useful. On the contrary, the Robot Games are &#8220;probably the most comprehensive testbed for humanoid physical abilities that we&#8217;ve seen,&#8221; as Center for Technology &amp; Statecraft research fellow Amelia Michael <a href="https://techstatecraft.substack.com/p/robotics-evals-should-test-hardware">points out</a>. This can be helpful for figuring out how good robots might be if AI models dramatically improve over the next year.</p><p>But it&#8217;s very difficult to predict the trajectory of general-purpose robots from benchmarks with a limited number of tasks.</p><h1>We don&#8217;t know the best US robot model</h1><p>Perhaps the biggest challenge to public robotics benchmarking today, though, is that there is very little of it among the top US companies.</p><p>One of the virtues of the Robot Games is that the event encouraged participation from some of China&#8217;s strongest robotics companies. In the US, there are much less of these broad-based competitions. In fact, there is almost no independent evaluation of the top robotics models at all.</p><p>On leaderboards such as RoboDojo, almost all of the American models listed are from organizations like the Allen Institute for AI and Nvidia, which have strong incentives (academic and commercial) to release open models. The most prominent exception is from Physical Intelligence, which openly released its &#960;<sub>0.5</sub> model in 2025.</p><p>So beyond demo videos and blog posts, we have little public information about how capable the models of other American companies are. Models like Generalist&#8217;s GEN-1.5, Dyna&#8217;s DYNA-2, and Skild&#8217;s S1 have all displayed impressive abilities in videos and blog posts. (Ditto for robots from the big makers like Figure or 1X.) But it&#8217;s difficult to turn that information into a systematic evaluation.</p><p>I expect this will change as robots start to be deployed in the real world (especially homes): we&#8217;ll at least be able to see how robots perform in settings where they can be observed independently. But for now, we&#8217;re sort of flying blind.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>RoboDojo has since updated its simulation leaderboard to separate the scores of vision-language-action (VLA) models and world models from agent-based systems like GPT-6 Astra.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-2" href="#footnote-anchor-2" class="footnote-number" contenteditable="false" target="_self">2</a><div class="footnote-content"><p>Though the increasing spatial capabilities of frontier LLMs likely helps with this.</p></div></div>]]></content:encoded></item><item><title><![CDATA[Tesla is finally scaling up driverless robotaxi service]]></title><description><![CDATA[Especially in Texas.]]></description><link>https://www.understandingrobots.org/p/tesla-is-finally-scaling-up-driverless</link><guid isPermaLink="false">https://www.understandingrobots.org/p/tesla-is-finally-scaling-up-driverless</guid><dc:creator><![CDATA[Timothy B. Lee]]></dc:creator><pubDate>Fri, 02 Oct 2026 15:32:17 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!lYl-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>I have a reputation as a Tesla skeptic, and something I regularly hear from Tesla owners is &#8220;you need to try the latest version of Tesla&#8217;s FSD technology &#8212; it wasn&#8217;t that good a year ago but it&#8217;s really impressive now.&#8221; I take comments like this seriously, and I try to drive (or ride) in Teslas as often as I can. But I also think people who say this tend to underestimate how difficult it is to judge autonomous systems based on casual observation.</p><p>I&#8217;m in San Francisco this week, and one of my goals has been to ride in as many autonomous vehicles as possible. So far I&#8217;ve taken three Waymo rides and one ride each from Zoox and Tesla. I plan to do a few more Tesla rides this weekend.</p><p>The rides so far have all been basically flawless. None of the vehicles made any major mistakes. In only one case &#8212; the Zoox ride &#8212; did a vehicle seem to make a minor mistake.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a></p><p>But here&#8217;s the thing: the industry has reached a point where we shouldn&#8217;t expect any mistakes to occur in a typical ride &#8212; or, for that matter, in 10 or 100 typical rides. AV companies are aiming to exceed human performance, and human drivers go tens of thousands of miles between serious crashes. So even if you observe a self-driving system driving flawlessly for many hours, this doesn&#8217;t tell you very much about whether its performance is at the cutting edge.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><p>For that you need data from large-scale deployments. Waymo, for example, says its vehicles have driven <a href="https://waymo.com/safety/impact/">more than 270 million driverless miles</a> through July 2026. That should be enough data to make statistically rigorous comparisons between human drivers and Waymo&#8217;s vehicles.</p><p>But there&#8217;s a bit of a chicken-and-egg problem when a new company wants to launch a commercial service. We&#8217;d like to know whether an AV system is safe <em>before</em> we allow large-scale deployment on public streets. But there&#8217;s no other way to get the necessary data. Even if a company drove tens of millions of miles on a private test track, the resulting data wouldn&#8217;t tell us very much; public roads are far more chaotic and unpredictable than any private facility.</p><p>And crash data is most informative when it comes from a full-scale commercial deployment. If a company only operates in a few designated routes, we can&#8217;t be sure its performance will generalize to other parts of a city. Similarly, a vehicle&#8217;s performance with safety drivers won&#8217;t necessarily predict its performance without them.</p><p>That&#8217;s not to say that testing on limited routes and with safety drivers is a bad idea &#8212; it just needs to be viewed as a stepping stone toward full-scale commercial deployment.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!lYl-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!lYl-!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg 424w, https://substackcdn.com/image/fetch/$s_!lYl-!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg 848w, https://substackcdn.com/image/fetch/$s_!lYl-!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!lYl-!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!lYl-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg" width="1456" height="1092" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/e22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1092,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!lYl-!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg 424w, https://substackcdn.com/image/fetch/$s_!lYl-!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg 848w, https://substackcdn.com/image/fetch/$s_!lYl-!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!lYl-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe22221b8-03db-4934-a6ae-7010e8bd0e29_2048x1536.jpeg 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">I took a selfie just before my first Zoox ride. Passengers face one another, which makes the ride extremely roomy. I could stretch my legs out almost straight. However, the high-backed seats meant I couldn&#8217;t really see out the front of the vehicle, giving it a bit of a claustrophobic experience. (Photo by Timothy B. Lee)</figcaption></figure></div><p>And that brings me to my robotaxi experiences this week. Zoox has yet to launch a full-scale commercial service in San Francisco &#8212; Zoox has an invite-only list of &#8220;Zoox Explorers&#8221; who are eligible to use its service for free in San Francisco. But those of us not in the Explorers program have to travel to the <a href="https://zoox.com/journal/rider-lounge-san-francisco">Zoox Rider Lounge</a>, a storefront that lets customers sign up for free Zoox rides. Rides must start in front of the store and end at one of a dozen or so hand-picked destinations.</p><p>So not only is Zoox&#8217;s operation far smaller than Waymo&#8217;s city-wide commercial service, Zoox may have chosen routes that would be relatively easy for its vehicles to complete.</p><p>In contrast Tesla serves a huge territory in the San Francisco Bay Area, ranging from San Jose in the South to Berkeley in the Northeast. There are no cherry-picked routes here. However, all of Tesla&#8217;s California Robotaxis have safety drivers behind the wheel.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!c95D!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a4e7b68-252e-4882-ad15-c3012b182a15_2048x1536.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!c95D!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a4e7b68-252e-4882-ad15-c3012b182a15_2048x1536.jpeg 424w, https://substackcdn.com/image/fetch/$s_!c95D!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a4e7b68-252e-4882-ad15-c3012b182a15_2048x1536.jpeg 848w, https://substackcdn.com/image/fetch/$s_!c95D!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a4e7b68-252e-4882-ad15-c3012b182a15_2048x1536.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!c95D!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a4e7b68-252e-4882-ad15-c3012b182a15_2048x1536.jpeg 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!c95D!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a4e7b68-252e-4882-ad15-c3012b182a15_2048x1536.jpeg" width="1456" height="1092" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/1a4e7b68-252e-4882-ad15-c3012b182a15_2048x1536.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1092,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!c95D!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a4e7b68-252e-4882-ad15-c3012b182a15_2048x1536.jpeg 424w, https://substackcdn.com/image/fetch/$s_!c95D!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a4e7b68-252e-4882-ad15-c3012b182a15_2048x1536.jpeg 848w, https://substackcdn.com/image/fetch/$s_!c95D!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a4e7b68-252e-4882-ad15-c3012b182a15_2048x1536.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!c95D!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a4e7b68-252e-4882-ad15-c3012b182a15_2048x1536.jpeg 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Me on my second-ever Tesla Robotaxi ride in the San Francisco Bay Area. The ride was flawless, but the safety driver took the wheel for the last 30 seconds as it was choosing a place to drop me off. He said this was standard procedure. (Photo by Timothy B. Lee)</figcaption></figure></div><p>These factors cause me to discount the generally excellent driving experience I had with both Zoox and Tesla in California. It&#8217;s easier to deliver a safe, pleasant drive on a few handpicked routes than citywide. And it&#8217;s easier to have a vehicle driver smoothly and confidently when a human driver is ready to intervene if the vehicle makes a mistake.</p><p>At this point, the only way to judge robotaxi performance is by large-scale, driverless commercial deployments. Waymo has done that in California. Tesla and Zoox have not.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><h2>The picture is different in Texas</h2><p>So far, California regulators have required all Tesla vehicles in California to have a safety driver behind the wheel. That&#8217;s not true in Texas, and Tesla <a href="https://techcrunch.com/2026/01/22/tesla-launches-robotaxi-rides-in-austin-with-no-human-safety-driver/?utm_source=chatgpt.com">began operating fully driverless vehicles</a> in Austin in January. At the time there were only a handful of driverless vehicles in Tesla&#8217;s Texas fleet &#8212; the rest still had human monitors in the passenger seat. By April, Tesla&#8217;s driverless fleet in Texas had grown to <a href="https://electrek.co/2026/04/30/tesla-robotaxi-unsupervised-finally-signs-ramping-up/?utm_source=chatgpt.com">around 25 vehicles</a>. Then over the summer, Tesla&#8217;s overall fleet ramped up dramatically. At the end of July, Tesla had 181 Model Y vehicles registered for Robotaxi operations in Texas. Today the number is 578.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!d0Vj!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F062dab63-e819-4c7d-846e-a17e8d097a6c_1920x1440.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!d0Vj!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F062dab63-e819-4c7d-846e-a17e8d097a6c_1920x1440.png 424w, https://substackcdn.com/image/fetch/$s_!d0Vj!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F062dab63-e819-4c7d-846e-a17e8d097a6c_1920x1440.png 848w, https://substackcdn.com/image/fetch/$s_!d0Vj!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F062dab63-e819-4c7d-846e-a17e8d097a6c_1920x1440.png 1272w, https://substackcdn.com/image/fetch/$s_!d0Vj!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F062dab63-e819-4c7d-846e-a17e8d097a6c_1920x1440.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!d0Vj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F062dab63-e819-4c7d-846e-a17e8d097a6c_1920x1440.png" width="1456" height="1092" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/062dab63-e819-4c7d-846e-a17e8d097a6c_1920x1440.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1092,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!d0Vj!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F062dab63-e819-4c7d-846e-a17e8d097a6c_1920x1440.png 424w, https://substackcdn.com/image/fetch/$s_!d0Vj!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F062dab63-e819-4c7d-846e-a17e8d097a6c_1920x1440.png 848w, https://substackcdn.com/image/fetch/$s_!d0Vj!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F062dab63-e819-4c7d-846e-a17e8d097a6c_1920x1440.png 1272w, https://substackcdn.com/image/fetch/$s_!d0Vj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F062dab63-e819-4c7d-846e-a17e8d097a6c_1920x1440.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>We don&#8217;t know how many of these vehicles were operating without in-car human supervision. But it&#8217;s notable that Tesla now has 158 Cybercabs registered in the state, including 32 that were added in the last week. These vehicles have no pedal or steering wheel and are designed exclusively for driverless operation. So Tesla&#8217;s driverless fleet in Texas is at least 158 vehicles, and possibly significantly more than that. Tesla also has a growing number of driverless vehicles in Florida.</p><p>We don&#8217;t yet have good data on the safety record of these vehicles, and we shouldn&#8217;t draw any firm conclusions about Tesla&#8217;s safety record until we do. It&#8217;s certainly possible that Tesla Robotaxis will be involved in serious crashes in the coming months and Tesla will be forced to slow its expansion. But at a minimum, the decision to scale up driverless service seems like a credible signal that <em>Tesla</em> believes its technology is safe enough for large-scale deployment.</p><p>For its part, Zoox has a commercial robotaxi service in Las Vegas using its purpose-built vehicles with no steering wheels or pedals. However, it&#8217;s a point-to-point service that travels to a fixed list of destinations such as hotels and the airport. This is more limited than the general taxi services offered by Waymo and Tesla.</p><p>A couple of weeks ago, Techcrunch <a href="https://techcrunch.com/2026/09/17/amazon-owned-zooxs-100-robotaxi-limit-in-nevada-is-about-to-disappear/?utm_source=chatgpt.com">reported</a> that Zoox had about 100 of its custom-designed vehicles nationwide. That&#8217;s fewer driverless vehicles than Tesla and a <em>lot</em> fewer than Waymo, which has <a href="https://techcrunch.com/2026/09/24/waymo-is-scaling-fast-heres-what-the-fleet-data-shows/">around 4,000 vehicles</a>. And we don&#8217;t know how many of them are being actively used to provide rides to the public.</p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>My Zoox vehicle approached a portable road sign that was blocking the right lane. There were no cars nearby, so the vehicle (which is symmetrical from front to rear) went into reverse for about 100 feet to give itself room to change lanes and then continue forward in the next lane over. But an experienced human driver would have presumably seen the obstacle ahead and changed lanes before reaching it.</p></div></div>]]></content:encoded></item><item><title><![CDATA[OpenAI’s Astra model is shockingly good at robotics]]></title><description><![CDATA[Will general-purpose LLMs control future robots?]]></description><link>https://www.understandingrobots.org/p/openais-astra-model-is-shockingly</link><guid isPermaLink="false">https://www.understandingrobots.org/p/openais-astra-model-is-shockingly</guid><dc:creator><![CDATA[Kai Williams]]></dc:creator><pubDate>Thu, 01 Oct 2026 18:16:25 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!Z69u!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>On September 3, OpenAI <a href="https://openai.com/index/gpt-6-astra/">released</a> the first GPT-6 model: Astra. As usual, the company described its new release as the &#8220;world&#8217;s most intelligent and aligned model.&#8221; But OpenAI didn&#8217;t just list the customary coding and agentic benchmarks. It also <a href="https://x.com/OpenAI/status/2095595741528125780">claimed</a> that Astra was significantly better than previous LLMs at controlling computers. &#8220;Anything you can do on a computer, Astra can do for you. Fast.&#8221;</p><p>OpenAI released videos of Astra fluently completing varied computer tasks, from formatting legal documents in a word processor to navigating the Texas DMV&#8217;s website. Of these demos, one category stood out: 3D modeling. OpenAI showed models that Astra had made of a house, a circuit board, a video game world, and a car transmission.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!mLI-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4af25769-a469-4186-aad0-319197a20f1b_2048x1639.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!mLI-!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4af25769-a469-4186-aad0-319197a20f1b_2048x1639.png 424w, https://substackcdn.com/image/fetch/$s_!mLI-!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4af25769-a469-4186-aad0-319197a20f1b_2048x1639.png 848w, https://substackcdn.com/image/fetch/$s_!mLI-!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4af25769-a469-4186-aad0-319197a20f1b_2048x1639.png 1272w, https://substackcdn.com/image/fetch/$s_!mLI-!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4af25769-a469-4186-aad0-319197a20f1b_2048x1639.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!mLI-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4af25769-a469-4186-aad0-319197a20f1b_2048x1639.png" width="1456" height="1165" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/4af25769-a469-4186-aad0-319197a20f1b_2048x1639.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1165,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!mLI-!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4af25769-a469-4186-aad0-319197a20f1b_2048x1639.png 424w, https://substackcdn.com/image/fetch/$s_!mLI-!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4af25769-a469-4186-aad0-319197a20f1b_2048x1639.png 848w, https://substackcdn.com/image/fetch/$s_!mLI-!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4af25769-a469-4186-aad0-319197a20f1b_2048x1639.png 1272w, https://substackcdn.com/image/fetch/$s_!mLI-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4af25769-a469-4186-aad0-319197a20f1b_2048x1639.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">A model of a car&#8217;s transmission made by GPT-6 Astra in FreeCAD. (Image from OpenAI&#8217;s announcement)</figcaption></figure></div><p>This jump in spatial reasoning prompted a question: might Astra be good at robotics? The answer was a resounding yes. In a viral Twitter <a href="https://x.com/cdngdev/status/2097339677128982873">post</a> the following week, OpenAI robotics employee Thijs Simonian set up Astra with a cheap robot arm and a set of paintbrushes and asked it to paint the Golden Gate Bridge from a reference image. After five iterations, the results were astonishingly good.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Z69u!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Z69u!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png 424w, https://substackcdn.com/image/fetch/$s_!Z69u!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png 848w, https://substackcdn.com/image/fetch/$s_!Z69u!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png 1272w, https://substackcdn.com/image/fetch/$s_!Z69u!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Z69u!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png" width="1456" height="798" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/bdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:798,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Z69u!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png 424w, https://substackcdn.com/image/fetch/$s_!Z69u!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png 848w, https://substackcdn.com/image/fetch/$s_!Z69u!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png 1272w, https://substackcdn.com/image/fetch/$s_!Z69u!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbdf8a5c1-ef8d-4b78-9503-0960eaec2c15_2048x1122.png 1456w" sizes="100vw"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Astra&#8217;s five attempts to paint the Golden Gate Bridge. (Screenshot from Simonian&#8217;s <a href="https://x.com/cdngdev/status/2097339678206861789">tweet</a>)</figcaption></figure></div><p>Simonian wasn&#8217;t the only person to notice this skill. Since Astra&#8217;s release, the Internet has reverberated with <a href="https://github.com/zjwzcx/Awesome-Astra-Embodied-AI">demos</a> of the model completing various robotics tasks: Astra <a href="https://openai.robocurve.org/gpt-6-astra/">picking and placing</a> robot blocks, Astra <a href="https://www.rednote.com/discovery/item/6aa0c4900000000012034a2f?xsec_token=ABYB6HItIwwYq0Yyi9-hwoM-vMalkFRmYMAkN0iUWpTM4=&amp;xsec_source=pc_search&amp;source=web_profile_page">slicing</a> cucumbers, Astra <a href="https://x.com/ZeYanjie/status/2098118164626501669">solving</a> a Rubik&#8217;s Cube in simulation, even Astra <a href="https://drivingbench.com/">driving a car</a> in a parking lot.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><p>Astra is so good at controlling robots, in fact, that it was briefly the <a href="https://robodojo-benchmark.com/report/gpt-6-astra-eval">top-rated robot-control model on the RoboDojo Benchmark</a>, ahead of specialized robotics models like &#960;<sub>0.5</sub> from <a href="https://www.understandingai.org/p/how-google-taught-llms-to-control">Physical Intelligence</a> and MolmoAct2 from the Allen Institute for AI.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a> Unlike previous general LLMs, Astra seems good at controlling robots directly, specifying the exact positions that the gripper needs to navigate to complete a task, rather than just giving high-level instructions that must be carried out by a lower-level robotics model.</p><p>Astra&#8217;s capabilities have big caveats at the moment. For instance, it has to pause for seconds at a time to think about its next movement &#8212; far too slow to complete most robotics tasks. But Astra&#8217;s recent performance has been jarring to roboticists. &#8220;Currently it&#8217;s not very useful,&#8221; said Yale PhD student Haoxiang You. But the capabilities of the new OpenAI model have &#8220;kind of shocked people,&#8221; he said.</p><p>Some researchers think that OpenAI&#8217;s and Anthropic&#8217;s models might become good enough to completely supplant <a href="https://www.understandingai.org/p/how-google-taught-llms-to-control">specialized robotics models</a>. &#8220;I feel it&#8217;s totally possible to have one large model that does both very good reasoning and a very good action output,&#8221; ETH Zurich PhD student Chong Zhang told me.</p><p>Others are more skeptical. Astra will not &#8220;just take over everything&#8221; in robotics, Princeton PhD student Zeyu Shen said in an interview. Astra is too slow and it&#8217;s not reactive enough to sudden changes in the environment. Astra is also far too large to run on board a robot, so robots would probably still need a small local model in case the internet connection goes down.</p><p>Whether or not Astra-type models eventually control robots directly, they are useful for robotics development today. I expect roboticists to use Astra to generate new simulation environments and training data. All this might significantly accelerate robotics progress.</p><h1>How many LLMs does it take to control a robot?</h1><p>Using LLMs to control robots is not a new idea. In fact, it&#8217;s older than ChatGPT.</p><p>In April 2022, researchers at Google released a system called <a href="https://arxiv.org/abs/2204.01691">SayCan</a> that explored how LLMs could be applied to robotics. The researchers realized that LLMs could break complex robotics tasks into a list of small steps. For instance, an LLM could identify that in order to clean a spill a robot would first need to &#8220;obtain a rag&#8221; and &#8220;navigate to the spill.&#8221; The LLM&#8217;s broad knowledge of the world would help it make better decisions; for instance, it would know that a placemat would not work well as a rag.</p><p>However, there was no way at the time for LLMs to directly control the robot. Some other piece of software would need to turn the subtask &#8220;pick up a rag&#8221; into a series of robot movements that actually pick up a rag.</p><p>It turned out that this lower-level challenge of generating movement trajectories, sometimes called the dexterity problem, was the bigger bottleneck to solving many tasks in the real world. While LLMs seemed useful as a <a href="https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow">&#8220;system two&#8221;</a> planner, it wasn&#8217;t clear whether they could become a good &#8220;system one&#8221; that can dexterously control a robot.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-2" href="#footnote-2" target="_self">2</a></p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><p>The next year, Google researchers found an approach to do just that. The researchers took an existing LLM with vision capabilities and trained it further on 130,000 robot demonstrations to make what they called a vision-language-action (VLA) model. That is, they trained the system to take in a text prompt and an image from the robot&#8217;s cameras and output a sequence of numbers: the position and rotation of the gripper at the end of the robot&#8217;s arm. (See Tim Lee&#8217;s <a href="https://www.understandingai.org/p/how-google-taught-llms-to-control">VLA explainer</a> at Understanding AI for more info.)</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!weEa!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadcd4292-f8ce-4065-bb1a-bdac5e1fc92c_2048x1320.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!weEa!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadcd4292-f8ce-4065-bb1a-bdac5e1fc92c_2048x1320.png 424w, https://substackcdn.com/image/fetch/$s_!weEa!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadcd4292-f8ce-4065-bb1a-bdac5e1fc92c_2048x1320.png 848w, https://substackcdn.com/image/fetch/$s_!weEa!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadcd4292-f8ce-4065-bb1a-bdac5e1fc92c_2048x1320.png 1272w, https://substackcdn.com/image/fetch/$s_!weEa!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadcd4292-f8ce-4065-bb1a-bdac5e1fc92c_2048x1320.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!weEa!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadcd4292-f8ce-4065-bb1a-bdac5e1fc92c_2048x1320.png" width="1456" height="938" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/adcd4292-f8ce-4065-bb1a-bdac5e1fc92c_2048x1320.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:938,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!weEa!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadcd4292-f8ce-4065-bb1a-bdac5e1fc92c_2048x1320.png 424w, https://substackcdn.com/image/fetch/$s_!weEa!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadcd4292-f8ce-4065-bb1a-bdac5e1fc92c_2048x1320.png 848w, https://substackcdn.com/image/fetch/$s_!weEa!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadcd4292-f8ce-4065-bb1a-bdac5e1fc92c_2048x1320.png 1272w, https://substackcdn.com/image/fetch/$s_!weEa!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadcd4292-f8ce-4065-bb1a-bdac5e1fc92c_2048x1320.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption"> A robot controlled by Google&#8217;s VLA RT-2 grabbing a banana and moving it to the number 3. (Screenshot from <a href="https://robotics-transformer2.github.io/">RT-2 project page</a>)</figcaption></figure></div><p>The first VLA, RT-2, had several versions trained from LLMs of different sizes, ranging from five to 55 billion parameters. The largest version, which was the most capable, had to be run on four AI chips located outside of the robot. RT-2 displayed all sorts of emergent capabilities and generalized fairly well. But the largest version could only output a single command one to three times a second, far too slow many demanding manipulation tasks.</p><p>Later VLA releases have been smaller and faster. Physical Intelligence&#8217;s first VLA, <a href="https://www.pi.website/blog/pi0">&#960;<sub>0</sub></a>, takes 73 milliseconds to output a one-second chunk of actions &#8212; and only has 3.3 billion parameters. The largest VLAs widely in use today only have about 15 billion parameters, far fewer than the estimated trillions of parameters of a model like GPT-6 Astra.</p><p>As a result, the small LLMs that form the backbone of VLAs do not have the general robustness and adaptability of frontier LLMs like GPT-6 Astra.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-3" href="#footnote-3" target="_self">3</a> Because they are trained on copious real-world data, VLAs can create plausible action trajectories. But they can&#8217;t necessarily plan over long time horizons or generalize to new settings.</p><p>This is fine for many of the current tasks that robots do &#8212; moving totes or folding packages does not require sophisticated planning. But to create general-purpose robots, you need a model that can handle chaotic open-world scenarios.</p><h1>Robotics companies are trying to make their models generalize</h1><p>So far I&#8217;ve been discussing the possibility that a general-purpose model could become good enough at low-level actions to supplant specialized robot models. Another possibility is that today&#8217;s special-purpose robotics models will become more generalized over time. That&#8217;s the vision of startups like Generalist and Skild, both of which demonstrated models in August that could complete a task from a single human demonstration, a skill called in-context learning.</p><p>Both models also displayed some amount of physical intuition. For instance, when Generalist&#8217;s GEN-1.5 model was tasked to place a block in a bowl that was covered with a piece of paper, GEN-1.5 removed the piece of paper without being directly asked to.</p><p>This progress was exciting; Generalist&#8217;s <a href="https://www.youtube.com/watch?v=1cllCVK-9lo">announcement video</a> showed clips of employees freaking out when the model displayed some new generalization capability.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!NHa4!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9faf86c2-ce4f-4293-b59d-ee4a2ab80b73_1267x714.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!NHa4!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9faf86c2-ce4f-4293-b59d-ee4a2ab80b73_1267x714.png 424w, https://substackcdn.com/image/fetch/$s_!NHa4!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9faf86c2-ce4f-4293-b59d-ee4a2ab80b73_1267x714.png 848w, https://substackcdn.com/image/fetch/$s_!NHa4!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9faf86c2-ce4f-4293-b59d-ee4a2ab80b73_1267x714.png 1272w, https://substackcdn.com/image/fetch/$s_!NHa4!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9faf86c2-ce4f-4293-b59d-ee4a2ab80b73_1267x714.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!NHa4!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9faf86c2-ce4f-4293-b59d-ee4a2ab80b73_1267x714.png" width="1267" height="714" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/9faf86c2-ce4f-4293-b59d-ee4a2ab80b73_1267x714.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:714,&quot;width&quot;:1267,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!NHa4!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9faf86c2-ce4f-4293-b59d-ee4a2ab80b73_1267x714.png 424w, https://substackcdn.com/image/fetch/$s_!NHa4!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9faf86c2-ce4f-4293-b59d-ee4a2ab80b73_1267x714.png 848w, https://substackcdn.com/image/fetch/$s_!NHa4!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9faf86c2-ce4f-4293-b59d-ee4a2ab80b73_1267x714.png 1272w, https://substackcdn.com/image/fetch/$s_!NHa4!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9faf86c2-ce4f-4293-b59d-ee4a2ab80b73_1267x714.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Generalist employees freak out when GEN-1.5 successfully places a block in a bowl. The model had been fine-tuned to use a brush to put the block in the bowl. When only given a dustpan, GEN-1.5 improvised; it pushed the block into the dustpan and then emptied the dustpan into the bowl. (Screenshot from Generalist&#8217;s <a href="https://www.youtube.com/watch?v=1cllCVK-9lo">announcement video</a>)</figcaption></figure></div><p>To Generalist&#8217;s CEO Pete Florence, the result vindicated the company&#8217;s approach of combining LLM training with large amounts of physical data.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-4" href="#footnote-4" target="_self">4</a> He told me that for the past couple of years, &#8220;there&#8217;s been this feeling like, ah man, is all of the generalization just coming from the language model-y stuff and we&#8217;re just bolting on this very thin layer?&#8221; But the results from GEN-1.5 were &#8220;washing away&#8221; this concern. &#8220;It feels like we are indeed now getting real, very hard to ignore levels of generalization&#8221; from the physical data, he said.</p><p>Meanwhile, Anthropic has been investigating how good its flagship models were at controlling models.</p><p>In July, the company <a href="https://www.anthropic.com/research/claude-plays-robotics">found</a> that Claude was pretty good at writing Python scripts to control the robots. For instance, Claude was able to program a quadruped to walk slowly through a maze.</p><p>But Claude and its general-purpose rivals were not yet good at directly controlling robots. On Anthropic&#8217;s tests with <a href="https://libero-project.github.io/main.html">LIBERO</a>, a well-known benchmark of manipulation tasks using a robot arm, the best performing general LLM was <a href="https://www.understandingai.org/p/why-anthropic-believes-its-latest">Claude Mythos Preview</a>, which succeeded 5.5% of the time. Compare that to MolmoAct, a specialized VLA that scored 86%. General-purpose LLMs were even worse at controlling unstable humanoids or quadrupeds.</p><p>Nevertheless, Anthropic found that general models were getting better over time. &#8220;Newer models have made real gains in direct manipulation and high-level policy control across the humanoid and quadruped embodiments we tested,&#8221; they wrote.</p><p>And once an LLM can directly do a task, it comes with a good deal of generalization capabilities. The day after Generalist announced its new model, independent robotics evaluator Robocurve <a href="https://x.com/chooi_jeq/status/2090453418460790939">found</a> that Opus 5 could also put a block in a bowl covered by a piece of paper by directly controlling a robot &#8212; though it took seven minutes, rather than seven seconds for a specialized model.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><h1>Astra is impressive but not deployable</h1><p>GPT-6 Astra was released about two months after Anthropic&#8217;s report and it was substantially more capable at robotics than its predecessors.</p><p>Robocurve co-founder Jay Chooi told me that he saw a &#8220;huge jump&#8221; when Astra came out. On one task, which involved placing a block into a bowl, the success rate jumped from 5% for Fable 5 to 95% for Astra.</p><p>It wasn&#8217;t just Robocurve. The Twitter demos we saw in the intro showed that Astra could solve all manner of robotics tasks directly, including tasks that almost certainly weren&#8217;t in Astra&#8217;s training data.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-5" href="#footnote-5" target="_self">5</a></p><p>This has prompted speculation that general LLMs might soon become capable enough to control deployed robots on their own, without help from a smaller VLA-type model.</p><p>On September 7, MIT professor Phillip Isola released an <a href="https://web.mit.edu/phillipi/www/writing/robot-use-agents.html#fn2">essay</a> that argued that LLMs could become &#8220;robot-use agents.&#8221; In the same way that GPT and Claude have been trained to use <a href="https://www.understandingai.org/p/how-ai-agents-got-good-at-using-tools">tools</a> to control a computer directly, models may soon puppet robots directly. That would dramatically speed the pace and diffusion of robotics progress, Isola argued, since every upgrade of these models would improve the capabilities of many robots simultaneously.</p><p>For now, Astra is still far too slow and isn&#8217;t great at dealing with complicated hardware, like hands. But inference is getting much faster: Robocurve <a href="https://github.com/robocurve/llm-token-speed">found</a> that the maximum LLM token generation speed has been increasing by a factor two to seven times each year, so we could see VLA-type speeds from LLMs by the end of the decade.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-6" href="#footnote-6" target="_self">6</a> And the capabilities of LLMs in this area have been improving rapidly.</p><p>Yet even if those predictions hold up, I don&#8217;t think LLMs will completely replace system one models like VLAs, at least in the medium term. It seems more likely that LLMs like Astra serve as effective orchestrators &#8212; similar in spirit to SayCan &#8212; without controlling robots directly. This is for a couple of reasons.</p><p>First, developers will almost certainly want a robotics model directly on the robot in case the Internet goes down. Astra today is far too large to run in a GPU that can fit on a robot and OpenAI doesn&#8217;t seem likely to let its weights be downloaded directly.</p><p>&#8220;I have a very strong view based on industry experience that robot brains need to run on the edge. Most manufacturing facilities have very poor internet, and same thing for most warehouses and most of the places where you want to deploy robots,&#8221; Leif Jentoft, the Head of AI Solutions at <a href="https://standardbots.com/">Standard Bots</a>, told me.</p><p>One possibility is a hybrid scenario with a big, remote model and a small model running directly on the robot. If the network connection goes down, the onboard model may be able to complete its immediate task and go into a safe position until connectivity is restored.</p><p>Second, the abilities of current LLMs still seem to complement VLAs more than replace them. Astra apparently has enough spatial capabilities to brute-force some manipulation tasks, but it seems to lack the intuitive dexterous capabilities of specialist robotics models.</p><p>For instance, while Astra was briefly the top robotics model on the RoboDojo leaderboard, its performance was barely correlated with that of specialized robotics models. Astra excelled at physically basic tasks that required semantic knowledge, but it struggled at tasks that required precise coordination or complex movements. VLAs were comparatively better at precision tasks and worse at semantic ones.</p><p>So combining Astra with a VLA can get the best of both worlds. Researchers associated with the Chinese startup Galbot <a href="https://anonymous-report-421.github.io/public-website/?lang=en&amp;view=1">found</a> that a hybrid system combining Astra with the VLA &#960;<sub>0.5</sub> performed better on RoboDojo tasks than either Astra or &#960;<sub>0.5</sub> alone.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!BSVz!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3893816f-5bad-496f-a929-3f6f9b55f10a_1920x1440.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!BSVz!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3893816f-5bad-496f-a929-3f6f9b55f10a_1920x1440.png 424w, https://substackcdn.com/image/fetch/$s_!BSVz!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3893816f-5bad-496f-a929-3f6f9b55f10a_1920x1440.png 848w, https://substackcdn.com/image/fetch/$s_!BSVz!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3893816f-5bad-496f-a929-3f6f9b55f10a_1920x1440.png 1272w, https://substackcdn.com/image/fetch/$s_!BSVz!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3893816f-5bad-496f-a929-3f6f9b55f10a_1920x1440.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!BSVz!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3893816f-5bad-496f-a929-3f6f9b55f10a_1920x1440.png" width="1456" height="1092" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/3893816f-5bad-496f-a929-3f6f9b55f10a_1920x1440.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1092,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!BSVz!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3893816f-5bad-496f-a929-3f6f9b55f10a_1920x1440.png 424w, https://substackcdn.com/image/fetch/$s_!BSVz!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3893816f-5bad-496f-a929-3f6f9b55f10a_1920x1440.png 848w, https://substackcdn.com/image/fetch/$s_!BSVz!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3893816f-5bad-496f-a929-3f6f9b55f10a_1920x1440.png 1272w, https://substackcdn.com/image/fetch/$s_!BSVz!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3893816f-5bad-496f-a929-3f6f9b55f10a_1920x1440.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>This may change in the future. We know that OpenAI is investing heavily in robotics: it is <a href="https://robotsandstartups.substack.com/p/east-bay-becoming-home-to-physical">renting</a> a 202,400-square-foot facility for robotics operations in the Bay Area and has &#8220;robotic workcells used in ongoing data acquisition operations,&#8221; according to a recent <a href="https://openai.com/careers/prototyping-lab-technician-robotics-san-francisco/">job posting</a>. Future versions of Astra might be trained on enough robotics data to be able to complete tasks that VLAs currently do.</p><p>But I won&#8217;t be surprised if we continue to see a hierarchical division of labor. After all, the brain has its separations: the prefrontal cortex plays a major role in high-level planning, while the cerebellum helps coordinate precise movements. And it would follow a precedent set by Google. The most recent <a href="https://deepmind.google/models/gemini-robotics/">Gemini Robotics release</a> included three models: two VLAs and a higher-level &#8220;embodied reasoning model&#8221; based on Gemini 3.5 Flash.</p><h1>Astra is helpful for other reasons</h1><p>It&#8217;s also possible that general LLMs don&#8217;t ever really end up controlling robots in production, at least not directly. There are other robotics paradigms, like world models, which don&#8217;t rely as heavily on text-based capabilities. Even if that happens, though, Astra might still end up helping train the next generation of robot models by creating valuable training data.</p><p>As I <a href="https://www.understandingai.org/p/robot-startups-are-trying-everything">covered in September</a> in Understanding AI, robotics companies are desperate for data to train effective robots.</p><p>One strategy is to collect millions of demonstrations of a human piloting a robot to complete a task. This is easy to train into a model, but it&#8217;s very expensive to collect.</p><p>In contrast, training robots in a computer simulation is much cheaper and easier to scale &#8212; provided you already have the environments. Developers can train the robot on programmatically generated trajectories in the simulation engine or let the robot train itself by <a href="https://www.understandingai.org/p/reinforcement-learning-explained">trial and error</a>.</p><p>However, it is still very expensive to make new simulated environments to train a robot in. Historically, simulations required substantial manual work where humans had to make assets for the simulation and specify physical properties such as the friction of each object. There have been some approaches to making assets programmatically, but it&#8217;s difficult to make a completely new environment from scratch, especially one which mirrors a real-world setting.</p><p>Astra&#8217;s uncanny ability to create 3D models might be enough to create the millions of simulated worlds necessary to train robots.</p><p>And Astra&#8217;s strong ability to solve robotics tasks &#8212; either directly as we saw above or using Python to code a robotic policy &#8212; could help create data for training VLAs. I talked with a group of researchers who released a paper called <a href="https://embodiedswe.github.io/">EmbodiedSWE</a>. They made a benchmark of difficult everyday tasks and found that coding models could write Python code that solved these tasks (VLAs struggled at this). From trajectories coded by Opus 5, they were able to fine-tune &#960;<sub>0.5</sub> to unscrew a lightbulb in a real lamp in two out of ten trials.</p><p>Waymo started as a deterministic software system. This enabled Waymo to build up the expertise and training data it needed to eventually train a model to drive the car instead. Perhaps Astra will enable future robotics projects to speed-run through this same process in other robotics domains.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>This comparison is probably too generous to GPT-6 Astra, for a couple of reasons (some of which I&#8217;ll cover later). First, many of the best robotics policies from <a href="https://www.understandingai.org/p/19-robotics-companies-to-watch">top startups</a> aren&#8217;t available for evaluators to study &#8212; I wouldn&#8217;t be surprised if e.g., <a href="https://www.pi.website/blog/pi07">&#960;<sub>0.7</sub></a> would perform better. Second, Astra&#8217;s official benchmark result was restricted to simulated environments because real-world testing stopped after it damaged some of RoboDojo&#8217;s hardware. Third, Astra&#8217;s performance has since been surpassed by specialized robotics models: it is currently seventh in the standings, though the top model is an LLM agent with access to &#960;<sub>0.5</sub>.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-2" href="#footnote-anchor-2" class="footnote-number" contenteditable="false" target="_self">2</a><div class="footnote-content"><p>The software controlling robots <a href="https://newsletter.semianalysis.com/i/215051663/layers-of-a-robot-model">is</a> often <a href="https://generalrobots.substack.com/p/so-you-want-to-do-robots-part-2-what/comments">viewed</a> as having a hierarchy of response times. Planning is a high-level task which doesn&#8217;t require quick responses. Dexterity is somewhere in the middle &#8212; the ideal command rate is somewhere between one to a hundred times a second. Below that is the software that takes joint positions and turns it into actual motor commands. That software needs to be running hundreds to thousands of times a second, and is sometimes referred to as &#8220;system zero.&#8221;</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-3" href="#footnote-anchor-3" class="footnote-number" contenteditable="false" target="_self">3</a><div class="footnote-content"><p>This is true also of other model architectures which fulfill the same role as a VLA, like world action models. They still have a relatively small number of parameters in order to be efficient to run in real time.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-4" href="#footnote-anchor-4" class="footnote-number" contenteditable="false" target="_self">4</a><div class="footnote-content"><p>Unlike RT-2 or Physical Intelligence&#8217;s models, Generalist doesn&#8217;t call its models VLAs, and mostly pre-trains the models from scratch. But there&#8217;s almost certainly text and web data alongside the hundreds of thousands of hours the company has collected of <a href="https://www.understandingai.org/p/robot-startups-are-trying-everything">UMI data</a>.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-5" href="#footnote-anchor-5" class="footnote-number" contenteditable="false" target="_self">5</a><div class="footnote-content"><p>Though Twitter does have some selection bias. Shen, the Princeton PhD student, told me that his advisor had tried to use Astra to fold a shirt overnight, but the robot had been very slow, balled the T-shirt up instead of folding it, and moved the robot&#8217;s arms to &#8220;pathological states.&#8221; That experiment was not reported on Twitter, however.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-6" href="#footnote-anchor-6" class="footnote-number" contenteditable="false" target="_self">6</a><div class="footnote-content"><p>Or even sooner. Robocurve found that the max token generation speeds of models that do as well as Fable on the Artificial Analysis benchmark was doubling every month. If that pace held up, LLMs could catch up to the current VLA pace within three to four months, Chooi said. I&#8217;m skeptical that pace can hold, though.</p></div></div>]]></content:encoded></item><item><title><![CDATA[Let’s learn about robots together]]></title><description><![CDATA[Introducing Understanding Robots.]]></description><link>https://www.understandingrobots.org/p/lets-learn-about-robots-together</link><guid isPermaLink="false">https://www.understandingrobots.org/p/lets-learn-about-robots-together</guid><dc:creator><![CDATA[Kai Williams]]></dc:creator><pubDate>Thu, 01 Oct 2026 17:07:16 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!jgIB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Back in 2025, when I was doing technical AI safety research, I heard a refrain over and over. &#8220;Robotics is getting much better.&#8221; But I was never quite clear on what that meant.</p><p>We&#8217;ve all seen AI get much better, very quickly. In the past four years, LLMs have gone from the engaging curiosity that was ChatGPT to agentic systems that have reinvented entire professions and might solve some of humanity&#8217;s grand intellectual challenges. </p><p>And the whole time, people have been tracking this through benchmarks, revenue figures, and new capabilities. If you understood the scaling laws in 2020, it was almost even possible to see into the future and predict what future systems might bring.</p><p>Despite its physical nature, robotics is much harder to understand and track. As one interviewee told me, robotics is &#8220;fifteen fields in a trench coat.&#8221; To build a good robot, a company must figure out AI development, software, hardware, operations, regulatory barriers. The list goes on. </p><p>It takes a lot to understand the current state of robotics. Over the past 10 months, I&#8217;ve traveled to New York, San Francisco and even China to see robots and learn more about the industry. I&#8217;ve talked with dozens of roboticists and, alongside Timothy B. Lee, I wrote a <a href="https://www.understandingai.org/p/i-spent-4000-on-a-robot-dog-from">five</a> <a href="https://www.understandingai.org/p/why-humanoid-robots-wont-catch-up">part</a> <a href="https://www.understandingai.org/p/how-google-taught-llms-to-control">series</a> <a href="https://www.understandingai.org/p/robot-startups-are-trying-everything">on</a> <a href="https://www.understandingai.org/p/19-robotics-companies-to-watch">robotics</a> at Understanding AI in September.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!jgIB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!jgIB!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg 424w, https://substackcdn.com/image/fetch/$s_!jgIB!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg 848w, https://substackcdn.com/image/fetch/$s_!jgIB!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!jgIB!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!jgIB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg" width="1456" height="886" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:886,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:336293,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.understandingrobots.org/i/218368108?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!jgIB!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg 424w, https://substackcdn.com/image/fetch/$s_!jgIB!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg 848w, https://substackcdn.com/image/fetch/$s_!jgIB!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!jgIB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54d368bc-9377-4738-a00f-083eb5834ea1_2834x1724.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Those AI people I talked with are right. Robotics is getting better. On Twitter every day, there&#8217;s a steady drumbeat of announcements. New companies. New fundraising. New capabilities. Over the past five years, we&#8217;ve seen the birth and explosion in humanoid robots, new types of AI models controlling robots, even new contenders in the nominally boring bits of the actuator supply chain.</p><p>But robotics also deals with the real world, and interacting with the real world is very difficult. It&#8217;s taken 15 years for self-driving cars to go from promising demos to mid-scale deployments. I wouldn&#8217;t be surprised if the same happens for robots.</p><p>So, alongside Tim, my editor at Understanding AI, I&#8217;m starting a part-time newsletter to track robotics in a more focused space. I won&#8217;t cover every new development in robotics; nor will I necessarily write about the big story of the week. But I aim, once a week or so, to write a story about something that&#8217;s happening in the intersection of robotics, AI, and daily life, and explain why it matters in understandable language.</p><p>If that sounds interesting, please click the button below and subscribe to Understanding Robots!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.understandingrobots.org/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.understandingrobots.org/subscribe?"><span>Subscribe now</span></a></p><p></p>]]></content:encoded></item></channel></rss>