Robotics for software engineers
👋 Hi, this is Gergely with a subscriber-only issue of the Pragmatic Engineer Newsletter. In every issue, I cover challenges at Big Tech and startups through the lens of engineering managers and senior engineers. If you’ve been forwarded this email, you can subscribe here. Robotics for software engineersWhat does it take to build and program robots? A look into the exciting, increasingly popular field of robotics. Guest post by humanoid robot expert, Sandor Felber
Today, there’s an ever-growing number of startups, scaleups, and established companies like Boston Dynamics producing increasingly capable robots, including ones that look humanoid. Tech giants like Tesla have been building humanoid robots, and Meta is expected to invest billions in the technology. These and other factors point to the possibility that, after artificial intelligence, robotics could be the “next big thing” in tech. But what’s the process of building intelligent robots, and what’s it like as a day-to-day job for the developers who do? It’s not always easy to find this out because robotics startups are famously shrouded in secrecy – and with such cutting-edge technology, it’s unsurprising. Academia is often a bit more open, so I turned to MIT Robot Learning Researcher, Sandor Felber. He’s a Humanoid Robot Learning Researcher at MIT in Boston, Massachusetts, and previously led a team that built a self-driving race car. Sandor’s also been a robotics intern at Tesla in Palo Alto, California, and a robotics R&D engineer at the Edinburgh Center of Robotics. Today, he takes us through:
This topic is intriguing because it combines software, hardware, and cutting-edge tech:
This is a guest post. If you’re interested in writing a deepdive for The Pragmatic Engineer – collaborating with Gergely on it – check here for more details. The bottom of this article could be cut off in some email clients. Read the full article uninterrupted, online. With that, it’s over to Sandor. You can follow Sandor on LinkedIn, and learn more about his work on his website. 1. Getting into robotics, a personal accountMy interest in robotics began in high school, where I wrote the junior equivalent of a dissertation on characterization methods for how electric motors behave under different conditions. This led me to pursue a degree in electrical and mechanical engineering at the University of Edinburgh, home to numerous renowned robotics and AI researchers. Building a driverless electric race car was one of my bigger projects. At uni, I joined the Edinburgh University Formula Student (EUFS) team, where we designed, built, and raced a driverless electric race car. From a roboticist’s perspective, this is considered a wheeled mobile robot.
We designed and built several versions of the car. Here’s a later model:
I started working in the electric powertrain team and was responsible for designing and implementing systems that generate and deliver power. It included:
Here’s the high-voltage battery pack I had a hand in designing:
I eventually became the powertrain team’s lead, and upon returning from Tesla the following year, moved on to direct all operations related to the driverless vehicle’s hardware design, and later served as president of the team of around 140 members.
The students in the team worked on the project part-time, and everyone went above and beyond their academic requirements to get hands-on experience. It was a student-led project – and an especially cool one; we built race cars that drove on actual Formula 1 tracks!
Along the way, I discovered a passion for control theory. Control systems engineers tend to concern themselves with crafting control strategies that ensure optimal performance, from spacecraft trajectories to insulin delivery in diabetics. In robotics, you could think of control theory as the invisible puppeteer of a robotic arm, except that instead of pulling strings, it's using mathematics to orchestrate every joint's motion in real time. As a robotic arm needs to smoothly pick up an egg without crushing it, control theory provides the mathematical "muscle memory" that turns crude motor commands into precise, graceful movements. It does this with constant sensor feedback and adjustments, with the approach adjusted depending on how sensitive the control system is to the various feedback signals. My interest in control theory and background in batteries, powertrains (vehicle motors), and electrical control systems from previous internships landed me one at Tesla's robotics department. Since then, I’ve worked with academic and industrial stakeholders on projects ranging from quadruped (quad- as in four, and -ped as in legged) dog-like robots, to humanoid systems. I’m currently at MIT’s Computer Science and Artificial Intelligence Lab (CSAIL), focusing on teaching humanoid robots to perform complex, real-world tasks using learned controllers, powered by neural networks. I’m particularly interested in embedding intelligence into robots by leveraging various types of learning, including supervised and unsupervised learning, offline and online deep reinforcement learning, and imitation learning. I use these approaches for humanoid robot locomotion, manipulation, whole-body control, and teleoperation (remotely controlling the robot). 2. Robotics industry overviewRobotics companies are raising more money than ever before. In the first eight months of 2024, global robotics companies attracted a record amount of $10.8 billion in funding, averaging $1.2 billion per month! What’s all the hype fuelling this surge of capital? In public debate, there’s no shortage of focus on labor shortages and stagnating productivity, which reduce growth and competitiveness. The US economy is estimated to be missing up to 4.6 million workers annually needed to maintain levels of supply and demand, according to a recent study. That number equates to 2% of the current US population. The same study suggests Germany needs to find an additional 1.6 million workers to keep current economic levels; 3% of its population. This is one driver of industrial robots’ increased adoption, which shows no sign of slowing down. Sustaining economic growth amid shrinking labor pools is one viable path for robotics which is getting a lot of attention, as evidenced in the increased adoption of industrial robots, with global installations reaching nearly 600,000 units in 2024, as per the World Robotics Report. This figure surpassed previous benchmarks and shows that industries struggling with labor shortages can return to growth.
If robots become ubiquitous, what will humans do? This is the most common concern I hear when I talk about my work. The question is an old one: on the New York subway, I stumbled upon a Holiday Nostalgia Ride, which offers a trip in an old subway car from between the 1930s and 1970s. My coach looked like it was from the 1960s, and inside was an advert about upskilling for “tomorrow’s jobs:”
The world has changed massively since then, thanks in part to automation, without which our standards of comfort wouldn’t be possible or affordable; think of the dishwasher that automates the washing up and helps canteens to offer a cheaper menu, than if they had to hire extra people to wash dishes and pay for excess water (new dishwashers are very efficient). Acquiring new skills in a quickly-changing world is as least as necessary today as it was back then; as illustrated by that subway ad from the mid-20th century. Industry vs academiaApproaches to research and development have always differed between industry and academia. Having worked in both environments, here’s how I see them compare in robotics: Academia: smaller “long-shot” bets, developed on a budget. Many projects take years to mature, due to the limited effort that a couple of post-docs, PhD students, and undergrads can dedicate. Industry: execute on proven concepts with substantial backing. Industry prioritizes execution on feasible concepts. Once a concept is assumed to work – as usually verified by a proof-of-concept (POC), also known as a “minimum viable product” in startup lingo – industry players can raise vast amounts of money to build it. Modern robotics may have reached an inflection point, where enough academic “long-shot” bets are delivering results, with feasible paths to building practical robots, including humanoid ones. Examples of bets made on robotic hardware:
3. Planning a robotics projectHere’s how I’ve seen robotics projects get done. Vision for a demo: many robotics projects start with the question: “What should the robot achieve when completed?” Apple co-founder Steve Wozniak’s “coffee test” is one such vision, now frequently referred to as the “New Turing Test.” His definition:
In practice, on most projects I’ve seen, the vision gets watered down or descoped as deadlines approach, and material or human resources become limiting factors. However a demo that passes the test above could well serve as a benchmark for artificial general intelligence (AGI). As part of that test, a physically-embodied machine must demonstrate intelligence by entering a random home and successfully making a cup of coffee with tools available in a previously unseen kitchen. After the vision is set, the next steps are: Planning: goals, demoable outcomes, and target applications or target environments. The requirements of a good demo in a target environment can be very high! Break things down. Translating deliverables into functional requirements. For each functional requirement, one or more features that show successful fulfillment need to be identified. When breaking things down, features are often separated into at least two subsets:
Critical vs nice-to-have thresholds can change over time. Hard constraints (“critical”) vs soft ones (“nice-to-have”) are often linked, and the difference lies in arbitrary measures of quantity and quality. Being clear about the “minimum needed” (critical) and “nice-to-have” requirement thresholds is good practice in robotics projects. These can change as the project develops, turning nice-to-haves into critical ones. As long as everyone on the project is notified and on-board when changes happen, then a project should progress without hiccups. 4. Development phaseDuring development, robotics engineers need to balance proof-of-concept experiments with the scaling of a project, including the supporting infrastructure. For example, one proof-of-concept experiment could be verifying that a parallel jaw gripper (the robot’s hand) can grasp a mug in specific setups. Scaling can be hard because there are lots of problems in the real world to solve. Scaling the above example of holding a mug can be seen as:... Subscribe to The Pragmatic Engineer to unlock the rest.Become a paying subscriber of The Pragmatic Engineer to get access to this post and other subscriber-only content. A subscription gets you:
|
Comments
Post a Comment