For most people, the task of identifying an object, picking it up, and placing it somewhere else is trivial. For robots, it requires the latest in machine intelligence and robotic manipulation.

That’s what MIT spinoff RightHand Robotics has incorporated into its robotic piece-picking systems, which combine unique gripper designs with artificial intelligence and machine vision to help companies sort products and get orders out the door.

“If you buy something at the store, you push the cart down the aisle and pick it yourself. When you order online, there is an equivalent operation inside a fulfillment center,” says RightHand Robotics co-founder Lael Odhner ’04, SM ’06, PhD ’09. “The retailer typically needs to pick up single items, run them through a scanner, and put them into a sorter or conveyor belt to complete the order. It sounds easy until you imagine tens of thousands of orders a day and more than 100,000 unique products stored in a facility the size of 10 or 20 football fields, with the delivery expectation clock ticking.”

RightHand Robotics is helping companies respond to two broad trends that have transformed retail operations. One is the explosion of e-commerce, which only accelerated during the Covid-19 pandemic. The other is a shift to just-in-time inventory fulfillment, in which pharmacies, grocery stores, and apparel companies restock items based on what’s been purchased that day or week to improve efficiency.

The robot fleet also collects data that help RightHand Robotics improve its system over time and enable it to learn new skills, such as more gentle or precise placement. Process and performance data feed into the company’s fleet management software, which can help customers understand how their inventory moves through the warehouse and identify bottlenecks or quality problems.

“The idea is that rather than looking at just the performance of a single operation, e-commerce firms can modify or overhaul the operational flow throughout the warehouse,” Odhner says. “The goal is to eliminate variability as far upstream as is feasible, making a simpler, streamlined process.”

Pushing the limit

Odhner completed his PhD in the lab of Harry Asada, MIT’s Ford Professor of Engineering in the Department of Mechanical Engineering, who Odhner says encouraged students to develop a broad familiarity with robotics research. Colleagues also frequently shared their work in seminars, giving Odhner a well-rounded view of the field.

“Asada is a very well-known robotics researcher, and his early work, as well as the projects I worked on with him, are very much fundamental to what we’re doing at RightHand Robotics,” Odhner says.

In 2009, Odhner was part of the winning team in the DARPA Autonomous Robotic and Manipulation Challenge. Many of the competing teams had MIT connections, and the entire program was eventually run by former MIT associate professor Gill Pratt. After making the semifinals of the MIT 100K competition in 2013 as “Manus Robotics,” the team was introduced to Mick Mountz ’87, founder of Kiva Systems (later acquired by Amazon), who encouraged the team to look at applications in supply chain and logistics.

Today, a significant amount of RightHand Robotics employees and leadership come from MIT. MIT researchers also accounted for many early customers, buying components Odhner’s team had invented during the DARPA program.

“Generally, we’ve been in such close proximity to MIT that it’s hard to avoid circling back there,” Odhner says. “It’s kind of a family. You don’t ever really leave MIT.”

At the core of the RightH and Robotics solution is the idea of using machine vision and intelligent grippers to make piece-picking robots more adaptable. The combination also limits the amount of training needed to run the robots, equipping each machine with what the company equates to hand-eye coordination.

“The technical part of what we do is we have to look at an unstructured presentation of consumer goods and semantically understand what’s in there,” Odhner says.

RightHand Robotics also utilizes an end-of-arm tool that combines suction with novel underactuated fingers, which Odhner says gives the robots more flexibility than robots relying solely on suction cups or simple pinching grippers.

“Sometimes it actually helps you to have passive degrees of freedom in your hand, passive motions that it can make and can’t actively control,” Odhner says of the robots. “Very often those simplify the control task. They take problems from being heavily over-constrained and make them tractable to run through a motion planning algorithm.”

The data the robots collect are also used to improve reliability over time and shed light on warehouse operations for customers.

“We can give people insights into their inventory, insights into how they’re storing their inventory, how they’re structuring tasks both upstream and downstream of any picking we’re doing,” Odhner says. “We have very good insight as to what may be a source of future problems, and we can feed that back to customers.”

Odhner notes that warehouse fulfillment could grow to be a much larger industry if throughput were improved.

“As consumers increasingly value the option of shopping online, more and more items need to get into a growing number of ‘virtual’ carts. The availability of people near order fulfillment centers tends to be a limiting factor for e-commerce growth. All of that is really indicative of a massive economic inefficiency, and that’s essentially what we’re trying to address,” Odhner says. “We are taking the least engaging tasks in the warehouse — things like sorter induction, where you’re just picking, scanning, and putting something on a belt all day long — and we’re working to automate those tasks to the point where you can take your people and you can direct them to things that are going to be more directly felt by the customer.”

Odhner also says more automated fulfillment centers offer improved measures to protect worker health and safety, such as ergonomic stations where goods are brought to workers for specialized tasks and increased social distancing. Rather than reducing the number of people employed in a warehouse, he says, “Ultimately, what you want is a system with people working in roles like quality control, overseeing the robots.”

Robots made easy

This year, the company is introducing the third version of its picking robot, which ships with standardized integration and safety features in an attempt to make deploying piece-picking robots easier for warehouse operators.

“People may not necessarily grasp the enormity of our progress in productizing this autonomous system, in terms of ease of integration, configuration, safety, and reliability, but it is huge because it means that our robot systems can be drop-shipped pretty much worldwide and get up and running with minimal customization,” Odhner says. “There is no reason why this can’t just come in a box or on a pallet and be set up by anyone. That’s our big vision.”