Armstrong Robotics wants to create general purpose kitchen robots, starting with dishwashing

At each full-service restaurant in the world, dishwashers spend hours every day cleaning hundreds of dishes to keep things running. Armstrong Robotics Inc. said it plans to use robotics and artificial intelligence to free these workers for more customer-facing tasks.

When the San Francisco-based startup first launched, its team was drawn to the challenges and opportunities that kitchen environments presented for robotics, said Axel Hansen, co-founder and CEO of Armstrong. The company’s system includes three off-the-shelf, seven degree-of-freedom (DoF) robot arms, each with a parallel access gripper on the end.

“We just kept coming back to kitchens and restaurants as this really interesting space, and especially dishwashing within that problem,” Hansen told The Robot Report. “Dishwashing is universal. There are a million restaurants in the U.S. Also, every household is dealing with dishwashing. No one likes doing it. Restaurants are really struggling to keep those roles filled. It costs them a lot of money.”

While dishwashing is a difficult task for robotics, Hansen asserted that simpler kitchen tasks, like frying or flipping burgers, provide less value to restaurants if that’s all the robot can do.

“There are very few restaurants that are flipping enough burgers or doing enough French fries to justify the kind of technology investment that we’re talking about,” he said. “In a full-service restaurant, dishwashing is a lot of work. It took us a long time. For the minimal viable product, in robotics, the bar is very high.”

Now, with a fresh $12 million funding round under its belt, Armstrong is ready to start ramping up deployments of its dishwashing robot.

“With five years of R&D that we’ve done, we’re able to really nail it,” Hansen said. “We’re now operating live in multiple restaurants, 24 hours a day, washing thousands of dishes autonomously.”

Perception is key to handling dishes

Working with dirty dishes presents a number of unique challenges for a robot. It must handle shiny plates, transparent glasses, and any number of substances on top of the dishes.

“We spent two and a half years just on the perception problem. How can we detect where dishes are in space, down to millimeter accuracy?” Hansen explained. “We’re dealing with glass, shiny metal, so we can’t just throw a RealSense [camera] in there.”

“We have 30 sensors in our robot and a neural network that we’ve trained with millions of examples of dishes that we’ve captured, either manually or now in the field,” he added.

Once it tackled the perception problem, Armstrong Robotics moved to figuring out how to actually get a grip on these dishes.

The right grip is all in the fingers

“There’s a lot of interest in humanoid-style hands,” Hansen said. “I would love to go that direction, but the cost and robustness are just not there for a commercial kitchen right now. We’ll pick cups out of a bus tub that’s literally full of spilled Gatorade, coffee, and water or whatever. So we’re literally going right into a tub of liquid.”

“We actually designed custom fingers for different situations. One of the hardest things to deal with is plates that are in a stack,” he continued. “It’s a very dynamic motion of how you actually peel a plate off of that. And of course, these plates aren’t clean. They’re dirty. They’ve got grease, one of the most challenging is any kind of butter or oil, which just makes them super slippery. We have a certain type of finger that enables a sort of thumb to slip in between the plates.”

Armstrong Robotics also carefully picked materials that could hold onto a greasy plate but also allow the arm to move fast enough to wash hundreds of dishes an hour.

“We have about a dozen different fingers that we’re able to choose between,” Hansen said. “For silverware, we have these magnet fingers that are actually able to magnetically pick up whole bunches of silverware at once. It’s been a really effective method for us, because these fingers are really cheap. We replace them on a monthly or every-other-month basis.”

“On the software side, the machine learned perception side gives us the ability to handle these dynamic situations, and the robot switches its fingers autonomously,” Hansen explained. “It sees plates in a stack, that’s one type of finger. If it sees a cup that’s on its side, that’s a different type of finger. When we have to lift up an empty rack, that’s a pretty heavy object, so we use a special type of finger for that.”

Armstrong said it plans to tackle the dishwashing vertical and then layer new tasks on top. | Source: Armstrong Robotics

Armstrong makes improvements based on deployments so far

Armstrong’s robots have been active for over a year and a half. The company said it has made a lot of improvements in that time.

“With most robots in the wild, I think the most important question to ask is: ‘Is someone babysitting the robot?’” Hansen said. When they first deployed, Armstrong took a much more hands-on approach.

“We basically had a role that we called an ambassador,” Hansen said. “It’s an employee of Armstrong, and they’re there to just fix problems with the robot.”

“The way that we really thought about that is, the restaurant only has so much patience for dealing with problems, and you’ll never get the problems down to zero,” he noted. “There’s a line when the robot has a rare enough issue that the restaurant would be willing to deal with it. A year and a half ago, we were clearly not there. We weren’t even running 24 hours a day. We started with a little mini shift in the afternoon. As we improved it, we expanded the hours of the robot.”

Now, however, Armstrong’s robot comes with a screen that alerts restaurant staffers to any issues, such as whether it ran out of dishes or something went wrong. The screen explains to staff how to fix the issue and keep operations running smoothly.

“We just installed our second [system] six months ago, it went immediately to 24 hours, and that one operates without any Armstrong ambassador, so it is fully independent. It’s just restaurant staff that deal with it,” Hansen said.

“I think the thing that has wowed me is the development of the speed of the robot from day one to where we are now,” Gary Kagan, chief operating officer at Armstrong Robotics, told The Robot Report. “Really, it just goes to the power of real-world data and utilizing that. We’re labeling so many images on a daily basis. We’re getting over 100,000 images from the dish rooms. Those are different angles, and whether it’s dry egg or a lot of ranch [dressing, they’re] all different, unique edge cases.”

“We have an incredible partner who helped us get there with our customer,” Kagan added. “We’re not going to say which customer it is, but it’s a large chain that we’re super happy to work with. Getting to 24/7 reliability required patience. From the nuances of the pads that we use to the finger strength, all of those things came into play to get to that level of reliability.”

Armstrong’s system uses AI to detect dirty dishes and determine the best ways to handle them. | Source: Armstrong Robotics

Dishwashing robots work around the clock

Armstrong Robotics aims to make its robots easy to install and out of the way of workers.

“We’re really proud of the installations we’ve done so far, because the restaurant has not shut down for a single minute as we were installing,” Kagan said. “That was really important, especially with our customer. They’re operating 24 hours. They literally are serving people at all times. So that was tricky, and it was something that we really had to be thoughtful with.”

“In terms of the install process, we look at it right now as a multi-day event,” Kagan said. “On Day 1, we just want to get the hardware in. Then, on Day 2, we make sure the software is aligned. Then we go.”

Armstrong tried to keep its maintenance schedule as inconspicuous for the restaurant as possible.

“There are really three companies that rent dish machines and provide chemicals. So the way they service restaurants and hospitality groups is they have somebody come in once a month to do a maintenance inspection, and they provide a two-hour guarantee,” Kagan explained. “If something happens with the dish machine, they have a 1-800 number, they call it, and somebody comes.”

“We’re providing the exact same thing. That way, we’re not recreating the wheel,” he said. “Once a month, we’re going to have somebody come in, inspect everything, make sure everything is looking good, and then if there is a situation, the restaurant has that number, and we’re going to have a staff person fix the problem.”

Armstrong’s latest funding round included participation from Lerer Hippeau, Bloomberg Beta, Next Play Ventures, Transmedia Capital, and WestWave Capital. | Source: Armstrong Robotics

Automation helps on the late shift

“We actually had this story from the restaurant owner. It was late on a Saturday night, and it was looking quiet, so they actually sent most of their staff home early,” Hansen said. “They had only two people, cooking, serving, cleaning, and doing everything.”

“Suddenly, the restaurant got a swarm of people totally unexpectedly. There must have been some concert or something nearby,” he said. “It would have been a situation where they would have just been completely KO’ed before. The owner was just telling us how amazing it was to see these two people keep this whole restaurant running. The dish room was running, and they didn’t run out of dishes because of the robot.”

The arms do not get in the way of the dish machine, and if restaurant workers need to do some hands-on dish work, they can simply press a button, and the arms will move up and out of the way. So far, restaurant staffers have been warming up to their new robotic co-worker.

“I remember the first time we ever installed, and the dishwasher looked at the robot and was like, ‘Oh my gosh. What does this mean for me?’ Honestly, he was worried,” Kagan recalled. “By the end of the second month, he called Armstrong his ‘little assistant,’ and we love that. He was then elevated to prep, and he did pots and pans. [The robot] elevated his role, he’s making more money, and he was in front of customers. It was something that was really, really exciting to see.”

Armstrong focuses on deployment before tackling new tasks

With fresh funding secured, Armstrong Robotics said it is focusing on deploying more robots and eventually expanding to new kitchen tasks.

“We have a product that the restaurant wants,” Hansen said. “All of our early customers are deploying additional robots, which for us is the most important metric. So it’s really about the deployment rate for us, and trying to just keep this flywheel going. The more data we get, the more time we’re able to spend optimizing the robot, the faster it is. The goal is, by the summer, to be installing one robot per month. That’s really our new deployment metric.”

“We’re being very thoughtful about who we work with,” added Kagan. “We’re looking for scalability, we’re looking for density, and we’re really looking for the right team. There are some incredible restaurateurs who are early adopters and are pushing us to get this into their dish room.”

When it comes to adding more tasks to the robot, Armstrong said it’s working with its customers to find the highest-value tasks to start with.

“Our vision is general-purpose robots starting with dishwashing,” Kagan said. “We want to solve a really hard problem that is frustrating to a lot of our customers, and then keep layering on other tasks. So our customers right now are asking us about frying. They’re asking us about silverware sorting and wrapping. They’re even asking us if we can clean the bathroom. So there are all of these tasks that people want someone else to do.”