AI-powered humanoid robot can serve you food, stack the dishes — and have a conversation with you

 Might be helpful to your living alone after your stroke, assuming it doesn't become like the robot in the movie; 'Robot and Frank'.

I really prefer the socially assistive robot in the movie 'Robot and Frank'.

AI-powered humanoid robot can serve you food, stack the dishes — and have a conversation with you

Figure 01 learned how to make coffee by watching a human do it, and now it can speak to you like a person.

In the new promotional video, a technician asks Figure 01 to perform a range of simple tasks in a minimalist test environment resembling a kitchen. (Image credit: Figure)

A self-correcting humanoid robot that learned to make a cup of coffee just by watching footage of a human doing it can now answer questions thanks to an integration with OpenAI's technology.

In the new promotional video, a technician asks Figure 01 to perform a range of simple tasks in a minimalist test environment resembling a kitchen. He first asks the robot for something to eat and is handed an apple. Next, he asked Figure 01 to explain why it handed him an apple while it was picking up some trash. The robot answers all the questions in a robotic but friendly voice.

Related: Watch scientists control a robot with their hands while wearing the Apple Vision Pro

The company said in its video that the conversation is powered by an integration with technology made by OpenAI — the name behind ChatGPT. It's unlikely that Figure 01 is using ChatGPT itself, however, because that AI tool does not normally use pause words like "um," which this robot does.

LATEST VIDEOS FROM livescience

With OpenAI, Figure 01 can now have full conversations with people-OpenAI models provide high-level visual and language intelligence-Figure neural networks deliver fast, low-level, dexterous robot actionsEverything in this video is a neural network: pic.twitter.com/OJzMjCv443March 13, 2024

Should everything in the video work as claimed, it means an advancement in two key areas for robotics. As experts previously told Live Science, the first advancement is the mechanical engineering behind dexterous, self-correcting movements like people can perform. It means very precise motors, actuators and grippers inspired by joints or muscles, as well as the motor control to manipulate them to carry out a task and hold objects delicately. 

Even picking up a cup — something which people barely think about consciously — uses intensive on-board processing to orient muscles in precise sequence.

RELATED STORIES

—This video of a robot making coffee could signal a huge step in the future of AI robotics. Why?

—Human-like robot tricks people into thinking it has a mind of its own

—Robot hand exceptionally 'human-like' thanks to new 3D printing technique 

The second advancement is real-time natural language processing (NLP) thanks to the addition of OpenAI's engine — which needs to be as immediate and responsive as ChatGPT when you type a query into it. It also needs software to translate this data into audio, or speech. NLP is a field of computer science that aims to give machines the capacity to understand and convey speech.

Rehab Robots Assist Post-Stroke Patients At BGU Lab - NoCamels

I really prefer the socially assistive robot in the movie 'Robot and Frank'.

 

Rehab Robots Assist Post-Stroke Patients At BGU Lab

By Simona Shemer, NoCamels April 28, 2022 7 minutes

Technology

The semi-humanoid Pepper robot is being used in The Cognition, Aging and Rehabilitation Lab at Ben-Gurion University of the Negev to help stroke patients.

New studies on rehabilitation support the theory that not all therapy needs to be hands-on.

Take stroke patients, for instance. While for most stroke patients, rehabilitation requires physical therapy as patients need to relearn simple motor activities like walking and sitting, occupational therapy (relearning daily activities like eating and dressing), and speech therapy, formal caregivers are not always available due to budget constraints or insufficient staff, and informal caregivers (spouses, family members) don’t always have the skill set or the patience.

Prof. Shelly Levy-Tzedek says socially assistive robots, or robots that assist using social cues, can help support and treat patients in rehabilitation from stroke or other conditions when human caregivers are not available. In her work leading The Cognition, Aging and Rehabilitation Lab at Ben-Gurion University, she regularly conducts studies with other researchers on how these social robots help improve the well-being of people who need rehabilitation.

Prof. Shelly Levy-Tzedek and her team at The Cognition, Aging and Rehabilitation Lab are using socially assistive robots to help stroke patients. Photo by Dani Machlis / BGU

Earlier this year, Levy-Tzedek, and fellow professors Ayelet Dembovski and Yael Amitai, published the methodology for their system and the initial results of a study that highlighted the use of socially assistive robots that support patients dealing with stroke. The team developed a robot-based gamified exercise platform for long-term post-stroke rehabilitation, came up with seven gamified based on functional tasks, and used the semi-humanoid robot Pepper manufactured by Softbank Robotics for the study.

The study looked at mixed attitudes towards the robots, motivation for use, and the differences in interaction between the patient and a human therapist vs. a patient and an assistive robot. The study was published in the journal Frontiers in Rehabilitation Science.

Socially assistive robots “help the person, not physically – they don’t move their limbs or they don’t move something in the world for the person – but they get them to do something themselves. So one of the biggest projects in the lab in the past few years has been a socially assistive robot that helps people who’ve had a stroke, do their exercise,” Prof. Shelly Levy-Tzedek tells NoCamels.

While the person can do the exercise on their own, Prof. Levy-Tzedek says socially assistive robots can provide extra benefits like motivation, companionship, and a gamified system that could make the patient feel like he is playing a game rather than relearning skills or completing tasks.

The other benefit is that the robot could be taken home in the future.

In the study, researchers collected and analyzed information from 23 patients (11 stroke patients and 12 informal caregivers) who participated in a total of six focus-group discussions. The patients answered questions regarding the use of a socially assistive robot to promote physical exercises during the rehabilitation process including the advantages and disadvantages, specific needs the robot would address, adaptions the patient would propose to include, and concerns regarding the technology.

A team headed by Prof. Shelly Levy-Tzedek used the semi-humanoid robot Pepper to help support stroke patients. Deposit Photos

“We found that the majority of the participants in both groups were interested in experiencing the use of a SAR (socially assistive robot) for rehabilitation, in the clinic and at home,” the authors wrote in the study.

“This is a study that we ran in the clinic with patients who have had a stroke, previously being healthy individuals. This was a pilot experiment in the lab, and then we ran it with actual stroke patients in the clinic. We did over two years during COVID-19,” she explains.

The clinical results have not yet been published, but Levy-Tzedek says they are “promising.”

“This is the first experiment of its kind in the world in that it’s a long-term experiment with stroke patients in the clinic with a social robot. So this hasn’t been done before. There were studies with stroke patients that were just one-off meetings with a robot which is a good first start, but You need to do the experiment in the long term because rehabilitation is a long-term endeavor. So you have to see how people react to it over the long term and whether the novelty wears off after a while. And then do people still continue and we see that they do,” she says.

Rehab robots assist stroke patients

Levy Tzedek, a biomedical engineer who studied at UC Berkeley and earned a Master of Science and PhD from Massachusetts Institute of Technology (MIT,) uses what she calls “off-the-shelf robots” and fits them with a platform developed in the lab. “Off-the-shelf” means they are commercially available.

“It’s not a robot we built. What we did is build a whole platform around it,” she explains. In the case of stroke patients, “this is a platform that helps people after stroke perform exercises.”

“Now, you might wonder, is it the robot that matters? Or is it the platform that we developed? People who used the computer instead of the robot to give them instructions and feedback – so exactly the same platform, but a computer was providing the instructions and the feedback [instead of the robot] – they also got better, but not to the same extent,” she explains, “So more people got better with a robot.”

People who have had a stroke often lose the ability to perform tasks that were trivial before something like buttoning their own shirt, slicing bread, or placing a jar on a shelf. These are things that have to be practiced thousands of times, and some of this practice is done in a clinic with a physical therapist or an occupational therapist, but a lot of the practice has to be done on its own.

“In general, when we look at compliance with physical therapy exercises, it’s around 30 percent. So we wanted to get people to do more self-exercise, but in a guided way – giving them motivation but also feedback. So that’s the idea behind this,” she says.

Stroke patients often lose the ability to perform daily actions with everyday objects. Tasks that were trivial before — like buttoning a shirt, slicing bread, or placing a jar on a shelf — are suddenly tasks that have to be practiced thousands of times. Some of the practice is done in a clinic with a physical therapist but a lot of it is done at home, where the patient needs to do it on his own.

Prof. Shelly Levy-Tzedek (left) and Lab Engineer Yuliya Berdichevsky. Photo by Dani Machlis / BGU

“So if they have to relearn how to button a shirt, they actually have to use a button and try to do the actual activity because they have to relearn how to coordinate their muscle activity. And just strengthening their muscles is not enough, using virtual reality is good, but not sufficient, they have to actually do the tasks that they’re trying to relearn how to do, which is why we use everyday objects in all of the practice,” Levy-Tzedek explains, “

The everyday objects have RFID (radio frequency identification) tags on them. RFID is a form of wireless communication that can identify an object. This is done so researchers can know where each item is placed and the robot can give feedback to the person.

“What they do is they give a task using the screen and also speech and they say to place the, in this case, orange, green and blue cups in this particular arrangement. And the person does that. And because we have these sensors, we know exactly where they place the objects and we and the robot can then give them feedback,” Levy-Tzedek says.

During the interview, Levy-Tzedek showed some of the setups where a patient was practicing various tasks using the help of a socially assistive robot. In one kitchen setup, the patient was tasked with placing kitchen items on a shelf. They had to place the items as well as remember where they should be placed. In another escape room setup, they had to find items as requested by the robot. In a third setup, a robot played Blackjack with the patient and the patient had to remember his cards and play the game correctly.

The value of a socially assistive robot

What is it about the robot that makes it better or different from a human? Levy-Tzedek says it’s “something that we’re trying to figure out.” Her team has done numerous in-depth interviews with people over time, including in the beginning and middle of the experiment, and even after they’ve completed it.

“We asked them about their experience and what they thought and it seems that at the same time they were treating it as a human but then also not as a human,” she explains, citing examples that the robot made mistakes (they would tell the patient he did something wrong when he was correct) and some were frustrated by the incident while others let it roll off their back. On the other hand, patients were afraid that a human therapist would be judgemental of them, even though therapists aren’t supposed to be judgemental. The robot could not be judgemental and this was a plus.

“At the same time, they treat it as a human, but then also not as a human. And they take the best out of each. And I will say, though, that people were most interested in continuing to work with a robot when they felt that they actually had a benefit to their functional rehabilitation and when they felt they got better when they would go home at the end of the session, and they were able to do something that they weren’t able to do before,” Levy-Tzedek explains, “That was the strongest predictor of how much they would want to come and then work with it again, and continue working with a robot. So it wasn’t just some sort of halo effect of using technology, but the actual benefit that they reaped from working with it.

How Personalized Interactions with Robots Could Help Stroke Victims

Good luck getting anything like this in your stroke hospital.The movie 'Robot and Frank' pretty well described personal robots, but they do need a moral filter.
http://www.hcanews.com/news/how-personalized-interactions-with-robots-could-help-stroke-victims
Personalizing interactions between humans and robots could help rehabilitate stroke victims by boosting patient use of tailored physical therapy programs, researchers have found.

A new study by researchers at the Ben-Gurion University of the Negev (BGU) in Israel has made a first step in developing an interactive movement protocol that fits an individual’s preferences, with the potential to make physical therapy more customized and effective.

“I think of it as a robotic revolution in rehabilitation,” said Dr. Shelly Levy-Tzedek, lead author of the report, published in Restorative Neurology and Neuroscience this month, and head of the Cognition, Aging, and Rehabilitation Lab in BGU's Department of Physical Therapy. “The goal is to have a rehabilitation protocol that includes the robot as a partner and as a coach so that people can do their exercises at home.”

The findings were presented to Israeli President Reuven Rivlin this week during a visit to the BGU campus.

The study used a “mirror game”, usually played in improvisational theater, in which an individual mimics the movement of the robot or vice versa. “People report that when they play the mirror game they feel a sense of togetherness and closeness. We’re the first to my knowledge to play the mirror game between a human and a robot,” Levy-Tzedek said.

The robotic arm was fitted with an Xbox Kinect camera to track the user’s arm. In the game, the user first led the robot, and then the roles were reversed, and finally the pair was instructed to move together.

The study found that the movement of the robot “primes” the human movement. In other words, the participants tended to mimic the robot behavior. “Priming seems to be a byproduct of the interaction, but it actually could be used as a feature that we plan for patients. If we want the patient to move in certain ways, we could accordingly program the robot to move in certain ways,” Levy-Tzedek said.

Researchers also found that participants were evenly split in their preference for either leading or following. This demonstrates a need for robotic physical therapy routines to be responsive to what a patient is most likely to find appealing and satisfying, Levy-Tzedek noted.

“Just like we have personalized medicine, which is tailored to the particular person, it’s the same with human-robotic interactions. It’s not just one size fits all,” Levy-Tzedek said.  “Some people are more competitive and like to lead, others less so. You can’t assume that one type of interaction would fit everyone.”

Researchers also found that participants enjoyed tracing smoother, recognizable movements like the infinity sign or circles than jerkier, robotic ones like dribbling a ball or zig zags.

Levy-Tzedek doesn’t envision robotic therapists replacing their human counterparts anytime soon. A robotic coach could, however, be invaluable in encouraging exercises at home and bridging the time between meetings with physical therapists.

“In that gap is where we come into the picture,” Levy-Tzedek said. “This was a first step in that direction of using rehabilitation in a gamified environment.”

Engaging patients in gamified physical therapy routines is important,  she said, because patients can often be reluctant to undertake the routines outside of scheduled appointments. “The rehabilitation process can be boring, painful, frustrating, and patients aren’t always motivated enough to do that.”

Almost 800,000 people in the US have a stroke every year, according to the American Stroke Association, or 1 every 40 seconds. Although Levy-Tzedek and her team are first focusing on that demographic, a robotic coach to encourage physically therapeutic routines could have benefits for a much broader user base. There is a “wide variety of conditions” that these findings could apply to, according to Levy-Tzedek, and not necessarily individuals who have a physical disorder.

“It turns out you can be an active couch potato. You can go to the gym three times a week and still suffer from disorders associated with sitting for long periods of time,” she added.

Sedentary behavior such as long hours sat at a desk or at home in front of the TV is associated with an increased risk of cardio-metabolic disease, all-cause mortality and a variety of physiological and psychological problems, according to the International Journal of Behavioral Nutrition and Physical Activity. Encouraging physical routines in the home could help combat that, Levy-Tzedek said.

Over recent years the cost of robotics has dropped dramatically, she said, anticipating that as soon as a decade from now most patients would be able to afford this technology.