Tiny device processes hand movement in real time, storing visual memories with brain-like efficiency

Engineers at RMIT University have invented a small “neuromorphic” device that detects hand movement, stores memories and processes information like a human brain, without the need for an external computer.

The findings are published in the journal Advanced Materials Technologies.

Team leader Professor Sumeet Walia said the innovation marked a step toward enabling instant visual processing in autonomous vehicles, advanced robotics and other next-generation applications for improved human interaction.

“Neuromorphic vision systems are designed to use similar analog processing to our brains, which can greatly reduce the amount of energy needed to perform complex visual tasks compared with digital technologies used today,” said Walia, Director of the RMIT Center for Opto-electronic Materials and Sensors (COMAS).

The work brings together neuromorphic materials and advanced signal processing led by Professor Akram Al-Hourani, who is Deputy Director of COMAS.

The device contains a metal compound known as molybdenum disulfide (MoS₂).

In their latest study, the team showed how atomic-scale defects in this compound can be harnessed to capture light and process it as electrical signals, like how neurons work in our brain.

“This proof-of-concept device mimics the human eye’s ability to capture light and the brain’s ability to process that visual information, enabling it to sense a change in the environment instantly and make memories without the need to use huge amounts of data and energy,” Walia said.

“Current digital systems, by contrast, are very power-hungry and unable to keep up as data volume and complexity increase, which limits their ability to make ‘true’ real-time decisions.”

Walia and Al-Hourani are corresponding authors of the research, and Mr. Thiha Aung, a Ph.D. scholar at RMIT, is the first author. RMIT has filed a provisional patent for the work.

Seeing the future in the wave of a hand

During experiments, the device detected changes in a waving hand’s movement, without the need to capture the events frame by frame. This is known as edge detection, which requires significantly less data processing and power.

Once the changes were detected, the device stored these events as memories like a brain.

The researchers conducted experiments in the spectrum visible to the human eye, which built upon the team’s previous neuromorphic research in the ultraviolet domain.

“We demonstrated that atomically thin molybdenum disulfide can accurately replicate the leaky integrate-and-fire (LIF) neuron behavior, a fundamental building block of spiking neural networks,” Thiha said.

The past UV work only involved the detection, memory making and processing of still images. In both the visible-spectrum and UV devices, memories could be reset so that devices were ready to perform the next task.

Potential applications

The team’s innovation could one day improve the response times of automated vehicles and advanced robotic systems to visual information, which could be crucial, particularly in dangerous and unpredictable environments.

“Neuromorphic vision in these applications, which is still many years away, could detect changes in a scene almost instantly, without the need to process lots of data, enabling a much faster response that could save lives,” Walia said.

“For robots working closely with humans in manufacturing or as a personal assistant, neuromorphic technology could enable more natural interactions by recognizing and reacting to human behavior with minimal delay,” Al-Hourani said.

Next steps

The team is now scaling up the proof-of-concept single-pixel device to a larger pixel array of MoS₂-based devices.

“While our system mimics certain aspects of the brain’s neural processing, particularly in vision, it’s still a simplified model,” Walia said. “We will optimize the devices to perform specific real-world applications with more complex vision tasks, and further reduce power consumption.”

The team plans to develop hybrid systems that integrate their analog technology with conventional digital electronics.

“We see our work as complementary to traditional computing, rather than a replacement,” Walia added. “Conventional systems excel at many tasks, while our neuromorphic technology offers advantages for visual processing where energy efficiency and real-time operation are critical.”

The team is also investigating materials other than MoS₂ that might extend capabilities into infrared, which could enable real-time tracking of global emissions and intelligent sensing of contaminants such as toxic gases, pathogens and chemicals.