New haptic patch transmits complexity of touch to the skin

• Link to: Northwestern Now Story

EMBARGOED UNTIL 11 A.M. EST (U.S.) ON WEDNESDAY, NOVEMEBER 6, 2024

• Thin, flexible device could help people with visual impairments ‘feel’ surroundings
• Device comprises a hexagonal array of 19 actuators encapsulated in soft silicone
• Device only uses energy when actuators change position, operating for longer periods of time on a single battery charge

EVANSTON, Ill. --- A Northwestern University-led team of engineers has developed a new type of wearable device that stimulates skin to deliver various complex sensations.

The thin, flexible device gently adheres to the skin, providing more realistic and immersive sensory experiences. Although the new device obviously lends itself to gaming and virtual reality (VR), the researchers also envision applications in healthcare. For example, the device could help people with visual impairments “feel” their surroundings or give feedback to people with prosthetic limbs.

The study was published today (Nov. 6) in the journal Nature.

The device is the latest advance in wearable technology from Northwestern bioelectronics pioneer John A. Rogers. The new study builds on work published in 2019 in Nature, in which his team introduced “epidermal VR,” a skin-interfaced system that communicates touch through an array of miniature vibrating actuators across large areas of the skin, with fast wireless control.

“Our new miniaturized actuators for the skin are far more capable than the simple ‘buzzers’ that we used as demonstration vehicles in our original 2019 paper,” Rogers said. “Specifically, these tiny devices can deliver controlled forces across a range of frequencies, providing constant force without continuous application of power. An additional version allows the same actuators to provide a gentle twisting motion at the surface of the skin to complement the ability to deliver vertical force, adding realism to the sensations.”

Rogers is the Louis A. Simpson and Kimberly Querrey Professor of Materials Science and Engineering, Biomedical Engineering and Neurological Surgery, with appointments in Northwestern’s McCormick School of Engineering and Northwestern University Feinberg School of Medicine. He also directs the Querrey Simpson Institute for Bioelectronics.

Rogers co-led the work with Northwestern’s Yonggang Huang, the Jan and Marcia Achenbach Professorship in Mechanical Engineering at McCormick; Hanqing Jiang of Westlake University in China; and Zhaoqian Xie of Dalian University of Technology in China. Jiang’s team built the small modifying structures needed to enable twisting motions.

Leveraging skin-stored energy

The new device comprises a hexagonal array of 19 small magnetic actuators encapsulated within a thin, flexible silicone-mesh material. Each actuator can deliver different sensations, including pressure, vibration and twisting. Using Bluetooth technology in a smartphone, the device receives data about a person’s surroundings for translation into tactile feedback — substituting one sensation (like vision) for another (touch).

Although the device is powered by a small battery, it saves energy using a clever “bistable” design. This means it can stay in two stable positions without needing constant energy input. When the actuators press down, it stores energy in the skin and in the device’s internal structure. When the actuators push back up, the device uses the small amount of energy to release the stored energy. So, the device only uses energy when the actuators change position. With this energy-efficient design, the device can operate for longer periods of time on a single battery charge.

“Instead of fighting against the skin, the idea was ultimately to actually use the energy that's stored in skin mechanically as elastic energy and recover that during the operation of the device,” said Matthew Flavin, the paper’s first author. “Just like stretching a rubber band, compressing the elastic skin stores energy. We can then reapply that energy while we're delivering sensory feedback, and that was ultimately the basis for how we create the created this really energy-efficient system.”

At the time of the research, Flavin was a postdoctoral researcher in Rogers’ lab. Now, he is an assistant professor of electrical and computer engineering at the Georgia Institute of Technology.

To identify the optimal design, Huang’s team conducted systematic computational modeling to examine how the device interacts with skin through electromagnetic actuation.

“It is essential that this bi-stability design can be universally applied to all types of human skin. Computational modeling enables precise optimization, ensuring this capability,” Huang said. “Achieving this solely through trial-and-error experiments would be impossible. Shupeng Li, a co-first author and postdoc in my group, led this modeling effort.”

Sensory substitution

To test the device, the researchers blindfolded healthy subjects to test their abilities to avoid objects in their path, change foot placement to avoid injury and alter their posture to improve balance.

One experiment involved a subject navigating a path through obstructing objects. As the subject approached an object, the device delivered feedback in the form of light intensity in its upper right corner. As the person moved nearer to the object, the feedback became more intense, moving closer to the center of the device.

With only a short period of training, subjects using the device were able to change behavior in real time. By substituting visual information with mechanical, the device “would operate very similarly to how a white cane would, but it's integrating more information than someone would be able to get with a more common aid,” Flavin said.

“As one of several application examples, we show that this system can support a basic version of ‘vision’ in the form of haptic patterns delivered to the surface of the skin based on data collected using the 3D imaging function (LiDAR) available on smartphones,” Rogers said. “This sort of ‘sensory substitution’ provides a primitive, but functionally meaningful, sense of one’s surroundings without reliance on eyesight — a capability useful for individuals with vision impairments.”

The title of the study is “Bioelastic state recovery for haptic sensory substitution.”