Gesture Recognition Using Ultrasound

Simple hand movements such as wiping, pulling or tapping have become common place through the popularity of the smartphone. These types of gesture controls, however, require access to a touch screen. Contactless solutions for man-machine communications are required in cases where a touch screen is not available or hands and fingers cannot be used. Systems assisting in speech recognition and interpretation in particular are already growing in popularity. However, these systems rely on quiet environments free from external noise disturbances and are sometimes unsuitable for use in public areas. Fraunhofer IPMS researchers are working on an alternative approach to provide non-contact, three-dimensional recording of distance, movement and gesture for communication with robots as well as in surgical areas and household systems.

Scientists have developed a micro-chip architecture that can generate and receive ultrasound up to 300 kHz. Reflected sound waves are analyzed by measuring, for example, how long it took the wave to travel between the sensor system and the reflecting object, or how frequencies shifted due to the Doppler effect. Evaluation of the ultrasound provides a spatial resolution for natural movements and gestures in the sub-centimeter range at distances up to half a meter. Fraunhofer IPMS representatives are certain that the ultrasonic transducer has advantages over competing optical sensor methods. Group leader Sandro Koch notes, “Compared to camera-based systems, our ultrasonic sensors enable the construction of significantly cheaper electronic and software systems due to longer signal transit times. Our transducers are not susceptible to stray light and allow for reliable data acquisition on optically transparent surfaces as well. Our systems are CMOS-compatible, are considerably more compact, and can be inexpensively produced in mass quantities.”

For this development, researchers are implementing a new class of electrostatic micro-electro-mechanical (MEMS) bending actuators which have been continuously further advanced for generating sound in micro-loudspeakers and use in micropumps since 2016. The Fraunhofer IPMS proprietary nano-e-drive (NED) principle utilizes the high forces of electrostatic fields in nanometer-sized electrode gaps to allow for mechanical movements with displacements in ranges of several microns. The chip surface as well as the complete component volume is used for sound generation. Sandro Koch explains, “Using the entire chip volume for sound generation enables us to produce very small components. Because hundreds of such devices can fit on a single wafer – and multiple wafers can be simultaneously processed in single process steps – the cost of manufacturing large volumes is potentially low.”

Fraunhofer researchers expect that high air volume flows that have been converted into high sound pressure will support further development to provide an increased signal-to-noise ratio for low-frequency ultrasonic transducers.  The resonance frequency and thus the detection range and spatial resolution can then be defined by the geometry of the NED bending actuators.

Possible fields of applications for ultrasound-based non-contact motion detection include uses in automation, safety and medical technology as well as the automotive and entertainment and household electronics industries. Fraunhofer IPMS will present a first functional demonstrator showing how ultrasound can support gesture recognition at the 2019 Sensor and Test Measurement Fair event in Nürnberg. Visitors can find the Fraunhofer IPMS exhibit at Booth 248 in Hall 5 from 25.-27. June.