Significant advancement in 2D and 3D human posture estimation using RGB cameras, LiDAR, and radars has been made possible by improvements in machine vision and machine learning algorithms. However, occlusion and lighting, which are prevalent in many exciting circumstances, have a negative impact on the estimation of human position from photographs. On the other hand, radar and LiDAR technologies require expensive and power-hungry specialized hardware. Additionally, there are serious privacy considerations when using these sensors in private spaces.
Recent studies have analyzed the use of WiFi antennas (1D sensors) for body segmentation and identification of key body points to overcome these limitations. This article further explores the use of the WiFi signal in conjunction with deep learning architectures, which are frequently used in computer vision, to estimate the dense correlation of human pose. In a study published by Carnegie Mellon University (CMU) scientists, they described WiFi DensePose, an artificial intelligence (AI) model that can identify the pose of numerous people in space using only WiFi transmitter signals. At the 50% IOU threshold, the algorithm achieves an average accuracy of 87.2 in studies using real-world data.
Since WiFi signals are one-dimensional, most existing techniques for WiFi person detection can only identify a person’s center of mass, and often can only detect one person. Three different receivers recorded three WiFi signals and the CMU method uses the amplitude and phase data of those signals. This generates a 3×3 feature map that is fed into a neural network that generates UV maps of human body surfaces and can locate and identify the poses of various people.
The approach employs three elements to extract human body surface UV coordinates from WiFi signals: First, the raw CSI signals are cleaned using amplitude and phase sanitization. After domain translation of sanitized CSI samples into image-like 2D feature maps, a two-branch encoder-decoder network is used. The UV map, a representation of the dense relationship between 2D and 3D people, is estimated using the 2D features after inputting a modified DensePose-RCNN architecture. The team uses transfer learning to reduce discrepancies between multi-level feature maps created by images and those produced by WiFi signals before training the lead network to optimize training of WiFi input networks.
A data set of WiFi signals and video recordings of scenarios with one to five people were used to test the performance of the model. The recorded scenes were both offices and classrooms. The researchers used previously trained DensePose models on the films to produce a false real reality, although there are no annotations on the video to serve as the underlying reality of the assessment. Overall, the model was only “able to successfully recognize the approximate locations of human bounding boxes” and the pose of the torsos.
The group identified two main categories of failure cases.
(1) The WiFi-based model is biased and likely to create faulty body parts when body positions are seen infrequently in the training set.
(2) Extracting accurate information for each subject from the amplitude and phase tensors of the entire capture is more difficult for the WiFi-based approach when there are three or more contemporaneous subjects in a capture.
The researchers believe that collecting more comprehensive training data will help solve both problems.
Work performance is still limited by the training data available in WiFi-based perception, particularly when considering multiple designs. In their next research, the scientists also intend to collect data from multiple designs and advance their efforts to predict 3D human body shapes from WiFi signals. Compared to RGB cameras and lidars, the WiFi device’s enhanced dense perception capabilities could make it a more affordable, illumination-invariant, and private human sensor.
review the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 14k+ ML SubReddit, discord channel, and electronic newsletterwhere we share the latest AI research news, exciting AI projects, and more.
Niharika is a technical consulting intern at Marktechpost. She is a third year student, currently pursuing her B.Tech from the Indian Institute of Technology (IIT), Kharagpur. She is a very enthusiastic individual with a strong interest in machine learning, data science, and artificial intelligence and an avid reader of the latest developments in these fields.
