The Internet is transitioning to an edge computing architecture to accommodate latency sensitive DNN workloads in the developing Internet of Things and mobile computing application domains. Unfortunately, large, high-precision DNN models cannot be operated at the edge due to their lack of computing capabilities, unlike in cloud environments. Therefore, past efforts have focused on moving some computing to the cloud to circumvent this restriction. However, this results in longer delays.
New research from Microsoft proposes REACT, a unique architecture that uses the edge and the cloud together to run redundant computations. To improve detection quality without compromising latency, they merge asynchronously received cloud inputs into the compute stream at the edge. This allows you to take advantage of the precision of the cloud without sacrificing the low latency of the edge.
The team uses a two-pronged approach to solve the problems of poor edge compute capability and loss of precision due to edge models.
- To begin with, edge object identification needs to be called only once every few frames due to the spatiotemporal correlation between successive video frames. Edge detection occurs every five frames. They use a fairly lightweight object tracking operation to bridge the gap between the two frameworks.
- Second, only certain frames are sent to the cloud asynchronously to increase the accuracy of the inference. Depending on network latency and the availability of cloud resources, edge devices do not get cloud detections for a few frames afterward.
- The most recent and previously unreported cloud detections are then combined with the current image. To “fast forward” to the current time, they use the cloud detection generated in a previous frame and feed it into a second instance of the object tracker. As long as there is no drastic change to the scene, the newly identified elements can be integrated into the current frame.
The team applied this method to a dataset of dashcam videos. His experiments used state-of-the-art computer vision methods to obtain detections of both local and remote elements. In addition, they use the widely used statistic in the field of machine vision known as [email protected] (0.5 IoU average mean precision) to assess the quality of object detections. They also analyzed two sets of data to determine how effective REACT was:
- As a drone-based surveillance system, VisDrone
- The D2City system is a dashcam-based driving assistance system.
Their test results show that REACT can provide up to 50% better results than reference methods. They also demonstrate that the edge and cloud models can complement each other and that the proposed edge-cloud merger approach can improve overall performance.
In addition to the light object tracking done at intermediate frames, the object detector only runs once every few frames. By doubling detection between the edge and the cloud, developers have more freedom to choose how often to run their applications on each platform while maintaining the same level of detection accuracy.
The researchers also note that having multiple edge devices using the same cloud-hosted model can spread the expense of using cloud resources across a larger population. In particular, the V100 GPU can support more than 60 concurrent devices simultaneously, assuming the application can support an average latency of up to 500ms.
While this work has primarily discussed its application to object detection, the team believes it can be applied in other situations, including human pose estimation, instance estimation, and semantic segmentation applications, for the “best of both worlds.”
review the Paper and Article. Don’t forget to join our 21k+ ML SubReddit, discord channel, and electronic newsletter, where we share the latest AI research news, exciting AI projects, and more. If you have any questions about the article above or if we missed anything, feel free to email us at [email protected]
🚀 Check out 100 AI tools at AI Tools Club
Tanushree Shenwai is a consulting intern at MarktechPost. She is currently pursuing her B.Tech at the Indian Institute of Technology (IIT), Bhubaneswar. She is a data science enthusiast and has a strong interest in the scope of application of artificial intelligence in various fields. She is passionate about exploring new advances in technology and its real life application.
