Efficient neural network backbones for mobile devices are often optimized for metrics like FLOP, or parameter count. However, these metrics may not correlate well with network latency when deployed to a mobile device. Therefore, we perform a thorough analysis of different metrics by implementing various mobile-friendly networks on a mobile device. We identify and analyze architecture and optimization bottlenecks in recent efficient neural networks and provide ways to mitigate these bottlenecks. To this end, we designed an efficient backbone MobileOne, with variants achieving sub-1ms inference time on an iPhone12 with 75.9% accuracy among the top 1 on ImageNet. We show that MobileOne achieves next-generation performance within efficient architectures while being many times faster on mobile devices. Our best model gets similar performance on ImageNet as MobileFormer and is 38 times faster. Our model gets 2.3% more top 1 accuracy on ImageNet than on EfficientNet with similar latency. Furthermore, we show that our model generalizes to multiple tasks: image classification, object detection, and semantic segmentation with significant improvements in latency and accuracy compared to existing efficient architectures when implemented on a mobile device.