This post is co-written with Andreas Astrom from Northpower.
northern power It provides reliable and affordable electricity and fiber internet services to customers in the Northland region of New Zealand. As an electricity distributor, Northpower aims to improve access, opportunity and prosperity in its communities by investing in infrastructure, developing new products and services and giving back to shareholders. Additionally, Northpower is one of New Zealand's largest infrastructure contractors, serving clients in transmission, distribution, generation and telecommunications. With over 1,400 employees working across 14 locations, Northpower plays a crucial role in maintaining essential services for customers driven by the purpose of connecting communities and building futures for Northland.
The energy industry is at a critical inflection point. There is a strong push from policymakers and the public to decarbonize the industry while balancing energy resilience with health, safety and environmental risks. Recent events, including Tropical Cyclone Gabrielle, have highlighted the network's susceptibility to extreme weather conditions and emphasized the need for climate adaptation with resilient infrastructure. Electricity distribution companies (EDBs) also face new demands with the integration of decentralized energy resources such as rooftop solar, as well as larger-scale renewable energy projects such as solar and wind farms. These changes require innovative solutions to ensure operational efficiency and continued resilience.
In this post, we share how Northpower has worked with its technology partner Sculpt to reduce the effort and carbon required to identify and remediate public safety risks. Specifically, we cover computer vision and artificial intelligence (ai) techniques used to combine data sets into a list of priority tasks for field teams to investigate and mitigate. The resulting panel highlighted that 141 power pole assets required action, out of a network of 57,230 poles.
Northern Power Challenge
Utility poles have guy wires that anchor the pole to the ground for added stability. These stay cables are intended to have an inline insulator to prevent the stay cable from becoming stressed, which would create a safety risk to people or animals in the area.
Northpower faced a major challenge in determining how many of its 57,230 power poles have live wires without insulators. Without reliable historical data, manual inspections of such a vast and predominantly rural network are labor-intensive and expensive. Alternatives such as helicopter inspections or field technicians require access to private property for safety inspections and are expensive. Additionally, the need for travel for technicians to physically visit each pole across such a large network posed a considerable logistical challenge, emphasizing the need for a more efficient solution.
Fortunately, some asset data sets were available in digital format and historical paper inspection reports, dating back 20 years, were available in scanned format. This archive, along with 765,933 inspection photographs of varying quality, some more than 15 years old, presented a significant data processing challenge. Processing these images and scanned documents is not a time- or cost-saving task for humans and requires high-performance infrastructure that can reduce time-to-value.
Solution Overview
amazon SageMaker is a fully managed service that helps developers and data scientists create, train, and deploy machine learning (ML) models. In this solution, the team used amazon SageMaker Studio to launch an object detection model available in amazon SageMaker JumpStart using the PyTorch framework.
The following diagram illustrates the high-level workflow.
Northpower chose SageMaker for several reasons:
- SageMaker Studio is a managed service with out-of-the-box development environments, saving time that would otherwise be spent configuring environments manually.
- SageMaker JumpStart took care of the setup and implemented the necessary machine learning jobs involved in the project with minimal configuration, saving even more development time.
- The labeling solution integrated with amazon SageMaker Ground Truth was suitable for large-scale image annotation and simplified collaboration with Northpower labeling staff.
In the following sections, we discuss the key components of the solution as illustrated in the diagram above.
Data preparation
SageMaker Ground Truth employs a human workforce of Northpower volunteers to annotate a set of 10,000 images. The staff created a bounding box around stay cables and insulators and the result was later used to train an ML model.
Training, validation and storage of models.
This component uses the following services:
- SageMaker Studio is used to access and deploy a pre-trained object detection model and develop code on managed Jupyter notebooks. The model was then fitted with training data from the data preparation stage. For a step-by-step guide to setting up SageMaker Studio, see amazon SageMaker Simplifies Setting Up amazon SageMaker Studio for Individual Users.
- SageMaker Studio runs custom Python code to augment the training data and transform the SageMaker Ground Truth metadata output to a format compatible with the computer vision model training job. The model is then trained using a fully managed infrastructure, validated, and published to the amazon SageMaker Model Registry.
- amazon Simple Storage Service (amazon S3) stores model artifacts and creates a data lake to host inference output, document analysis output, and other data sets in CSV format.
Model implementation and inference.
In this step, SageMaker hosts the ML model on an endpoint used to run inference.
A SageMaker Studio notebook was again used after inference to run custom Python code to simplify the data sets and render bounding boxes on objects based on criteria. A custom scoring system was also applied in this step and was also represented in the final image, allowing for an additional human quality control step for low confidence images.
Data analysis and visualization.
This component includes the following services:
- An AWS Glue crawler is used to understand the dataset structures stored in the data lake so that amazon Athena can query them.
- Athena allows the use of SQL to combine inference output and asset data sets to find higher risk items.
- amazon QuickSight was used as a tool for both the human quality control process and to determine which assets needed a field technician to be sent for physical inspection.
Understanding the document
In the final step, amazon Textract digitizes historical paper asset evaluations and stores the result in CSV format.
Results
PyTorch's trained object detection model enabled detection of stays and insulators on utility poles, and a SageMaker post-processing job calculated a risk score using an amazon Elastic Compute Cloud (EC2) m5.24xlarge instance with 200 concurrent Python threads. This instance was also responsible for rendering the score information along with an object bounding box in an output image, as shown in the following example.
Writing the confidence scores to the S3 data lake along with the historical inspection results allowed Northpower to run analyzes using Athena to understand each image classification. The sunburst graph below is a visualization of this classification.
Northpower classified 1,853 poles as high priority risks, 3,922 as medium priority, 36,260 as low priority and 15,195 as lowest priority. These could be viewed in the QuickSight dashboard and were used as input for humans to review the highest risk assets first.
At the conclusion of the analysis, Northpower found that 31 poles required the installation of stay wire insulators and a further 110 poles required field investigation. This significantly reduced the cost and carbon usage involved in manually verifying each asset.
Conclusion
Remote asset inspection remains a challenge for regional EDBs, but the use of computer vision and artificial intelligence to discover new value from previously unused data was key to Northpower's success on this project. SageMaker JumpStart provided deployable models that could be trained for object detection use cases with minimal data science expertise and overhead.
Discover the publicly available basic models offered by SageMaker JumpStart and accelerate your own machine learning project with the following step-by-step tutorial.
About the authors
Scott Patterson He is a Senior Solutions Architect at AWS.
Andreas Astrom He is the Head of technology and Innovation at Northpower
