Machine learning and automatic image recognition. These are the ingredients with which CQM has made an image application for the inspection of train tracks intelligent. The self-learning system makes it possible to automatically detect defects on the track, so that only 20% of all images need be viewed by people. As well as efficiency, this also delivers improved maintenance, and thus greater track safety and availability. And another thing: an algorithm doesn’t blink.
Inspectation is a division of VolkerRail (a VolkerWessels company) that specializes in the monitoring of infrastructure. For ProRail, the company inspects roughly half of all rail routes in the Netherlands. A part of this involves the inspection of rails and points. Previously this was done by people who literally walked the track to carry out the checks. A dangerous job, which is why ProRail asked all contractors to carry out inspections using an inspection train. Inspectation also has one of these inspection trains, and will soon have a second, including cameras that take photos of the track every half-meter. Each section of the track is checked this way every 6-10 weeks. This generates a huge number of images that all have to be viewed by inspectors from Inspectation. Intensive and monotonous work that nevertheless must be carried out with the utmost precision. As Clemens Schoone, business consultant at Inspectation, puts it, “You can barely blink your eyes or you’ll miss something.”
Proof of concept
CQM developed a planning system for inspection trains for ProRail and it was the many images the inspection train produced that inspired former CQM senior consultant, Huub van den Broek, to automate the inspection process. “At CQM we’re very enthusiastic about the combination of image recognition and machine learning. The image inspection at Inspectation was an ideal project to put the strength of that combination into practice."
Clemens was keen to cooperate on a proof of concept. “The vast majority of images from the inspection train contain no so-called ‘squats’ (instances of damage to the track). So the manual viewing of these images isn’t just very intensive, it’s also largely unnecessary. If this can be automatized in a reliable way using modern technology, then we want to take advantage of that.”
State-of-the-art
Inspectation had already equipped its inspection train with a state-of-the-art camera system, putting it at the forefront of the sector. “It means the images are excellent and deliver a high-quality data set, an absolute must for machine learning,”, explains Huub. "What’s more, Google has recently released a deep-learning toolkit that in 2017 beat the world champion in the complex board game, Go. At CQM, we put this TensorFlow toolkit to work for Inspectation. In addition, we’ve acquired the latest hardware that can handle the many high-resolution images and complex mathematical calculations. We’ve used this to train the system, over various phases, to recognize squats. Working with the Inspectation specialists, we then did lots of test runs and tweaks to gradually improve the system. We did this manually, but the continuous perfecting of the algorithm will soon also be done automatically by the self-learning system.”
Since then, CQM has in the proof-of-concept phase proved convincingly that the system works and the algorithm has now been incorporated into the existing image application of Inspectation. “The inspectors still see the same pictures and inspect them using the trusted method,” explains Huub, “but in the same image they now also see a graph that’s controlled by our algorithm. The graph flags up all likely squats. At the moment, the system recognizes the squats almost perfectly. We’re now in the phase of fine-tuning the threshold values in the detection, and reducing possible false negatives and false positives to a minimum. Ultimately, the goal is that the inspectors only have to view pictures on which the system has detected a suspicious mark. This means that some 80% of the images can then be skipped. In other words, an inspector can do their work five times more efficiently.”
Safe
The key question is, of course, whether a system is safe when most of the work is done by machines. “Yes, I’m now 100% convinced of this,” affirms Clemens. “A machine can do this work with greater accuracy and reliability than a human being. CQM has demonstrated this. But obviously we will carry out random checks to continuously monitor quality. And 20% of the images are still being viewed by humans, plus there are numerous periodic technical checks.”
And the track itself, is that now also safer? “Absolutely,” says Clemens. “Because the quality of the inspection has risen, so the level of track maintenance is also improved. We see emerging defects earlier and can intervene early on, if necessary. This not only increases safety but also the availability of the track. Which in turns means less passenger disruption due to maintenance.”
With the new technology, Inspectation aims to become far and away the best rail inspector in the Netherlands, and hopes to employ the system for other types of track inspections. For CQM, the project paves the way for many other applications combining machine learning and image recognition, for example in the medical world. As Huub points out, “One medical specialist may see a few thousand X-rays of a certain condition during his entire career. Imagine if you were to put the billions of photos made by all the medical specialists in the world into one image-recognizing and self-learning system...”
Want to know more about machine learning and automatic image recognition?
Contact Minou. She'll be only too happy to tell you all about it!
