Post on 11-Jul-2020
Deep Learning of Railway
Track Faults using GPUs
Swiss Federal Railways (SBB)
Swiss Center for Electronics and Microtechnology
(CSEM)
Nathalie Rauschmayr (CSEM)
Matthias Hoechemer (CSEM)
Marcel Zurkirchen (SBB)
Stefan Kenzelmann (SBB)
Maitre Gilles (SBB)
GTC 2018, Santa Clara
Defect Detection.
Fingerprinting.
Monitoring the conditions of the Swiss rail network.
Need for automation
▪ Manual inspection is infeasible
▪ Ever increasing traffic leads to faster attrition
▪ New train types (Cargo, High Speed and Tilting
Trains…)
cause the development of railway faults that have
not been observable 10 years ago
Multiple specially equipped trains «Diagnosis Trains»
▪ Travelling up to 100 mph
▪ Multiple high resolution cameras and other sensors
Big Data Deep Learning of Railway Track Faults using GPUs 2
Some Statistics
• 15k trains per day
• 1.2 million rides per day
• Size of rail network: 4000 miles
Monitoring the conditions of the Swiss rail network.
Deep Learning of Railway Track Faults using GPUs 3
Diagnosis train operates since 2007, BUT
▪ Software generates too many false positives/negatives
▪ Railway experts have to filter all of them by hand
System has not been of much use until now
Railcheck project.
Deep Learning of Railway Track Faults using GPUs 4
Our Project Goals: Use Deep Learning to:
reduce the time for onsite visual inspections to a minimum
minimize the time experts spend on false positives
increase network safety
Human Being / BusinessMachine LearningDiagnosis train
Supervised Learning
Unsupervised Learning
Coniditon monitoring
DFZ since 2006
IBN gDFZ 2018
Railcheck project.
Deep Learning of Railway Track Faults using GPUs 5
Joint project between Swiss Center for Electronics and Microtechnology (CSEM) and
Swiss Federal Railways (SBB)
CSEM: Swiss R&D Lab specialized in microtechnology, system engineering, robotics,
automation and computer vision mainly doing applied R&D for industry (www.csem.ch)
The Input Data.
Deep Learning of Railway Track Faults using GPUs 6
Challenges:
▪ Changing weather conditions: rain, snow, ice
▪ Artefacts: leaves, dirt
▪ Different forms and shapes
Small artefacts Rain TurnoutsIn the street
Snow
Preprocessing.
Deep Learning of Railway Track Faults using GPUs 7
Tensorflow Object Detection API
Transfer Learning on faster R-CNN inception resnet in order to segment railway
surfaces and clamps
Anomaly Detection.
Deep Learning of Railway Track Faults using GPUs 8
Next Step: Clustering of railway components
Anomaly: components that do not fit to any cluster
Clustering with Generative Adversarial Networks (GAN):
Generator DiscriminatorRandom
Noise
Fake Data
Training Data
Anomaly Detection.
Deep Learning of Railway Track Faults using GPUs 9
Generator and Discriminator learn the underlying structure of the data
Search for similar images (Clustering) – Example:
Input Output: most similar images
Anomaly Detection.
Deep Learning of Railway Track Faults using GPUs 10
Pre-Clustering of normal railway components
Train Convolutional Autoencoder per Cluster
Input Image Reconstructed Difference
Anomaly
Fault Classification.
Deep Learning of Railway Track Faults using GPUs 11
Major Problem:
▪ Very little training data < 100
▪ Tricky to define fault category: in principal 20 categories
Was simplified to 6 categories and later on to 4
▪ A lot of variation
Welding Joint Surface Defect
Fault Classification.
Deep Learning of Railway Track Faults using GPUs 12
Major Problem:
▪ Very little training data < 100
▪ Tricky to define fault category: in principal 20 categories
Was simplified to 6 categories and later on to 4
▪ A lot of variation
Wheel SlipSquatNot Defect
Fault Classification.
Deep Learning of Railway Track Faults using GPUs 13
Labelling faults: Tedious,
▪ railway surface does mostly not contain any fault (luckily)
▪ faults can be easily overlooked
Speed the labelling process up
Increased training data for faults by a factor 100 within a month
Transfer Learning & Inference
Inspect & Correct Output
Add to Training
Data
Fault Classification.
Deep Learning of Railway Track Faults using GPUs 14
Major Problem: bias, wrong categories, different expert opinions
The worst faults are sometimes the ones that are nearly invisible (e.g. squats)
Squat Surface Defect Surface Defect
Fault Classification.
Deep Learning of Railway Track Faults using GPUs 15
Solution: Crowd Decision Making
Experts can view output of neural network via a website and give feedback
Add to Training
Data
Transfer Learning & Inference
Inspect & Correct Output
StatisticalAnalysis
Railway Experts
Surface Defect
Fault Classification.
Deep Learning of Railway Track Faults using GPUs 16
Even with little training data network achieves astonishing results
▪ Detection rates:
Joint 99%, Welding 57%, Surface Defect 65%
No training data: Squat, Wheel Slip
Surface Defect Surface Defect Welding Tiny surface defect
Fingerprinting.
Deep Learning of Railway Track Faults using GPUs 17
«Diagnosis trains» go routes multiple times per year
▪ Which fault is new?
▪ Which fault is already recorded in DB?
Challenge:
▪ Faults change over time
▪ Environmental conditions can be different
▪ Faults << GPS accuracy
Fingerprinting.
Deep Learning of Railway Track Faults using GPUs 18
Input faulty image into
pretrained model
Reduce size of
feature vector
Locality Sensitive
Hashing
Check for images within GPS
Range and compares hashes
20162017
Are these surface
defects identical?
Fingerprinting.
Deep Learning of Railway Track Faults using GPUs 19
Recording faults over time helps to understand
▪ How do they develop
▪ How to prevent them
November
2016
February
2017
April
2017
May
2017
June
2017
August
2017
Summary.
Deep Learning of Railway Track Faults using GPUs 20
Summary.
Deep Learning of Railway Track Faults using GPUs 21
Highly parallel problem:
▪ Create more instances of Fault- and Anomaly-Detectors depending on available
GPUs
▪ Ideally: real-time processing (120-160 km/h) HPC