JK Jung2018/05

Applications of Real-time Object Detection on NVIDIA Jetson TX2

自主創新Rapid Innovation

綠能環保Sustainable Energy

雲端應用Cloud Solutions

移動生活Mobile Lifestyle

新興市場Emerging Markets

JK Jung (鍾俊魁) @ IIoT Center AI Team, Inventec

2

• Blog: https://jkjung-avt.github.io/

• GitHub: https://github.com/jkjung-avt/

NVIDIA JETSON TX2 FORSMART CITY APPLICATIONS

Inventec Confidential 3

4

Cloud

NVR/Server

ControlCenter

Smart Camera (IVS)

SOS Emergency

AI Gateway

LED Light

Solar Power

Battery

Display Panel

Smart Streetlights

Sensors

Real Deployment at Taoyuan Industrial Park

5

Illegal Parking Detection

Smart Streetlight

IP-CAM * 2

WiFiAntenna

IoT Gateway

IVS (TX2)

More Deployment Cases

6

Parking Lot Vehicle CountingTraffic Counting

Traffic Counting Dashboard (Control Center)

7

WeeklyReport

HourlyReport

DEVELOPING OBJECT DETECTION ALGORITHMS ON NVIDIA JETSON TX2

Inventec Confidential 8

Faster R-CNN (FRCN)

9

Courtesy of https://blog.csdn.net/majinlei121/article/details/53870433

Single Shot Multibox Detector (SSD)

10

Courtesy of https://arxiv.org/pdf/1512.02325.pdf

Applying Object Detection Models on Jetson TX2

• To run Faster R-CNN on Jetson TX2: https://jkjung-avt.github.io/faster-rcnn/

• To run SSD on Jetson TX2: https://jkjung-avt.github.io/ssd/

• Observations:– Faster R-CNN is more accurate and could pick up smaller objects

– But Faster R-CNN is too slow (1~2 fps) for real-time edge analytics

– Training with more data does improve accuracy (mAP) of the models

• To improve inference speed of the object detection models:– Using faster CNN feature extractors

– Applying TensorRT: https://developer.nvidia.com/tensorrt

– Designing the model with less anchor boxes

– Trade-off (input image size) between mAP and inference time

11

Input Image Dimension

VOC2007 mAP

Inference Speed on Jetson TX2

Comments

VGG16 (original) 1000x600 0.69+ 900 ms

GoogLeNet 1000x600 0.69 480 ms

GoogLeNet +TensorRT

1280x720 0.69 200 ms

12

Faster R-CNN

SSDInput Image Dimension

VOC0712 mAP

Inference Speed on Jetson TX2

Comments

VGG16 (original) 300x300 0.72 160 ms

VGG16 + TensorRT

300x300 0.72 75 ms

GoogLeNet 300x300 0.70 60 ms

GoogLeNet +TensorRT

300x300 0.70 28 ms > 30 fps

FUTURE DIRECTIONS

Inventec Confidential 13

Future Directions

• People counting and tracking

• Boat/vessel counting at the harbor

• Water level monitoring (flooding alert)

• More advanced event detection about people:– Fight

– Crime, robbery, etc.

– Fall and anesthesia detection for elderly

• More advanced event detection for vehicles and roads:– Traffic collision

– Unloading cargos from trucks or vans

– Scattered material, or wandering animals

– Road construction

14

Anomaly Detection

THANK YOU!

Questions and Answers