VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

y

SHRP 2 Summer Safety Symposium SHRP 2 Summer Safety Symposium S06/S07 Contractors MeetingS06/S07 Contractors Meeting

Washington, DCWashington, DC

DAS UpdateDAS Update

Tom DingusJuly 13, 2010

1

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yDAS UpdatesDAS Updates

DAS Overview

Technology Enhancements

Procurement Process & Acceptance Testing

Driver ID Status

NIST Rodeo EvaluationNDS

2

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yData Acquisition SystemData Acquisition System

Multiple Videos•

Machine Vision Eyes Forward Monitor

•

Machine Vision Lane Tracker

Accelerometer Data (3 axis)

Rate Sensors (3 axis)

GPS•

Latitude, Longitude, Elevation, Time, Velocity

Forward Radar•

X and Y positions•

Xdot and Ydot Velocities

Cell Phone•

ACN, health checks, location notification

•

Health checks, remote upgrades

Illuminance sensor

Passive alcohol sensor

Incident push button

Video

Audio (only on incident push button)

Turn signals

Vehicle network data•

Accelerator•

Brake pedal activation•

ABS•

Gear position•

Steering wheel angle•

Speed•

Seat Belt Information•

Airbag deployment•

etc

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

y

VTTI Machine Vision Lane Tracking VTTI Machine Vision Lane Tracking

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

y

Mask_Perclose

VTTI Driver Monitor used for drowsiness detection (post hoc, compressed video sample)

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yNextGen Data Acquisition NextGen Data Acquisition SystemSystem

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yDAS StatusDAS Status

Updates to system•

Solid State Drive•

Lower failure rate

•

Higher capacity -> increased time between drive swap

•

FCC system upgrades•

Upgraded system to be consistent with requirements needed to comply with FCC regulations

7

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yDAS Configuration ExampleDAS Configuration Example

Ford Focus Test Vehicle•

Head Unit

•

Main Unit•

Radar

8

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yHead Unit and Passenger ViewHead Unit and Passenger View

9

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yRadar MountRadar Mount

10

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yMain Unit On Packing ShelfMain Unit On Packing Shelf

11

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yDAS Procurement Process DAS Procurement Process --

IFQCIFQC

Step 1 -

Invitation for Qualified Contractors (IFQC)•

Purpose: establish a Qualified Contractor List (QCL) of eligible Contract Manufacturers (CMs) based on specific criteria

•

Process: applicants rated based on criteria by selection committee made up of representatives from TRB, VT and NIST

12

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yDAS Procurement Process DAS Procurement Process --

RFPRFP

Step 2 -

Request for Proposal (RFP)

•

Purpose: receive proposals to select one of the qualified CMs for the manufacturing and support of the Data Acquisition System (DAS)

•

Key Criteria:•

Price of Goods•

Price of Services•

Ability to meet delivery requirements•

Services Offered•

Prototype Evaluation•

Selection Process:•

Initial Screening •

Plant Visit •

Prototype Evaluation 13

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yAcceptance Testing OverviewAcceptance Testing Overview

•

Three competing contract manufacturers (CM)•

Each CM’s system exposed to identical conditions by installing one system from each manufacturer in each test vehicle.

•

Three test vehicles•

All three had•

3 head units•

3 rear cameras•

3 radar interface boards•

3 power connections•

1 radar

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

y““HydraHydra””

Vehicle ConfigurationVehicle Configuration

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yVehiclesVehicles

2008 Ford Fusion2006 Chevy Malibu

2007 GMC Sierra Pickup

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yProcessProcess

•

A 52 mile / ~ 1.5 hr route was driven repeatedly.•

Route included•

Rough gravel road (6 mi / ~ 20 min)•

Paved roads -

city / interstate / 2 lane / 4 lane

•

1 short ignition on/off/on cycle (< 15 sec)•

2 long ignition on/off/on cycles (> 5 min)

•

Vehicles were in process 19 -22 hrs per day (includes stops when changing drivers, fueling, and during ignition cycles).

•

System hard drives were swapped frequently.•

The three vehicles traveled together.

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yTotalsTotals

VEHICLE MILES COVERED HOURS OF DRIVING

CHEVROLET MALIBU 3,143 64.5

GMC SIERRA TRUCK 3,683 81

FORD FUSION 3,938 89.5

TOTAL 10,764 235

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yDAS Procurement OverviewDAS Procurement Overview

Status•

Contract Manufacturer (CM) selection complete following formal bid process

•

RFP designed to ensure ability of CM to build and support a quality product in a timely manner

Cost•

Final price from CM is within budget

Schedule•

PO has been issued and CM is on track to deliver to support S07 installation schedule

19

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

y

SHRP2 NDS: Driver IdentificationSHRP2 NDS: Driver Identification

1.9 million hours of video …

who’s driving?

20

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yFacial RecognitionFacial Recognition

Facial recognition apps process digital images to detect faces in a “query”

image, then use pattern matching

algorithms to compare it against a gallery of “reference” images to determine a probable match

Algorithms tuned to minimize False Acceptance and Rejection Rates

Developed for use in a constrained environment where variables such as lighting, head pose, and facial expression are carefully controlled •

widely used in the security arena for “mug shot”

identification

The SHRP2 NDS environment is NOT constrained!

21

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

ySHRP2 NDS: an unconstrained SHRP2 NDS: an unconstrained

environment environment

Variable lighting conditions•

unevenly lighting•

strong shadows and washed out faces•

night driving with IR illumination

Variable facial expressions

Variable head pose

Facial occlusions•

hand motion•

coffee/drink cup•

sunglasses

Image density •

number of pixels between the eyes

22 22

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

y

SHRP2 NDS will produceSHRP2 NDS will produce……

23

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yFace Recognition solutionsFace Recognition solutions

Vendors are starting to loosen limits inherent in unconstrained environments by addressing issues related to:•

off-axis head poses

•

variable lighting conditions•

IR illumination

•

occlusions•

face expression

•

skin color

24 24

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yDriver ID Vendor EvaluationsDriver ID Vendor Evaluations

Animetrics, Inc. –

in discussion to evaluate one of several

products:•

Animetrics90 SDK •

FaceR CredentialME•

http://animetrics.com

STAR Technologies -

evaluated and found to be acceptable,

though processing intensive•

STARFace SDK•

http://www.startechnologies-usa.com

Pittsburgh Pattern Recognition –

evaluated and rejected:

could not accommodate head pose angles typically found in SHRP2 NDS•

FTR-SDK (Face Recognition Software Development Kit)•

http://pittpatt.com25

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yNIST and Rodeo EvaluationNIST and Rodeo Evaluation

Objectives: 1.

Provide NIST a dataset to assess the S06 DAS configuration

2.

Provide a dataset to Iowa State to determine the feasibility of using the S06 DAS to supplement roadway data

3.

Provide a dataset for other researchers to assess for use in upcoming s08 RFPs

26

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yDASs DASs

Three Systems running simultaneously

Synchronized via GPS time1.

S06 DAS as previously shown •

Chevy Malibu

2.

NIST instrumentation•

Battery packs required due to current draw•

S06 DAS designed to shut down if current draw is too high

3.

Differential GPS and Crossbow running on a separate laptop•

One ground truth comparison•

GPS: Omnistar Service•

Crossbow: angle, rate, accel @ 100Hz

27

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yNIST InstrumentationNIST Instrumentation

28

Challenge: NSIT instrumentation likely impedes s06 lane tracker

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yRural Route and Urban RouteRural Route and Urban Route

Leesburg Route (~ 41 miles each direction)

Arlington Route (urban canyon)    

(~ 11 miles each direction)

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yProcedureProcedure

The Rural route will be run with all three instrumentation packages•

NIST instrumentation removed in the field and sent back with chase van

Rural route will be rerun with s06 DAS and Differential GPS/ Crossbow packages•

Highly surveyed routes will be run 3 times each to allow comparisons

Urban route will be rerun with s06 DAS and Differential GPS/ Crossbow packages•

Highly surveyed routes will be run 3 times each to allow comparisons

30

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yDriver and Vehicle SampleDriver and Vehicle Sample

General Sample 1600 DASs•

Driver Demographic is the most important criteria

•

Exception need 230 of each vehicle type

Advanced Vehicle Sample 350 DASs•

the vehicle is the most import criteria

•

Vehicle-specific sampling will be done to ensure good representation of critical areas•

Driver Assist including collision avoidance and adaptive cruise control

•

Infotainment including Sync, navigation and integrated communication

31

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

y

Participating OEMSParticipating OEMS

General Fleet

32

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yWhat is needed from the What is needed from the

nonnon--advanced vehicle fleet?advanced vehicle fleet?

The codes necessary to obtain information from the vehicle network and the vehicles models that should be included in the sample•

2002 or newer vehicles

•

Initial selection = biggest seller for each OEM•

Including vehicles in the sample that support CAN

•

Each OEM provided feedback on the list including model and year

Worked with each OEM to put an NDA in place so we could have acess

to the codes

Negotiated with each participant OEM on what information they would provide

33

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yOEM participantsOEM participants

GM

Ford

Nissan

Honda

Hyundai

Kia

Chrysler

Toyota

Audi/VW

34

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yBiggest Remaining ChallengesBiggest Remaining Challenges

Programming our instrumentation to understand all the different CAN messages•

Determining when each vehicle updated protocol or messages and how often

Determine a method to reliably obtain all the model features for each OEM. •

VIN

•

Trim

Sampling newer vehicles in the fleet than we originally wanted

35

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

y

Participating OEMSParticipating OEMS

Advanced Fleet

36

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yRecommendations on Advanced Recommendations on Advanced

Vehicle FleetVehicle Fleet

A recommended sample of 50 advanced vehicles from each OEM•

Recommend the vehicle model, trim and features

•

Determine what information from the vehicle network can be shared in the public database

Help obtain the names and phone numbers of people who own the vehicles with these features at the data collection sites.

37

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yOEM participantsOEM participants

GM

Ford

Nissan

Honda

Toyota

Audi/VW

38

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yAdvanced VehiclesAdvanced Vehicles

Hybrid vehicles•

Pedestrian and bicyclist interactions•

Commuting behavior•

Driving behavior

In-vehicle communication systems and advanced infotainment system•

Interaction type, duration, •

When and when interacts•

Safety…. Etc

Driver monitoring, feedback, collision warning systems•

System on•

Alerts

39

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yDrive Assist Features IncludedDrive Assist Features Included

Front & Rear Park Assist

Forward Collision Alert

Adaptive Cruise Control

Lane Departure Warning

Side Blind Zone Alert

Rear Video

Collision Imminent Braking

Pre-crash

Includes: Feature activation & alert occurrence at a minimum

40

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yHybrid, communication, and InfotainmentHybrid, communication, and Infotainment

Hybrids are in the fleet

OnStar

and Sync for communication

Infotainment•

Mode –

Voice, Center Stack, Steering Wheel

•

What is being requested (HVAC)•

Not adjust to 76 degrees

41

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yBiggest Remaining Challenges (1)Biggest Remaining Challenges (1)

Huge amount of information available on the CAN and multiple CAN networks in use

One OEM over 600 messages of interest•

Reducing the sample down to something reasonable but still keep necessary information for researchers

•

Programming our instrumentation to understand all the different CAN messages

•

Determining when each vehicle updated protocol or messages and how often

Determine a method to reliably obtain all the model features for each OEM. •

Negotiating what information with each OEM that we can put in the researcher database

42

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

yBiggest Remaining Challenge (2)Biggest Remaining Challenge (2)

Some vehicles are so new that they are new not currently in the fleet.•

Advantage is newest possible systems

•

Disadvantage is not many in the fleet.•

We are looking at shifting advance vehicle instrumentation later in the flow to accommodate.

•

Assumption: Not acceptable to delay instrumentation

43

VTTI

Driv

ing

Tran

spor

tatio

n w

ith T

echn

olog

y

Questions?

Thanks!

44