Dedicated Software Algorithms for 3D Clouds of

Points

AQSENSE develops and commercializes 3D image acquisition and processingtechnologies that allow high speed in-line 100% production inspection, for the

Machine Vision Industry.

Introduction

Were are we ?

Parc Científic i Tecnològic de la UdG - 17003 Girona (Catalonia - Spain)

Presentation Outline

Part II: SAL3D Tools Overview

Part I: SAL3D PCM

Part III: Applications Examples

✔ Accurate Light Stripe Detection✔ Easy and Fast Profiles Merging✔ Metric Calibration for Linear and Angular Scanning✔ Measuring Geometric Primitives✔ Ultra-Fast Alignment for Pose Determination and Dimensional Control✔ Importing Models from CAD Designs✔ Integrating Multiple 3D Views into a Single Mesh✔ Easy Visualization of Rangemaps, COPs and ZMaps

✔ The ZMap as a “FLAT” representation of a COP

✔ Measuring the Diameter of a Tilted Circle✔ Dimensional Inspection

PART I

SAL3D Point Cloud Manager (PCM)

Point Cloud Manager (PCM)

Tool for management for 3D data: COPs and ZMaps

COP: Organized container for 3D data coming from different sources:

● Laser triangulation● Time-of-flight● Structured Light● Stereo vision

PCM: Zmap, the flat representation of the COP data

Orthogonal projectiononto the Z plane

A COP is an organized set of 3D points, expressed in X, Y and Z coordinates.It's like a set of 3 “images” of floating point numbers, representing the X, Y and Z sets of coordinates

A COP is an organized set of 3D points, expressed in X, Y and Z coordinates.It's like a set of 3 “images” of floating point numbers, representing the X, Y and Z sets of coordinates

A ZMap is the projection of a COP onto the Z=0 plane.Its horizontal and vertical coordinates are proportional to the actual X and Y coordinates of the COP, while its contents are the Z coordinate values.

A ZMap is the projection of a COP onto the Z=0 plane.Its horizontal and vertical coordinates are proportional to the actual X and Y coordinates of the COP, while its contents are the Z coordinate values.

The ZMap as a “FLAT” representation of a COP

Rangemap ZMap

Preserves metric properties

Independent of acquisition source

Can be used by 2D tools

Can be used to locate 3D features using 2D tools, in 3D space

Removes perspective distortion

YES

YES

ALWAYS

YES

NO

INDIRECTLY

SOMETIMES

NO

NO

YES

Rangemap vs. ZMap

PCM: Zmap, the flat representation of the COP data

Zmap data allows 3D measurements using common 2D tools

SAL3D contains examples to how to export 3D data to standard2D libraries ( Halcon, MIL and OpenCV).

Zmap factors allow the conversion from pixel data to metric data:

● Length● Diameter● Area and volume of defects● etc.

PART II

SAL3D Tools Overview

SAL3D Tools Overview

SAL3D Tools Overview

Accurate Light Stripe Detection

Grey level representation

Determinationof the maximumvalue positionup to 1/64th ofa pixel

A single point is obtained for each line across the laser stripe.This point is obtained at subpixel accuracy at a maximum of 1/64 pixel.Lab tests revealed 5 microns in Z, with a FOV of 130mm.

Range Map as the accumulation of laser profiles

Accurate Light Stripe Detection

Intensity profile of the stripe imageThe maximum intensity is in the middle of the stripe image.

The maximum intensity is NOTin the middle of the stripe image.

Intensity profile of the stripe image

COG & Peak

COGPeak

Ideal Case

Real Case

COGPeakCOG

PeakCOG Peak

Accurate Light Stripe Detection

The maximum intensity is NOTin the middle of the stripe image.

Intensity profile of the stripe image

● Presence of noise● Nature of surface material● Coatings

COGPeak

Accurate Light Stripe Detection

Light stripe detection

COG

Peak Detector

Better detection allows smaller details to be visible.

8x to 10x better detection accuracy.(3 orthogonal distance measurements of a plane reconstructed object)

SAL3D Tools Overview

MERGER Tool: Easy and Fast Profiles Merging

This area is not visible by camera A

A B

ADD MORECAMERAS

SAL3D Tools Overview

Metric Calibration for Linear and Angular scanning

Linear Scanning Angular Scanning

RANGEMAP CLOUD OF POINTS (COP)

Metric Calibration for Linear and Angular scanning

Multi-camera systemfor large objects@ high resolution

Single camera systemfor large objects@ low resolution

Special Multi-camera system for 360º viewscanning

Metric Calibration for Linear and Angular scanning

RANGEMAPRANGEMAP

DIRECT3D

REPRESENTATION

3DREPRESENTATION

AFTER METRICCALIBRATION A rangemap can create an

“illusion” of 3D. Not only metric coordinates are missing, but perspective distortion makes the shape completely different from what it really is.

A rangemap can create an “illusion” of 3D. Not only metric coordinates are missing, but perspective distortion makes the shape completely different from what it really is.

SCANNING A TILTEDCYLINDER

Metric Calibration for Linear and Angular scanning

The robot scans the part using one of its angular articulations.Using the accurate positioning of the robot, multiple scans can be overlapped and integrated with SAL3D Integrator Tool

Geometric Tool

The tool to allow you to made 2D measurements on planes not parallel to the 3D sensor.

Zmap from the acquired COPZmap after plane fitting and

correction on the acquired COP

Geometric Tool

This tool also allows planarity measurements and sphere fitting.

Ultra-Fast Alignment for Pose Determinationand Dimensional Control of all Parts Produced

Dimensional correctness inspection

Surface inspection

Quantification of tolerances in 3D, for complex, a-priory unknown scanned surfaces

Parts Pose Determination for picking operations

SAL3D Tools Overview

Ultra-Fast Alignment for Pose Determinationand Dimensional Control of all Parts Produced

A Part's Model and its Pose

The Scanned Part and its Pose

{M}

{P}

MTP

Match3D determines a part's pose, relative to a known model of it, in a small fraction of a second.Both Model and Part Cloud Of Points (COPs) consisting of 1 or more million points.Versions: Match3D and Match3D with Coarse.

Match3D determines a part's pose, relative to a known model of it, in a small fraction of a second.Both Model and Part Cloud Of Points (COPs) consisting of 1 or more million points.Versions: Match3D and Match3D with Coarse.

Ultra-Fast Alignment for Pose Determinationand Dimensional Control of all Parts Produced

Differences (or distances) in mm between the model and the scanned surfaceDISPARITY MAP

DISPARITY MAP

SAL3D can compute the difference between two Clouds Of Points, generating a disparity map.It's a floating point value image which directly conveys 3D metric localized information on the difference between model and part.

Cross-section of a shape deviation

SAL3D Tools Overview

Importing Models from CAD Designs

CAD Designs in IGES or STL formats can be imported to SAL3D COPs for being used in Match3D ApplicationsCAD Designs in IGES or STL formats can be imported to SAL3D COPs for being used in Match3D Applications

SAL3D Tools Overview

Integrating Multiple 3D Views into a Single Mesh

Multiple views aligned and overlapped

Single mesh after integration

A Mesh can be stored as a .PLY or .STL file with a simple function call.

Reverse Engineering, Rapid Prototyping, Archeology, etc...

SAL3D Tools Overview

Area Computation Tool

Applications

● Volume measure

● Density checking: With the help of the Area Tool the volume of an object is obtained. Then, knowing its weight, the density can be easily calculated

● Slicing: When automatically slicing food to be packed, neither extra material is desired nor can fewer product be given to the user. Computing the piece volume allows a precise cut with the exact weight.

Area of a closed profile

Area with respects to a reference line

Easy Visualization of Rangemaps, COPs and ZMaps

SAL3D allows the instantiation of viewer windows with their corresponding controls

PART III

Application Examples

Example #1: Measuring the Diameter of a Tilted Circle

Strategy:

1. Obtain a model of the part2. Align all parts, one after one, with the obtained model3. Align all parts to the Z=0 plane, using4. Generate a ZMap and feed into a 2D circle measurement tool5. Compute the hole diameter (in pixels)6. Convert the diameter in pixels into millimeters using the ZMap factors

Example #1: Measuring the Diameter of a Tilted Circle

MODEL PLANE Z=0

Getting a model of the partAlign the Model with plane Z=0 (ROBUST PLANE or arbitrary)

Example #1: Measuring the Diameter of a Tilted Circle

Align the part with the model using Match3DAlign the part with the Z=0 to generate a ZMapAlign the part with the model using Match3DAlign the part with the Z=0 to generate a ZMap

BEFORE ALIGNMENT AFTER ALIGNMENT

3D Transformations can be easily combined for efficiency

Example #1: Measuring the Diameter of a Tilted Circle

Generate a ZMap for each part after alignment with the Z=0 plane.

Example #1: Measuring the Diameter of a Tilted Circle

The Circle Diameter can be computed in pixels and converted to mm (ZMap factors)

Example #2: Dimensional Inspection

PSEUDO-COLOR REPRESENTATION GRAY-LEVEL REPRESENTATION

DISPARITY MAPDISPARITY MAP

Example #2: Dimensional Inspection

Disparity Map

Example #2: Dimensional Inspection

Disparity MapBinarization

Example #2: Dimensional Inspection

Disparity MapBinarization

Blob segmentation by size

Example #2: Dimensional Inspection

Disparity MapBinarization

Blob segmentation by size

mmPerGrayLevel = 0.024

Gray Level

mm

SUMMARY

SAL3D as a C++ Standard Library or integrated in CVB.

SAL3D technology is proven and present in different industries such as Food, Dental, Metal, ...

SAL3D technologies cover a very wide range of 3D MV applications.

Ease of Use, Robustness, Accuracy and Speed are the four pillars of AQSENSE's Technologies, embedded in SAL3D.

SAL3D can be easily combined with Standard 2D Machine Vision Libraries to enable a huge number of applications.