3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura...
-
Upload
3d-icons-project -
Category
Internet
-
view
374 -
download
0
description
Transcript of 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura...
![Page 1: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/1.jpg)
3D Processing and metadata
inges1on at POLIMI
Gabriele Guidi*
Sara Gonizzi Barsan1
Laura Loredana Micoli
Politecnico di Milano ‐ Mechanical Engineering Dept.
![Page 2: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/2.jpg)
Project
• 3 years EU-ICT pilot project
• Aim: supply Europeana with 3D items such as:
– Archaeological sites
– Architectures
– Monuments
– Artifacts
Including UNESCO World Heritage assets
http://3dicons-project.eu
![Page 3: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/3.jpg)
Project numbers
• 16 Partners
• 11 Countries
• 3000 3D models + metadata
• 36 months (30 months acquisition phase)
• Project: 100 models/month (average)
• POLIMI unit: 537 3D models + metadata
• POLIMI unit: 18 items/month (average)
Massive digitization project, each production step �
has to be greatly optimized! �
![Page 4: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/4.jpg)
Massive Digitization (Libraries)Book
(physical object)
Bibliographic record (descriptive metadata)
create/convert
2D scan
Digital object
storing
Repository
(technical metadata)
Metadata Record
Digital object url
![Page 5: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/5.jpg)
Massive Digitization (3DICONS)
storing
Repository
(technical metadata)
CH Asset(physical object)
3D capture
3D model(digital object)
(descriptive metadata)
create/convert
Digital object url
Metadata Record
![Page 6: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/6.jpg)
Source of Heritage Assets for POLIMI �The Archaeological Museum in Milan
The architectural structure Settled upon a complex stratification of archeological ruins, tangible sign of the ancient role of Milan as Capital of the Western Roman Empire
The content
1000+ archaeological items including: • epigraphs • statues
• mosaics • furniture
• potteries related to Greek, Etruscan, Roman and Medieval periods
![Page 7: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/7.jpg)
• Specific skill required
The complex ar1cula1on of data requires a high level of
exper1se in the archaeological/historical field
• Time consuming process Collec1ng the informa1on required for arranging
suitable descrip1ve metadata might require more 1me
than allowed by the project dura1on
Descriptive metadata (85%)
Technical metadata definition (1%)
3D acquisition and modeling (14%)
Metadata creation
✗ Only Heritage Assets with pre-existing metadata
have been chosen: conversion instead of creation�
![Page 8: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/8.jpg)
• POLIMI source of metadata is SIRBeC (Information System of Cultural Heritage of the Lombardia Region)
• All records can be exported in xml format
• The SIRBeC data structure is compliant with the CARARE metadata schema, used by 3DICONS as reference for structuring their metadata
• Only metadata mapping has to be designed
Metadata conversion
![Page 9: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/9.jpg)
Data collec1on workflow
![Page 10: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/10.jpg)
3D data collection
Image‐based
modelling
triangula1on‐
based systems TOF system
Small
texturized
objects
Small un‐
texturized
objects
Buildings
(77%)
(14%)
(9%)
![Page 11: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/11.jpg)
SFM• SFM is therefore the most used technology in
this project
• but SFM is nearly a "black box" giving an output with little of no way of intervention on the final output
• the only controllable inputs are good quality images
We need an optimized image acquisition protocol
in order to maximize the quality of 3D output �
![Page 12: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/12.jpg)
Possible imaging problems
• Image blurring due to: – Movement on shooting
– Wrong focusing
– Limited Depth of Field
• Lighting/dynamic range – Backlights/mixed color temp.
– Light spots
– Highlights
• Confusing scene elements – Painted walls/mosaics
– High contrast elements around the subject
![Page 13: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/13.jpg)
DEPTH OF FIELD
![Page 14: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/14.jpg)
Aperture tests on a small artifact
• Camera: Canon 5D mkII
• Sensor: Full frame CMOS 21.1 Mpixel
• Lens: 50mm macro
• Manual focusing on the
left eye @ x10
• Camera-target distance: 22.5 cm
• Avg. GSD: 28µm
15 cm
7.5 cm
![Page 15: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/15.jpg)
Aperture F 2.5
DOF @22.5 cm
2.4 mm
Focal Plane (FP)
FP + 23mm
![Page 16: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/16.jpg)
Aperture F 5.6
DOF @22.5 cm
5.1 mm
FP
FP + 23mm
![Page 17: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/17.jpg)
Aperture F 11
DOF @22.5 cm
10.2 mm
FP
FP + 23mm
![Page 18: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/18.jpg)
Aperture F 22
DOF @22.5 cm
20.4 mm
FP
FP + 23mm
![Page 19: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/19.jpg)
Aperture F 32
DOF @22.5 cm
28.8 mm
FP
FP + 23mm
![Page 20: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/20.jpg)
SFM/matching at different aperturesFront Le^ Right
• Automatic identification of tie points• Image orientation• Dense color cloud image matching (high)
All processing was made with AGISOFT Photoscan
![Page 21: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/21.jpg)
Matchable points vs. aperture
F Tie
points
Dense cloud
size
2.5 200 420637
Focal zone: 22.4 cm – 22.6 cm
![Page 22: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/22.jpg)
Matchable points vs. aperture
F Tie
points
Dense cloud
size
2.5 200 420637
5.6 408 1286552
Focal zone: 22.2 cm – 22.8 cm
![Page 23: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/23.jpg)
Matchable points vs. aperture
F Tie
points
Dense cloud
size
2.5 200 420637
5.6 408 1286552
11 694 2827132
Focal zone: 22 cm – 23 cm
![Page 24: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/24.jpg)
Matchable points vs. aperture
F Tie
points
Dense cloud
size
2.5 200 420637
5.6 408 1286552
11 694 2827132
22 848 3854555
Focal zone: 21.4 cm – 23.6 cm
![Page 25: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/25.jpg)
Matchable points vs. aperture
F Tie
points
Dense cloud
size
2.5 200 420637
5.6 408 1286552
11 694 2827132
22 848 3854555
32 1033 4117599
Focal zone: 21.1 cm – 24.1 cm
![Page 26: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/26.jpg)
LIGHTING/DYNAMIC RANGE
![Page 27: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/27.jpg)
Light filtering on shiny surfaces
No filter
Circular
polarizer
![Page 28: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/28.jpg)
Light filtering
No filter Circular
polarizer
![Page 29: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/29.jpg)
Matchable points vs. filtering
Filtering Tie
points
Dense
cloud size
None 978 1502661
Circular
polarizer
1051 1572751
+5% matched points
![Page 30: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/30.jpg)
HDR processing
• Enhances details in images containing both overexposed and underexposed areas
• Allows therefore to increase the number of points in image matching
• Since the SW manages jpegs only the full HDR is tone mapped and converted in 24 bit RGB
• This allows to increase by the number of matchable points in the darker areas
![Page 31: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/31.jpg)
HDR Processing example
0.60s, f/32Exposure OK
2.50s, f/32+2 stop
0.15s, f/32-2 stop
Tone mappedHDR
• 4 groups of 3 shots have been taken from different orientations
• SFM with:
– Properly exposed shots
– Corresponding one mapped shots
![Page 32: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/32.jpg)
Matchable points vs. processing
Processing Tie
points
Dense
cloud size
None 5467 5637516
HDR 5646 5858700
+4% matched points
![Page 33: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/33.jpg)
CONFUSING SCENE ELEMENTS�
![Page 34: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/34.jpg)
Masking images
Without mask
Withmask
![Page 35: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/35.jpg)
Photo shooting with backgrounds
![Page 36: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/36.jpg)
Corresponding landing page
![Page 37: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/37.jpg)
A few examples - Archaeological Museum, Milan
Texturized mesh models from SFM
![Page 38: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/38.jpg)
Good practices adopted
Issues
• Image blurring due to – Movement on shooting
– Wrong focusing
– Limited Depth of Field
• Lighting/dynamic range – Backlights/mixed color temp.
– Light spots
– Highlights
• Confusing scene elements – Painted walls/mosaics
– High contrast elements around the subject
Shooting/pre-processing solutions
– Tripod
– Manual focusing @ 10x
– Small apertures (16-32)
– Light shielding panels
– Mask post processing
– HDR/Polarizer filter
– Black/white background • hides confusing elements
• speeds up masking
![Page 39: 3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli](https://reader033.fdocument.pub/reader033/viewer/2022060202/559b7fcd1a28ab0e768b46be/html5/thumbnails/39.jpg)
Conclusions• The best practices of a massive
3D digitization project has been shown
• Metadata conversion from preexisting sources was needed for quickly generating searchable material
• SFM is a key technology for shortening 3D digitization to a sustainable level
• No many intervention is possible on SFM, the only actual action is improving image quality
• Proper imaging protocols may increase the 3D model quality and the success rate, in possible bad environmental conditions.