Streaming Meshes
description
Transcript of Streaming Meshes
Martin Isenburg
UC Berkeley
Streaming Meshes
Peter Lindstrom
LLNL
Indexed Formats
# triceratops.obj## 2832 vertices# 2834 polygons#v 3.661 0.002 -0.738v 3.719 0.347 -0.833v 3.977 0.311 -0.725v 4.077 0.139 -0.654 ⋮ ⋮ ⋮ ⋮ f 2806 2810 2815 2821f 2797 2801 2811 2805f 2789 2793 2802 2796f 2783 2794 2788 ⋮ ⋮ ⋮ ⋮
2806
Original Orderings
showing the first 20to 40 percent of thetriangle array
Large Meshes
3D scans isosurfaces
Large Meshes
3D scans isosurfaces
Mesh Pieces
6.1 GB
Input Problem
186 million vertices= 2 GB
372 milliontriangles= 4 GB
Indexed Mesh
6 GB
Input Problem
186 million vertices= 2 GB
372 milliontriangles= 4 GB
Indexed Mesh
6 GB
External Memory Mesh
12 GB
Input Problem
186 million vertices= 2 GB
372 milliontriangles= 4 GB
Indexed Mesh Triangle Soup
12 GB6 GB
Papers from 1997 - 2005• Viz 1997, “I/O Optimal Isosurface Extraction”, Chiang, Silva
“unfortunately, data sets are often given in formats that contain indices to vertices”
• SIGGRAPH 2000, “Out-of-Core Simplification”, Lindstrom“operate on a triangle soup in which each triangle”
• SIAM 2003, “Adaptive Vertex Clustering”, Schaefer, Warren“we also assume that the mesh is in a triangle soup format”
• GI 2003, “Decimation of Massive Meshes”, Wu, Kobbelt“in this format (dubbed “triangle soup” in [14]) every triangle”
• TVCG 2003, “External Memory Mesh”, Cignoni et al.“from a triangle soup (list of faces, not indexed)”
• SIGGRAPH 2004, “Tetra Puzzles”, Cignoni et al.,“we assume […] triangle soup, a flat list of triangles”
• SIGGRAPH 2005, “Far Voxels”, Gobbetti, Marton “[…] we assume […] represented as a triangle soup”
Large Model Distribution
“Digital Michelangelo Project” Stanford University
“3D Gallery” Visual Computing Lab, ISTI
Reading Indexed Input • one-pass
– memory-map the vertex array
– rely on paging system of OS
– will “fail” on incoherent input
[Chiang & Silva 1997]• multi-pass– sort triangle array on each index field
– replace index with vertex
– large disk space and I/O overhead
Output ProblemPieces
372 milliontriangles= 4 GB
186 million vertices= 2 GB
Indexed
Writing Indexed Output• use computer with lots of memory
– Stanford used an SGI Onyx2
• output piece by piece– concatenate in final pass
• memory-map the mesh– number of vertices known a priori ?
• output into two separate files– concatenate in final pass
Format Conversion
standard indexed formats streaming formatspieces external soup
372 milliontriangles= 4 GB
186 million vertices= 2 GB
possibly not IO-efficient
IO-efficient
+ finalization !!!
Streaming Formats
Streaming Formats• physical
– water in a pipe
– drip coffee
• digital– streaming formats
• audio
• video
• geometry
Two Types of Streaming• progressive
• non-progressive
Non-Progressive Streaming• consume immediately
• potentially without end
• keep small buffer
• delete data if no longer needed
small window
Streaming Mesh Formats
interleave introduce finalize
v 1.32 0.12 0.23v 1.43 0.23 0.92v 0.91 0.15 0.62f 1 2 3finalize 2 v 0.72 0.34 0.35f 4 1 3finalize 1 ⋮ ⋮ ⋮ ⋮
vertex # 2finalized
not used bysubsequent
triangles
vertex # 2introduced
not used byprecedingtriangles
active
Reading Streaming Formats mesh.open ( “mesh.sma” );
while ( event = mesh.read_event() ) switch ( event ) case TRIANGLE:
// look-up v[0], v[1], v[2] in hash case VERTEX: // put new vertex in hash case FINALIZE: // remove finalized vertex from hash
mesh.close ();
• span– maximal “time” a vertex stays active how “long” vertices remain in the buffer
• do not “bloat” width and span !!!– introduce late and finalize early
• width– maximal number of active vertices how many vertices need to be buffered
Streaming Mesh Definitions
Writing Streaming Formats• isosurface
– 235 million vertices
– 469 million trianglesover
8 Gigabyte
• marching cubes– extract layer by layer
– output elements as extracted
– finalize vertices of previous layer
Richtmeyer-Meshkov instability simulation at LLNL
Layout Diagramtriangles
vert
ices
Coherence in Indexed MeshesIndexed Streaming
visualization of thecoherence
vertex spans
triangle spans
tells us how difficult itis to convert to stream
Layout Diagram: Buddha
triangles
vert
ices
20 MB
Layout Diagram: Dragon
triangles
vert
ices
20 MB
Layout Diagram: Lucy
triangles
vert
ices
508 MB
Layout Diagram: St. Matthew
triangles
vert
ices
6,760 MB
338 MB
676 MB
vertex span
Visualizing high vertex spans
Topologic and Geometric Sortdepth-first breadth-first
along one axisz-order
Output Qualities• low width & low span
– sweep-line
– FIFO (queue)
• low width, high span– block by block
– FILO (stack)
• high width, high span– random access
Streaming with minimal width• graph theory
– minimization is NP hard
– heuristic: “spectral sequencing”
Stream-Processing
unprocessed region
Stream-Processing Model
in-corebuffer
processed region
Stream-Processing Tasks• tasks that process mesh elements
one at a time– e.g. for each triangle t …
• tasks that only require access to local neighbors– e.g. for each triangle t of vertex v …
• tasks that are order independent– e.g. collapse edges shorter than
Streaming Simplification
“A Stream Algorithm for … ” [Wu & Kobbelt ‘03]
“Large Mesh Simplification …” [Isenburg et al. ‘03]
Performance
“External Memory Management and Simplification of Huge Meshes” [Cignoni et al. ‘03]
12 GBconstruct
11 hrs
simplify80 MB
14 hrs
Octree-based External Memory Mesh
(figure courtesy of Paolo Cignoni)
15 MB
45 min
compared to:
Streaming Compression
“Streaming Compression of Triangle Meshes”[Isenburg, Lindstrom & Snoeyink ‘05]
Performance
“Out-of-Core Compression of Gigantic Polygon Meshes” [Isenburg & Gumhold ‘03]
11 GBconstruct
7 hrs
compress384 MB
4 hrs
compared to:
12 MB
28 min
344 MB 392 MBfile sizeOut-of-Core Mesh
Pipelined Streaming
P1P1 P2P2 P3P3
P1P1P2P2
P3P3
P1P1 P2P2 P3P3
• pipelined stream-processing
Pipelined Stream-Processing• conventional processing
P1P1 P2P2 P3P3
– super-linear speedup
– minimal end-to-end I/O delay
– optimal disk caching
Demo Pipeline
256
256
256
regular volume grid
smextract | smclean | smsimp | smcompress
P2P2 P3P3P1P1 P4P4
grid.raw mesh.smc
v 1.32 0.12 0.23v 1.43 0.23 0.92v 0.91 0.15 0.62f 1 2 3finalized 2 v 0.72 0.34 0.35f 4 1 3finalized 1 ⋮ ⋮ ⋮ ⋮
Conclusion
Indexed Formats do not Scale
9 GB1 MB
Small Change … Big BenefitsStandard Format Streaming Formats
Current / Future Work• more stream modules
– streaming re-meshing
– streaming smoothing
– streaming feature extraction
– streaming stripification, …
• stream other geometry data– volumetric meshes / grids
– points in 2D and 3D
Streaming Volume Meshes• format
– like surface mesh format
– straight-forward
• applications– streaming iso-surface extraction
– streaming simplification
– streaming compression
– streaming mesh refinement, …
Volume Mesh Compression
torso
fighter
“Tetrahedral Mesh Compression with the Cut-Border Machine” [Gumhold et al. ‘99]
140 MB
662 sec
6 MB
12 sec
1.81 3.56
compress
bits / tet
compared to:
Layouts of Volume Meshes
torso.off v = 168,930 tet = 1,082,723 (19 MB)
fighter.off v = 256,614 tet = 1,403,504 (25 MB)
rbl.off v = 730,273 tet = 3,886,728 (70 MB)
Streaming Points• format
– need “spatial finalization” • e.g. no future points in this “space”
• applications– streaming surface reconstruction
– streaming re-sampling
– streaming smooting
– streaming compression, …
Streaming Delaunay
450 MB
250 sec
4 MB
125 sec2D Delaunay triangulate6 million terrain points
2D Delaunay 3D Delaunay
Acknowledgements• meshes
– Stanford University
– Cyberware
• support– NSF grant 0429901
"Collaborative Research: Fundamentalsand Algorithms for Streaming Meshes."
– U.S. DOE / LLNL # W-7405-Eng-48
Which models have such layouts?
Thank You
demo & API : http://www.cs.unc.edu/~isenburg/sm
Why?
Read the paper!