Barycentric Differential
Partial differential = relationship between two coordinate systemschange in barycentric coordinate related to change in screen space
-1-
Screen space (x, y) Barycentric space (ฮฒ, ฮณ)
๐ ๐ท๐ ๐โฆ
๐ ๐ฝ๐ ๐ฆ
๐ ๐ฝ๐ ๐ฅ
Detecting Edge
-2-
If any test of four corner is failed, it is edge!Sampling more rays for the case
Edge Detection Example
Geometric edge detection, not visual edge Recklessly modeled object
-3-
Image Comparison (0-depth)
-4-
1X non-sample 4x proposed 4X super-sampling
Image Comparison (4-depth)
-5-
1X non-sample 4x proposed 4X super-sampling
Performance Comparison
-6-
1x 4x ours 4x SS 8x ours 8x SS0
5000000
10000000
15000000
20000000
25000000
# of rays (0 depth)
PrimaryShadow
1x 4x ours 4x SS 8x ours 8x SS0
5000000
10000000
15000000
20000000
25000000
30000000
35000000
# of rays (4depth)
PrimaryShadowReflectionRefraction
1x 4x-ours 4x-ss 8x-ours 8x-ss
0-depth 6.54 11 25.68 16.69 50.2
4-depth 11.05 18.82 42 29.3 82.13
โข Rendering Time
โข # of Rays
Todos
๋ ผ๋ฌธ๋ช ์ ์ถ์ฒ ์ํ
A Latency-efficient Stencil/depth Buffer System for Mobile 3-D Rendering Processors
IEICE Express On review
An Effective Depth Data Memory System Using an Escape Count Buffer for 3D Rendering Processors
IEICE Express Accepted
๋ฌด๊ฒ ์ค์ฌ ์ขํ๊ณ์ ํธ๋ฏธ๋ถ์ ์ด์ฉํ ๊ด์ ์ถ์ ๊ธฐ๋ฒ์ ํ ์ค์ฒ MIPMAP
๋ ๋ฒจ ์ ํ ์๊ณ ๋ฆฌ์ฆ๊ฒ์ํํ ๋ ผ๋ฌธ์ง ๊ณํ ์ค
-7-
๋ ผ๋ฌธ ์์ฑ ๊ณํโ Adaptive antialiasing using ray differentials
โข ์ถ๊ฐ ์คํ ํ์ , PBRT ์ ์ฉ ํ์ ๋ ผ๋ฌธ ์์ฑ
2011/06Jeongsoo Park
The Graduate SchoolYonsei University
Department of Computer Science
The Design of Acceleration Algorithm for Real-time Ray Tracing
Agenda
Introductionโ Benefit of Ray Tracingโ Why todayโs issue?โ Contributions
Background & Related Workโ Basics of Ray Tracingโ Acceleration technique
Ray Differentials for Adaptive Level-of-Detailโ Derivation of ray differentialโ Ray-diff for texture filteringโ Ray-diff for adaptive antialiasing
Probability Heuristics for Ray Tracing Accelerationโ Traditional SAHโ Surface Probability Heuristics
Conclusion & Future Work
-9-
Introduction
Application of computer graphics is getting wider
Quality of computer graphics is getting real and faster
-10-
Toy story, 1995, 11 hours Crysis, 2011, 30 ms
Game Movie Medical AR/VR
Introduction (cont.)
Ray tracing is capable of high degree of realistic imageโ Support reflection, refraction, caustics naturallyโ Calculate physically-correct lighting effectโ Simulate interference between primitives in the scene
-11-
Background (cont.)
Ray tracing is basically โSEARCHโ algorithmโ Rasterization sorts the primitive into 2-D grid of buckets(pixels)
-12-
Processing PRIMITIVES to SORT onto image plane
Z-buffered rasterization
Processing RAYS to SEARCH contributing objects
Ray tracing
Ray differential unit for antialiasing
Introductionโ Ray tracing is regular sampling texture aliasingโ Solutionโข Stochastic strategy
More rays per pixel, EXPENSIVE (Distribution ray tracing, Path tracing)โข Filtering strategy
Filtering out the area within a pixel or using pre-filtered texture How to know minification ratio
-13-
Ray differential unit for antialiasing (cont.)
Related worksโ Distance based [BMRT1996]โข Ray is linearly diverging โ twice the distance, twice the differenceโข Used in RenderMan [Apodaca1999]โข Only valid for PRIMARY RAYโข Not support reflection/refraction of ray
-14-
Ray differential unit for antialiasing (cont.)
Tracing ray differentialโ Partial derivatives of rayโs starting point (P) and directions (D)โ Difference of P and D in accordance with that of x and y.
-15-
๐ ๐ฅ
๐ (๐ฅ , ๐ฆ)
๐ (๐ฅ+๐๐ฅ , ๐ฆ )
๐ท(๐ฅ , ๐ฆ )
๐ท(๐ฅ+๐ ๐ฅ , ๐ฆ )๐ ๐ท โฒ โฒ๐๐
๐ ๐ซ โฒ โฒ๐ ๐
๐ ๐ท โฒ๐ ๐
๐ ๐ซ โฒ๐ ๐
Ray differential unit for antialiasing (cont.)
For primary ray
-16-
View
Right
Up
Differentiating
Ray differential unit for antialiasing (cont.)
For reflection ray
-17-
Differentiating
ND Phit
Drefl
Ray differential unit for antialiasing (cont.)
For refraction ray
-18-
Differentiating
ND Phit
Drefr
Ray differential unit for antialiasing (cont.)
Proposed differential unit operates in three modes (raytype)
-19-
Ray differential unit for antialiasing (cont.)
Proposed differential unit operates in three modesโ Similar pipeline process ( 1DIV= 2MUL)
-20-
Primary ray
DOT3 (1)
DIV (1)
DOT3(2)
SQRT (1)
MUL (3)
MUL (12)
ADD (6)
MUL (1)
MUL (1)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
Reflection ray
DOT3 (1)
MUL (1)
MUL (3)
MUL (1)
MUL (3)
ADD (6)
DOT3(1)
Refraction ray
DIV (1)
MUL (1)
MUL (1)
ADD (1)
MUL (2)
MUL (12)
ADD (6)
DOT3 (1)
DOT3(2)
Ray differential unit for antialiasing (cont.)
Application of ray differential (Texture filtering)โ Calculate length of projected area withโข R1= , R2 =
โ Level of textureโข LOD = [Heckbert1983]
-21-
Ray differential unit for antialiasing (cont.)
Image comparisonโ Distance method excessively blurs (blue)
,or does not filter (red)
-22-
Distance Method Ray Differential MethodBase Level (l=0 always)
Base Level (l=0 always) Distance Method Proposed method
Ray differential unit for antialiasing (cont.)
Image comparison (cont.)โ Distance method could not support convergence/divergence of rayโ Proposed method selects appropriate level of filtered texture
-23-
Base Level (l=0 always) Distance Method Proposed method
Surface Probability Heuristics
Introductionโ If ray tracing is a SEARCH algorithm, pre-calculated indexing is desirableโ Acceleration structure is index of geometry distributionโข Spatial - GRID[Klimaszewski1997], KD-tree[Fussel1998]โข Object - BVH[Goldsmith1987]
โ KD-tree is known as the best AS for rendering speed [Havran2000]โข Redundant primitive reference
-24-
Surface Probability Heuristics (cont.)
Related worksโ Object median[Goldsmith1987]
โ Spatial medianโข Bad for skewed distribution
โ Surface Area Heuristics [Mac-donald1990]โข Split minimizing a cost functionโข Empty space cut, but sweeping
โ Hybrid approach[Shevtsov2007]โข Upper-level : medianโข Lower-level : SAH
-25-
โProposed division scheme is enhancement of SAH
Surface Probability Heuristics (cont.)
Surface Area Heuristicsโ Probability that arbitrary ray goes through space VL
=
โ Then, cost of traversing this node
-26-
KT = Cost of visiting this nodeKI = cost of intersection testTL|R = Number of primitives
V
VL VR
p
Surface Probability Heuristics (cont.) Proposed scheme
โ Calculate all probabilities of inbound and outbound sub-surface
โ Probability of ray come IN with โaโ
โ Probability of ray go OUT with โbโ
โ Probability of ray hit left
โฆ
โ Probability of hit โbothโ (left to right)
)โฆ
-27-
a e
b f
c g
d h
i j
development figure
a
b
e
f
j
โ
Surface Probability Heuristics (cont.)
Experimental resultโ ~ 82.3% reduction in tree size
โ ~69.3% reduction in primitive redundancy
-28-
kitchen sponza0
200000400000600000800000
10000001200000140000016000001800000
# prim. index
SAHProposed
kitchen sponza0
100000
200000
300000
400000
500000
600000
700000
Tree size
SAHProposed
Surface Probability Heuristics (cont.)
Experimental resultโ ~ 15.8% reduction in tree size
โ 3% overhead in rendering performance
-29-
kitchen sponza020406080
100120140160180200
# traversal
SAHProposed
kitchen sponza0
20
40
60
80
100
120
rendering time
SAHProposed
Conclusions
This research focuses on the research of algorithm design for memory reduction and visual improvement for real-time ray trac-ing โ Effective filtering kernel estimator by ray differentialโ Hardware design that switch it operating mode in accordance with type of
rayโ Advanced heuristics that reduces required memory for acceleration struc-
ture
-30-
BACKUP
-31-
The Design of Acceleration Al-gorithm for Real-time Ray Trac-
ing
Jeong-soo Park
2011/05/31Media System Lab.Yonsei Univ.
* ๋ชฉ์ฐจ
1. Introductionโ Benefit of Ray Tracing
โ Why todayโs issue?
โ Contributions
2. Background & Related Workโ Basics of Ray Tracing
โ Technical Categorization
โข RT execution platform
โข Acceleration technique
3. Ray Differentials for Adaptive Level-of-Detailโ Derivation of ray differential
โ Ray-diff for texture filtering
โ Ray-diff for adaptive antialiasing
4. Probability Heuristics for Ray Tracing Acceleration
5. Simulation Environment for Graphics Acceleration Hardware
6. Conclusion & Future Work
Introduction
* Benefit of Ray Tracing
โข High degree of realistic image
โ Support reflection, refraction, caustics naturally
โ Calculate physically-correct lighting effect
โ Simulate interference between primitives in the scene
35
Ray tracing is one of global illumination algorithm to synthesize high degree realistic images as well as to
process 3D scene more efficiently than Z-buffer algorithm
Image of nVidia. :Optix : INTERACTIVE RAY TRACING ON NVIDIA QUADRO PROFESSIONAL GRAPHICS SOLUTIONS,โ developer.nvidia.com, 2009
* Barrier of Ray Tracing
โข Computation
โ All operations are based on real
numbers
โ High amount of computations
โข For example:โ Image resolution : 1280 x 720,
Scene size : 100K primitives, 10
propagations
โ 1280 x 720 x 100K x 10 = 858G
times intersection test and shading
36
However ray tracing has several barriers to achieve in real-time performance, which are its types and amount of
computations.
objectresolution
R N
propagation
D
Color O(#pixel * #obj * #propagation)
* Why Todayโs Issue
โข Hardware performance growth
โ Raw performance of CPU/GPU in-
creased
โข GPU as massive parallel architec-
ture
โ More parallelism exploited
โข Even CPU starts to expand parallel-
ism
โข Ray tracing algorithm is highly
parallel
37
Even so, there are several trends that realizes real-time performance of ray tracing.
*Nvidia developer center
Parallel architect`ure in commodity PC*
*Larrabee architecture
*Intel, โLarrabee: A Many-Core x86 Architecture for Visual Computing,โ SIGGRAPH 2008
* Why Todayโs issue
โข Industrial demands for advanced, realistic graphics
โ In addition to be getting complex, it becomes more real-time
38
Even so, there are several trends that realizes real-time performance of ray tracing.
Expendable(1999)Quake 3(1999) Half-Life 2(2004)
Real-time Rendering (Game)Crysis 2(2011)
A Bugs Life(1998)Toy Story(1995) Car(2006)Finding Nemo(2003)
TextureMapping
BumpMapping HDRI
RayTracing
High Quality Rendering (Movie)
Background & Related Work
* Basics of Ray Tracing
โข Ray tracing processes RAYS to SEARCH corresponding primitives
โ Basically searching algorithm
โ Z-buffered algorithm processes primitives to sort its projective position
Ray tracing algorithm is essentially searching algorithm that finds the primitive that contributes to the screen
Processing PRIMITIVES to SORT onto image plane
Processing RAYS to SEARCH contributing objects
Z-buffered rasterization Ray tracing
40
* Acceleration Algorithms
There are several works for accelerating ray tracing to real-time performance.
41
โข Ray tracing Acceleration works
โ Acceleration structure : pre-indexing primitives with auxiliary data structure
โ Core acceleration : optimize efficiency of RT core algorithm (traversal & intersection
test)
โ Execution platform : exploiting hardware resources efficiently Search Algo-
rithmRay tracing
Acceleration Structure
โข Grid[Kalojanov11], BVH[Lauterbach09], KD-tree, โข Fast build/update[Kalojanov09, Pantaleoni10]
Core Acceleration โข MLRTA[Reshtov05], Packet[Boulos08] โข Fast IST[Havel09]
Platform โข CPU/GPUโข GPU[Parker10, Aila09], Multi-core[Wald10, Seiler08]
โข GPU build [Lauterbach09]
Ray Differentials for Adaptive
LOD
* Introduction
โข Regular sampling of ray tracing aliasing problem
โข Solution
โ Stochastic sampling
โข More rays per pixel, EXPENSIVEโ Eg.) Distribution ray tracing, Path tracing
โ Texture filtering (like rasterization)
โข Filtering texels in a pixel, or choose from prefiltered texture map (MIPMAP)
โข Filtering kernel is rayโs footprint on texture space
How to know rayโs footprint along ray traversal
Like other rendering algorithm, ray tracing is another regular sampling technique. It also suffers from aliasing
problem.
43
* Related works
โข BMRT[Gritz96]
โ Estimating difference based on ray
traversal distance
โ Only valid for primary ray
โ Could not support convergence/di-
vergence of ray ( lens effect )
โข Tracing ray differential[Igehy99]
โ Derivatives of world coordinate po-
sition
โ Used for geometry subdivision in
offline renderer [Christensen03,
Christensen06]
Several works has been announced to eliminate aliasing problem in texture mapping
44
* Tracing Ray Differential
โข Essentially tracing derivatives of rayโs starting point (P) and its direction (D)
โ Keep track of whether ray is expanding or shrinking
45
๐ ๐ฅ
๐ (๐ฅ , ๐ฆ)
๐ (๐ฅ+๐๐ฅ , ๐ฆ )
๐ ๐ท (๐ , ๐ )
๐ ๐ซ (๐ , ๐ )๐ท(๐ฅ , ๐ฆ )
๐ท(๐ฅ+๐ ๐ฅ , ๐ฆ )
* Derivation of Ray Differential
Ray generation Ray differential
Primary ray
Reflection Ray
Refraction Ray
Barycentric Coor-dinate
Ray differential is partial derivatives of ray parameters of x, y. Hence ray differentials for each types of ray
could be found by differentiating ray generation equation.
46
* Ray differentials for texture filtering
โข Longer edge (R1 or R2) is selected
for filtering kernel size
โข For Mipmapping, appropriate level
of mipmap is calculated as:
47
size =
๐๐๐๐๐=(๐๐๐ ) log๐(๐๐๐๐)
* Ray differential pipeline design
โข Ray differential unit needs 3 DOT3, 17 MUL, 10 ADD, 1 DIV, 1 SQRT
48
At ray generation stage, differential unit calculates ray differential according to type of ray.
Primary ray
DOT3 (1)
DIV (1)
DOT3(2)
SQRT (1)
MUL (3)MUL (12)
ADD (6)
MUL (1)
MUL (1)
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
Reflection ray
DOT3 (1)
MUL (1)
MUL (3)
MUL (1)MUL (3)
ADD (6)
DOT3(1)
Refraction ray
DIV (1)
MUL (1)
MUL (1)
ADD (1)
MUL (2)MUL (12)
ADD (6)
DOT3 (1)
DOT3(1)
* Result
49
Scene comparison (no filtering โ distance based โ ray differential )
BMRT Method Ray Differential MethodBase Level (l=0 always)
Base Level (l=0 always) BMRT Method Ray Differential Method
*
Base Level (l=0 always) BMRT Method Ray Differential Method
* Result
51
Scene comparison (no filtering โ distance based โ ray differential )
Base Level (l=0 always) BMRT Method Ray Differential Method
Conclusion & Future work
* Conclusion & Future works
โข Ray differential could estimated size of ray footprint as the ray propagates
โข By differentiating function of world-to-texture space transformation, it is possible
to calculate footprint on the texture
โข Texture space footprint could be used to compute:
โ Filtering kernel size
โ Level of mipmap
โข Adaptive supersampling by differential scheme
โ Transform ray differential barycentric differential
53
Backup
* Benefit of Ray Tracing
โข Benefit โ High Image Quality
โข Support reflection, refraction, caus-
tics naturally
โข Calculate lighting effect globally
โ Efficient processing
โข Logarithmic behavior to scene
complexity
โข Only process the primitive that con-
tributes to final image
55
Ray tracing is one of global illumination algorithm to synthesize high degree realistic images as well as to
process 3D scene more efficiently than Z-buffer algorithm
*Daniel Pohl, โRay Tracing and Gaming,โ PC perspective articles from Intel, Jan. 2008
Z-buffer algorithm
Ray Tracing
ComputationBehavior of graphics algorithm*
*
์ฑ๋ฅ
์ฌ์ค์ฑ
56
ํ์ฌ ๊ทธ๋ํฝ์ค ๊ธฐ์ ์ ๋ฌธ์ ์
โข์ง์ญ ์กฐ๋ช ๊ธฐ๋ฐ์ผ๋ก ์ธํ ์ฌ์ค๊ฐ์ ํ๊ณโข ๊ฐ ๋ฌผ์ฒด์ ๊ด์๊ณผ์ ๊ด๊ณ๋ง ๊ณ ๋ ค
โข Eg. 1โ ์ ๋๋ฆฌ์์ง๋ 3โ ์ ๊ทธ๋ฆผ์ ํํ ๋ถ๊ฐโข์ธ์ ํ ๋ฌผ์ฒด๊ฐ์ ์์น ๊ด๊ณ๋ฅผ ํํํ์ง ๋ชปํจ
โข ํน์ ๋ฌผ์ฒด๋ฅผ ์ฒ๋ฆฌํ ๋ ๋ค๋ฅธ ๋ฌผ์ฒด์ ์์น๋ฅผ ๋ชจ๋ฆโข Eg. 4 ์ ๋น์น 1 ์ ์์์ ํํ ๋ชปํจ
โข์ฑ๋ฅํจ์จ์ด ๋ฎ์โข ๋น๊ฐ์ ์์ญ์ ๋ํด ๋ถํ์ํ ๋ ๋๋ง์์ ์ํ
โข Eg. 3 ์ ์ํด ๊ฐ๋ ค์ง๋ 2 ์ ์์ญ๊น์ง ๋ ๋๋ง์ํโข ์ฌ์ค๊ฐ ๊ฐ์ ์ ์ํ ์ค๋ณต ๋ ๋๋ง
โข Eg. 5 ๋ฅผ ๋ค์ ๋ ๋๋งํ์ฌ 1 ์ ๋ํ ๋ฐ์ฌ๋ฅผ ํํโข๋ฌผ์ฒด์ ์์ ์ ํ ๋ณต์ก๋๋ฅผ ๊ฐ์ง
โข ์ฅ๋ฉด์ ๊ตฌ์ฑํ๋ ๋ชจ๋ ๋ฌผ์ฒด (N ๊ฐ ) ์ ๋ํด ๋ ๋๋ง ์ํ (Lin-ear complexity: O(N) )
โข ๊ทธ๋ํฝ application ์ ์์ ๋ณต์ก๋๋ ๊ธ๊ฒฉํ ์ฆ๊ฐ์ถ์ธ
5โ
3
2
1
Graphics Pipeline
2
1โ
3โ
N 5
4
* Result
57
Scene comparison (no filtering โ distance based โ ray differential )
BMRT Method Ray Differential MethodBase Level (l=0 always)
Base Level (l=0 always) BMRT Method Ray Differential Method
* Global Illumination ๊ธฐ๋ฒ์ ์ฅ์
58
์ฑ๋ฅ
์ฌ์ค์ฑโข์ค๊ฐ์๋ ์ง์ ์กฐ๋ช ํจ๊ณผ ์ฌํ
โข ๊ด์ ์ ์ฐ์์ถ์ ์ ํตํด ์ค์ ์ ์ ์ฌํ ๋น์ ๋ฐ์ฌ , ๊ตด์ , ๊ทธ๋ฆผ์๋ฅผ ์ฌํ (simulation)
โข Eg. 1 ์์ ๊ด์์ ๋ํด ๊ทธ๋ฆผ์ ray ์ถ์ , 4 ์์ 1์ ๋ํ ๋ฐ์ฌ ray ์ถ์
โข๊ฐ์ ์กฐ๋ช ํจ๊ณผ ์ฌํ โข ๊ด์์์ ์์ฑ๋๋ ๊ฐ์ ์กฐ๋ช ํจ๊ณผ ์ง์
โข Eg. 3 ์ ์ํ ๊ฐ์ ์กฐ๋ช ํจ๊ณผ๋ก 1 ์ ์์์ด ๋ฐ๋
โขํจ์จ์ ์ธ ๋ ๋๋ง ์ฒ๋ฆฌโข ๋ฌผ์ฒด๊ฐ ์๋ ์์ฑํ ๊ด์ ์ ์์คํ ์์ ์ฒ๋ฆฌโข ๊ด์ ์ญ์ถ์ ์ผ๋ก ํ ์์ ์์ ๊ฐ์์์ญ๋ง ๋ ๋๋ง ์ํ
โข๊ณ์ฐ ๋ณต์ก๋๊ฐ ๋ฎ์โข ๊ณต๊ฐ์์ ๋ฐฐ์น๋ ๊ฐ ๋ฌผ์ฒด๋ฅผ tree ๊ตฌ์กฐํํ์ฌ ๋ฌผ์ฒด ์ (N) ์
๋ํด Log(N) ์ ๋ณต์ก๋๋ฅผ ๊ฐ์งโข ์ฅ๋ฉด ๋ณต์ก๋ ์ฆ๊ฐ์ ํจ๊ณผ์ ์ผ๋ก ๋์
1
23
1
Shadow ray
Eye ray
Reflection ray
Refraction ray
4
* Edge detection using ray differential
โข For each sub-pixel, check conditions:
Barycentric coordinate differential could be used to estimate hit-point of neighbor rays
๐๐๐ ๐ฅ
๐๐๐ ๐ฆ
3 4
1 2
* Why todayโs issue
โข Hardware performance growth
โ Raw performance of CPU/GPU in-
creased
โข GPU as massive parallel architec-
ture
โ More parallelism exploited
โข Even CPU starts to expand parallel-
ism
โข Ray tracing algorithm is highly
parallel
Even so, there are several trends that realizes real-time performance of ray tracing.
CE/ ํ์ค
CE/ ํ์ค
CE/ ํ์คGflops growth
CPUGPU
Gtx 480
Gtx 200
G92
G80
Core2 XeonI7 980
* ๋ชฉ์ฐจ
1. Introduction
โ Benefit of Ray Tracing
โ Why todayโs issue?
โ Contributions
2. Background & Related Work
โ Basics of Ray Tracing
โ Technical Categorization
โข RT execution platform
โข Acceleration technique
3. Ray Differentials for Adaptive LOD
โ Derivation of ray differential
โ Ray-diff for texture filtering
โ Ray-diff for adaptive antialiasing
4. New Heuristics for Accelerating RT
5. Simulation Environment for Advanced Computer Graphics
6. Conclusion & Future Work
* ๋ชฉ์ฐจ
โข Introduction
โข Background & Related Work
โข Ray Differentials for Adaptive LOD
โข New Heuristics for Accelerating RT
โข Simulation Environment for Advanced Computer Graphics
โข Conclusion & Future Work
* Result
63
Scene comparison (no filtering โ distance based โ ray differential )