Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents •...
Transcript of Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents •...
![Page 1: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/1.jpg)
Denoising for Monte Carlo Renderings
Bing Xu 徐冰2020.03.19
![Page 2: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/2.jpg)
Contents• Background knowledge
• Monte Carlo Integration for Light Transport Simulation
• Various Ways to Reduce Variances (noise)
• Sampling & Reconstruction for MC Renderings
• Image-space Denoising (biased)
• Adversarial Monte Carlo Denoising with Conditioned Auxiliary Feature Modulation
• Motivation & Contributions
• Performance & Evaluation
• Limitations & Future work
![Page 3: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/3.jpg)
Background Recap
camera info, lighting, geometries, textures
Photorealistic Rendering
[Scene from Kujiale]
![Page 4: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/4.jpg)
Monte Carlo Path Tracing
! Physically based! Very general: Monte Carlo estimators help to get rid of the high dimensionality
of the problem! Convergence is guaranteed
! Disadvantages:! Slow convergence: variance ~1/sqrt(N)! sparse sampling => noise
![Page 5: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/5.jpg)
How to reduce variances within time limits! Sampling
! Importance sampling! Adaptive sampling! Various sampling operators….
! Reconstruction (balance between bias & variance)! A prior methods: Analyze light transport equations for individual samples, reconstruction filters
based on analysis. [Zwicker et.al. 2015]! A posterior methods: Ignorant of light transport effects, reconstruction based on empirical statistics.
! Others! Control variates! MCMC
![Page 6: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/6.jpg)
Primary focus
❏ “A posterior” method [ Zwicker et al. 2015] ❏ Low sample counts ( 4spp, 16spp, 32spp ..)❏ Guided by per-pixel auxiliary feature buffers (albedo , normal, depth..)
❏ Much cheaper! ❏ Contain rich information
❏ CNN based - possible to involve much larger pixel neighbourhoods while
improving speed.
![Page 7: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/7.jpg)
Sample rays for each pixel
Image-space denoising seconds
Keep sampling to convergence hours/days
Rendered image with 4spp (MC path tracing) Noisy free image
![Page 8: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/8.jpg)
Adversarial Monte Carlo Denoising with Conditioned Auxiliary Feature Modulation
BING XU, KooLab, Kujiale, ChinaJUNFEI ZHANG, KooLab, Kujiale, ChinaRUI WANG, State Key Laboratory of CAD & CG, Zhejiang University, ChinaKUN XU, BNRist, Department of Computer Science and Technology, Tsinghua University, ChinaYONG-LIANG YANG, University of Bath, UKCHUAN LI, Lambda Labs Inc, USARUI TANG, KooLab, Kujiale, China
![Page 9: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/9.jpg)
Motivation & Contribution
![Page 10: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/10.jpg)
[Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder]Loss function = spatial loss*a + gradient loss*b + temporal loss*c
Better results for high frequency area
Original a, b, c Use larger b at the begining
Motivation 1 : Loss automation
![Page 11: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/11.jpg)
Me
Reconstructed image
weights of loss combination
Network Retrain
Motivation 1 : Loss automation
![Page 12: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/12.jpg)
Motivation 1 : Loss automation
Me
Reconstructed image
weights of loss combination
Network Retrain
CriticNetAdversarial loss
![Page 13: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/13.jpg)
Visual perceptual quality
Lower pixel-wise loss (mostly used) != Better visual perceptual quality
Ideal case:
A differentiable metric which naturally reflects human visual system.
Reality:
No direct definition or knowledge of the data distribution
Then we can take advantage of implicit models.
![Page 14: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/14.jpg)
Adversarial MC denoising framework
DenoisingNet
DenoisingNet
CriticNet
Noisy Diffuse
Noisy Specular
Output Diffuse
Output Specular
Output
GT Diffuse
GT Specular
Auxiliary Features
CriticNet
![Page 15: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/15.jpg)
Adversarial MC denoising framework
DenoisingNet CriticNet
Noisy SpecularOutput Specular
GT Specular
Auxiliary Features
![Page 16: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/16.jpg)
Adversarial MC denoising framework
DenoisingNet CriticNet
Noisy Specular
Auxiliary Features
Output Specular
GT Specular
G: DenoisingNet D: CriticNet
![Page 17: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/17.jpg)
Training Details & Datasets❏ WGAN-GP and auxiliary features
help stabilize GAN’s training.❏ Datasets
KJL indoor scenes by FF RendererTungsten scenes by Benedikt Bitterli https://benedikt-bitterli.me/resources/ released by Disney
![Page 18: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/18.jpg)
Noisy color image Reconstructed noise-free image
Auxiliary feature buffers
Image-space denoising:
Motivation 2: How to use the auxiliary features more wisely?
![Page 19: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/19.jpg)
Motivation 2: How to use the auxiliary features more wisely?
Noisy color image Reconstructed noise-free image
Conditioning On
Auxiliary feature buffers
Image-space denoising:
![Page 20: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/20.jpg)
Noisy color image Reconstructed noise-free image
Conditioning On
Auxiliary feature buffers
Image-space denoising:
Expectations:
1. To extract more clues from auxiliary feature buffers.
2. To explore the correct relationship between noisy image and aux features.
Motivation 2: How to use the auxiliary features more wisely?
![Page 21: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/21.jpg)
Noisy color image Reconstructed noise-free image
Conditioning On
Auxiliary feature buffers
Image-space denoising:
Extract deep features using NN.
Try more complex interaction to model the relationship.
Expectations:
1. To extract more clues from auxiliary feature buffers.
2. To explore the correct relationship between noisy image and aux features.
Motivation 2: How to use the auxiliary features more wisely?
![Page 22: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/22.jpg)
Concatenation on all layers
Different Ways of Network Conditioning:Traditional approach: Concatenation based conditioning. [Bako et al. 2017 ; Chaitanya et al. 2017]
Input layer
Auxiliary features
Linear
OutputCon
cate
natio
n
Motivation 2: How to use the auxiliary features more wisely?
[Dumoulin et al. 2018]
![Page 23: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/23.jpg)
[Dumoulin et al. 2018]
Concatenation on all layers Conditional biasing
Input layer
Auxiliary features
Linear
OutputCon
cate
natio
n
Auxiliary features
Input layer
Linear
Mapped to bias vector
Output
Motivation 2: How to use the auxiliary features more wisely?
Different Ways of Network Conditioning:Traditional approach: Concatenation based conditioning. [Bako et al. 2017 ; Chaitanya et al. 2017]
![Page 24: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/24.jpg)
Conditional scaling
Auxiliary features
Input layer
Linear
Mapped to bias vector
Output
[Dumoulin et al. 2018]
Motivation 2: How to use the auxiliary features more wisely?
![Page 25: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/25.jpg)
Conditional biasing
Conditional scaling
[Dumoulin et al. 2018]
Motivation 2: How to use the auxiliary features more wisely?
Different Ways of Network Conditioning:Traditional approach: Concatenation based conditioning. [Bako et al. 2017 ; Chaitanya et al. 2017]
![Page 26: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/26.jpg)
Conditional biasing
Conditional scaling
[Dumoulin et al. 2018]
Shazam!
Motivation 2: How to use the auxiliary features more wisely?
Different Ways of Network Conditioning:Traditional approach: Concatenation based conditioning. [Bako et al. 2017 ; Chaitanya et al. 2017]
![Page 27: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/27.jpg)
Auxiliary Buffer Conditioned Modulation
![Page 28: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/28.jpg)
Other details❏ Auxiliary feature buffers:
❏ Can be obtained from GBuffer or at first bounce of path tracer.❏ Extensible, you can try more.
❏ Diffuse/Specular decomposition (same as in KPCN)❏ A simplified light path decomposition. ❏ Attention: specular here is not the accurate specular but (color - diffuse)❏ Necessary if calculating an untextured color buffer
![Page 29: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/29.jpg)
Complete Framework
![Page 30: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/30.jpg)
Results & Performance
![Page 31: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/31.jpg)
Evaluation
SOTA Baselines:
NFOR [Bitterli et al. 2014], KPCN [Bako et al. 2017], RAE [Chaitanya et al. 2017]
![Page 32: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/32.jpg)
Examples of public scenes
More results with a html interactive viewer can be seen on http://adversarial.mcdenoising.org/interactive_viewer/viewer.html
![Page 33: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/33.jpg)
Examples of public scenes
More results with html interactive viewer can be seen on http://adversarial.mcdenoising.org/interactive_viewer/
viewer.html
![Page 34: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/34.jpg)
Reconstructed diffuse results
![Page 35: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/35.jpg)
Reconstructed specular results
![Page 36: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/36.jpg)
Reconstruction performance
For 1280x720 image:
Ours: 1.1s (550ms for diffuse/specular) single 2080Ti
KPCN: 3.9s single 2080Ti
NFOR: more than 10s, 3.4GHz Intel Xeon processor
![Page 37: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/37.jpg)
Analysis & Discussion
![Page 38: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/38.jpg)
Effectiveness of the adversarial loss and critic network
Control groups:❏ L1 loss (KPCN tests many loss functions L1, L2, SSIM etc and L1 shown to
be the best)❏ L1 with adversarial loss
![Page 39: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/39.jpg)
Effectiveness of the adversarial loss and critic networkL1 Loss L1 and Adversaria Loss
![Page 40: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/40.jpg)
Effectiveness of the adversarial loss and critic networkL1 Loss L1 and Adversaria Loss
![Page 41: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/41.jpg)
Effectiveness of auxiliary feature buffers
![Page 42: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/42.jpg)
Effectiveness of feature conditioned modulation
No auxiliary features Concatenate the auxiliary features & noisy color as fused input
Full model of CFM Reference
![Page 43: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/43.jpg)
Previous work & Proposed conditioned feature modulation
❏ Traditional feature-guided filtering: ❏ generally based on joint filtering or cross bilateral filtering [Bauszat et al.2011]❏ handcrafted assumption on the correlation between the low-cost auxiliary features and noisy image
❏ Learning based approaches: concatenation as fused input❏ Limit the effectiveness of auxiliary features to early layers❏ amounts to biasing
❏ Combination of conditional biasing and scaling:❏ perform scaling and shifting at different scales❏ point-wise shifting modulates the feature activation.❏ point-wise scaling selectively suppresses or highlights feature activation.
![Page 44: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/44.jpg)
Effectiveness of feature conditioned modulation
![Page 45: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/45.jpg)
Diffuse and specular decomposition
Reflection is not reconstructed well without separating diffuse and specular components
Reflection is well reconstructed by separating diffuse and specular components
![Page 46: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/46.jpg)
Convergence discussion
![Page 47: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/47.jpg)
Limitation, future work, conclusion
![Page 48: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/48.jpg)
Limitations
![Page 49: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/49.jpg)
Future work
❏ Network optimization & speedup❏ Model simplification❏ Custom-precision❏ Model pruning
❏ Temporal coherence❏ Explore more complex relationship between noisy input and auxiliary features
❏ Attention mechanism❏ Hypernetworks
❏ More rendering effects❏ Depth of field❏ Motion blur..
❏ How to do without large training set? (expensive)
![Page 50: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/50.jpg)
Conclusion
❏ Adversarial learning framework for MC denoising problem❏ Shed light on exploring the relationship between auxiliary features and noisy
images by neural networks.❏ Open source code and weights released on http://adversarial.mcdenoising.org.
![Page 51: Denoising for Monte Carlo Renderings · Renderings Bing Xu 徐冰 2020.03.19 Contents • Background knowledge • Monte Carlo Integration for Light Transport Simulation ... Reality:](https://reader033.fdocument.pub/reader033/viewer/2022051811/601d7f7ad7b3b56024730e6f/html5/thumbnails/51.jpg)
Thank you!AcknowledgementWe gratefully thank the anonymous reviewers for their constructive suggestions, and Qing Ye, Qi Wu, Junrong Huang for helpful discussions and cluster rendering support. This work is partially funded by National Key R&D Program of China (No. 2017YFB1002605), NSFC (No. 61872319, 61822204, 61521002), Zhejiang Provincial NSFC (No. LR18F020002), CAMERA - the RCUK Centre for the Analysis of Motion, Entertainment Research and Applications (EP/M023281/1), and a gift from Adobe.