670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 ·...
Transcript of 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 ·...
![Page 1: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/1.jpg)
Convolu'on and Non-‐parametric Density Es'ma'on
![Page 2: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/2.jpg)
Correla'on vs Convolu'on
• Correla'on: – Primary use: matching
• Convolu'on: – Models perturba'ons in the image genera'on process.
![Page 3: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/3.jpg)
Visualizing Image Filtering
hCps://developer.apple.com/library/ios/DOCUMENTATION/Performance/Conceptual/vImage/Convolu'onOpera'ons/Convolu'onOpera'ons.html
![Page 4: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/4.jpg)
Point Spread Func'ons
![Page 5: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/5.jpg)
A Pachinko Machine
![Page 6: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/6.jpg)
There goes the family fortune
![Page 7: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/7.jpg)
Maximum Likelihood Parameter Es'ma'on
• To the board! • Worked through how to find the maximum likelihood mean for a sample from a Gaussian distribu'on with unit variance and unknown mean. – Write down expression for likelihood of all the data points. – Take deriva've (of log likelihood) with respect to unknown mean.
– Set to 0 and solve. • Result: maximum likelihood mean is the empirical mean of the sample!
![Page 8: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/8.jpg)
Histogram dependence on bin posi'on
hCp://www.mvstat.net/tduong/research/seminars/seminar-‐2001-‐05/
![Page 9: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/9.jpg)
Gaussian Density Func'on (you might as well learn it now!)
![Page 10: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/10.jpg)
Likelihood of a new point x under a par'cular sample’s Gaussian
![Page 11: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/11.jpg)
Kernel Density Es'mate built from 12 samples.
![Page 12: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/12.jpg)
Kernel Density Es'mate
![Page 13: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/13.jpg)
KDE as convolu'on
• The density is the convolu'on of the sample with the kernel:
• p(x) = conv(S,kernel).
![Page 14: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/14.jpg)
Kernel Density Es'ma'on
hCp://www.mvstat.net/tduong/research/seminars/seminar-‐2001-‐05/
![Page 15: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/15.jpg)
Kernel Density Es'ma'on
hCp://www.mvstat.net/tduong/research/seminars/seminar-‐2001-‐05/
![Page 16: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/16.jpg)
Kernel Density Es'ma'on
hCp://www.mvstat.net/tduong/research/seminars/seminar-‐2001-‐05/
![Page 17: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/17.jpg)
Comparison of the histogram (le_) and kernel density es'mate (right) constructed using the same data. The 6 individual kernels are the red dashed curves, the kernel density es'mate the blue curves. The data points are the rug plot on the horizontal axis. (WIKIPEDIA)
![Page 18: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/18.jpg)
randn('seed',8192); x = [randn(50,1); randn(50,1)+3.5]; [h, iat, xgrid] = kde(x, 401); figure; hold on; plot(xgrid, iat, 'linewidth', 2, 'color', 'black'); plot(x, zeros(100,1), 'b+'); xlabel('x') ylabel('Density func'on') hold off;
![Page 19: 670 Unit3 Align conv NPDE - UMass Amherstelm/Teaching/ppt/670/670... · 2013-11-04 · 670_Unit3_Align_conv_NPDE.pptx Author: Erik Learned-Miller Created Date: 11/4/2013 2:09:43 PM](https://reader034.fdocument.pub/reader034/viewer/2022043010/5fa05ea767ca9c3f051b6382/html5/thumbnails/19.jpg)
Es'ma'ng the kernel “bandwidth”
• Use leave-‐one-‐out cross valida'on. • For each point, calculate probability under density es'mate with all the other points. This is the “leave-‐one-‐out es'mate” of that point.
• Now consider the product of the probabili'es of each point under its leave-‐one-‐out es'mate.
• Find the variance of the Gaussian kernel which maximizes the leave-‐one-‐out product.