Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method...
-
date post
15-Jan-2016 -
Category
Documents
-
view
221 -
download
0
Transcript of Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method...
![Page 1: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/1.jpg)
Independent Component Analysis
From PCA to ICABell Sejnowski algorithmKurtosis methodDemonstrations
![Page 2: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/2.jpg)
![Page 3: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/3.jpg)
![Page 4: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/4.jpg)
![Page 5: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/5.jpg)
![Page 6: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/6.jpg)
![Page 7: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/7.jpg)
![Page 8: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/8.jpg)
![Page 9: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/9.jpg)
![Page 10: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/10.jpg)
![Page 11: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/11.jpg)
Bell and Sejnowski 1995
Consider y=g(x)+noise with f depending on w
I(y;x)=H(y)- H(y|x)
H(y|x)=E_x E_y|x [-log P(y|x)]
![Page 12: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/12.jpg)
![Page 13: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/13.jpg)
![Page 14: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/14.jpg)
![Page 15: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/15.jpg)
![Page 16: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/16.jpg)
![Page 17: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/17.jpg)
![Page 18: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/18.jpg)
![Page 19: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/19.jpg)
ICA based on KurtosisOja and Hyvarinen
![Page 20: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/20.jpg)
![Page 21: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/21.jpg)
![Page 22: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/22.jpg)
![Page 23: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/23.jpg)
![Page 24: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/24.jpg)
![Page 25: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/25.jpg)
![Page 26: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/26.jpg)
![Page 27: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/27.jpg)
![Page 28: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/28.jpg)
![Page 29: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/29.jpg)
![Page 30: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/30.jpg)
![Page 31: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/31.jpg)
Independent Component Analysis
Perform “blind separation” of signals recorded at multiple sensors
Use minimal assumptions about the characteristics of the signal sources.
An overview of applications of ICA to biological data and general data mining,Computational Neurobiology Laboratory Salk Institute, La Jolla CA (April, 1999).
Enter [Enter] to advance, [up-arrow] to rewind.
![Page 32: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/32.jpg)
Principle: Maximize Information
• Q:Q: How to extract maximum
information from multiple visual
channels?
Set of 144 ICA filters
• AA: ICA does this -- it maximizes
joint entropy & minimizes
mutual information between output
channels (Bell & Sejnowski, 1995).• ICA produces brain-like visual filters
for natural images.
![Page 33: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/33.jpg)
ICA versus PCA
• Independent Component Analysis (ICA) finds directions of maximal independence in non-Gaussian data (higher-order statistics).
• Principal Component Analysis (PCA) finds directions of maximal variance in Gaussian data (second-order statistics).
![Page 34: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/34.jpg)
Example: Audio decomposition
Play Mixtures Play Components
Perform ICA
Mic 1
Mic 2
Mic 3
Mic 4
Terry Scott
Te-Won Tzyy-Ping
![Page 35: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/35.jpg)
Electroencephalography (EEG)
• ICA separates
brain signals from
artifacts.
Artifacts
Brainsignals
• Allows monitoring
of multiple brain
processes.
• Permits study of
brain activity in
noisy conditions.
![Page 36: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/36.jpg)
Functional Brain Imaging
• Functional magnetic
resonance imaging (fMRI)
data are noisy and
complex.
I C A C o m p o n e n t T y p e s
S u s t a i n e d t a s k - r e l a t e d
( a )
T r a n s i e n t l yt a s k - r e l a t e d
( b )
S l o w l y - v a r y i n g
( c )
Q u a s i - p e r i o d i c
( d )
A b r u p t h e a dm o v e m e n t
( e )
A c t i v a t e dS u p p r e s s e d
S l o w h e a dm o v e m e n t
( f )
• ICA identifies concurrent
hemodynamic processes.
• Does not require a priori
knowledge of time courses
or spatial distributions.
![Page 37: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/37.jpg)
Data Mining
• ICA was applied to Armed Forces Vocation Aptitude Battery (ASVAB) test scores and Navy Fire Control School grades.
• ICA may suggest more efficient and balanced selection criteria.
• Two ICA components contributed to final school
grade.
![Page 38: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/38.jpg)
This presentation by
• Scott Makeig, Naval Health Research Center, San Diego
• Tzyy-Ping Jung, Institute for Neural Computation,
UCSD, La Jolla CA
• Te-Won Lee, Salk Institute, La Jolla CA
• Sigurd Enghoff, Salk Institute
• Terrence J. Sejnowski, Salk Institute & UCSD
![Page 39: Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations.](https://reader035.fdocument.pub/reader035/viewer/2022062322/56649d635503460f94a46085/html5/thumbnails/39.jpg)
From Barak Pearlmutter Contextual ICA The first demo applies the Contextual ICA blind source
separation algorithm. Lucas Parra and I digitally extracted ten five-second clips from ten audio CDs. These were digitally mixed, without time delays or echos, and with random gains, to form the output of a synthetic microphone. Ten such microphone outputs were synthesized. These synthetic microphone outputs formed the input to the Bell-Sejnowski Independent Components Analysis algorithm. The sources are somewhat separated in the output of the Bell-Sejnowski ICA algorithm, but not fully.
The same synthetic microphone outputs were then used as input to our new cICA algorithm (see my publications page for technical details). The sources are almost fully separated in the output of cICA.