Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data...

93
新新新新新新新新新新新新新 Data and Data Analysis in the New I Data and Data Analysis in the New I T World T World 新 新 新新新新新新 新新新新新新新新 Norden E. Huang Research Center for Adaptive Data Analysis National Central University 2008

Transcript of Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data...

Page 1: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

新資訊世界的數據和數據分析Data and Data Analysis in the New IT WorldData and Data Analysis in the New IT World新資訊世界的數據和數據分析

Data and Data Analysis in the New IT WorldData and Data Analysis in the New IT World

黄 鍔國立中央大學 數據分析研究中心

Norden E. HuangResearch Center for Adaptive Data Analysis

National Central University

2008

黄 鍔國立中央大學 數據分析研究中心

Norden E. HuangResearch Center for Adaptive Data Analysis

National Central University

2008

Page 2: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

什麼是數據 ? What is data?What is data?什麼是數據 ? What is data?What is data?

DataData (plural of Datum) [(plural of Datum) [Latin: data – what is givenLatin: data – what is given] Inf] Information; Facts, evidence, records, statistics, etc. from ormation; Facts, evidence, records, statistics, etc. from which conclusions can be formed.which conclusions can be formed.

Information in a form suitable for storing and processing Information in a form suitable for storing and processing by a computer.by a computer.

數據就是信息 ,事實 ,證物 ,記錄 ,统計資料等等可用來推導結論

DataData (plural of Datum) [(plural of Datum) [Latin: data – what is givenLatin: data – what is given] Inf] Information; Facts, evidence, records, statistics, etc. from ormation; Facts, evidence, records, statistics, etc. from which conclusions can be formed.which conclusions can be formed.

Information in a form suitable for storing and processing Information in a form suitable for storing and processing by a computer.by a computer.

數據就是信息 ,事實 ,證物 ,記錄 ,统計資料等等可用來推導結論

Page 3: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Data, data, everywhere!Data, data, everywhere!

數據 , 數據 , 到處都是數據 !數據 , 數據 , 到處都是數據 !

Page 4: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

GoogleTrendGoogleTrend©© : Happy : Happy

Page 5: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

GoogleTrendGoogleTrend©© : Love : Love

lovelove     

Page 6: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

GoogleTrendGoogleTrend©© : Crisis : Crisis

crisis  1.00  

Page 7: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

DataData

““In God we trust”; everyone else has tIn God we trust”; everyone else has to show data.o show data.“我們相信上帝” ;其他人都得有數據 .

““In God we trust”; everyone else has tIn God we trust”; everyone else has to show data.o show data.“我們相信上帝” ;其他人都得有數據 .

Page 8: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

為什麼要分析數據 ?Why do we need to analyze data?Why do we need to analyze data?

為什麼要分析數據 ?Why do we need to analyze data?Why do we need to analyze data?

Data are the only connects between us and the Data are the only connects between us and the real world.real world.

Therefore, Therefore, data analysisdata analysis is the only way we can f is the only way we can find out what is the truth.ind out what is the truth.數據是我們與真實世界唯一的連繫數據分析就成了唯一尋求真理之道

Page 9: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

數據分析和數據處理之不同Data Processing and Data AnalysisData Processing and Data Analysis

ProcessingProcessing [proces < L. Processus < pp of Procedere = Proceed: [proces < L. Processus < pp of Procedere = Proceed: propro- forward + - forward + cederecedere, to go, to go] : A particular method of doing some] : A particular method of doing something.thing.( 數據 ) 處理只是照特定方法做特定的動作 .

AnalysisAnalysis [Gr. [Gr. anaana, up, throughout + , up, throughout + lysislysis, a loosing, a loosing] : A separatin] : A separating of any whole into its parts, especially with an examination of thg of any whole into its parts, especially with an examination of the parts to find out their nature, proportion, function, interrelatioe parts to find out their nature, proportion, function, interrelationship etc.nship etc.( 數據 ) 分析是將整體分為部份、特別指檢驗部份以決定各部份之特性、成份、機能和相關性 .

ProcessingProcessing [proces < L. Processus < pp of Procedere = Proceed: [proces < L. Processus < pp of Procedere = Proceed: propro- forward + - forward + cederecedere, to go, to go] : A particular method of doing some] : A particular method of doing something.thing.( 數據 ) 處理只是照特定方法做特定的動作 .

AnalysisAnalysis [Gr. [Gr. anaana, up, throughout + , up, throughout + lysislysis, a loosing, a loosing] : A separatin] : A separating of any whole into its parts, especially with an examination of thg of any whole into its parts, especially with an examination of the parts to find out their nature, proportion, function, interrelatioe parts to find out their nature, proportion, function, interrelationship etc.nship etc.( 數據 ) 分析是將整體分為部份、特別指檢驗部份以決定各部份之特性、成份、機能和相關性 .

Page 10: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Motivations for alternatives: Motivations for alternatives: Problems for Traditional MethodsProblems for Traditional Methods

Physical processes are mostly nonstationaryPhysical processes are mostly nonstationary

Physical Processes are mostly nonlinearPhysical Processes are mostly nonlinear

Data from observations are invariably too Data from observations are invariably too shortshort

Physical processes are mostly non-repeatable.Physical processes are mostly non-repeatable.

Ensemble mean impossible, and temporal mean Ensemble mean impossible, and temporal mean might not be meaningful for lack of might not be meaningful for lack of stationarity and ergodicity. Traditional stationarity and ergodicity. Traditional methods are inadequate.methods are inadequate.

Page 11: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Some New Approaches for Some New Approaches for Data AnalysisData Analysis

Categorization and classificationCategorization and classification

Quantification of complexityQuantification of complexity

Time-Frequency Analysis with Time-Frequency Analysis with Adaptive Basis. Adaptive Basis.

Page 12: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

1. Categorization or 1. Categorization or ClassificationClassification

The work by Linnaeus is the key to the progThe work by Linnaeus is the key to the progress in biological sciences.ress in biological sciences.

複雜數據分類複雜數據分類

Page 13: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Carl Linnaeus (Carl von Linné) Carl Linnaeus (Carl von Linné) 1707-1778

God created, Linnaeus organized.

With the exception of Shakespeare and Spinoza, I know no one among the no longer living who has influenced me more strongly.

Johann Wolfgang von Goethe

Page 14: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

數據分析的的第一步

一個數據分析的例子

數據分析的的第一步

一個數據分析的例子

Categorization or Categorization or ClassificationClassification複雜數據分類

Page 15: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

基於重複模式的 複雜數據分類Categorize complex signals based on the occurrenCategorize complex signals based on the occurren

ces of repetitive patterns:ces of repetitive patterns: Based on the work of Albert Yang (VGHTPE) anBased on the work of Albert Yang (VGHTPE) an

d C. K Peng (Harvard, Medical School)d C. K Peng (Harvard, Medical School)

Illustration of this Approach forcomparison of human literary texts

Repetitive patterns: words

Yang et al. Phys Rev Lett 2003; 90:108103; Physica A 2003:329:473-83; J Comput Biol 2005; 12:1103Peng et al. Chaos 2007 (in press)

Page 16: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Dis-Similarity Index

Page 17: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Phylogenetic Tree Distance matrix method

多基因源樹分析

Similar

Not similar

Page 18: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Phylogenetic Tree多基因源樹分析

Page 19: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

誰寫的莎翁劇本 ?Who Wrote Shakespeare’s Plays?Who Wrote Shakespeare’s Plays?

Shakespeare Marlowe

Page 20: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

這分析法就像把所有的字都丟到菓汁機一樣

It is like taking all the words and throwing them in the blender

~ a leading Shakespeare scholar

Page 21: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

文章比較分析Comparison of Human TextsComparison of Human Texts

WordWordRank Rank

(The Winter's (The Winter's Tale)Tale)

Rank Rank (Bonduca)(Bonduca)

THETHE 11 33

II 22 22

ANDAND 33 11

TOTO 44 44

OFOF 55 77

YOUYOU 66 2222

AA 77 55

MYMY 88 1010

THATTHAT 99 99

NOTNOT 1010 1515

Page 22: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

個體出現頻比圖Rank Comparison MapsRank Comparison Maps

Shakespeare vs. Shakespeare Shakespeare vs. Fletcher

Ran

k (C

ymbe

line

)

Ran

k (B

ondu

ca)

Page 23: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden
Page 24: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden
Page 25: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Authorship Problem

Chinese literary debate:Dream of the Red Chamber

Page 26: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Authorship Problem

Page 27: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

其他的數據分類應用 Other Applications of CategorizationOther Applications of Categorization

生物醫學應用Bio-medical ApplicationsBio-medical Applications

Page 28: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

應用於基因序列

Application to Genetic SequencesApplication to Genetic Sequences

Picture obtained from www.genetic-programming.org

Page 29: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

DNA “WordsDNA “Words 詞 ( 辭 ) ””

TACCCCCACTGTCAACCCAACACAGGCATG……

WordWord FrequencyFrequency RankRank

CCCCCC 633633 11

CCTCCT 543543 22

CTACTA 526526 33

AAAAAA 524524 44

ACCACC 515515 55

…… …… ……

5’ 3’

Page 30: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

個體出現頻比圖Rank Comparison MapsRank Comparison Maps

Same Species Different Species

Rank: Mitochondiral DNA (Human)

0 10 20 30 40 50 60

Ran

k: M

itoch

ondi

ral D

NA

(H

uman

)

0

10

20

30

40

50

60

AAA

AAT

AAC

AAG

ATAATTATC

ATG

ACA

ACT

ACC

ACG

AGA

AGT

AGCAGG

TAA

TAT

TAC

TAG

TTA

TTT

TTC

TTGTCA

TCT

TCCTCG

TGA

TGT

TGC

TGGCAA

CAT

CAC

CAG

CTA

CTT

CTC

CTG

CCA

CCTCCC

CCG

CGA

CGT

CGC

CGGGAA

GAT

GAC

GAG

GTA

GTT

GTC

GTG

GCA

GCT

GCC

GCGGGA

GGT

GGC

GGG

Rank: Mitochondiral DNA (Gorilla)

0 10 20 30 40 50 600

10

20

30

40

50

60

AAA

AAT

AAC

AAG

ATAATTATC

ATG

ACA

ACT

ACC

ACG

AGA

AGT

AGCAGG

TAA

TAT

TAC

TAG

TTA

TTT

TTC

TTGTCA

TCT

TCCTCG

TGA

TGT

TGC

TGGCAA

CAT

CAC

CAG

CTA

CTT

CTC

CTG

CCA

CCTCCC

CCG

CGA

CGT

CGC

CGG GAA

GAT

GAC

GAG

GTA

GTT

GTC

GTG

GCA

GCT

GCC

GCGGGA

GGT

GGC

GGG

Page 31: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

生理數據測試Quiz on physiologic dynamicsQuiz on physiologic dynamics

Heart Failure Heart Failure

Normal Atrial Fibrillation

• Loss of dynamical fluctuations is bad 失去動態變化是不好的失去動態變化是不好的• Not all dynamical fluctuations are good 但是但是 ,, 不是所有的動態變化都是好的不是所有的動態變化都是好的

Hea

rt R

ate

(bpm

)H

eart

Rat

e (b

pm)

Hea

rt R

ate

(bpm

)H

eart

Rat

e (b

pm)

Time (min) Time (min)

Page 32: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

D = 0.10

Health vs. Health

D = 0.25

Health vs. Disease

人類心率分析人類心率分析Comparison of Human HeartbeatComparison of Human Heartbeat

Page 33: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

人類心率的多基因源樹分析Phylogenetic Tree of Human HeartbeatPhylogenetic Tree of Human Heartbeat

Page 34: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

甴數據分類 到 數據定性與定量 From classification to From classification to

qualification and Quantificationqualification and Quantification

From relative comparisons to From relative comparisons to absolute values absolute values

Page 35: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

2. Quantification of 2. Quantification of ComplexityComplexity

生物系统都是非常複雜的生物系统都是非常複雜的All biological systems are extremely cAll biological systems are extremely c

omplex.omplex.

Page 36: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

量測複雜性 Quantification of ComplexityQuantification of Complexity

生物系统都是非常複雜的生物系统都是非常複雜的All biological systems are extremely cAll biological systems are extremely c

omplex.omplex.

Page 37: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

The complexity of a biological system should be a The complexity of a biological system should be a measure of the system’s measure of the system’s capacitycapacity to adapt and to adapt and function in an ever changing environment.function in an ever changing environment.

The system that can adapt to the most external The system that can adapt to the most external challenges (stresses) will have the best advantage challenges (stresses) will have the best advantage for survival.for survival.

複雜性是生物系统對無時不變環境適應的結果有適應性才能生存

什麽是複雜性 ?

What is Complexity?What is Complexity?

Page 38: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

AllAll biological systems will evolve to increase their biological systems will evolve to increase their dynamical capacity (complexity).dynamical capacity (complexity).

所有的生物都由演化而增加複雜性

複雜性增加的原則

Principle of Increasing ComplexityPrinciple of Increasing Complexity

Corollary:Corollary:Aging and disease will degrade a system’s complexityAging and disease will degrade a system’s complexity衰老和疾病都會使複雜性減低

Page 39: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Biological systems are constantly perturbed by external stimuli even under basal conditions.

Thus, the dynamical responses to these challenges will reflect a system’s complexity.

對外界刺激動態反應就表現它的複雜性對外界刺激動態反應就表現它的複雜性

如何測量複雜性如何測量複雜性 ??How to Quantify How to Quantify ComplexityComplexity??

Goal: To quantify dynamical complexity of a physiologic system by studying its fluctuations

目的目的 : : 甴研究數據的變動給複雜性定量甴研究數據的變動給複雜性定量

Page 40: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

diso

rder

orde

rgo

od v

aria

bili

ty

Page 41: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

order disorder

傳統熵的定量法

Conventional entropy measure

order disorder

理想的複雜性定量

Expected complexity measure

熵能給複雜性定量嗎 ?Can entropy be used as a complexity measure?Can entropy be used as a complexity measure?

Page 42: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

orde

rdi

sord

er

critical point

T

Scale-invariant symmetry

Page 43: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

複雜性定量的新方法A New Way to quantify complexity, MSEA New Way to quantify complexity, MSE

Multi-Scale Entropy (MSE) is a quantitative measure to estimate the complexity of a system through examining the information richness of its output signal on multiple scales.Costa, Goldberger, Peng: Phys Rev Lett 2002;89:068102 Phys Rev Lett 2003;91:119802 Phys Rev Lett 2004;92:089804 Phys Rev E 2005; 71:021906

多尺度熵多尺度熵 : : 複雜性定量法複雜性定量法 , , 用以檢驗多尺度數用以檢驗多尺度數據據

Page 44: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

計算多尺度熵 : Calculating Sample EntropyCalculating Sample Entropy

ln(patterns of length m) – ln(patterns of length m+1)

Pattern

Page 45: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

多尺度熵多尺度熵 : : 白噪音和 白噪音和 1/f1/f 噪音噪音 MSE - White and 1/MSE - White and 1/f f noises noises

1/f noise is more complex than white noise

Whitenoise

1/f noise

0.8

1.2

1.6

2

2.4

2.8

0 4 8 12 16 20

Scale factor

Sam

pE

n

Page 46: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

那一組數據更複雜 ? Which is the Most Complex?

Multiscale Entropy Analysis

AF

Healthy

CHF

0.5

0.8

1.1

1.4

1.7

2

2.3

0 4 8 12 16 20Scale factor

Sam

pEn

a) Healthy (n=18)b) Chronic Heart Failure (n=15)c) Atrial Fibrillation (n=9)

* Phys Rev Lett 2002;89:068102

Order Disorder

Two Patterns ofPathologic Breakdown

Healthy Dynamics: Multiscale Fractal Variability

Page 47: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Elderly

Young

0.6

0.8

1

1.2

1.4

1.6

1.8

0 4 8 12 16 20

Scale factor

Sam

pE

n多尺度熵分析健康的年青人和老人

MSE Analysis for Healthy Young vs. ElderlyMSE Analysis for Healthy Young vs. Elderly

Elderlyn=20; age 69±3

Young n=20; age 32 ±6

Page 48: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

壓力中心實驗 Center of Pressure (COP) ExperimentsCenter of Pressure (COP) Experiments

Experiment I – Analysis of Experiment I – Analysis of COP (sway) dynamicsCOP (sway) dynamics

15 healthy young15 healthy young22 healthy elderly22 healthy elderly22 fallers22 fallers

Experiment II – Noise-Experiment II – Noise-Enhanced Human Balance Enhanced Human Balance ControlControl * *##

15 healthy young 15 healthy young 12 healthy elderly12 healthy elderly

*A Priplata, J Niemi, J Harry, LA Lipsitz, and JJ *A Priplata, J Niemi, J Harry, LA Lipsitz, and JJ Collins. Lancet 2003;Collins. Lancet 2003;362362:1123.:1123.

## A Priplata, J Niemi, M Salen, J Harry, LA A Priplata, J Niemi, M Salen, J Harry, LA Lipsitz, and JJ Collins. PRL Lipsitz, and JJ Collins. PRL 2002;2002;8989:238101:238101

Page 49: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

壓力中心實驗數據Example: center of pressure dataExample: center of pressure data

young

elderly

Page 50: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

搖擺時間序列數據 : 那一組數據最複雜 ?Sway time series: which is most complex?Sway time series: which is most complex?

Young

Elderly

Faller

-60

-55

-50

-45

-40

0 5 10 15 20 25 30Time (sec)

Dis

plac

emen

t (m

m)

-80

-70

-60

-50

0 5 10 15 20 25 30

Time (sec)

Dis

plac

emen

t (m

m)

-45

-40

-35

-30

-25

0 5 10 15 20 25 30

Time (sec)

Dis

plac

emen

t (m

m)

-51

-50

-49

-48

-47

-46

8 9 10 11 12

-71

-70

-69

-68

-67

-66

9 10 11 12 13

-36

-35

-34

-33

-32

-31

12 13 14 15 16

Anteroposterior direction

Page 51: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

複雜性分析Complexity Analysis of the Sway Time SeriesComplexity Analysis of the Sway Time Series

Complexity: young > elderly > fallerComplexity: young > elderly > fallerp value < 0.001p value < 0.001

Page 52: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

3. Time-Frequency 3. Time-Frequency Analysis: Adaptive BasisAnalysis: Adaptive Basis

Spectral analysis for Nonlinear Spectral analysis for Nonlinear and nonstationary data and nonstationary data

Page 53: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

數據有趨勢該如何處理 ? What if the data has a trend?What if the data has a trend?

Problem with nonstationarityProblem with nonstationarity非平穩性問題非平穩性問題

Page 54: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Effect of trend on MSE analysisEffect of trend on MSE analysis

Nonstationarities defined on scales larger than those considered for thNonstationarities defined on scales larger than those considered for the MSE study may also substantially alter the results.e MSE study may also substantially alter the results.

1 2 3 4 5 6 7 8 9 100

1

2

3

Scalar factor

Sam

ple

Entr

opy

1/f noise

1/f noise with linear trend

0 500 1000 1500 2000 2500 3000-5

0

5

10

15

20

1/f noise

1/f noise with linear trend

Page 55: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Two Sets of DataTwo Sets of Data

Page 56: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

The State-of-the-ArtsThe State-of-the-Arts現階段的能力 :

“ 一個經濟學者的趨勢也是另一個經濟學者的週期” ““ OOne economist’s trend is another economist’s cyclene economist’s trend is another economist’s cycle” ”

Engle, R. F. and Granger, C. W. J. 1991 Engle, R. F. and Granger, C. W. J. 1991 Long-run Economic RelationshipsLong-run Economic Relationships. . Cambridge University Press.Cambridge University Press.

簡單週期簡單週期 Simple trend – straight lineSimple trend – straight line

隨機週期隨機週期 ---- 每季一條直線每季一條直線 Stochastic trend – stStochastic trend – straight line for each quarterraight line for each quarter

Page 57: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Philosophical ProblemPhilosophical Problem

名不正則言不順

言不順則事不成Without a proper definition, Without a proper definition,

logic discourse would be impossible.logic discourse would be impossible.Without logic discourse, Without logic discourse,

nothing can be accomplished.nothing can be accomplished.

ConfuciusConfucius

孔夫子孔夫子

名不正則言不順

言不順則事不成Without a proper definition, Without a proper definition,

logic discourse would be impossible.logic discourse would be impossible.Without logic discourse, Without logic discourse,

nothing can be accomplished.nothing can be accomplished.

ConfuciusConfucius

孔夫子孔夫子

Page 58: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

趨勢的定義 : Definition of the TrendDefinition of the Trend

Within the given data span, the trend is an intrinsically fitted monotonic function, or a function in which there can be at most one extremum.在一定的時段中 , 趨勢是遵照有適應性所定的單調方程或是只有一個極值的方程 The trend should be determined by the same mechanisms that generate the data; it should be an intrinsic and local property.

Being intrinsic, the method for defining the trend has to be adaptive. The results should be intrinsic (objective); all traditional trend determination methods give extrinsic (subjective) results.

Being local, it has to associate with a local length scale, and be valid only within that length span as a part of a full wave cycle.

Page 59: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

一個有適應性的基New Adaptive BasisNew Adaptive Basis

The Hilbert-Huang Transform:The Hilbert-Huang Transform:

Empirical Mode DecompositionEmpirical Mode Decomposition

Page 60: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

經驗模式分解 Empirical Mode Decomposition: Empirical Mode Decomposition:

Methodology : data and m1Methodology : data and m1

Page 61: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

經驗模式分解 Empirical Mode Decomposition: Empirical Mode Decomposition: Methodology : IMF c1Methodology : IMF c1

Page 62: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

全球温度異常值 Global Temperature Anomaly Global Temperature Anomaly 1856 to 20031856 to 2003

Page 63: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

內禀函數 IMF Mean of 10 Sifts : CC(1000, I)IMF Mean of 10 Sifts : CC(1000, I)

Page 64: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

數據與趨勢 Data and Overall Trends : Data and Overall Trends : EMD and LinearEMD and Linear

Page 65: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

趨勢的變率 Rate of Change Overall Trends : ERate of Change Overall Trends : EMD and LinearMD and Linear

Page 66: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Vostok 冰層温度數據Vostok Ice Core Temperature DataVostok Ice Core Temperature Data

Data length 3,311 points Data length 3,311 points covering 422,766 Years BPcovering 422,766 Years BP

Page 67: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Will SUVs start the next ice age?Will SUVs start the next ice age?

Page 68: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

DataData

Page 69: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Modes 1 to 8: ≤ 10k year Modes 1 to 8: ≤ 10k year periodicitiesperiodicities

Page 70: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden
Page 71: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Data & Data & modes modes 11 to 13 : ≥100k 11 to 13 : ≥100k

year year periodicitiesperiodicities

Page 72: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Data & Data & modes 10 to 13 : 40k & ≥100k year periodicities modes 10 to 13 : 40k & ≥100k year periodicities

Page 73: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Data & Data & modes 9 to 13 : 20k, 40k, 100, 400k, 400+k modes 9 to 13 : 20k, 40k, 100, 400k, 400+k yearsyears

Page 74: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Data & Data & modes 8 to 13 : 10k, 20k, 40, 100k, 400k, modes 8 to 13 : 10k, 20k, 40, 100k, 400k, & 400+k year periodicities& 400+k year periodicities

Page 75: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Data & Data & modes 1 to 7 : < 10k years periodicities modes 1 to 7 : < 10k years periodicities

Page 76: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

小結 : SummarySummary

Most of the signal limited to Milankovitch cycles.Most of the signal limited to Milankovitch cycles.Currently we are on a high temperature plateau according Currently we are on a high temperature plateau according to the Milankovitch cycle.to the Milankovitch cycle.Anthropogenic effects could be critical at this plateau regiAnthropogenic effects could be critical at this plateau region of temperature to tip the balance.on of temperature to tip the balance.

所有温度尺度都與 Milankovitch 週期吻合目前地球正處於高温期在這關鍵時期 , 人類產生的効果可能改變平衡

Page 77: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

新法和傳統法的對比New vs. Traditional ApproachNew vs. Traditional Approach

FFT vs. HHTFFT vs. HHT

Page 78: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

傅立葉 Jean-Baptiste-Joseph FourierJean-Baptiste-Joseph Fourier

1807 “On the Propagation of Heat in Solid Bodies”

1812 Grand Prize of Paris Institute

“Théorie analytique de la chaleur”

‘... the manner in which the author arrives at these equations is not exempt of difficulties and that his analysis to integrate them still leaves something to be desired on the score of generality and even rigor.’

1817 Elected to Académie des Sciences

1822 Appointed as Secretary of Académie

paper published

Fourier’s work is a great mathematical poem. Lord Kelvin

Page 79: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Comparisons: Comparisons: Fourier, Hilbert & WaveletFourier, Hilbert & Wavelet

Page 80: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

語音分析語音分析 Speech AnalysisSpeech Analysis Hello : DataHello : Data

Page 81: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Four comparsions DFour comparsions D

Page 82: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

經驗模式分解另一例An Example of EMDAn Example of EMD

Page 83: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

一天的長度一天的長度Length Of Day DataLength Of Day Data

Page 84: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

禀性函數禀性函數 LODLOD : : IMFIMF

Page 85: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

平均年週期平均年週期Mean Annual Cycle & Envelope: 9 CEI CasesMean Annual Cycle & Envelope: 9 CEI Cases

Page 86: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Current Efforts and ApplicationsCurrent Efforts and Applications

Non-destructive Evaluation for Structural Health Non-destructive Evaluation for Structural Health MonitoringMonitoring – ((DOTDOT, NSWC, DFRC/NASA, KSC/NASA Shuttle, , NSWC, DFRC/NASA, KSC/NASA Shuttle,

THSRTHSR))Vibration, speech, and acoustic signal analysesVibration, speech, and acoustic signal analyses– (FBI, and DARPA)(FBI, and DARPA)

Earthquake EngineeringEarthquake Engineering– (DOT)(DOT)

Bio-medical applicationsBio-medical applications– ((Harvard,Harvard, Johns Hopkins,Johns Hopkins, UCSD, NIH, NTU, VHT, UCSD, NIH, NTU, VHT,

AS)AS)Climate changesClimate changes– (NASA Goddard, NOAA, (NASA Goddard, NOAA, CCSPCCSP))

Cosmological Gravity WaveCosmological Gravity Wave– (NASA Goddard)(NASA Goddard)

Financial market data analysisFinancial market data analysis– (NCU)(NCU)

Theoretical foundationsTheoretical foundations– (Princeton University and Caltech)(Princeton University and Caltech)

Page 87: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

彭加箂 Henri PoincaréHenri Poincaré

Science is built up of factsScience is built up of facts**, , as a house is built of stones; as a house is built of stones;

but an accumulation of facts is no more a science but an accumulation of facts is no more a science than a heap of stones is a house. than a heap of stones is a house.

科學是事實科學是事實 (( 數據數據 ) ) 組成的組成的 正如房屋是石頭堆砌的正如房屋是石頭堆砌的但是一堆事實但是一堆事實 (( 數據數據 )) 不是科學不是科學 正如一堆石頭不是房屋一樣正如一堆石頭不是房屋一樣* * Here facts are indeed our data.Here facts are indeed our data.

Page 88: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

科學家的職責在於細心地聆聽自然而不是命科學家的職責在於細心地聆聽自然而不是命令自然如何運作令自然如何運作 理查 理查 費曼費曼

The job of a scientist is to listen carefully to nature, not to tell nature how to behave.

Richard Feynman

推薦一個有適應性的數據分析法A Plea for Adaptive Data AnalysisA Plea for Adaptive Data Analysis

Page 89: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

聆聽就得有適應性地聽聆聽就得有適應性地聽 ,, 這才能使自然唱出這才能使自然唱出自己的歌聲自己的歌聲 , , 而不是逼著自然就範於自己先而不是逼著自然就範於自己先入為主的觀念入為主的觀念 ..

To listen is to use adaptive method and let the data sing, and not to force the data to fit preconceived modes.

推薦一個有適應性的數據分析法A Plea for Adaptive Data AnalysisA Plea for Adaptive Data Analysis

Page 90: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Adaptive data analysis is the only logic method Adaptive data analysis is the only logic method for scientists and engineers to unlock the for scientists and engineers to unlock the

secrets of the nature processes.secrets of the nature processes.

有適應力的數據分析是科學與工程上了解有適應力的數據分析是科學與工程上了解自然的奧秘唯一合理的分析方法自然的奧秘唯一合理的分析方法

Page 91: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

Many of the most significant and interesting Many of the most significant and interesting challenges of the modern world require challenges of the modern world require boundary-crossing collaborations among boundary-crossing collaborations among scientists and scholars with widely different scientists and scholars with widely different

fields of expertise.fields of expertise. Allison RichardAllison Richard

Vice Chancellor, Cambridge UniversityVice Chancellor, Cambridge University

Page 92: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

ConclusionConclusion

We are drowning in data in this new IT We are drowning in data in this new IT World. To survive, we have to adaptivWorld. To survive, we have to adaptive and learn to analysis the data adaptie and learn to analysis the data adaptively.vely.

在新資訊世界裡在新資訊世界裡 ,, 我們會被數據淹沒我們會被數據淹沒 .. 要生存我們就要適應要生存我們就要適應 . . 對排山倒海的數據對排山倒海的數據 ,, 我們就要用一個有適應性的數據分析法我們就要用一個有適應性的數據分析法 ..

Page 93: Data and Data Analysis in the New IT World 新資訊世界的數據和數據分析 Data and Data Analysis in the New IT World 黄 鍔 國立中央大學 數據分析研究中心 Norden

謝謝 謝謝

ThanksThanks