du 30/11/16 au 02/12/16 à Grenoble, France The Analog Data · 3 Institut Mines-Télécom Data...
Transcript of du 30/11/16 au 02/12/16 à Grenoble, France The Analog Data · 3 Institut Mines-Télécom Data...
Institut Mines-Télécom
The Analog Data Assimilation (¡AnDA!)Colloque Nationale d’Assimilation de Donnéesdu 30/11/16 au 02/12/16 à Grenoble, France
Pierre Tandeo (Télécom Bretagne, Brest)En collaboration avec :P. Ailliot, R. Fablet, R. Lguensat
Institut Mines-Télécom2
Data Assimilation: general context
2
Nonlinear state-space model:
→ hidden state → dynamical model → noisy/partial obs.
01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
■ Model issues• not enough (small scales, param.)• need to include error (bias, noise)• model badly/not known
■ AnDA idea• avoid the use of model• data-driven approach• emulate the model
Institut Mines-Télécom3
Data Assimilation: 2 points of view
3 01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
Institut Mines-Télécom4
Motivations: large datasets in oceanography
4 01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
■ Satellite data• interpolations• daily• mesoscale
■ Surface variables• Temperature
(40 years)• Height
(20 years)• Salinity
(10 years)
Institut Mines-Télécom
Catalog (analogs & successors)
5 01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
Temperature(SST)
Height(SSH)
Salinity(SSS)
Lorenz-63
Institut Mines-Télécom
Analog Forecasting (loc. const. & Gaussian)
6 01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
Lorenz-63
Find analogs and compute weights: Approximate transition distribution:
Institut Mines-Télécom7
Analog Forecasting
01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
■ Experiment• dt=0.08 (6h atmos.)• k=50 analogs, Gaussian assumpt.
■ Interests• estimate both mean and cov• low-cost procedure
Institut Mines-Télécom
AnDA = Analog Forecasting + Filtering Method
8
Step 0:
- start from last analysis step (time t)- N members with same weights (EnKF)- N particles with different weights (PF)
Step 1:
- find K nearest analogs(a1, …, aK)- compute their weights (w1, …, wK)- use appropriate distance/kernel
Step 2:
- combine the K successors (s1, ..., sK) using (w1, …, wK)- different analog forecasting strategies (constant, incremental, linear)
Step 3:
- different sampling schemes (Gaussian, multinomial)- compare forecasts to noisy observations- recombine N members (EnKF) or particles (PF)
t ← t+101/12/16 Colloque National d’Assimilation de Données 2016
30/11-02/12, Grenoble, France
Institut Mines-Télécom
Classic VS Analog Data Assimilation
9
■ Experiment• simulated data
(Lorenz-63)• 1 obs. variable
(variance R=2)• partial observations
(every 8 time steps)• K=50 analogs• perfect catalog
■ Result• equivalent
performance for large enough catalog
Classic Data AssimilationAnalog Data Assimilation
Ensemble Kalman FilterParticle FilterEnsemble Kalman Smoother
01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
Institut Mines-Télécom
Analog Data Assimilation with noisy catalog
10
■ Experiment• same as previously• 3 catalogs with different variances• K=50 analogs, AnEnKS
01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
■ Result• robust to the presence of
noise in the catalog
Var=0.5Var=1Var=2
Institut Mines-Télécom
Analog Data Assimilation for model evidence
11
■ Experiment• same as previously• 3 catalogs with different parameters• obs. generated with 1• K=50 analogs, AnPF
01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
■ Result• able to retrieve the good
parameterization:1 (61%), 2 (27%), 3 (12%)
1 = (10,28,8/3)2 = (13,28,8/3)3 = (7,28,8/3)
Institut Mines-Télécom
Local VS Global Analog Data Assimilation
12
■ Experiment• simulated data
(Lorenz-96)• 20 obs. variables
(noise R=2)• partial observations
(every 4 time steps)• analog forecasting
(k=50, 10^3 Lorenz-96 times)
■ Result• local strategy
overperforms the global one
01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
Institut Mines-Télécom13
AnDA: timeline
13
Conference paper (Climate Informatics): The Analog Ensemble Kalman Filter and Smoother, Tandeo et al.Sep2014
Journal paper (submitted): The Analog Data Assimilation, Lguensat et al.Nov2016
Book chapter (Machine Learning and Data Mining Approaches to Climate Science): Combining analog method and ensemble data assimilation: application to the Lorenz-63 chaotic system, Tandeo et al.
Jan2015
Conference paper (Climate Informatics): The analog data assimilation: application to 20 years of altimetric data, Tandeo et al.Sep2015
Journal paper (Physical Review X): The Ensemble Kalman Filtering without a Model, Hamilton et al.Mar2016
Conference paper (OCEANS): Using archived datasets for missing data interpolation in ocean remote sensing observation series, Lguensat et al.Jun2016
01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
Institut Mines-Télécom14
AnDA: collaborations
14
Télécom Bretagne, UBO, UBS, Rennes 1
Univ. Buenos Aires(climate)
Univ. ColoradoNCAR/IMAGe
Ocean Univ. of China(oceanography)
McGill Univ.(meteorology)
Methodology
Applications
IFREMER(oceanography)
Univ. Corrientes
01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
Pathfinder SST
Rainfall radar
SPEEDY simulations
AVISO/GCM SSH
Institut Mines-Télécom
Conclusions & Perspectives
15
■ Conclusions• data-driven assimilation
─ use historical datasets─ satellite data, simulations
• various implementations─ analog forecasting─ local/global analogs─ filtering method
• Pyton library on github • AnDA paper on researchgate
■ Perspectives• Methodology
─ other methods (EKF, HMM, 4D-VAR) ─ spatio-temporal dynamic kernels─ distance metric learning
• Applications─ realistic examples─ oceanography, climate, meteorology─ other fields?
• collaborations are welcomed!
01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France1515
Institut Mines-Télécom
The birthplace of the Analog Data Assimilation
16
Thank you for your attention!Any questions?
01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
Data Science & Environment
3-7 July 2017, Brest, France
Workshop + Summer Schoolwebsite
Institut Mines-Télécom
Additional slides
17 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
Institut Mines-Télécom18
Analog Method
■ State of the art• Lorenz (1969)“atmospheric predictability using naturally occurring analogs from past records”
• Van Den Dool (1994)“it would take order 10^30 years to find analogues that match over the entire Northern Hemisphere 500mb height field to within current observational error”
■ Revival• Amount of data
─ from satellite─ numerical simulations
• Data mining tools─ recent machine learning methods─ fast nearest neighbor search
• Scientific interest for few years─ Yiou, Climate Dynamics (2013)─ Delle Monache et al., MWR (2015)─ Dernott & Wikle, Environmetrics (2015)─ Atencia & Zawadzki, MWR (2015)─ Zhao & Giannakis, Nonlinearity (2016)
01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France
Institut Mines-Télécom
AnDA: Analog Forecasting strategies
19 01/12/16 Colloque National d’Assimilation de Données 201630/11-02/12, Grenoble, France