Technische Implementation von CERA Hannes Thiemann Max-Planck-Institut für Meteorologie Modelle und...
-
Upload
porsche-heidelberger -
Category
Documents
-
view
102 -
download
0
Transcript of Technische Implementation von CERA Hannes Thiemann Max-Planck-Institut für Meteorologie Modelle und...
Technische Implementation von CERA
Hannes ThiemannMax-Planck-Institut für Meteorologie
Modelle und Datenhannes.thiemann @ zmaw.de
Jena, 24. Januar 2007
Inhalt
Aufgabe und Motivation
Umsetzung Datenbanken
Anbindung an das HSM
Ausblick
Klimasystem
Klimamodell: Grid
Klimamodell: Auflösung
T42 (300 km)
T106 (120 km)
Datenmengen
Horizontalauflösung des Klimamodells
T42: 128 * 64 = 8192 Punkte pro Globalfeld T106: 160 * 320 = 51200 Punkte pro Globalfeld
Erforderliche Speichereinheiten (GRIB Format)
Horizontalfeld (Zugriffseinheit): 17.1 kB (T42) / 100.1 kB (T106)
Unix Filegröße für monatsweise akkumulierte Ergebnisse mit 6 Std. Speicherintervall und 300 2d Variablen (Physikalische Einheit):
616 MB (T42) / 3500 MB (T106)
240 Jahre Modellintegration (Logische Einheit): 1.7 TB (T42) / 10 TB (T106)
Umsetzung Datenbanken
The Winter TopTen Program identifies the world’s largest and most heavily used databases.
….. Congratulations on achieving Grand Prize award winner status (1) in Database Size, Other, All and TopTen Winner status Database Size, Other, Linux;Workload, Other, Linux in Winter Corp.'s 2005 TopTen Program! .......
(1) Grand prizes are awarded for first place winners in the All Environments categories only.
WDCC's CERA DB has been identified as the largest Linux DB.
Wintercorp (2005) - DB Size: Scientific, Archive, and other
Company Size (TB)
DBMS Platform System Vendor
Max-Planck 222 Oracle Federated/SMP NEC
USGS/EROS 17 Oracle Centralized/SMP Sun
USGS/EROS 17 Oracle Centralized/SMP Sun
HP 1 NonStop SQL Centralized/MPP HP
T-Systems 1 Oracle RAC Centralized/Cluster Sun
See: www.wintercorp.com
Wintercorp (2005) - DB Size: Data Warehouse
Company Size (TB)
DBMS Platform System Vendor
Yahoo 100 Oracle Centralized/SMP Fujitsu Siemens
AT&T 1) 94 Daytona Federated/SMP HP
KT IT-Group 50 DB2 Centralized/Cluster IBM
LGR 25 Oracle Centralized/SMP HP
Amazon 25 Oracle RAC Centralized/Cluster HP
See: www.wintercorp.com1) 330 GB Norm. Data Volume
Oracle 9.2 single instance running on TX7 Enterprise Edition Partitioning Option Advanced Security
24 Tbyte disk attached to database nodes Database size ~260 Tbyte (logical) Database nodes connected to HSM system Data accessible on the internet 800 named users worldwide Daily access 300 GB/Day (average) New data 250 GB/Day (average)
CERA: Some Facts
BLOB
SX-6
AsAmA 16way AsAmA 4way
DXSM DXDB
Oracle DBDiskXtenderDisk cache
PostprocessingSystemraw
META + DataGFSEnvironment
GFS/Server
AsAmA 4way
DXDB. . .
Oracle DB
AsAmA 16way
GE Network
Users
GFS/Server GFS/Server
DXSN DXSN
BLOB
DXDM DXDM
Climate Model
1.Climate Model writes raw output (GFS I/O)
AP
GFS/Client
Post Process Application
2.PP reads raw data (GFS I/O)
PP writes data (local I/O)AP
OCI Application
3.OCI reads data (Local I/O)
AP
Local disk
Migration &
Staging
Oracle ApplicationServer
5.Data inquiry (OCI)
Oracle AS
© NEC Corporation
4.OCI writes BLOB (via networks)
Oracle InstanceOracle Instance
Level 1 - Interface:Metadata entries(XML, ASCII)+ Data Files
Level 2 – Interf.:Separate filescontaining BLOBtable data in application adapted structure(time series ofsingle variables)
Experiment Description
Pointer toUnix-Files
Dataset 1Description
Dataset nDescription
BLOB DataTable
BLOB DataTable
WDCC Data Topology
BLOB DB Table corresponds to scalable, virtual file at the operating system level.
Datenbanken: Aufteilung
11121311415
16
MetadatenDaten
EnterpriseUserSecurity
OID
Entry
Reference
Status
Distribution
Contact Coverage
Parameter
SpatialReferenceLocal Adm.
Data Access
Data Org
100.000 Tabellen800 GB
Data matrix of model experiment
Model variables
Mod
el R
un T
ime
2 D: small BLOBS (16 KB)
3 D: large BLOBS (3 MB)
Raw data file: direct model output (0.7 – 16.2 GB)
Each columm is one BLOB Table and one META Table in CERA-DB
Raw
data file in D
KR
Z A
rchive
T2M Precip SLP2D variables . . Temp
Water vapour
3D variables . .
T1T2T3.......Tn...................Tend
MetadataTable
Blob_idBlob_sizeStart_dateBlob_minBlob_max
Blob_mean
Structure of metadata tables
Informationen um Einfache Anfragen ohne Zugriff
auf Daten selbst zu beantworten.
Konsistenz zu den Daten selbst überprüfen zu können.
Qualitätskontrollen durchzuführen.
Liegen auf Disk
Metadaten erlauben die Abbildung der blob_id auf die wirkliche Modellzeit
BLOB DataTable
blob_idblob_data
Structure of blob tables
Range Partitioning
Table Partition 1
Table Partition 2
Table Partition n
…
blob_id 1 .. n
blob_id n+1 .. m
blob_id m+1 .. k
…
Time t0 .. tn
Time tn+1 .. tm
Time tm+1 .. tk
…
Datafile 1
Datafile 2
Datafile n
…
Umsetzung: HSM
Anbindung an das HSM
TBS - RW
TblPartition 1
TBS - RW
TblPartition 2
dxdb
TBS - RO
TblPartition 1
All tablespaces are moved
“at once” to dxdb
MigoutMigin
Migout / Migin
Migout takes place after files haven’t been modified for x minutes
Only one migout process per dxdb-filesystem Migin takes place immediately after a file is requested.
Only parts accessed are retrieved from the backend storage.
One migin process per requested file.
dxdb
LWM
HWM
Purging
Criteria for purging
Size of datafiles doesn’t matter Except: “small” datafiles can stay on disk
Time not modified (easy for read only tablespaces) Time not touched
Oracle has the tendency to touch data files quite often
Oracle parameter read_only_open_delayed could be an option
Prerequisite: 2 copies on tape
Inside the datafile
Primary Key
Lob Index
Table
Blob data
Header 128k
Frontend versus Backend
Header 128k
Filesystem Frontend HSM Backend
Header 128k
Part 1 = 512 MB
Part 2 = 512 MB
Retrieving data
4
Header 128k
3 1
2 5
Tape Request
Usage: Downloads
Downloads per year
0100000200000300000400000500000600000700000800000900000
1999 2000 2001 2002 2003 2004 2005 2006
Statistics: Size
Database Size
0
50
100
150
200
250
300
1998 1999 2000 2001 2002 2003 2004 2005 2006
Year
TB
yte
Ausblick: Globalmodell T213 (Atmosphäre)
Horizontalauflösung des Klimamodells
T213: 640 * 320 = 204800 Punkte pro Globalfeld T106: 160 * 320 = 51200 Punkte pro Globalfeld
Erforderliche Speichereinheiten (GRIB Format)
Horizontalfeld (Zugriffseinheit): 400.1 kB (T213) / 100.1 kB (T106)
Unix Filegröße für monatsweise akkumulierte Ergebnisse mit 6 Std. Speicherintervall und 300 2d Variablen (Physikalische Einheit):
14000MB (T213) / 3500 MB (T106)
240 Jahre Modellintegration (Logische Einheit): 40 TB (T213) / 10 TB (T106)
Ausblick: Regionalmodell Auflösung und Datenmengen
REMO-UBA-Modellgebiet
Orography
•Auslösung: 10x10 km
•Datenmenge: 5 TB / 100 Jahre (nur Bodenfelder)
Vielen Dank!