GEO Grid: Federating Geospatial Data andGEO Grid ... Grid OGFprint.pdf(PostGIS, oracle + geo, …)...
Transcript of GEO Grid: Federating Geospatial Data andGEO Grid ... Grid OGFprint.pdf(PostGIS, oracle + geo, …)...
www.geogrid.org
GEO Grid: Federating Geospatial Data andGEO Grid: Federating Geospatial Data and Integrating Services
Satoshi Sekiguchi, Yoshio Tanaka, Isao KojimaGrid Technology Research CenterGrid Technology Research Center
National Institute of Advanced Industrial Science and Technology(AIST), Japan
1
www.geogrid.org
What is the AIST ?What is the AIST ?
One of the largest Nat’l gLabs in Japan
3,500 employee+3,000 staff
Research topics includeEnvironmentMate ialMaterialBio/Life scienceStandards (JIS/ISO)( )GeologyGeologySemiconductor deviceIT/Computer ScienceIT/Computer Science
AIST Tsukuba Main Campus7 other campuses across Japan
IT/Computer ScienceIT/Computer Scienceetc.
NIST+USGS+DOE l b Narita
Tsukuba50km
40km
2
labs+NIH NaritaTokyo
50km
50km
www.geogrid.org
What is the GEO Grid ?What is the GEO Grid ?The GEO (Global Earth Observation) Grid is aiming at providing an E-Science Infrastructure for worldwide Earth Sciences communities to accelerate GEO sciences based on the concept communities to accelerate GEO sciences based on the concept that relevant data and computation are virtually integrated with a certain access control and ease-of-use interface those are enabled by a set of Grid and Web service technologies enabled by a set of Grid and Web service technologies.
AIST: OGF Gold sponsor (a founding member)
AIST: OGC Associate member (since 2007)
Satellite Data GeosciencesGrid
Technologies
Geo* ContentsApplications
Geology Data
GIS dataEnvironmentConservation
RDisaster
3
In-situ data Resources mitigation
www.geogrid.org
ASTER : Main ASTER : Main content content of AIST GEO Gridof AIST GEO GridASTER : Main ASTER : Main content content of AIST GEO Gridof AIST GEO Grid
ASTER was developed by METI with AIST scientific & engineering supportsGlobal land coverage of Digital Elevation Model of 15m spatial resolution Global land coverage of Digital Elevation Model of 15m spatial resolution Excellent geo-location accuracy
Easy to mosaic (or make a seamless image/DEM) Easy to overlay to GIS dataEasy to overlay to GIS data
Powerful spectral analysis VNIR 3 bands (+backward 1band)/SWIR 6 bands/TIR 5 bands
20002000~
4
www.geogrid.org
ASTER Global land coverageASTER Global land coverage
5
ASTER archive 1.4 million images = 140 TBMETI/NASA
www.geogrid.org
Digital Elevation Model of 15m spatial resolution Digital Elevation Model of 15m spatial resolution L d lid l L d lid l ti ti i ASTER DEMti ti i ASTER DEMLandslides volume Landslides volume estimation using ASTER DEMestimation using ASTER DEM
Synthetic bird eyes view with ASTER image and DEM
Volume : 0.06 km3, area : 0.5 km2
at DEM difference > 50 m
6
14NOV2000 11OCT2005
www.geogrid.org
ASTER excellent geoASTER excellent geo--location accuracylocation accuracy
Boulder, CO
7
Easy to overlay GIS data (ex. Road map) without adjustment manually
,
www.geogrid.org
DevelopmentDevelopment--caused change at caused change at SuvarnabhumiSuvarnabhumi AirportAirport(near Bangkok, Thailand)(near Bangkok, Thailand)( ea a g o , a a d)( ea a g o , a a d)
2000/11/02
2002/07/03
2004/10/28
2006/02/042006/02/04
8
Data achieved since 2000
www.geogrid.org
More More effective use of ASTER effective use of ASTER DataDataIncorporate with geospatial information to overlay
Integrated data search service across multiple satellite i d imagery data setsInteroperable with GIS based on OGC services and others
WMS, WFS, WCS, CSWISO 19115, GML, GeoSciML
Compliance with owners’ access control policy of Compliance with owners access control policy of data/serviceGrid Security Infrastructure – AuthN, AuthZ, AccountingAccounting
Integrating with processing (computing) servicesH d/ li ht i l ti WPS G id RPC tHeavy and/or light simulations: WPS, Grid RPC, etcResource management – infra providerComplex workflow support in portals incl. data access,
9
p pp p ,simulation execution, visualization, etc.
www.geogrid.org
ASTER ASTER –– FORMOSATFORMOSAT--2 Integrated Data Search2 Integrated Data SearchGet Get satellite satellite imagery of a particular imagery of a particular location from federated databases location from federated databases Get Get satellite satellite imagery of a particular imagery of a particular location from federated databases location from federated databases
R lt
FORMOSAT-2 OrbitFORMOSAT-2 Orbit Google EarthGoogle EarthYesNo
ASTERASTER FORMOSAT-2FORMOSAT-2
Result
Input Key:
PeriodLat/Lon
GEO G id P t lGEO G id P t lSpot ImageSpot Image
Cloud GEO Grid Portal(ASTER DB)
GEO Grid Portal(ASTER DB)
p g(FORMOSAT-2 DB)
p g(FORMOSAT-2 DB)
Result
Input Key
P i dResult
Date
PeriodLat/LonCloudetc
10
ImageOneImageOne
www.geogrid.org
11
www.geogrid.org
Overlay GISOverlay GISyy
12
www.geogrid.orgEnergy Cone Simulation
GEO Grid use case GEO Grid use case scenarioscenarioVolcano Volcano monitoringmonitoringVolcano Volcano monitoringmonitoring
M l i
←3D+Temp
13
AP photo
MelapiASTER
www.geogrid.org
火砕流火砕流 メラピ山オルソメラピ山オルソ パラメータ設定パラメータ設定
14
www.geogrid.org
火砕流火砕流 メラピ山オルソメラピ山オルソ 処理結果処理結果
15
www.geogrid.org
A Workflow example A Workflow example
““Disaster prevention and mitigation (Volcano)Disaster prevention and mitigation (Volcano)””In-situ observations by
CCOP membersHazard Map for
Evacuation planningMonitoring of crustal
deformation by PALSAR
Disaster prevention and mitigation (Volcano)Disaster prevention and mitigation (Volcano)
GEO
ASTER
16
High resolution DEM provided from ASTER
Simulation of lava and/or pyroclastic flow on GEO Grid
www.geogrid.org
A Workflow example A Workflow example
““Disaster prevention and mitigation (Landslide)Disaster prevention and mitigation (Landslide)””Rain fall
(AMEDAS/Fieldserver)
Early warning systembased on Susceptibility map
Geology map (GSJ)Disaster prevention and mitigation (Landslide)Disaster prevention and mitigation (Landslide)
GEOMET
ASTER
High resolution DEM La ge scale Comp te sim lation
17
High resolution DEM provided from ASTER
Large scale Computer simulation using actual landslide DB
www.geogrid.org
But, it was not so simple …
More sophisticated security architecture
Data access and integration
18
www.geogrid.org
GEO G idGEO Grid
Applications Disastermitigation Land slides, floodmitigation
Environmentmonitoring Global warming, CO2 flux estimation
Full L0 ASTER on disk
Oil, Gas Natural resourceexploration
Contents Satellite Imagery
Geology archives Japan, SE Asia
Full L0 ASTER on diskMODIS on disk (East Asia)
gy p ,
Sensors AsiaFlux, Field server
ITInfrastructure Software
Security, data access, service registry, resource mgmt., Weg GIS, Workflow, U/I Portal, etc.
19
Hardware Storage, ServersCluster computers
www.geogrid.org
GEO Grid Players & ToolsGEO Grid Players & Toolsyy
Player Tools Product
End Users Web browser GEO Grid Client
Portal & VO mgr Project Goal
GEO GridPortal Dev. KitVO t l
Customized Web portalVO Management SystemVO mgr
Data Provider
Scientific data
VO tools
G O G dGEO Grid Data Service
Provider
Resource P id
data
Application, IT
GEO GridService Dev. Kit
GEO Grid Hosting ServiceProvider IT resource
G O G d ost g Se ce
GEO Grid Tool kit
20
GEO Grid Tool kit
www.geogrid.org
Disaster iti ti
Environment it i
Natural resource l tiApplication mitigation
Workflow, Portal, PSE
monitoring exploration Application
Framework
Data Access + ComputingService Integration
R i t
Job Submission
A i
Brokering
D t T f
Storage Services
Open Grid
Structured Data Access
Registry
Info. Services Resource Mgmt.
Accounting Data TransferOpen Grid Service
Data and computing as services Accessg
Service
Security, Messaging, CommunicationConnectivity
21
Computing, Data & Storage Structured DataRelational/XML/RDF/WebDB
Resource
www.geogrid.org
AIST GEO Grid Architecture (IT Infrastructure)AIST GEO Grid Architecture (IT Infrastructure)l
Disas
preven
Ecolo
Agricul
Geolo
Applications
sterntion
ogy
lture
ogy
Visualizationservice
VO management
Std Web Service I/F incl. basic grid service
VO hosting services
Interoperable i GEON
GG US
GML/G-XML/OGCTiled Wall Display
Data Grid Computing Grid
Base Map Application hosting engine
GS
J data
GIS
data
US
GS
data
FieldS
ensors
gfarm
ASTER MODISASTER & MODIS
DSM/DEMPALSAR
hosting engineOverlayable GIS data PALSAR
Geometric, Radiometric &Atmospheric CorrectionMosaic & Composite,
Tsukuba (Japan)
MAP on Demand
AIST Supercluster
210TB /P4 3.5TB(raid5)/node x 60C
AIT(Thailand)
Image Processing
22
L0 or L1L1A data will not be in public
NASA(USA)
AISTAIST Supercluster
(11TFlops)
HATOYAMA (Japan)ASTER
MODIS
PALSAR
www.geogrid.org
Catalogue Service OGC-compliantclient
① Search Catalogue
httpCSW
Web serverRDB (postgres, oracle,…)
client
Java Script +
Viewer(google map, deegree, …)
http
httphttp
pGoogleMap API, OGC Client
Web Client
PC② Query metadata
OGSA-DAI CSW / WMS+
MetadataPC
③ Retrieve Data
http
Web serverGeo-featured DB
(PostGIS, oracle + geo, …)
Grid client
W b Cli t
Portlet
③ Retrieve Datahttp
OGC-compliant Services(WMS, WFC, WCS)
DataWeb Client
PC③ Computation
httpGRAM(WMS, WFC, WCS)
Web serverGeo-featured DB
(PostGIS, oracle + geo, …) Computation
GRAMWeb serverApplications
WPS
23
Data(satellite, GIS, …)
ComputationComputers
www.geogrid.org
Security issuesSecurity issues
24
www.geogrid.org
FOUR categories of Data Access AttributeFOUR categories of Data Access AttributeggALLMany
OPENCat. 2
Open to ManCat. 1
Open to AllOPEN Open to Many(Black list)
Open to All(Public)
CLOSECat. 3
Close to Many(White list)
Cat. 4Close to All
(Private)(White list) (Private)
25
www.geogrid.org
You might need to have like thisYou might need to have like this
local
Private Virtual Organization (VO) enabled by GRID In-Situ
Cat 3 Project/Community PEN
Cat 2 Domain CommonCluster
ASTER
Internet
Cat 1 Public
26
Cat. 1 Public
www.geogrid.org
Security Architecture Design PolicySecurity Architecture Design Policy
Introduces concept of VO (Virtual Organization)Data and computation are provided as “services” via Data and computation are provided as services via standard protocols and APIs.A VO is created dynamically by integrating available services and resources according to the interests and requirements of the VO. User level Authentication and VO level AuthorizationUser-level Authentication and VO-level Authorization
User’s right is managed (assigned) by an administrator of his belonging VO.administrator of his belonging VO.Access control to a service is configured by the service provider according to the publication
li Th ti f th policy. There are some options of the access control
VO-level, Group/Role-based, User-level, etc.
27
VO level, Group/Role based, User level, etc.Scalable architecture for the number of users.
www.geogrid.org
Overview and usage model of the GEO Grid systemOverview and usage model of the GEO Grid system
28
www.geogrid.org
account
Security ArchitectureSecurity Architecture
GAMA
CACA
AccountDB
M P
GEO GridAdmin
login “username /Password”
creation User Aw/o certificate GEO Grid Portal
GAMA
MyProxy X.509 long-livedcertificates
User Bw/ certificate
BB
login by certificate X 509 proxy
credential repository
AA
AABB
AA
X.509 proxytifi t
certificate X.509 proxycertificates
Anonymous User
anonymouslogin
AABB
VOMS VOMS serverserver
certificatesw/ VOMS attributes
Project
y
GEO GridMeta Data CatalogueVirtual DB
2 Fetch Meta Data info
1. simple query
blacklistwhitelist
Admin
ASTER
Meta D
ata
2. Fetch Meta Data info
3. Data Access with appropriate method
grid-mapfileconfig file
LCASLCMAPS
Filesystem ACL ASTERa
PDP #1
PDP #2
Filesystem ACLAuthZ
29
ASTER(JPEG)
File ASTER(WMS)
ASTER(WFS)Service
ProviderPDP #n
www.geogrid.org
Data IntegrationData Integration
30
www.geogrid.org
GEO Grid data integration GEO Grid data integration -- at a glanceat a glance
CSW WFSWPS WMS
OGC Standards
Workflow engines OGSA-DQPOGC SQLSpatial querySupport of OGC standards
S i
Distributed JoinsService Coordination
Support of OGC standardswith
Grid-based federated DatabasesOGSA-DAI@AISTServicesServices
OGSA-DAI
D t S i
OGSA-WebDB
Data Services GSJ
ASTERData Resources
GTRC
MODIS
Data Services
31
Parallel SQL processing
www.geogrid.org
Data Integration Requirements(1)Data Integration Requirements(1)g q ( )g q ( )★ Support of Geo-Spatial Standards
OGC standardsCSW for catalog serviceCS o cata og se ceWMS,WFS,WCS for geospatial data servicesWPS for data processingOGC ti l OGC spatial query
Data Format standardsData Format standardsData
GML and other XML data formatM t d t ( h )Metadata(schema)
ISO19115,Dublin Core, ebXML etc.
32
www.geogrid.org
Data Integration Requirements(2)Data Integration Requirements(2)g q ( )g q ( )★ Support of Distributed Database Federation
Support of multi-level distributed database federation
Local = Different Resources under one ServiceEnterpriseGlobal = Distributed Services over the worldGlobal = Distributed Services over the world
Support of Heterogeneous Resourcespp gRelational databasesXML databasesWeb databasesWeb databases
33
www.geogrid.org
Data Integration Requirements(3)Data Integration Requirements(3)g q ( )g q ( )★ Support of the Grid Technology
Support of the VO conceptVO b d d t b th i ti / it VO based database authorization/security management
Support of Grid Standards & MiddlewaresOGF Standards & MiddlewareOGF Standards & Middleware
WS-RF/Globus, MDS,GSI(VOMS), OASYS/W3C/
BPEL, Service Registry(ebXML/UDDI)
34
www.geogrid.org
ApproachApproachpppp
Multi-level-distributed data integrationg→ Service-based Database Integration
MiddlewareMiddleware
S t f VO tSupport of VO concept→ VOMS enabled Database Middleware
OGSA-DAI with VOMS
Support of Grid Standard & MiddlewareSupport of Grid Standard & Middleware→ Grid based Database Middleware
35
www.geogrid.org
A sample of ASTER DEM image on Tiled DisplayA sample of ASTER DEM image on Tiled Display
36
www.geogrid.org
37
www.geogrid.org
38