GSN Global Sensor Networks for Environmental Data Management
-
Upload
jean-paul-calbimonte -
Category
Education
-
view
295 -
download
2
description
Transcript of GSN Global Sensor Networks for Environmental Data Management
Global Sensor NetworksAn evolving middleware for sensor data
stream processing
Jean-Paul Calbimonte, Julien Eberle, Sofiane Sarni, Karl Aberer
LSIR EPFLMountain Observatories 2014, Reno Nevada
16.07.2014
http://gsn.epfl.ch
Sensor deployments everywhere
MountainsGlaciersSnow regionsSea CoastalAgriculture…
DIYMobileParticipatory
We want the data
• Open data repositories• Accessible research datasets• Discoverability• Reuse datasets• Metadata
GSN: Global Sensor Networks
GSN: Global Sensor Networks
Help managing sensor datasets
Help publishing the data
Help making the data discoverable and reusable
GSN in a nutshell
• Middleware: Sensor network deployment
• Virtual Sensor (VS): Process streaming data
• Hosts & manages multiple VSs
6
Where is GSN?
7
• Sensor Network:• Sensing• In network
• Data/query Processing• Filtering• Aggregation
• Data Management:• Data:
• Management.• Publishing.• Expensive processing.• Archives.
DataManagement
SensorNetwork
GSN goes here
Collecting data from different sources
8
• GSN works in a distributed fashion• Data can be kept locally• Break data silos
• Put sensor data on the webGSN nodes
GSN Distributed Deployment
9
Integrity Service
Access ControlGSN/Web/Web-Services
Notification ManagerQuery ProcessorQuery RepositoryStorage Manager
Virtual Sensor ManagerInput Stream Manager
Stream Quality ManagerLife Cycle Manager
Pool Of Sensing Devices
GSN Virtual Sensors
10
• A virtual sensor, any kind of data producer• a real sensor, a wireless camera, a desktop computer, GPS sensor, network traffic,
etc.• combination of other virtual sensors.• Logical view of the sensor network.• Described in an XML file:
• Functional/non-function properties.
Source 1
Source 2…
Source n
Application logicand processing
Output Stream
Virtual S
ensor
Virtual Sensor configuration
11
<virtual-sensor name="room-monitor" > <addressing> <predicate key="geographical">BC143</predicate> <predicate key="usage">
room monitoring</predicate> </addressing> <life-cycle pool-size="10" /> <output-structure> <field name="image" type="binary:image/jpeg" /> <field name="temp" type="int" /> </output-structure> <storage permanent="true" history-size="10h" /> <input-streams> <input-stream name="cam"> <stream-source alias="cam" storage-size="1“
sampling-rate=“1”> <address wrapper=“tinyos2.x"> <predicate key=“host">tinybox.epfl.ch </predicate> <predicate key=“port">9001</predicate> </address> select * from WRAPPER </stream-source><stream-source alias="temperature1“ storage-size="1m“ sampling-rate=“1”>
<address wrapper="remote"> <predicate key="type">temperature</predicate> <predicate key="geographical">BC143-N
</predicate> </address> select AVG(temp1) as T1 from WRAPPER </stream-source> <stream-source alias="temperature2“ storage-size="1m“> <address wrapper="remote"> <predicate key="type“>temperature</predicate>
<predicate key="geographical“>BC143-S
</predicate> </address> select AVG(temp2) as T2 from WRAPPER </stream-source> <query> select cam.picture as image, temperature.T1 as temp from cam, temperature1 where temperature1.T1 > 30 AND temperature1.T1 = temperature2.T2 </query> </input-stream> </input-streams></virtual-sensor>
Data in GSN through Wrappers
12
Common abstractions, independent of applications, hardware
Simple integration & data correlation.
5140
GS
N
Various Applications
Plug & Play deployment
On-the-fly reconfiguration
GS
N
GSN
Some available mappings
13
• HTTP generic wrapper• devices accessible via HTTP GET or POST requests, e.g., the AXIS206W wireless camera
• Serial forwarder wrapper• enables interaction with TinyOS compatible motes (standard access in TinyOS)
• USB camera wrapper• local USB connection.• supports cameras with OV518 and OV511 chips.
• RFID wrapper• access to Texas Instruments Series 6000 S6700 multi-protocol RFID readers• Alien Technologies long range RFID reader 8950 EU.
• WiseNode wrapper• access to WiseNode sensors (CSEM, Switzerland, http://www.csem.ch/)
• Generic UDP wrapper• any device using the UDP protocol
• Generic serial/bluetooth wrapper• supports sensing devices which send data through the serial port, e.g., EPuck robots, etc.
Wrappers: Lines of Code
14
50RFID reader (TI)
50Generic HTTP
300Wired camera
180Generic serial
45Generic UDP
75WiseNode
160TinyOS
Lines of codeWrapper type
Open source project: Available in Github
• Open Source License• Mainly in Java• Community Support• Used in several projects
Releases available in Github
So what can I do with it?
• Get data from my sensors (API, web interface) • Store and archive the data• Put it online, available for download• Put it online, available for discovery and querying• Apply post-processing to the data• Combine different data sources• Use the data from an R script• More stuff…
Data Validation through Measurements and Modelling over Multiple Scales
LagrangianDispersion Model
High resolution urbanatmospheric pollution maps
Model InputTerrain, meteorology, source strength, background
Sensor DataCrowd-sensors, mobile sensors, monitoring stations
GSN Storage
19
• Centralized RDBMS
Trends: Data & Users
Evaluate a NoSQL solutionScalability Fault-tolerance Performance
GSN Storage Extension
20
LSIR-Cloud
HBASE Java API Client
Experimental Platform
CPU: 24 cores x 2.3GHzMem: 64 GBDisk: 2.8 TB
Nodes: 8CPU: 8 x 12 cores x 2.3GHzMem: 32 GBDFS Disk: 43 TBNetwork: 1 Gbps
HDFS Cluster
Put / Get
HBASEExporter
HBASE Wrapper
HBASE Query Handler
User Requests
VS data
Store VS data
Read VS data
Execute Query
SSN Ontology with other ontologies
21
W3C SSN Ontology
tool for modeling our sensor datacombine with domain ontologies
GSN Access Control (AC)
• VS has an owner: decides user access
22
VS: Virtual Sensor
AC ISSUES REASON
Private VS Features not visible VS Availability should be provided
No Notifications Faster responses, if notified
No Access Time Limitations Enable owner to control access
Manual VS management Automation of the VS activation
No AC in REST services Enable alternative data access
More things we’re doing
• Integration: integrate with Geo-enabled repositories (e.g. GeoNetworks)
• Standards: NetCDF, OGC standards, OpenDAP• Metadata: add semantics to the data• Web standards: RDF and Linked Data• tinyGSN: for mobile devices
Some example of Use
• OpenIoT: Smart agriculture, manufacturing, etc.• SwissExperiment: environmental sensing• PlanetData: traffic data observation• OpenSense: air quality measurements• Permasense: mountain and snow observatory• Etc..
OSPER - Swiss ExperimentOpen support platform for environmental research
Multidisciplinary research team• Real world data + problems
Facilitating research in:• Precipitation patterns in mountains• Evaporation in Africa• Return periods of Natural Hazards• Stream flows in Alpine catchments• Permafrost in the Alps
managing environmental sensor data &metadataPlatform
http://swiss-experiment.chD
ata
heterogeneous sensing devices
summarization, filtering, compression, interpolation
continuous processing, streaming, geospatial, aggregation
pattern discovery, correlation, regression
metadata management, semantics
data services, visualization, standards
acquisition
processing
querying
analysis
discovery
provision
OpenSense2
global concernhighly location-dependenttime-dependent
Crowdsourcing High-Resolution Air Quality SensingAir Pollution
Accurate location-dependent and real-time information on air pollution is needed
Integrated air quality measurement platform
- Heterogeneous devices and data- Human activity assessment, lifestyle and health data
• Link high-quality and low-quality data • Integration of pure statistical models and physical
dispersion models• Better coverage through crowdsensing• Incentives for crowd data provision• Finer temporal and spatial resolutions• Utilitarian approach for trade-off between model
complexity, privacy and accuracy• Higher accuracy of pollution maps models
http://opensense.epfl.ch
Institutional stationsOpenSense infrastructure
Personal mobile sensors CrowdSense
OpenIoT FP7Open Source Cloud solution for the Internet of Things
http://openiot.eu
Established Open-source platform for IoT
• Integrate sensors & things with cloud computing • Configure, deploy and use IoT services• Auditing/assessing privacy of IoT apps in the cloud• Semantic annotations of internet-connected objects• Energy-efficient data harvesting• Publish/subscribe for continuous processing and
sensor data filtering• Mobility of sensors and QoS aspects in IoT
https://github.com/OpenIotOrg/openiot
Use cases and validation scenariosSmart
Manufacturing Campus Guide Air Monitoring Agriculture Sensing
Thanks a lot!
Global Sensor Networks
Jean-Paul Calbimonte
LSIR EPFL
http://gsn.epfl.ch