Competence Center Agent Core Technologies

Or: what agents can be good for

DATA SIM Summer School 2014

Electric Vehicles vs. (Micro) SmartGrids

Marco Lützenberger

22. Juli 2014

2 22. Juli 2014

►Technische Universität Berlin – Berlin Institute of Technology

►DAI-Lab

►~150 employees

Post-docs, PhD Students, undergraduates

►Separated into competence centres

IRML, NEMO, SEC, COG, EDU, ACT

Aims of this Talk

3 22. Juli 2014

►Some optimization problems

►Science

►Agents

Agenda

4 22. Juli 2014

►Why are Electric Vehicles important for us (as a researcher)?

►Part One: The Driver of an Electric Vehicle

A user-centric approach

►Part Two: Electric Vehicle Fleets

A provider-centric approach

A Demonstration

The Electric Vehicle

5 22. Juli 2014

►the (electric) vehicle

►regular vs. electric

► Interesting:

the (flaw of the) battery makes the difference

Optimise range and charging intervals

the features of the battery

Utilisation of renewable energy, grid load balancing, minimising emissions

►different stakeholders

Mini Cooper Mini E (V2G)

Power 90 kW/122HP 150 kW/204HP

Torque 160 Nm 220 Nm

Weigth 1090 kg 1.465kg

Acceleration 9.1 s 8.5 s

Maximum Speed 203 km/h 152 km/h

Range 740 km 250 km

Battery 40 l 35 kWh

Charging 1-2 min 2.4 h (230V, 50A)

3.8 h (230V, 32A)

10.1h (230V,12A) source: www.mini.de

Stakeholders and their interest in EVs

6 22. Juli 2014

►„Common“ stakeholders

The driver

The vehicle manufacturer

►„Uncommon“ stakeholders

The battery manufacturer

The charging station operator

The energy provider

The government

►Conflicting interests!

Driver Battery/Vehicle Charging station Energy provider Government

mobility lifetime money grid safety image

money service availability exploiting renewable

energy

reducing emissions

image reducing

emissions utilisation

regulatory energy

distribution

image

The researcher

7 22. Juli 2014

►The challenge: different stakeholders, different interests

►The task: bring them together

►The aim: maximise the stakeholders profit

Driver

Battery/Vehicle

Manufacturer

Charging Station

Operator

Energy Provider

Government

Developing a Solution - Variables

8 22. Juli 2014

►Driver

scheduled and unscheduled appointments

►Vehicle/Battery manufacturer

charging profile, feeding profile, CO2 fingerprint and consumption

►Charging station operator

amount, characteristics, local grid infrastructure (LLM)

►Energy provider

local and global grid infrastructure, availability prognoses, CO2 fingerprint

►Government

amount of vehicles, CO2 emissions

The Problem... In a nutshell

9 22. Juli 2014

►Different stakeholder

driver, vehicle/battery manufacturer, charging station operator, energy provider, (government)

►Different interests

mobility, CO2 efficiency, lifetime, service, utilisation of infrastructures, utilisation of wind energy, money, image, ...

►The aim: maximise the stakeholders profit

►Developing such system is…

Modelling

Implementation

Deployment

Monitoring

► logical distribution, autonomy, reactivity, proactivity, interaction

► { The | A } solution: The agent paradigm

►Consider the stakeholders as (software-) agents

Easy/difficult

Agentoriented Software Engineering

10 22. Juli 2014

► „The Agent“ as constituting concept

► What is the definition of an Agent? There is no (common) definition!

Wooldridge and Jennings (1995): […] the term agent is used to denote a hardware or (more usually) software-based computer system that enjoys the following properties:

Autonomy: agents operate without the direct intervention of humans or others, and have some kind of control over their actions and internal state

Social ability: agents interact with other agents (and possibly humans) via some kind of agent-communication language

Reactivity: agents perceive their environment, (which may be the physical world, a user via a graphical user interface, a collection of other agents, the INTERNET, or perhaps all of these combined), and respond in a timely fashion to changes that occur in it;

Pro-activeness: agents do not simply act in response to their environment, they are able to exhibit goal-directed behaviour by taking the initiative.

► Why agents?

AOSE Methodologies, Documentation, Development Tools, Frameworks, Monitoring Tools

JADE, JACK, Jason, JASDL, Janus, Jadex, JIAC, 3APL, Cougaar, …

► Back to the problem: autonomy, reactivity, proactivity, interaction, logical distribution

The W2V2G System I - Design

11 22. Juli 2014

► User Agent(s) Accesses mobility patterns (derived, upcoming), detect derivations

► Car Agent(s) Current state, vehicle and battery characteristics and constraints,

charging and feeding control ► Charging Station Agent(s)

Local grid management, infrastructure information, charging and feeding control

► Energy Provider Agent Information about (global) grid load and available wind energy

► System Functionality

Energy Management (W2V, V2G, Controlling) ► Additional functionality:

Route Planning, Booking, etc.

The W2V Algorithm

12 22. Juli 2014

►Backend Software

►Triggered by Vehicle Agent (VA) and User Agent (UA)

Calculated consumption (VA) < Minimum SOC (UA)

The W2V Trigger I

13 22. Juli 2014

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Expected SOC

Minimum

The W2V Trigger II

14 22. Juli 2014

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Expected SOC

Minimum

The W2V Algorithm

15 22. Juli 2014

►Backend Software (BS)

►Triggered by Vehicle Agent (VA) and User Agent (UA)

Calculated consumption (VA) < Minimum SOC (UA)

►Preceeding Time intervals are examined

Effect of charging time on energy progression (BS – UA – VA)

Grid state (BS – Energy Provider Agent (EA))

W2V - Filtering

16 22. Juli 2014

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Expected SOC

Minimum

The W2V Algorithm

17 22. Juli 2014

►Backend Software (BS)

►Triggered by Vehicle Agent (VA) and User Agent (UA)

Calculated consumption (VA) < Minimum SOC (UA)

►Preceding time intervals are filtered

Effect of charging time on energy progression (BS – UA – VA)

Grid demand (BS – Energy Provider Agent (EA))

►Remaining time intervals are assessed

Wind energy (BS – EA)

Utilise renewable energy

Local grid state (BS – Charging Station Agent)

Grid load balancing

W2V - Filtering

18 22. Juli 2014

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Expected SOC

Minimum

0

2

1 3 5 7 9 11 13 15 17 19 21 23

W2V - Charging

19 22. Juli 2014

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Expected SOC

Minimum

The W2V Algorithm

20 22. Juli 2014

►Backend Software (BS)

►Triggered by Vehicle Agent (VA) and User Agent (UA)

Calculated consumption (VA) < Minimum SOC (UA)

►Preceding (violation) time intervals are filtered

Effect of charging time on energy progression (BS – UA – VA)

Grid state (BS – Energy Provider Agent (EA))

►Remaining time intervals are assessed

Wind energy (BS – EA)

Local grid state (BS – Charging Station Agent (CA))

►Vehicle is mainly charging renewable energy

►Additional consumption serves for load peak grading

The V2G Algorithm

21 22. Juli 2014

►Backend Software

►Triggered by User Agent or by Vehicle Agent (BS – UA – VA)

Detected change in mobility pattern

►Potential time intervals are analysed (BS – EA – UA – CA)

Grid load expected wind energy (EA)

Quotient > 0.9 discarded

Availability (CA)

►Constraint check for identified feeding intervals (UA – VA)

SOC violation?

If not possible compensation by charging (W2V)

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Expected SOC

Minimum

W2V - Charging

22 22. Juli 2014

Original W2V Strategy

V2G Feeding Intervals

V2G Compensation

Results

23 22. Juli 2014

►Field test Evaluation

►No SOC violation

►A few grid violations (our fault)

►Mini Cooper S CO2 Emissions (estimated): 18.126 gram

►Mini E (user controlled charging): 4.283,53 gram

►Mini E (W2V2G application): 2364,57 gram

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

Mini E (W2V2G) Mini E (User Controlled) Mini Cooper S

Implementation Details

24 22. Juli 2014

►Java Intelligent Agent Componentware V (JIAC V)

►Framework with a focus on industrial applications/projects

►Reliability, robustness, scalability, modularity, reusability

►Merging agents and services

►Third party API integration

Java Intelligent Agent Componentware

25 22. Juli 2014

► features

reliable communication, extensibility, reuse, performance, monitoring, maintenance, documentation, comprehensive tool support, state-of-the-art concepts/paradigms

►project requirements

robustness, scalability, support for service management, monitoring, extensibility, SOA, Cloud, webservices, OSGi Bundles, ...

►extensibility by modular assembly tailored solutions

component based architecture (agent/node beans)

►state-of-the-art libraries and languages

Java, Spring, ActiveMQ, JMX, …

The JIAC V Framework – Architecture

26 22. Juli 2014

►Agent platform agent nodes (+node beans) agents agent beans

►Runtime deployment (Spring)

►Java based implementation

►agent interaction by (ActiveMQ)

service invocation (SOA), messages, custom protocols

►Knowledge

tuple-space based memory

►runtime monitoring (JMX), ASGARD

JIAC Applications - Nodes

27 22. Juli 2014

►Default Nodes

JMX, Secured JMX, Service Directory, Registry

►Component Specification in Spring parent description

JMX capable parent node

Agent references

Agent description

JIAC Applications - Agents

28 22. Juli 2014

►Default Agents

Simple Agent, non-blocking agent, custom

►Component specification in Spring parent description

non-blocking parent agent

bean reference

JIAC Applications – (Agent) Beans

29 22. Juli 2014

►Specification in Spring

► Implementation in Java (extends AbstractMethodExposingBean)

fully qualified java name

Bean attribute

JIAC V – Agents

30 22. Juli 2014

►agent standard components

execution cycle, local memory, communication adaptors

►component based architecture

agent behaviours and capabilities in AgentBeans

► flexible activation schemes

regular, life cycle, observers, action methods

►AgentBeans and NodeBeans

►available AgentBeans (and NodeBeans)

communication, JADL++ interpreter, Drools rule engine, migration, persistence, load measurement and –balancing, user management, human agent interface, webserver, webservice gateway, OSGi gateway

Lessons Learned

31 22. Juli 2014

► It worked!

►CO2 was decreased

►Agent technology supported:

Transparent distribution

Distributed development

Programming behaviour

►Not everything was good!

Communication

Planning performance

► It was NOT a trivial problem!

2 Years of research, 1.5m€ funding

Lessons Learned

32 22. Juli 2014

►Stakeholder

Driver Battery/Vehicle Charging station Energy provider Government

mobility lifetime money grid safety image

money service availability exploiting renewable

energy

reducing emissions

image reducing

emissions utilisation

regulatory energy

distribution

image

Electric Vehicles and Micro SmartGrids

33 22. Juli 2014

►Wish list:

Not one but many (electric) vehicles

Valid information on vehicle utilisation

Consuming AND producing infrastructure

Ability to (temporarily) store electric energy

► (Electric) car sharing + the Micro SmartGrid

►The vision

Use EVs and local storage to ‘buffer’ surpluses of energy

Make the grid autarkic

Area of application: Companies, car sharing enterprises

Test Site Setup

22. Juli 2014 ISGT 2012 34

▶ Real-life test system of ‚Micro Smart Grid‘

Photovoltaic 50 kWp

Wind Turbines 5 kWp

Hydrogen Fuel Cell 1 kWel, 1 kWth

Stirling Engine 1 kWel, 16 kWth

Grid Buffer Battery 140 - 160 kWh; 18 kW

13 Electric Vehicle charging stations with distinct specifications, mostly 16 A, 400 V

Single Point of Common Coupling at 630 kVA transformer

The (Second) Problem

35 22. Juli 2014

►Factors:

Consumption

Production (wind and solar energy)

Vehicle utilisation

►The challenge:

Not one but many vehicles

Vehicles are REALLY required (time critical)

Minimise grid procurement

Maximise utilisation of renewable energy minimise C02 emissions

Counter the Second Problem

36 22. Juli 2014

►W2V2G Approach

Inapplicable time critical environment

►Deterministic optimisation

Brute force?

Complex (NP-hard problem) time critical environment

►Stochastic optimisation

Possible! …but is it good?

Evolution strategy

Modelling – Formulate the problem

37 22. Juli 2014

►Consider charging schedules (the arrangement of charging and feeding processes) as ‘population’

►Populations are moduled as follows:

►A ‘process graph’ contains all energy consumption and generation processes

►These are modelled as activities (duration+energy demand)

►Activities are linked to ‘inventory resources’ (EVs or charging stations)

Constraints: maximum load, minimum capacity, ...

►Minimize the externally procured energy

Depending on a (dynamic) tariff

Optimsation Algoritm (Evolution Strategy)

38 22. Juli 2014

► (μ/ρ + λ) strategy

►Generate initial population of μ individuals

►Based on these μ ‘parents’, λ ‘offsprings’ are generated...

by recombining the best measured selection of ρ parents

and slightly altering ‘mutating’ the result

► Initial population is created by very simple scheduler (charge when vehicles return)

►Mutate by shifting individual or groups of activities to another place in the process plan

►Recombination difficult due to many dependencies

►Measure the quality and mutate, again

►Terminate when there is no increase in quality

Problems

39 22. Juli 2014

►Results are ok!

►Well know problem of stochastic approaches

Local optima

Algorithm gets ‚stuck ‘

►Solution: Not one but many problem solvers (agents)

Develop interaction protocol

Distribute agents on local (multi-core) machine

Optimisation protocol

40 22. Juli 2014

►Optimisation client

Proposes optimisation job

►Optimisation server

Accepts (or rejects) optimisation job

Performance

41 22. Juli 2014

►Quality versus populations

►Quality versus time

Conclusion

42 22. Juli 2014

►Agents can not only be used for physical distribution, but also for logical distribution

►Avoided well known problem of stochastic optimisation

►Exploited multi-core architecture

►No ‘real’ agency

autonomy? social ability? reactiveness? pro-activeness?

►recombination

Demonstration

43 22. Juli 2014