2014.05.21 Ken’ichi Ohmichi

NEC Solution Innovators, Ltd.

Agenda • Who am I

• OpenStack

• OpenStack RESTful API

• Nova RESTful API

• Issues of Nova RESTful API

• Going Solutions

• Current Situation

• Summary

2

$ who am i

3

• Software developer

• Join OpenStack community for its quality assurance since 2012

• Working Areas – OpenStack QA (Quality Assurance)

– OpenStack Compute (Nova)

4

5

• OpenStack components

6

Neutron Network

Nova Compute

Glance Image

Cinder Block storage

・・・ ・・・

Horizon Dashboard

Keystone Identity

OpenStack

• OpenStack community – Rapid Growth

7

0

200

400

600

800

1000

1200

1400

A B C D E F G H I

Companies

Developers

OpenStack Versions

2013 2012 2011 2010 2014

• Community development – Importance of code review

• Can know the internal implementation

• Can know the trend

• Can get folks

– Importance of communication

• Daily : IRC or e-mail

• Weekly: IRC meeting

• Version: F2F at Design Summit

8

9

• RESTful API – Scalable and stateless

– Simple and consistent interfaces

• OpenStack provides many features through RESTful API

10

RESTful API

OpenStack Server

POST http://<api-server>/servers

RESTful Client

Create a virtual

machine

Hypervisor (kvm, etc)

VM 01

VM 02

• Many RESTful client libraries are available for OpenStack – Python － .NET

– Java － …

• Dashboard and commands request operations via RESTful APIs

OpenStack Server

RESTful API RESTful Client library

Commands $ nova .. $ cinder ..

Horizon Dashboard

• Each OpenStack component also is connected via RESTful APIs

Neutron Network

Nova Compute Glance

Image

Horizon Dashboard

Keystone Identity

Nova RESTful API

Commands $ nova .. $ cinder ..

13

• OpenStack core functions – Create a VM (virtual machine)

– Delete a VM

– Attach a volume to a VM

– …

• Scalable

• Stable

• Current API version is v2

• Nova v2 API contains 250+ APIs (Icehouse)

• API usage – Create a VM (virtual machine)

Nova v2 API

Nova server

RESTful Client

URL

{ “server”: { “name”: “VM01”, “flavorRef”: “42”, “imageRef”: “d57886..” } }

Body

http://<api-server>/servers

POST Hypervisor (kvm, etc)

VM01

16

• Inconsistent interfaces – Inconsistent naming

• “instance” or “server”, “tenant” or “project”

• CamelCase, or snake_case

17

{ “instanceAction”: [ { “action”: “reboot”, “events”: [], “instance_uuid”: “b48316c5-[..]”, “project_id”: “147”, [..] }

“Show insance actions” API

{ “server”: { “accessIPv4”: “”, “accessIPv6”: “”, “id”: “893c7791-[..]”, “name”: “test-server”, “tenant_id”: “a54313c7-[..]”, [..] } }

“Show server” API

CamelCase CamelCase

snake_case snake_case

• Inconsistent interfaces – Return incorrect success codes

• Should return 202(ACCEPTED) but some APIs return 201(CREATED) even if not complete to create a resource yet.

18

• Lacks of input validation – Cause internal errors, output a lot of error logs

in server, and return useless error messages

• Ex) Pass non-dict as metadata to “create a VM”

19

Nova v2 API

Nova API server

RESTful Client

{ “server”: { “name”: “VM01”, “flavorRef”: “42”, “imageRef”: “d57886..”, “metadata”: “non-dict” } }

Body

POST

Internal Error

The server has either erred or is incapable of performing the requested operation

HTTP500

# this bug has been already fixed.

• Lacks of input validation – Ignore unexpected data which a client sends,

and the request operates without some details which a client expects

20

Nova v2 API

Nova API server

RESTful Client

{ “server”: { “name”: “VM01”, “flavorRef”: “42”, “imageRef”: “d57886..” }, “os:scheduler_fints”: { “group”: “group-01” } }

Body

POST

typo

Create a VM without scheduler

hints HTTP202

21

• Nova v3 API （Havana -> Juno) – Re-define consistent naming

– Re-define consistent success codes

– Comprehensive input validation with JSONSchema

22

v3 API （Experimen

tal）

JSONSch

ema

input v

alid

atio

n

v2.0 API

（Current）

Nova Internal

implementation

Nova API Server Weak Validation

Inconsistent interfaces

Strong Validation

V2 API Client

V3 API Client

Consistent interfaces

Internal Error

• JSONSchema – Common data format definition library

– Clear, human- and machine-readable data format definition

23 23

{ “get_console_output”: { “length”: 100 } }

Correct request data

{ “type”: “object”, “properties”: { “get_console_output”: { “type”: “object”, “properties”: { “length”: { “type”: “integer”, “minimum”: -1 } } } }, “additionalProperties”: False, “required”: [“get_console_output”] } Data format definition with JSONSchema

Pass

• JSONSchema – Contains basic validation features in nature

– Ex) minimum

24 24

{ “get_console_output”: { “length”: -100 } }

Invalid request data

{ “type”: “object”, “properties”: { “get_console_output”: { “type”: “object”, “properties”: { “length”: { “type”: “integer”, “minimum”: -1 } } } }, “additionalProperties”: False, “required”: [“get_console_output”] } Data format definition with JSONSchema

Fail

• JSONSchema – Can deny undefined parameters

– Ex) additionalProperties: False

25 25

{ “get_console_output”: { “length”: 100 }, “undefined”: 100 }

Invalid request data

{ “type”: “object”, “properties”: { “get_console_output”: { “type”: “object”, “properties”: { “length”: { “type”: “integer”, “minimum”: -1 } } } }, “additionalProperties”: False, “required”: [“get_console_output”] } Data format definition with JSONSchema

Fail

• Use JSONSchema for input validation – Strong validation, then avoid internal errors

– Return useful error message if a client sends invalid format data

26 26

before JSONSchema validation

The server has either erred or is incapable of

performing the requested operation

Invalid input for field/attribute length.

Value: foo. foo is not of type ’integer'

after JSONSchema validation

• Use JSONSchema for input validation – Clean up logic code by separating validation

code

– Validation code also can be reduced

27 27

try: min_count = int(str(min_count)) except ValueError: msg = _('min_count must be an integer value') raise exc.HTTPBadRequest(explanation=msg) if min_count < 1: msg = _('min_count must be > 0') raise exc.HTTPBadRequest(explanation=msg)

'min_count': { 'type': 'integer', 'minimum': 1 }

Validation code of v2 API Validation code of v3 API

• Input validation development history – 2013.03 The prototype proposal

– 2013.04 The first proposal in Portland summit

– 2013.07 The framework patch was approved, but merger failed

– 2013.08 Do not merge * 2

• Input validation development history – 2013.08 Investigated input validations of all API

parameters and proposed necessary features.

– 2013.11 The second proposal in Hong Kong summit. We have gotten a consensus.

– 2013.11 The framework patch has been merged.

Necessary to merge API definition patches still.

• Input validation development status – Merged patches: 14

– Necessary to be reviewed/merged patches: 20

31

• Stopped v3 API development – Backward incompatibility against v2 API

– Large maintenance cost due to 250+ APIs for each API version

– Huge mail thread to keep/drop v3 API

32

v3 API （Experimen

tal）

JSONSch

ema

input v

alid

atio

n

v2.0 API

（Current）

Nova Internal

implementation

Nova API Server

Backward incompatibility

Strong Validation

V3 API Client

V2 API Client

Internal Error

• Proposed v2.1 API – Translate v2 request/response and use v3

implementation

– Strong API validation

– Keep maintenance cost low

33

v3 API （Experimen

tal）

JSONSch

ema

input v

alid

atio

n

v2.0 API

（Current）

Nova Internal

implementation

Nova API Server

v2.1 API （v2 – v3 Translator）

Keep compatibility with strong validation

V3 API Client

V2 API Client

Internal Error

Strong Validation Backward incompatibility

• Consensuses after Atlanta Summit – Implement v2.1 API in Juno cycle

– Implement JSONSchema validation for v2.1 API

– The future of v3 API is unclear now

34

35

• Nova v2.1 API will be available after Juno – Strong input validation

• The future of Nova v3 API is unclear – Consistent interfaces

– Strong input validation

• Join OpenStack community

– Code review is important

– Communication is important

36

37