MIRANTIS 2012 PAGE 1 CONFIDENTIAL MIRANTIS MIRANTIS 2012 CONFIDENTIAL MIRANTIS MIRANTIS 2013
OpenStack Overview
Paul Roberts
Principal Solutions Architect, Mirantis
MIRANTIS 2012 PAGE 2 CONFIDENTIAL MIRANTIS
Network security startup Acquired by MCI in 2005
Sun Microsystems Alumni Hosted many internal services such as Shared
Shell Carpathia Hosting
Cloud Architect with >48PB under management Coraid
Principal Architect enabling customers to simplify their complex storage architectures
Mirantis Helping customers design fully operationalized
and automated clouds
Who am I?
MIRANTIS 2012 PAGE 3 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Meetup Goals
Understand current OpenStack trends Understand OpenStack purpose and use cases Understand the OpenStack ecosystem
Definition History Programs (previously called Projects)
Understand OpenStack architecture Logical architecture Provision virtual machine (VM) request flow Components details
MIRANTIS 2012 PAGE 4 CONFIDENTIAL MIRANTIS MIRANTIS 2013
What is Cloud?
Cloud compu?ng has transformed the way storage, networking, and compute services are delivered.
MIRANTIS 2012 PAGE 5 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Traditional Public Cloud is Not Cheap
The public cloud is phenomenal if you need its elas?city, but if you dont if you do a consistent amount of workload its far, far beUer to go in-house. Eric Frenkiel, MemSQL/Wired
[Things] that need really high performance, in terms of [input and output] and reading and wri?ng to memory really belong on bare-metal servers or private setups. John Engates, CTO Rackspace/Wired
Versus what wed get on the cloud, [private hos?ng is] somewhere between 70 and 100 ?mes cheaper. John Hall, CTO Tradesy/Wired
MIRANTIS 2012 PAGE 6 CONFIDENTIAL MIRANTIS
OpenStack Mindshare
MIRANTIS 2012 PAGE 7 CONFIDENTIAL MIRANTIS
Cloud Job Trends
MIRANTIS 2012 PAGE 8 CONFIDENTIAL MIRANTIS MIRANTIS 2013
However, AWS still run-away winner
MIRANTIS 2012 PAGE 9 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Oh, Docker.
MIRANTIS 2012 PAGE 10 CONFIDENTIAL MIRANTIS MIRANTIS 2013
In the end Developers Win.
MIRANTIS 2012 PAGE 11 CONFIDENTIAL MIRANTIS MIRANTIS 2013
What is OpenStack?
As described by Wikipedia:
OpenStack is a cloud computing project aimed at providing an
infrastructure as a service (IaaS).
MIRANTIS 2012 PAGE 12 CONFIDENTIAL MIRANTIS MIRANTIS 2013
What is OpenStack?
As described by the OpenStack Foundation:
Aims to produce the ubiquitous Open Source Cloud Computing platform that
will meet the needs of public and private clouds regardless of size, by
being simple to implement and massively scalable.
MIRANTIS 2012 PAGE 13 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Cloud Exposed Capabilities (SPI Model)
Data Center (Hardware, Servers, Networking)
Software as a Service (SaaS): browser or thin client
access
Platform as a Service (PaaS): remote login, to install
applications
Infrastructure as a Service (IaaS): Provision CPU, RAM, VM
MIRANTIS 2012 PAGE 14 CONFIDENTIAL MIRANTIS MIRANTIS 2013
OpenStack Capabilities
Virtual machines (VMs) on demand provisioning snapshotting
Networks Storage for VMs and arbitrary files Multi-tenancy
quotas for different projects, users user can be associated with multiple projects
MIRANTIS 2012 PAGE 15 CONFIDENTIAL MIRANTIS
OpenStack History
Date Rel Programs Type Note
Jul 2010 N/A PoC * Rackspace Hos?ng & NASA joint launch Oct 2010 Aus4n Nova, Swih PoC Feb 2011 Bexar Nova, Glance, Swih PoC
Apr 2011 Cactus Nova, Glance, Swih PoC ** 6 month development cycle starts
Sep 2011 Diablo Nova, Glance, Swih Prod 1st produc?on release (Cactus) at
Internap (10/27)
Apr 2012 Essex Nova, Glance, Swih, Horizon, Keystone Prod Common web UI and shared authen?ca?on mechanism added
Sep 2012 Folsom Nova, Glance, Swih, Horizon, Keystone, Quantum, Cinder Prod OpenStack Founda?on
Established
Apr 2013 Grizzly Nova, Glance, Swih, Horizon, Keystone, Quantum, Cinder Prod Ceilometer and Heat incuba?on
projects added
Oct 2013 Havana Nova, Glance, Swih, Horizon, Keystone, Neutron, Cinder, Heat, Ceilometer Prod Quantum is renamed to Neutron
Apr 2014 Icehouse Nova, Glance, Swih, Horizon, Keystone, Neutron, Cinder, Heat, Ceilometer, Trove, Savanna, Ironic, Marconi
Prod Limited upgrade path from Grizzly is available
* Pre-July 2010 is predicated by Rackspace Cloud Files project (Swih), NASA Nebula project (Nova)
MIRANTIS 2012 PAGE 16 CONFIDENTIAL MIRANTIS MIRANTIS 2013
OpenStack Integrated Programs
Compute (Nova) Networking (Neutron) Object Store (Swift) Block Storage (Cinder) Image Service (Glance) Identity (Keystone) Dashboard (Horizon) Telemetry Service (Ceilometer) Orchestration Service (Heat) Database Service (Trove)
Core
Shared Services
Storage
MIRANTIS 2012 PAGE 17 CONFIDENTIAL MIRANTIS MIRANTIS 2013
OpenStack Incubation Programs
Data Processing (Sahara) Queue Service (Marconi) Bare Metal (Ironic)
MIRANTIS 2012 PAGE 18 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Each OpenStack Program
Is also a top-level OpenStack component Has an elected Project Technical Lead (PTL) Has separate developers and design teams Has a well defined public API
With the exception of Horizon, which is the Web GUI, all other projects have a RESTfull (JSON/HTTP) API
Common generic API/Infrastructure (Oslo) Has a separate database and isolated persistent
layer
MIRANTIS 2012 PAGE 19 CONFIDENTIAL MIRANTIS
OpenStack Architecture: Begining (Cactus)
MIRANTIS 2012 PAGE 20 CONFIDENTIAL MIRANTIS
OpenStack Architecture: 2 years later
MIRANTIS 2012 PAGE 21 CONFIDENTIAL MIRANTIS
Communication Types HTTP AMQP SQL 3rd-party
UI: Horizon or CLI
Heat Heat API
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Every OpenStack service exposes access to res6ul API via HTTP Each ac>on treated as distributed transac>on, state built as MQ messages Each service updates its own DB with state informa>on as ac>ons are performed Direct access calls, ex. Plugins,
NetApp, Nicira, etc.
MIRANTIS 2012 PAGE 22 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Part 1 Recap
OpenStack open source software for building IaaS OpenStack release cycle is every 6 months OpenStack is an umbrella over multiple independent
programs (components)
All OpenStack components talk RESTful API Most OpenStack components have dedicated DB
(SQL) and MQ (QP), some talk to 3rd party components using their native APIs
MIRANTIS 2012 PAGE 23 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Use case: Provision VM
Most common and complex process Interacts with most of OpenStack components
MIRANTIS 2012 PAGE 24 CONFIDENTIAL MIRANTIS
Initial State Assumes Project is created, provisioning quota is available, user has an access to Horizon/CLI Cloud Operator, DevOp, etc.
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
MIRANTIS 2012 PAGE 25 CONFIDENTIAL MIRANTIS
Step 1: Request VM Provisioning via UI/CLI
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Cloud Operator, DevOp, etc.
User logs in to UI Species VM params: name, avor, keys, etc. and hits "Create" buWon
MIRANTIS 2012 PAGE 26 CONFIDENTIAL MIRANTIS
Step 2: Validate Auth Data Horizon sends HTTP request to Keystone. Auth info is specied in HTTP headers.
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
MIRANTIS 2012 PAGE 27 CONFIDENTIAL MIRANTIS
Step 2: Validate Auth Data - Success Keystone sends temporary token back to Horizon via HTTP.
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
MIRANTIS 2012 PAGE 28 CONFIDENTIAL MIRANTIS
Step 3: Send API Request to Nova API
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Horizon sends POST request to Nova API (signed with given token).
MIRANTIS 2012 PAGE 29 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Auth Token Usage Neutron
MIRANTIS 2012 PAGE 30 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Keystone Architecture
OpenStack Services
Catalog Backend
Token Backend
Policy Backend
Assignments Backend
Iden?ty Backend
Creden?als Backend
Rule management interface and rule-based authoriza>on
Contains temporary tokens
Contains endpoint registry
Contains users and groups
Contains creden>als, e.g. EC2 tokens
Contains domains, projects, roles and role assignments
Deploys with its own DB but can also be subs>tuted with LDAP or other EAS Keystone API
MIRANTIS 2012 PAGE 31 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Nova API Characteristics
Exposes REST API via HTTP. Provides system for managing multiple APIs on
different sub-domains. EC2-compatiblestarting to be deprecated Compute APIall innovation happens here
The only "allowed" way to interact with Nova. StatelessHA-ready.
MIRANTIS 2012 PAGE 32 CONFIDENTIAL MIRANTIS
Step 4: Validate API Token
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Nova API sends HTTP request to validate API token to Keystone.
MIRANTIS 2012 PAGE 33 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Nova Database
In theory can be any relational database Most of the deployments are done with MySQL or
PostgreSQL
Nova API talks to database via SQLAlchemy (python ORM (Object Relational Mapper))
Database HA should be done via external tools: Galera Multi-Master replication Model for MySQL (MMM)
MIRANTIS 2012 PAGE 34 CONFIDENTIAL MIRANTIS
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Step 6a: Publish Provisioning Request
Nova API makes rpc.cast to Scheduler. It publishes a short message to scheduler queue with VM info.
Request has been validated, but no ac>on has been taken yet, i.e. which host, IP address, etc.
MIRANTIS 2012 PAGE 35 CONFIDENTIAL MIRANTIS
Step 7: Pick up Provisioning Request
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Scheduler picks up the message from MQ.
MIRANTIS 2012 PAGE 36 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Nova Scheduler
Nova Scheduler is a daemon, which
determines, on which compute host the request should run.
Only provisioning time componentnot like VMwares Distributed Resource Scheduler (DRS)
Typically co-located with the Cloud Controller
MIRANTIS 2012 PAGE 37 CONFIDENTIAL MIRANTIS
Step 8a: Schedule Provisioning
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Scheduler fetches informa>on about the whole cluster from database, lters, selects compute node and updates DB with its ID
MIRANTIS 2012 PAGE 38 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Nova Scheduler: Filtering
Anity, An>-anity, etc.
Eliminate inapplicable hosts
MIRANTIS 2012 PAGE 39 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Nova Scheduler: Examples
Based on Host statically configured properties SimpleCIDRAffinityFilter AvailabilityZoneFilter
Based on already running individual VMs SameHostFilter DifferentHostFilter
Collocate/Distribute group of VMs GroupAffinityFilter, GroupAntiAffinityFilter
Based on Host resources left CoreFilter,
AggregateCoreFilter RamFilter,
AggregateRAMFilter DiskFilter
Based on Host load IoOpsFilter NumInstancesFilter
Based on image used ImagePropertiesFilter
Write your own
MIRANTIS 2012 PAGE 40 CONFIDENTIAL MIRANTIS
Step 8b: Provision Scheduled
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Scheduler publishes message to the compute queue (based on host ID) to trigger VM provisioning
MIRANTIS 2012 PAGE 41 CONFIDENTIAL MIRANTIS
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Step 9a: Start VM Provisioning Nova Compute gets message from MQ
MIRANTIS 2012 PAGE 42 CONFIDENTIAL MIRANTIS
Nova Compute Drivers
Nova Compute
XCP
VM
VM
VMWare
PowerVM
VM
VM
libvirt
KVM
VM
VM
Xen
VM
VM
Qemu
VM
VM
LXC
Container
Container
Allows mul>ple hypervisor types per cloud. Libvirt / KVM is most commonly used in deployment
Maintained by Microso_
Maintained by IBM
Bare Metal
Docker
Container
Container
XenAPI
Nai>ve support comes in Icehouse PXE Tilera
Somewhat experimental
Maintained by Citrix
HyperV
VM
VM
ESXi
VM
VM
vSphere
VM
VM
MIRANTIS 2012 PAGE 43 CONFIDENTIAL MIRANTIS
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Step 9b: Start VM Provisioning Nova Compute makes rpc.call to Nova Conductor for informa>on on VM from DB
MIRANTIS 2012 PAGE 44 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Nova Conductor
Eliminates remote DB access (security) Horizontal scalability: spawn multiple worker threads operating in parallel (performance) Hides DB implementation/schema from the Nova Compute (upgrades) Possible offloading of long-running operations from other services, not just Nova Compute Beneficial for operations that cross multiple compute nodes (migration, resizes)
controller node
DB
nova-conductor
compute node
nova-compute rpc.call()
MIRANTIS 2012 PAGE 45 CONFIDENTIAL MIRANTIS
Step 10: Configure Network
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Nova Compute makes a call to Neutron API to provision network for the instance
MIRANTIS 2012 PAGE 46 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Neutron
Provides a flexible API (POST / GET) for service providers or their tenants to manage OpenStack network topologies. Create networks, associate VMs, set routers, etc.
Presents a logical API and a corresponding plug-in architecture that separates the description of network connectivity from its implementation.
One can still choose to stay with nova-network (Essex approach) or to go with Neutron.
MIRANTIS 2012 PAGE 47 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Neutron Server
Neutron CLI Nova Horizon
Neutron
Neutron Architecture
Queue
Neutron DB
Optional depending on plugin.
Neutron L3 Agent
HTTP AMQP SQL 3rd-party
Neutron DHCP Agent
Local vSwitches
Neutron L2 Agent
Runs on each Compute Node. Optional depending on plugin.
L3&DHCP Agents Scheduler
Optional
Other Network Services
FWaaS, VPNaaS, LBaaS, etc.
Neutron Metadata Agent
Neutron Plugin
SDN Controller, etc.
Optional depending on plugin.
MIRANTIS 2012 PAGE 48 CONFIDENTIAL MIRANTIS
Step 10: Configure Network (Continued) Neutron congures IP, gateway, DNS name, L2 connec>vity, etc.
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
MIRANTIS 2012 PAGE 49 CONFIDENTIAL MIRANTIS
Step 11: Request Volume
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
It is assumed a volume is already created. Nova Compute contacts Cinder to get volume data. Can also aWach volumes a_er VM is built.
MIRANTIS 2012 PAGE 50 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Open Stack Storage Concepts
Ephemeral storage: Persists until VM is terminated Accessible from within VM as local file system Used to run operating system and/or scratch space Managed by Nova
Block storage: Persists until specifically deleted by user Accessible from within VM as a block device (e.g. /dev/vdc) Used to add additional persistent storage to VM and/or run operating system Managed by Cinder
Object storage: Persists until specifically deleted by user Accessible from anywhere Used to add store files, including VM images Managed by Swift
MIRANTIS 2012 PAGE 51 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Cinder Resources
Volumes: Persistent R/W Block Storage devices Can be attached to VMs as secondary storage Can be root store to boot VMs Can be attached only to one instance at a time Keep their state independent of instances
Snapshots: Read-only point in time copy of a volume Can then be used to create a new instance
Backups: An archived copy of a volume
MIRANTIS 2012 PAGE 52 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Cinder
Cinder Architecture
Cinder DB
Queue
Cinder Volume
Backend Storage Devices Scheduler
Cinder API
Cinder CLI Nova Horizon HTTP AMQP SQL 3rd-party
Cinder Backup
Object Storage
MIRANTIS 2012 PAGE 53 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Cinder Volume Driver iSCSI:
Dell EqualLogic EMC VMAX/VNX Hitach HDS HP 3PAR (StoreServ) HP / Lefthand SAN (StoreVirtual) Huawei T/Dorado/HVS IBM Storwize family/SVC/XIV LVM (Reference Implementation) Nexenta NetApp SolidFire VMware VMDK Windows Server 2012 Zadara
GlusterFS NFS (volumes as sparse files)
IBM General Parallel File System (GPFS) (volumes as sparse files):
GPFS NSD
ATA over Ethernet (AoE): Coraid
Fibre Channel: NetApp HP 3PAR (StoreServ) Huawei T/Dorad/HVS IBM Storwize family/SVC/XIV VMware VMDK
NFS (volumes as sparse files): NFS Nexenta NetApp VMware VMDK Zadara XenAPI Storage Manager
RADOS Block Devices (RBD): Ceph
Shared SAS: VMware VMDK
Scale Out File System (SOFS) (volumes as sparse files): Scality
VirtIO (Local raw storage) (volumes as sparse files)
MIRANTIS 2012 PAGE 54 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Cinder Backup Drivers
Swift Ceph IBM Tivoli Storage Manager (TSM)
MIRANTIS 2012 PAGE 55 CONFIDENTIAL MIRANTIS
Step 11: Request volume (Continued)
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Nova Compute sets up the host mount if needed & instructs the Hypervisor to use vol. as a new block device
MIRANTIS 2012 PAGE 56 CONFIDENTIAL MIRANTIS
Step 12: Request VM Image from Glance
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Nova Compute requests VM image from Glance via Image ID
MIRANTIS 2012 PAGE 57 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Glance
"The Glance project provides services
for discovering, registering, and retrieving virtual machine images."
MIRANTIS 2012 PAGE 58 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Glance Summary
Images-as-a-Service. Can use multiple back-ends for image storage. Can store the same image in multiple locations. Supports multiple image formats.
MIRANTIS 2012 PAGE 59 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Glance API
Glance CLI Nova Horizon
Glance
Glance Architecture
HTTP AMQP SQL 3rd-party
Glance Registry
Glance DB
Store Adapter
Swift
Cinder
File System
Amazon S3
HTTP
GridFS
Sheepdog
Ceph (RBD)
MIRANTIS 2012 PAGE 60 CONFIDENTIAL MIRANTIS
Step 13: Get Image URI from Glance
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
If image with given image ID can be found - return URI HTTP Get URI
MIRANTIS 2012 PAGE 61 CONFIDENTIAL MIRANTIS
Step 14: Direct Image File Copy
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Nova Compute can download image using URI, given by Glance, directly from Swift
MIRANTIS 2012 PAGE 62 CONFIDENTIAL MIRANTIS
Step 14 alternative: Image Copy through Glance
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
To leverage Glance Server caching mechanism and addi>onal access restric>on, the Image copy can go through Glance
MIRANTIS 2012 PAGE 63 CONFIDENTIAL MIRANTIS
Step 15: Start VM Rendering via Hypervisor
In case of KVM / libvirtd this is a single XML VM cong le
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Nova Compute creates a command to Hypervisor and delegates VM rendering to Hypervisor.
MIRANTIS 2012 PAGE 64 CONFIDENTIAL MIRANTIS
Step 16: VM is UP
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Nova Compute sends a message to Nova Conductor to update DB with VM state
MIRANTIS 2012 PAGE 65 CONFIDENTIAL MIRANTIS
Step 17: User is Happy
Nova
Nova DB
Queue Nova API
Scheduler
Conductor
UI: Horizon or CLI
SwiG
Object Store
Proxy Server
Keystone KeystoneAPI
Keystone DB
Glance Glance API
Glance Registry
Glance DB Neutron
Neutron DB
Queue Neutron API
Scheduler
Plugin/Agent
Compute Node Compute Node
Network
VM
Hypervisor
nova-compute
Network Node DHCP/IPAM
Router/GW
Block Storage Node Storage
Cinder Cinder API
Scheduler
Cinder Backup Cinder DB
Queue
Cinder Vol
Ceilometer Ceilometer
API Collector
Agent
Horizon polls Nova API for VM status and power state, which is taken from Database.
MIRANTIS 2012 PAGE 66 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Recap:
Users log into Horizon and initiates VM creation Keystone authorizes Nova initiates provisioning and saves state to DB Nova Scheduler finds appropriate host Neutron configures networking Cinder provides block device Image URI is looked up through Glance Image is retrieved via Swift VM is rendered by Hypervisor
MIRANTIS 2012 PAGE 67 CONFIDENTIAL MIRANTIS MIRANTIS 2013
In the end Developers Win.
MIRANTIS 2012 PAGE 68 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Special Offer for OpenStack DC Meetup
Sign up for any OpenStack class by Mirantis
in Washington, DC in 2014 and save 10% off the ticket price.
To redeem your discount, use the code DC_Meetup_2014
Course schedule in Washington, DC:
OpenStack Bootcamp with Exam (OS110) Sep 30 Oct 3
OpenStack Bootcamp with Exam (OS110) Nov 11 - 14
OpenStack Fundamentals (OS50) Nov 17
OpenStack Bootcamp II (OS200) Nov 18 - 20
For complete schedule, course description, and registration visit training.mirantis.com
MIRANTIS 2012 PAGE 69 CONFIDENTIAL MIRANTIS MIRANTIS 2013
Questions and Comments?
Paul Roberts Twitter: pauljrob [email protected]
Top Related