Post on 11-Apr-2017
Docker PaaS ( Mesos+AWS )
Oskars Gavriševs@2015
Agenda :
● Overview of Docker Paas solutions in market.
● Mesos in details.
● Torture of Mesos high availability.
● Lessons learned from AWS deployment.
Docker PaaS
Docker PaaS• Kubernetes ( By Google )
• Mesos ( By Apache Foundation )
----------------------------------------------------
• EC2 Container Service ( By Amazone )
• CoreOS ( By 3x devs )
• Stratos ( By Apache )
• Rancher ( By Rancher Labs )
• SmartOS
Kubernetes concepts. • Pods
• Masters
• Minions
• Replication controllers
• Service
• Labels
• Kubelets
Minions !
Kubernetes architecture.
Kubernetes minion.
Kubernetes on open source pills .
Amazon EC2 container service (ECS) , concepts.
• Cluster
• Container instance
• Task definition
• Task
• Service
ECS architecture.
ECS disadvantages.
• Scheduling algorithm is not transparent.
• Limited docker functionality (No “Host” networking).
• Only one container per one host with ELB.
• No Labels.
• No Health checks.
ECS, know how.
• How one container can start other container on same host ?
ECS, know how (2).
CoreOS
Rancher
Mesos in details
Mesos Architecture
Mesos Master(s)
• High availability
• Masters form quorum.
• Leader election.
• Register frameworks, nodes.
• Min. 3x masters (1x off), 5x recommended ( 2x off )
Mesos Slaves
● Provides resources.
● Can consist of multiple executors.
● Executors executes tasks.
Mesos Frameworks
Mesos Frameworks (2)
Marathon ( Long running services )
Hadoop ( Big Data Processing )
Chronos / Jenkins ( Batch scheduling )
ElasticSearch ( Data storage )
* http://mesos.apache.org/documentation/latest/frameworks/
What so special with this mesos ?
High availability
• If master fails tasks can continue to run.
• New leader election.
• Failed master can recover jobs.
Health checks
Component disconnection handling
• Slave disconnects from Zookeeper (don’t know which master is leader).
Component disconnection handling (1)
• Master disconnects from Zookeeper (enters leaderless state).
Warning suicidal master !
Component disconnection handling(2)
• Slave doesn’t respond to master.
Component disconnection handling(3)
• Master disconnects from nodes.
Component disconnection handling(4)
• Leading master dies.
Better resource utilization
Better resource utilization (2)
Mesos API
• http://master:5050/master/state.json
Task labels.
Constrains
DEMO
CPU / Meme / Disk quotas
CPU / Meme / Disk quotas (2)
DEMO
Marathon
Marathon docker scaling
• Scale up / down with one click.
Marathon port mapping and LB
DEMO
Chronos
Job dependencies
DEMO
Lessons learned
Lessons
• Unable to deploy on AWS ECS due to ‘host != networking’.
Lessons (2)
• It's almost impossible to pass dynamic variables to container.
https://github.com/ogavrisevs/Mesos-Docker-Cluster