Puppet Camp Tokyo 2014: Kickstack: A pure-Puppet rapid deployment system for OpenStack -,
Aws + Puppet = Dynamic Scale
-
Upload
puppet -
Category
Technology
-
view
631 -
download
2
Transcript of Aws + Puppet = Dynamic Scale
![Page 1: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/1.jpg)
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Shiva NSolutions Architect, AWS
@shivamaan
AWS + Puppet = Dynamic Scale
![Page 2: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/2.jpg)
Application Rate of Change Is Increasing
Source 2013 State of DevOps Report Puppet Labs, IT Revolution Press
Top Performers30x more frequent code deployments
Of those with DevOps implemented > 12 month:
More than 25% can deploy on demand
![Page 3: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/3.jpg)
~11.6sMean timebetween deployments (weekday)
~1,079Max number of deployments in a single hour
~10,000Mean number of hosts simultaneously receiving a deployment
~30,000Max number of hosts simultaneously receiving a deployment
DEPLOYMENTS AT AMAZON.COM
![Page 4: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/4.jpg)
75%Reduction in outages triggered by software deployments since 2006
90%Reduction in outage minutes triggered by software deployments
Business Value of Frequent Deployments
~0.001%Software deployments cause anoutage
![Page 5: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/5.jpg)
example.com – Online auction
AWS Sydney Region
Public subnet App subnetHAProxy Vertx
Glassfish
Data subnet
Puppet Stack
ELKStackMgmt subnetCloudFront S3 SES SNS
Route 53
Jenkins
![Page 6: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/6.jpg)
Let’s build
[ ] OS [ ] Patches[ ] OS Configuration[ ] VM Templates (Amazon Machine Image)[ ] VPC[ ] Networking (Subnets, ELBs)[ ] Network Security (NACLs and SGs)[ ] Software[ ] Application (Code)[ ] Services
![Page 7: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/7.jpg)
Start with a Base Image
Machine Images can come from several sources..- AWS Provided (Quick Start)- AWS Marketplace- Community AMIs- Build your own AMIs- Migrate On Premise Image We need a server to
build an image to build a server
![Page 8: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/8.jpg)
Let’s build
[X] OS [ ] Patches[ ] OS Configuration[ ] VM Templates (Amazon Machine Image)[ ] VPC[ ] Networking (Subnets, ELBs)[ ] Network Security (NACLs and SGs)[ ] Software[ ] Application (Code)[ ] Services
![Page 9: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/9.jpg)
Layers to Golden Image
CI-‐CD Pipeline- git push puppetmanifests- Packer/AMINator- puppet apply- Test image- Register AMI, meta-‐data in CMDB
![Page 10: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/10.jpg)
Let’s build
[X] OS [X] Patches[X] OS Configuration[X] VM Templates (Amazon Machine Image)[ ] VPC[ ] Networking (Subnets, ELBs)[ ] Network Security (NACLs and SGs)[ ] Software[ ] Application (Code)[ ] Services
![Page 11: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/11.jpg)
Cloudformation
![Page 12: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/12.jpg)
Let’s build
[X] OS [X] Patches[X] OS Configuration[X] VM Templates (Amazon Machine Image)[X] VPC[X] Networking (Subnets, ELBs)[X] Network Security (NACLs and SGs)[ ] Software[ ] Application (Code)[ ] Services
![Page 13: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/13.jpg)
Build Puppet Stack
- EC2 Metadata service$ curl http://169.254.169.254/latest/user-datahiera_role=PuppetMasteraws s3 cp s3://puppetmanifests /etc/puppet/modules
- Self awareness by hiera magic- Self manifestation with puppet apply
![Page 14: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/14.jpg)
Build Software Stack
- CloudFormation- RDS- Elasticache- S3- CloudFront- Route 53- ELB- SNS- Autoscaling groups
- PuppetMaster- Build ELK Stack (PuppetForge)- Build Glassfish layer (Custom types and providers)- Build Vertx layer (Custom types and providers)- Build HAProxy layer (PuppetForge)
![Page 15: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/15.jpg)
Let’s build
[X] OS [X] Patches[X] OS Configuration[X] VM Templates (Amazon Machine Image)[X] VPC[X] Networking (Subnets, ELBs)[X] Network Security (NACLs and SGs)[X] Software[ ] Application (Code)[ ] Services
![Page 16: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/16.jpg)
Application Deployment
define glassfish::app ($user = 'glassfish',$action = 'deploy',$bindaddr = '127.0.0.1',$mgmtport = '8080',$deploy_source = '${glassfish::real_instance_basedir/deployments/${name}.war',) {require glassfishif ($action == 'deploy') {app { "${name}":ensure => present,mgmtport => $mgmtport,user => $user,source => $deploy_source,
}}
}
![Page 17: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/17.jpg)
Let’s build
[X] OS [X] Patches[X] OS Configuration[X] VM Templates (Amazon Machine Image)[X] VPC[X] Networking (Subnets, ELBs)[X] Network Security (NACLs and SGs)[X] Software[X] Application (Code)[X] Services
![Page 18: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/18.jpg)
example.com – Online auction
AWS Sydney Region
Public subnet App subnetHAProxy Vertx
Glassfish
Data subnet
Puppet Stack
ELKStackMgmt subnetCloudFront S3 SES SNS
Route 53
Jenkins
![Page 19: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/19.jpg)
Let’s run
[ ] Tune[ ] Release[ ] Orchestrate and Automate
![Page 20: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/20.jpg)
Decision, Decisions…
- In place deployments (Puppet run)- MCollective- CodeDeploy
- Immutable deployments - Blue-‐Green deployment- AMI baking
![Page 21: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/21.jpg)
AWS CodeDeploy
![Page 22: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/22.jpg)
Bake an AMI Puppet +
Sweet spot
To AMI or not to AMI
Start an instance
Configure the instance
Create an AMI from your instance
Start new ones from the AMI
Launch an instance
Use metadata service to get hiera roles
Puppet run performs post launch config
![Page 23: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/23.jpg)
CI Pipeline
• Vagrant• Jenkins
– Minor release• CodeDeploy release to environment• Packer build Golden AMI* -‐> Update DynamoDB -‐> Schedule ASG refresh
– Major release• Packer build Golden AMI* -‐> Update DynamoDB -‐> Refresh ASG
* Golden AMI – Glassfish + HAProxy + Vertx + Applications with all services turned off
![Page 24: Aws + Puppet = Dynamic Scale](https://reader034.fdocument.pub/reader034/viewer/2022051709/586fd8681a28ab18428b55bb/html5/thumbnails/24.jpg)
Thank You