2012年度 計算サーバ利用講習会 6/15 13:00-16:00...SMPサーバ プロセッサの スケーラビリティ メモリの スケーラビリティ 2012 年度 計算サーバ利用講習会
Auto Scaling x Spot Instances によるスケーラビリティと コストカット
-
Upload
akitsukada -
Category
Technology
-
view
6.174 -
download
1
Transcript of Auto Scaling x Spot Instances によるスケーラビリティと コストカット
- 1. Auto Scaling x Spot Instances 2015-06-03 @akitsukada ()
- 2. @akitsukada AWS Lambda #iOS #AWS #Rails #MySQL #Redis CAC() (CTO) About me ()
- 3. (^q^)
- 4. Purpose of today 1. Auto ScalingMin-Max Spot&Reserved Instances 2.
- 5. Purpose of today 1. Auto ScalingSpot Instances : [AWS] & http://www.slideshare.net/AmazonWebServicesJapan/20131023- aws-meisterregeneraterispotpublic 2. () 3.
- 6. 1. / 2. () 3. 4. 5. 6. 1. / 2. () 3. 4. 5. 6. Agenda
- 7.
- 8. AutoScaling, Spot Instances, Reseved Instances 1. 2. ( 3.
- 9. AutoScaling, Spot Instances, Reseved Instances 1. 2. ( 3. 1. 4.5 2. 4.5 3. 1
- 10. ()
- 11. () ()
- 12.
- 13. Twitter(BtoC) =100 =3050% =110%/ (313) : /
- 14. Ruby on Rails (Apache+Passenger) HTTP(S) Worker m3.2xlarge 8@1a,1c Rails+Resque**, Rails+Resque+ResqueScheduler*** WebAPIResqueSchedulerJob c4.2xlarge 10@1a,1c WebAPI on EC2 Worker on EC2* / *Worker2014/08, AWS Startup Tech Meetup http://j.mp/20140817_aws_tsukada **ResqueGitHub Redis ***ResqueSchedulercronResqueJobKick
- 15. WorkerWebAPI AWS cloud Amazon RDS Amazon SES ElastiCache (Redis for Resque) Internet users Elastic LoadBalancing users Availability Zone instances Availability Zone instances /
- 16. WorkerWebAPI AWS cloud Amazon RDS Amazon SES ElastiCache (Redis for Resque) Internet users Elastic LoadBalancing users Availability Zone instances Availability Zone instances / 1.AB @mention 2.B
- 17. WebAPI AWS cloud Amazon RDS Amazon SES ElastiCache (Redis for Resque) Internet Elastic LoadBalancing Availability Zone instances Availability Zone instances User B User A Worker 1 Job mention / 1.AB @mention 2.B
- 18. WebAPI AWS cloud Amazon RDS Amazon SES ElastiCache (Redis for Resque) Internet Elastic LoadBalancing Availability Zone instances Availability Zone instances User B User A Worker 1.AB @mention 2.B 1 Job /
- 19. / WebAPI Sum Requests (ELB)
- 20. / (24,000req/min) (600req/min) WebAPI Sum Requests (ELB) / (40)
- 21. / (40) () (600req/min) (10,000req/min) WebAPI Sum Requests (ELB) / (24,000req/min)
- 22. / (40) () WebAPI Sum Requests (ELB) / (600req/min) (10,000req/min) (24,000req/min) 8 ! 1 8 7 6 5 4 3 2 1
- 23.
- 24.
- 25. Auto Scaling EC2 (^o^) $9,195113/ *201505EC2
- 26. needed
- 27. ()
- 28. Auto Scaling Spot Instances JobAuto Scaling Spot Instances() SpotAuto Scaling Group SpotReserved1 WorkerWebAPI Spot Instances Reserved InstancesAuto Scaling m3.2xlarge (8vCPU/26ECU/30GiB) m3.large x 4 c4.2xlarge (8vCPU/31ECU/15GiB) c4.large x 4 (Push)Scheduled Action ()
- 29. AutoScaling Spot Instances WorkerWebAPI Spot Instances Reserved InstancesAutoScaling m3.2xlarge (8vCPU/26ECU/30GiB) m3.large x 4 c4.2xlarge (8vCPU/31ECU/15GiB) c4.large x 4 (Push)Scheduled Action () JobAutoScaling Spot Instances() SpotAuto Scaling Group SpotReserved1
- 30. /Reserved Spot Instance SpotSpot Auto Scaling Group Spot Instance 1 Reserved (Max20) SpotStuckJob Job ... Spot()
- 31. /Reserved Spot Instance SpotSpot Spot Instance 1 Reserved (Max20) SpotStuckJob Job ... Spot() 1
- 32. WorkerWebAPI () ELB ELB* Spot /Reserved ELBHealth Check *http://bit.ly/1KxDZBx - ELB[Qiita] (25) AMI chef** threshold Spot Request*** **http://amzn.to/1Eniqlm - AutoScale ***http://aws.amazon.com/jp/ec2/faqs/#How_many_instances_can_I_run_in_Amazon_EC2
- 33. Auto Scaling Reserved Instances
- 34. Auto Scaling/WebAPI
- 35. Name: WEB-API-ASG ELB: web-api-balancer Min: 4 Max: 36 Auto Scaling Group Name: WEB-API-LC-v1 AMI: ami-XXXXXXX Instance Type: m3.large Launch Configuration Scale In Alarm Comparison Operator: LessThanThreshold Evaluation Periods: 3 Period: 300 Statistic:Maximum Metric Name:CPUUtilization Threshold: 30 Unit:Percent Cloud Watch Alarm Scale Out Alarm Comparison Operator: GreaterThanThreshold Evaluation Periods: 1 Period: 300 Statistic:Maximum Metric Name:CPUUtilization Threshold: 65 Unit:Percent Adjustment Type: ChangeInCapacity Scaling Adjustment: 2 Cooldown: 300 Scaling Policy Health Check Grace Period: 300 Availability Zones: ap-northeast-1a, ap-northeast-1c /Auto Scaling/WebAPI WebAPI
- 36. Name:event-push-start Min: 22 Max: 36 Reccurence: 45 1 * * 1 Name:event-push-stop Min: 4 Max: 36 Reccurence: 30 2 * * 1 Scheduled Actions # 10:45(JST) # Min/Max(yamlDRY) # 11:30(JST) config yaml API config WebAPI /Auto Scaling/WebAPI
- 37. Auto Scaling WebAPI Sum Requests (ELB) /Auto Scaling/WebAPI Req
- 38. Auto Scaling WebAPI Sum Requests (ELB) WebAPI Healthy Hosts (ELB) /Auto Scaling/WebAPI
- 39. Auto Scaling WebAPI Sum Requests (ELB) WebAPI Healthy Hosts (ELB) Req /Auto Scaling/WebAPI
- 40. Auto Scaling WebAPI Sum Requests (ELB) WebAPI Healthy Hosts (ELB) Req Scheduled- Action /Auto Scaling/WebAPI
- 41. Auto Scaling WebAPI Sum Requests (ELB) WebAPI Healthy Hosts (ELB) Req Scheduled- Action HostsAZ Average 1a&1c 4362x2(AZ)=4 18x2(AZ)=36 /Auto Scaling/WebAPI
- 42. Auto Scaling/ Worker&Worker-Spot
- 43. Scale In Alarm Comparison Operator: LessThanThreshold Evaluation Periods: 1 Period: 300 Statistic:Average Metric Name:CPUUtilization Threshold: 35 Unit:Percent Scale Out Alarm Comparison Operator: GreaterThanThreshold Evaluation Periods: 1 Period: 300 Statistic:Average Metric Name:CPUUtilization Threshold: 70 Unit:Percent Name: WORKER-ASG ELB: - Min: 1 Max: 26 Health Check Grace Period: 300 Availability Zones ap-northeast-1a, ap-northeast-1c Name: WORKER-LC-v1 AMI: ami-XXXXXXX Instance Type: c4.large Spot Price: - Adjustment Type:ChangeInCapacity Scaling Adjustment: 2 Cooldown: 300 /Auto Scaling/Worker&Worker-Spot WorkerAuto Scaling Group Launch Configuration Cloud Watch Alarm Scaling Policy
- 44. Scale In Alarm Comparison Operator: LessThanThreshold Evaluation Periods: 1 Period: 300 Statistic:Maximum Metric Name:CPUUtilization Threshold: 25 Unit:Percent Scale Out Alarm Comparison Operator: GreaterThanThreshold Evaluation Periods: 1 Period: 300 Statistic:Maximum Metric Name:CPUUtilization Threshold: 60 Unit:Percent Name: WORKER-ASG-SPOT ELB: - Min: 1 Max: 26 Name: WORKER-SPOT-LC-v1 AMI: ami-XXXXXXX Instance Type: c4.large Spot Price:0.147 Adjustment Type:ChangeInCapacity Scaling Adjustment: 2 Cooldown: 300 /Auto Scaling/Worker&Worker-Spot Worker-SpotAuto Scaling Group Launch Configuration Cloud Watch Alarm Scaling Policy Health Check Grace Period: 300 Availability Zones ap-northeast-1a, ap-northeast-1c
- 45. Scale In Alarm Comparison Operator: LessThanThreshold Evaluation Periods: 1 Period: 300 Statistic:Maximum Metric Name:CPUUtilization Threshold: 25 Unit:Percent Scale Out Alarm Comparison Operator: GreaterThanThreshold Evaluation Periods: 1 Period: 300 Statistic:Maximum Metric Name:CPUUtilization Threshold: 60 Unit:Percent Name: WORKER-ASG-SPOT ELB: - Min: 1 Max: 26 Name: WORKER-SPOT-LC-v1 AMI: ami-XXXXXXX Instance Type: c4.large Spot Price:0.147 Adjustment Type:ChangeInCapacity Scaling Adjustment: 2 Cooldown: 300 /Auto Scaling/Worker&Worker-Spot Worker-SpotAuto Scaling Group Launch Configuration Cloud Watch Alarm Scaling Policy Health Check Grace Period: 300 Availability Zones ap-northeast-1a, ap-northeast-1c http://www.slideshare.net/AmazonWebServicesJapan/20131023-aws-meisterregeneraterispotpublic/72
- 46. Scale In Alarm Comparison Operator: LessThanThreshold Evaluation Periods: 1 Period: 300 Statistic:Maximum Metric Name:CPUUtilization Threshold: 25 Unit:Percent Scale Out Alarm Comparison Operator: GreaterThanThreshold Evaluation Periods: 1 Period: 300 Statistic:Maximum Metric Name:CPUUtilization Threshold: 60 Unit:Percent Name: WORKER-ASG-SPOT ELB: - Min: 1 Max: 26 Name: WORKER-SPOT-LC-v1 AMI: ami-XXXXXXX Instance Type: c4.large Spot Price:0.147 Adjustment Type:ChangeInCapacity Scaling Adjustment: 2 Cooldown: 300 /Auto Scaling/Worker&Worker-Spot Worker-SpotAuto Scaling Group Launch Configuration Cloud Watch Alarm Scaling Policy Health Check Grace Period: 300 Availability Zones ap-northeast-1a, ap-northeast-1c WorkerWorker-Spot http://www.slideshare.net/AmazonWebServicesJapan/20131023-aws-meisterregeneraterispotpublic/72
- 47. 1. Max= 2. Min=Max*(Req/Req)+ 3. Threshold(CPU Utilization) Top=CPU Bottom=1Max CPU 4. Period, Evaluation Periods, Cooldown * * /Auto Scaling/
- 48. Reserved Instances /Reserved Instances
- 49. /Reserved Instances Reserved Instances AutoScaling Billing Infomation * * http://calculator.s3.amazonaws.com/index.html
- 50. /Reserved Instances Reserved Instances
- 51. /Reserved Instances Reserved Instances Min 4 / n%xm% y xReserved
- 52. /Reserved Instances Reserved Instances 1080% All Upfront
- 53. http://j.mp/summit2015_tsukada_sheet /
- 54. http://j.mp/summit2015_tsukada_sheet
- 55. http://j.mp/summit2015_tsukada_sheet (2) -858792!!
- 56. http://j.mp/summit2015_tsukada_sheet
- 57. http://j.mp/summit2015_tsukada_sheet 3
- 58. http://j.mp/summit2015_tsukada_sheet 12 +8389656 3
- 59.
- 60. RDS(Reserved Instances), ElastiCache(Reserved Cache Nodes), Cloudfront(Reserved Capacity), DynamoDB(Reserved Capacity) AWS Black Belt Tech & http://www.slideshare.net/AmazonWebServicesJapan/20150325-aws- blackbeltrispotpublic Spot Instances c1.medium Worker Lambda Lambda/Job SpotSpotFleet API () SQLRDS
- 61.
- 62. !!!!!!!!!! Lambda HTTP/2 Scale ...?
- 63. Notes
- 64. THE END 2015-06-03 @akitsukada ()