Amazon Web Services Building Blocks for Drupal Applications and Hosting
-
date post
21-Oct-2014 -
Category
Technology
-
view
6.164 -
download
1
description
Transcript of Amazon Web Services Building Blocks for Drupal Applications and Hosting
• IntroduceyoutoAWS
• Showyouhowthepiecesfittogether• Providebackgroundforsecondhalfofthiswebinar• AnsweryourAWSandcloudcompuRngquesRons
GOALSFORTODAY
• AWS=“AmazonWebServices”
• Oneofourthreebusinesses:– OnlineRetail– SellerBusiness(Marketplace)
– WebServiceInfrastructure
• PayAsYouGoPricingModel• TwoLevels:
– InfrastructureasaService(IaaS)– Pla_ormasaService(PaaS)
INTRODUCINGAWS
Amazon Web Services
Or “AWS” for short…
• Business
• Security
• Presence
• Technical
BUILDINGTHECLOUD
• Amazon’sBrandName
• Self‐serveSignup• UserAccounts• PayAsYouGo• CreditCardBilling
BUILDINGTHECLOUD‐BUSINESS
• SecurityisJob#1forUs• DesignedforMulR‐Tenancy• SAS70CerRficaRon• SecurityWhitePaper
– PhysicalSecurity– LogicalSecurity
• PublishedBestPracRces• hfp://aws.amazon.com/security
BUILDINGTHECLOUD‐SECURITY
BUILDINGTHECLOUD‐PRESENCE
Ashburn,VA/Dallas,TX/LosAngeles,CA/Miami,FL/Newark,NJ/PaloAlto,CA/Seafle,WA/St.Louis,MO/Amsterdam/Dublin/Frankfurt/London/HongKong/Tokyo/Singapore/NewYork
USEast(NorthernVirginia)USWest(NorthernCalifornia)Europe(Dublin)AsiaPacific(Singapore)Moretocome…
AWSRegions:
AWSCloudFrontLocaRons
BUILDINGTHECLOUD‐TECHNICAL
• AmazonEC2:on‐demandcomputepower– Obtainandbootnewserverinstancesinminutes– Scalecapacityupordownasneeded– $0.085perhourandup
• Keyfeatures:– SupportforWindows,Linux,andOpenSolaris– Completeflexibilitytoinstallanyopensourceorcommercialsoqware– SupportsallmajorwebandapplicaRonpla_orms– MulRpleAvailabilityZones– ElasRcIPAddresses– ElasRcBlockStorage– ElasRcLoadBalancing– AutoScaling– AmazonCloudWatch
AMAZONELASTICCOMPUTECLOUD
• CreateanewrelaRonaldatabasewithoneAPIcall• OffloadcommonadministraRvetaskstoAWS• UseexisRngcodeandtools• ScaleupprocessingorstoragewithoneAPIcall
• Recoverdatabaseusingsnapshotorlogfile
• MySQL5.1
AMAZONRELATIONALDATABASESERVICE(RDS)
2010Jan Feb Mar Apr May Jun Jul
»AmazonEC2ReservedInstanceswithWindows,ExtraLargeHighMemoryInstances»AmazonS3VersioningFeature»ConsolidatedBillingforAWS»LowerpricingforOutboundDataTransfer
»AmazonSNS
»SingaporeRegion»RDSinEurope»ELBsessions
AWSPACEOFINNOVATION–2010
»RDSConsoleSupport» S3ReducedRedundancyStorage» RDSMulR‐AZ» CloudFrontStreamingLogs» VPCinEurope
»AWSSoluRonPages
»RDSSSLSupport»CloudWatchEBSMetrics»S3ConsoleSupport»Import/ExportAPI»CloudFrontHTTPSSupport»CloudFrontPOPinNY»CloudFrontPriceReducRon»ElasRcMapReduceupdates
»??»??»??
• AWSTrends:– ConRnuousinnovaRon– AddiRonalservices– AddiRonalfeatures– AddiRonallocaRons
• ThisistheRmetolearnandstarttousethecloud!
FINALTHOUGHTS
• Createdeveloperaccountathfp://aws.amazon.com
• Downloadaccesskeys,SDKs,developertools
• BuildanddeployapplicaRon
• AWSBlog:hfp://aws.typepad.com
GETTINGSTARTEDWITHAWS
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Hosting Drupal on AWS
Barry Jaspan Senior Architect, Acquia [email protected] @bjaspan
http://acquia.com/resources/acquia-tv
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Acquia Hosting Overview • Managed web hosting for Drupal
• High performance, scalability, availability
• Combined infrastructure- and application-level support from one vendor
• Single or multi-tier • Load balancers, web nodes, database clusters,
memcached, network filesystems, ...
• One instance, separate instances, or any combination
• Automated • We provisioned for a “20M uniques on day one”
customer ... in a few minutes.
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Load balancing
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Load balancing on AWS • No hardware LB is available
• Round-robin DNS to web nodes is bad for high-availability
• Elastic Load Balancer (ELB) • CNAME requires http://www.TLD
• Limited flexibility (but now with stickiness!)
• Elastic IP address (EIP) + software LB • Allows http://TLD
• Only one EIP per server
• We use nginx for LB and Varnish for caching
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Software load balancer
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Shared !les
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
File sharing on AWS • Drupal needs a POSIX filesystem for user-
uploaded files
• Elastic Block Store (EBS) is single-instance
• rsync does not scale past N=2 web nodes
• S3FS is worse than useless for this • Too slow, non-POSIX, unreliable, unsupported
• A network filesystem is required • NFS?
• More later...
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Basic web stack
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
HA load balancer • Load balancer might fail
• Elastic IP • Requires a hot spare
• EIP must be reassigned to a hot spare
• Elastic Load Balancer • For us, can’t talk directly to web nodes
• nginx provides virtual host flexibility, concurrency protection, etc
• Varnish provides caching
• Also requires a hot spare (or second active instance)
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
HA load balancer with EIP
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
HA !lesystem • NFS
• Cannot tolerate server failure
• Not replicated • DRBD with virtual IP reassignment won’t work
• AFS • Tolerates server failure
• Not replicated
• GlusterFS • Replicated and distributed
• GPL, commercially supported, under active dev
• Easy to use, tricky to optimize just right
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
HA !lesystem with GlusterFS
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
HA database • Multi-AZ Relational DB Service (RDS)
• Failover takes 3 minutes
• Active/passive master MySQL replication • Must avoid “split-brain” operation
• All web nodes must fail over simultaneously
• Heartbeat with virtual IP reassignment not possible
• Failover mechanism must itself be HA • Using MySQL proxy just leads to the same problem
• Requires synchronous app-level failover
• We store db election in GlusterFS • Any other HA replicated data store would do
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
High-availability web stack
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Machine con!g • All these instances need to be built
• Incremental custom AMIs • Time consuming during development/debugging
• Poor fit for source-code control
• Leads to mystery configurations
• Automated configuration scripts/language • Incremental updates are simple
• Good fit for source-code control
• Known process from a clean, base AMI
• Combining the two may be best
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Static con!g with puppet
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Elasticity • Elastic scalability means dynamic
configuration • Balancers: info on all web nodes
• Web nodes (Drupal): info on all db servers
• FS servers: info on all EBS volumes to use
• FS clients: info on all FS servers to use
• All (re-)started instances get a new IP • EIPs are limited and cost more to use
• All servers need current internal IPs
• Admins need current external IPs
• Requires frequent manual updates, or automation
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Dynamic data-driven con!g
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Email from AWS • AWS IPs are on many spam blacklists
• Can’t reliably be used to send mail
• Requires a non-cloud mail server or service provider
• Note: Jeff tells me this situation has improved
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Colo mail server
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Everything else • I’ve only talked about cloud-specific issues
• To self-host (anywhere), you also need: • Backups
• 24x7 monitoring
• Ops staff
• Caffeine
• Ibuprofen
© 2010 Acquia, Inc. Amazon Web Services Building Blocks for Drupal Applications and Hosting
Management and ops stack