60分鐘完送百萬edm,背後雲端ci/cd實戰大公開
-
Upload
kai-chu-chung -
Category
Software
-
view
1.228 -
download
0
Transcript of 60分鐘完送百萬edm,背後雲端ci/cd實戰大公開
60分鐘完送百萬EDM,背後雲端CI/CD實戰大公開
HELLO!I am Cage Chung
I am here because I like to share my experiences. You can find me at http://kaichu.io / QNAP 雲端應用部資深工程師
https://www.facebook.com/groups/GCPUG.TW/
https://plus.google.com/u/0/communities/116100913832589966421
[您知道”GCPUG”要怎麼唸嗎?為什麼會有一隻狗在 Logo裡面呢? ]
Google Cloud Platform User Group的縮寫是GCPUGGCPUG直接唸成G.C.P.U.G?當然可以!
但它也可以分開來,唸成 G.C. PUG喔~
Pug,指的是巴哥犬,所以 GCPUG的Logo中間才會有一隻可愛的巴哥犬喲。
下次聽到別人說G.C. PUG 的時候,您就可以大聲說:「我也是G.C. PUG社團成員!」
17th Meetup
大綱
◉ EDM 二三事 & 送信服務架構◉ CI (持續整合) & CD (持續部署) 整合◉ Demo/Cloud Source Repositories, Stackdriver Debugger◉ 小提示 & 學習資源
EDM 二三事 & 送信服務架構Let’s start with the first set of slides 1
寄送 Email 好簡單 - API Call
# Google App Engine Send Email example (Python)
from google.appengine.api import app_identity
from google.appengine.api import mail
def send_approved_mail(sender_address):
# [START send_mail]
mail.send_mail(sender=sender_address,
to="Albert Johnson <[email protected]>",
subject="Your account has been approved",
body="""Dear Albert:
Your example.com account has been approved. You can now visit
http://www.example.com/ and sign in using your Google Account to
access new features.
Please let us know if you have any questions.
The example.com Team
""")
# [END send_mail]
send_approved_mail('{}@appspot.gserviceaccount.com'.format(app_identity.get_application_id()))
客戶最關心的事 - 送達率
Sender(Bob)
Recipients(Alice / Sunrain)
Email Reputation
Sender Score Average Delivered
0-60 21%
61-70 67%
71-80 68%
81-90 92%
91-100 95%
[Your Email Reputation](http://www.slideshare.net/emaildelivered/email-reputation-2)[5 Ways to Check Your Sending Reputation | SendGrid](https://sendgrid.com/blog/5-ways-check-sending-reputation/)
◉ Authentication
◉ Bounce management
◉ List cleanliness
◉ User engagement (recipient
feedback)
EDM 實際會面臨到的種種問題...
Email Reputation
SpamminessWhiteList
Shared IP
DK(IM)/SPF
Blocks
Invalid Emails
Blacklists
SpamDedicated IP
Scores
[Best Practices in Email Deliverability - Act-On Software](https://www.act-on.com/ebook/email-marketing-deliverability-ebook/)
[垃圾邮件检测结果](http://www.mail-tester.com/web-eBR3ar&lang=zh)
Mail Tester
Sendgrid 如何幫我們送信
[Email Message Flow Explained - SendGrid | SendGrid](https://sendgrid.com/docs/Classroom/Basics/Email_Infrastructure/email_flow.html)
Sendgrid 遊戲規則
[IP Warmup Schedule - SendGrid Documentation | SendGrid](https://sendgrid.com/docs/API_Reference/Web_API_v3/IP_Management/ip_warmup_schedule.html)
[SendGrid | Heroku Dev Center](https://devcenter.heroku.com/articles/sendgrid)
送信服務架構
我們所使用到的資源
App Engine
BigQuery
Clou
d St
orag
e
DataStore
Clou
d En
dpoi
nts
Sendgrid
A fast, economical and fully managed data warehouse for
large-scale data analytics
BigQuery
Cloud Datastore is a highly-scalable NoSQL database for
your web and mobile applications.
DataStore
A powerful platform to build web and mobile apps that
scale automatically.
App Engine
A Powerful, Simple and Cost Effective Object Storage Service.
Cloud Storage
Delivering your transactional and marketing email through one reliable platform.
Sendgrid
Create RESTful APIs on App Engine accessible to iOS, Android and Javascript clients.
Cloud Endpoints
App Engine
BigQuery
Clou
d St
orag
e
DataStore
Clou
d En
dpoi
nts
Sendgrid
UI 彈性化擴展層、排程器、 Tasks 處理。針對不同任務的機器予給優化的資源,並可自動進行擴展
EDM Server
App Engine
BigQuery
Clou
d St
orag
e
DataStore
Clou
d En
dpoi
nts
Sendgrid
Cloud logging / sendgrid webooks (tracking - bounce, click, open, spam) 的資料值串接 BigQuery, 作後序分析
資料分析
App Engine
BigQuery
Clou
d St
orag
e
DataStore
Clou
d En
dpoi
nts
Sendgrid
檔案暫存
EDM html 檔案、連絡人清單暫存區
App Engine
BigQuery
Clou
d St
orag
e
DataStore
Clou
d En
dpoi
nts
Sendgrid
RESTful API endpoint
提供 RESTful API 給前端應用程式
https://<gcp-project-id>.appspot.com/_ah/api/explorerhttp://localhost:8080/_ah/api/explorer
App Engine
BigQuery
Clou
d St
orag
e
DataStore
Clou
d En
dpoi
nts
Sendgrid
Sendgrid
送信服務商,提供豐富的報表、EDM 相關的 know how,亦有 webhook 可以串接
App Engine
BigQuery
Clou
d St
orag
e
DataStore
Clou
d En
dpoi
nts
Sendgrid
Metadata 儲存
Nosql datastore. 儲存 EDM 送信服務過程中相關 Metadata. 連絡人清單、Log 等
架構
queue
MetadataEDM Server
App Engine
Task Queues
Cloud Storage
endpoints API(OAuth 2)
CloudeDatastore
CSV
upload recipients
Store/retrieve metadata
RESTful APIs for recipients
mail send
cron
...
# dispatch.yaml
application: your-project-id
dispatch: - url: "*/favicon.ico" module: default
- url: "your-project-id.appspot.com/" module: default
- url: "*/tasks/*" module: tasks
- url: "*/googleae8f4bcce8bec00c.html" module: ownership
- url: "*/*" module: "default"
# default/app.yaml
automatic_scaling:
min_idle_instances: 1
# tasks/app.yaml
instance_class: F4
automatic_scaling:
min_idle_instances: 2
min_pending_latency: 30ms
max_pending_latency: 300ms
# max_concurrent_requests: 80
# max_idle_instances: automatica # default value
# min_pending_latency: 30ms # default value
# max_pending_latency: automatic
# max_concurrent_requests: 50
EDM Server
EDM 檔案
聯絡人清單
Queue
API
Retry
fail
Retry cron
Schedule cron SuccessLog
流程示意圖 failLog
CI(持續整合) & CD (持續部署) 整合Let’s start with the second set of slides 2
Image by[每个架构师都应该研究下康威定律](http://www.infoq.com/cn/articles/every-architect-should-study-conway-law)
Devops: CI & CD (Docker, gitlab, GCP)
Bitbucket
Github
Jenkins
Travis CI
CircleCI
Docker
Gitlab
Cloud Source Repositories
Stackdriver Debugger
Origin imgae from and modified [DevOps:持續整合&持續交付(Docker、CircleCI、AWS)](http://blog.amowu.com/2015/04/devops-continuous-integration-delivery-docker-circleci-aws-beanstalk.html)
簡單腳本
◉ Local Development / Test◉ Gitlab-ci-multi-runner
◉ Local runner exec test - docker◉ Gitlab runner test docker◉ gcloud deploy
◉ Google App Engine◉ Google Cloud Source Repositories◉ Stackdriver Debugger
Gitlab Project Runner
Gitlab Project Variables
Register Gitlab runner
$ gitlab-ci-multi-runner register
WARNING: Running in user-mode.
WARNING: The user-mode requires you to manually start builds processing:
WARNING: $ gitlab-runner run
WARNING: Use sudo for system-mode:
WARNING: $ sudo gitlab-runner...
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/ci):
https://gitlab.com/ci
Please enter the gitlab-ci token for this runner:
<your-gitlab-project-runner-register-token>
Please enter the gitlab-ci description for this runner:
[Kai-Chus-MBP]: edm
Please enter the gitlab-ci tags for this runner (comma separated):
gae,python,edm
Registering runner... succeeded runner=kvwzB1cE
Please enter the executor: ssh, shell, parallels, docker, docker-ssh, virtualbox:
docker
Please enter the default Docker image (eg. ruby:2.1):
cage1016/gitlab-ci-gcp:v1.6
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
.gitlab-ci.yamlimage: cage1016/gitlab-ci-gcp:v1.6
before_script:
- export CLOUDSDK_CORE_DISABLE_PROMPTS=1
- export CLOUDSDK_PYTHON_SITEPACKAGES=1
- export GCP_PROJECT=cage-20160705-edm
- export GCP_PROJECT_VERSION=uat
types:
- test
- deploy
test:
stage: test
script:
- sh ./scripts/tests.sh
deploy:
stage: deploy
script:
- sh ./scripts/deploy.sh
only:
- develop
.gitlab-ci.yaml (./scripts/tests.sh)
#!/usr/bin/env bash
virtualenv env
source env/bin/activate
# install Google App engine test env packages from requirements.testing.txt
pip install -r requirements.testing.txt
echo $GCLOUD_SERVICE_JSON_KEY > tasks/cage-20160705-edm-cc07ec5bf115.json
# run tests
py.test tasks/tests
.gitlab-ci.yaml (./scripts/deploy.sh)
#!/usr/bin/env bash
echo $GCLOUD_KEY > key.json
gcloud auth activate-service-account $GCLOUD_ACCOUNT --key-file key.json
gcloud --quiet config set project $GCP_PROJECT
# tasks module
pip install -r tasks/requirements.txt -t tasks/lib/
gcloud --quiet preview app deploy tasks/app.yaml --no-promote --version $GCP_PROJECT_VERSION --project
$GCP_PROJECT
# gen-repo-info-file
gcloud app gen-repo-info-file
git push google develop
Local runner exec test docker
// [install gitlab-ci-multi-runner](https://goo.gl/a99Aa4)
// list and register gitlab-ci-runner
$ gitlab-ci-multi-runner list
Listing configured runners ConfigFile=/Users/cage/.gitlab-runner/config.toml
edm Executor=docker Token=<your-token> URL=https://gitlab.com/ci
// prepare a docker-machine for gitlab-ci-multi-runner
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
runner - virtualbox Running tcp://192.168.99.100:2376 v1.11.0
// run a gitlab-ci-multi-runner test via docker
$ gitlab-ci-multi-runner --debug exec docker test --env "GCLOUD_KEY=$GCLOUD_KEY" --env " GCLOUD_ACCOUNT=
$GCLOUD_ACCOUNT" --env="GCP_PROJECT_VERSION=$GCP_PROJECT_VERSION"
$ git remote add google https://source.developers.google.com/p/[PROJECT_ID]/r/default
// generates a file named source-context.json, which contains information about the version of the source
code used to build the application
$ gcloud preview app gen-repo-info-file
// push code to google cloud source repositories
$ git push google master Google Cloud Source Repositories - List
Google Cloud Source Repositories - Source Code
Stackdriver Debugger | Source Code
Stackdriver Debugger | Debug
Image by [每个架构师都应该研究下康威定律](http://www.infoq.com/cn/articles/every-architect-should-study-conway-law)
Demo/Cloud Source Repositories, Stackdriver Debugger Let’s start with the third set of slides 3
小提示 & 學習資源Let’s start with the fourth set of slides 4
“5G Google cloud storage for App engine for FREE
<project-id>.appspot.comstaging.<project-id>.appspot.com
“App Engine Admin API
Cloud Source Repositories API Google Cloud Storage
學習資源
◉ [Best Practices in Email Deliverability - Act-On Software](https://www.act-on.com/ebook/email-marketing-deliverability-ebook/)
◉ [Getting Started - SendGrid Documentation | SendGrid](https://sendgrid.com/docs/index.html)
◉ [Newsletters spam test by mail-tester.com](http://www.mail-tester.com/)
◉ [Your Email Reputation](http://www.slideshare.net/emaildelivered/email-reputation-2)
學習資源
◉ Deploying to App Engine or Managed VMs from Travis CI◉ [Deploying to App Engine or Managed VMs from Travis CI -
YouTube](https://www.youtube.com/watch?v=7U4jjRw_AJk&feature=youtu.be)
◉ [GoogleCloudPlatform/continuous-deployment-demo](https://github.com/GoogleCloudPlatform/continuous-deployment-demo)
學習資源
◉ GCP◉ [Cloud Source Repositories - Private Git Repositories —
Google Cloud Platform](https://cloud.google.com/source-repositories/)
◉ [Stackdriver Debugger - Production Debugging - Google Cloud Platform — Google Cloud Platform](https://cloud.google.com/debugger/)
◉ [Managed VMs lab · kaichu.io](http://kaichu.io/2015/09/managed-vms-lab/)
DevOps (Development & Operations)
Install the necessary apps with one-click on demand and the system will automatically finish the set-up processQuickly build a development environment ideally suited for collaborationOne NAS with Multiple projects
Recommended apps for quick installation : Private Registry, GitLab, and Jenkins.
THANKS!Any questions?
You can find me athttp://kaichu.io / [email protected]