Using google appengine_1027
-
Upload
wei-sun -
Category
Technology
-
view
1.358 -
download
3
Transcript of Using google appengine_1027
![Page 1: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/1.jpg)
使用 GoogleAppEngine建立个人信息中心
weibo.com/sagasw2011-11
![Page 2: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/2.jpg)
Agenda
1 )云计算那些事儿 2 ) Google AppEngine 3 ) GAE个人信息中心
4 ) Resources
![Page 3: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/3.jpg)
调查
1 ) Python语言? 2 ) Google AppEngine? 3 )云计算平台?
![Page 4: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/4.jpg)
1 )云计算那些事儿
![Page 5: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/5.jpg)
• IaaS (Infrastructure as a Service) o Amazon EC2 , Microsoft Azure
• PaaS (Platform as a Service) o GoogleAppEngine,新浪AppEngine , HeroKu
• SaaS (Software as a Service)o Zoho , Google Apps,易度
![Page 6: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/6.jpg)
云平台比较
http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf
![Page 7: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/7.jpg)
调查 - 虚拟化软件
![Page 8: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/8.jpg)
调查 - 公有云计算平台
![Page 9: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/9.jpg)
新浪微博 @_ 浪潮之巅 _ :
中国很多大公司 ( 主要是央企 ) ,对云计算理解就是 IDC和设备。 美国的 case 都是应用。美国这点认识比较一致: IDC 和最基础的东西让 Google , Facebook 和 Amazon 去做。
http://weibo.com/2142166543/xtKu40qgP
![Page 10: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/10.jpg)
![Page 11: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/11.jpg)
http://addons.heroku.com/
![Page 12: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/12.jpg)
2 ) Google AppEngine
![Page 13: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/13.jpg)
Google AppEngineGo language, Python, Java (JRuby, Lua, Rails etc.) 优点:
o UrlFetch,Memcache,TaskScheduleo XMPP,Mail,Crono MapReduce,Channelo 支持 OAuth认证或 Google 账号o Image处理 o 域名绑定o 易使用,易管理o 免费限额大方
缺点 : o 无全文搜索o 调整后的付费方案o 需要熟悉新 SQL操作方式
![Page 14: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/14.jpg)
适用于:Wiki,CMS,博客 , 微博 , 手机游戏后台 , 应用服务器 , 问答网站 , 电子书发布 ,SNS, 网店 , 工作流
不适合: 音频视频服务 , 大文件 , 密集计算
![Page 15: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/15.jpg)
DjangoWebApp2 Tornado
https://github.com/facebook/tornado/tree/master/demos/appengine
Web Framework
![Page 16: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/16.jpg)
Python library
BeautifulSoup, PythonTwitter, ReadabilityPIL, PyCrypto, lxml等等
可直接用于 AppEngine?
• 不依赖 C 扩展• 不使用数据库• 不访问本地资源
http://pypi.python.org/pypi
![Page 17: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/17.jpg)
Java on AppEngine
Play! Framework
http://www.slideshare.net/gasolin/play-framework-on-google-app-engine https://github.com/guillaumebort/play-gae
Scala Lift Framework
http://lift-example.appspot.com/ http://www.scala-lang.org/node/1826
JavaScript
http://www.appenginejs.org/
![Page 18: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/18.jpg)
JRuby on Google App Enginecode.google.com/p/appengine-jruby/github.com/jruby/jruby-rack
![Page 19: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/19.jpg)
Lua in Appengine
http://code.google.com/p/aelua/http://code.google.com/p/jillcode/http://code.google.com/p/kahlua/https://github.com/krka/kahlua2http://sourceforge.net/projects/luaj/
![Page 20: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/20.jpg)
企业使用 ?
•比较容易扩展•管理方便 •用于特定任务 •更推荐 EC2 http://code.google.com/appengine/business/ http://code.google.com/appengine/docs/premier/index.html
![Page 21: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/21.jpg)
![Page 22: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/22.jpg)
GAE SDK 1.5.5
• Python2.7• 支持多线程,可使用 threading• 支持并发请求• 可以访问 bytecode• 支持更多 Python lib( PIL v1.1.7)。• 前台请求的时间 30秒增加到 60秒, URLFetch请求时间从 10秒增加到 60秒。
• URLFetches的 POST参数大小从 1MB增加到 5MB。
From Keakon's blog
![Page 23: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/23.jpg)
GAE编程心得 • 尽量使用 memcache• 组合 Appengine & VPS• 通过 service访问 VPS数据库, GAE进行缓存• 使用 google或 VPS搜索• Nginx反向代理 AppEngine(必须的)
VPS seller and coupon http://www.lowendbox.com/ Install Debian, Nginx in VPS http://www.howtoforge.com/
![Page 24: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/24.jpg)
![Page 25: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/25.jpg)
3 )使用 GAE建立个人信息中心
![Page 26: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/26.jpg)
个人常用的信息服务• TwitterWeb客户端,可以同步到新浪腾讯网易搜狐
• RSS 聚合阅读• 获取网页 (urlfetch),解析短网址• Cron Jobs• 在线个人笔记,在线个人博客• 网络小说阅读• 推送文章到 Kindle• 发送天气短信给自己• 生日提醒(邮件,聊天工具,短信等多种手段)• 监控 iphone购买网页的变化 • 相册• proxy
![Page 27: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/27.jpg)
![Page 28: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/28.jpg)
个人信息平台已经实现的功能
1 )选择现成代码进行修改( V2EX,Micolog, GaeBBS)2 )同步发推 , 新浪微博功能 ,t.co扩展3 )定期采集 RSS功能4 )支持 Google Reader share5 ) HN文章全文阅读, RSS Feed输出6 )网页内容变化,通过短信提醒7 )支持 XMPP协议( gtalk)客户端自动聊天
![Page 29: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/29.jpg)
新浪微博 腾讯微博 import mechanize br = mechanize.Browser() br.open("http://abcd.com/openqqsdk/qqweibopost.php") br.select_form(nr=0) strform = status.encode('utf-8') br.form['text'] = str br.submit()
VPS上搭建微博 PHP SDK作中转
![Page 30: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/30.jpg)
BeautifulSoap使用
from BeautifulSoup import BeautifulSoupVALID_TAGS = ['strong', 'em', 'p', 'ul', 'li', 'br', 'img', 'ol', 'blockquote']
def sanitize_html(value): soup = BeautifulSoup(value) for tag in soup.findAll(True): if tag.name not in VALID_TAGS: tag.hidden = True
return soup.renderContents()
![Page 31: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/31.jpg)
https://gist.github.com/1298152
![Page 32: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/32.jpg)
![Page 33: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/33.jpg)
![Page 34: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/34.jpg)
![Page 35: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/35.jpg)
![Page 36: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/36.jpg)
![Page 37: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/37.jpg)
![Page 38: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/38.jpg)
https://gist.github.com/1312657
![Page 39: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/39.jpg)
![Page 40: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/40.jpg)
资源使用情况(优化前)
![Page 41: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/41.jpg)
资源使用情况(优化后)
![Page 42: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/42.jpg)
Mobile01 洁净版阅读功能设计全过程
• 分析网页内容• 设计抓取方式• HTML readability
– viewtext.org, www.instapaper.com/m?u=
• 下载图片• 添加文章
![Page 43: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/43.jpg)
![Page 44: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/44.jpg)
![Page 45: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/45.jpg)
![Page 46: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/46.jpg)
![Page 47: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/47.jpg)
![Page 48: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/48.jpg)
![Page 49: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/49.jpg)
4 ) Resources
![Page 50: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/50.jpg)
其它云平台
Amazon EC2 Heroku(免费 + 付费)Microsoft Azure Sina AppEngine Node AppEngine (内测中)Yo2Lua Lua AppEngine (内测中)
![Page 51: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/51.jpg)
Useful links
http://googleappengine.blogspot.com/http://www.v2ex.com/go/gaehttp://www.neopythonic.blogspot.com/http://blog.notdot.net/http://www.billkatz.com/http://www.franciscosouza.com/tag/google-app-engine/http://highscalability.com/google-architecturehttps://groups.google.com/forum/#!forum/google-appenginehttp://www.reddit.com/r/AppEngine/
![Page 52: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/52.jpg)
可用于微创新的项目
http://code.google.com/p/appscale/ http://code.google.com/p/app-engine-site-creator/http://code.google.com/p/google-app-engine-samples/http://code.google.com/p/fantasm/https://github.com/metachris/feedmailerhttp://code.google.com/p/jaikuengine/http://code.google.com/p/appengine-mapreduce/http://code.google.com/p/rietveld/https://github.com/xuming/micologhttps://github.com/livid/v2exhttp://code.google.com/p/gae-bbs/
![Page 53: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/53.jpg)
Kindlereader
一个定时将 Google reader发送至 kindle的工具master 分支为单用户版 ( 基于 python), 运行于 Linux, Mac OSX, Windows gae 分支为运行于 Google app engine 的多用户版 , demo: http://reader.dogear.mobi
https://github.com/jiedan/kindlereader/tree/gae
![Page 54: Using google appengine_1027](https://reader035.fdocument.pub/reader035/viewer/2022081400/5552bf49b4c905920f8b4752/html5/thumbnails/54.jpg)
Q & A