Keisuke Nishitani, Solutions Architect, Amazon Web
Services Japan K.K.
2015.11.21
Automated Testing on AWS Device Farm
{
"Name" : "","Twitter" : "@Keisuke69","Profile" : {
"Role" : "Solution Architect","Customers": [ "Web Services", "Start-up
],"Services" : [
"AWS Lambda","Amazon API Gateway","All Mobile Services
]}
}
AWS Device Farm Q&A
AWS Device Farm Q&A
48%
32%
2.74 2014 4 80%
Failing to meet mobile app user expectations - A mobile app user survey from Dimensional Research (2015/3/30)http://go.saas.hp.com/apppulse-mobile-survey
Mobile Marketing Costs See Unprecedented Year-over-Year Growth, According to the Fiksu Indexes (2015/5/29)https://www.fiksu.com/about/news-releases/fiksu-indexes-mobile-marketing-costs-see-unprecedented-year-over-year-growth
OS
Open Signal - Android Fragmentation Visualized (2014/8)http://opensignal.com/reports/2014/android-fragmentation/
OS
Android 11,868 2014 8 18,796
Android 38% Jelly Bean, 39 % KitKat, 12% Lolipop, 10% Gingerbread Ice Cream Sandwich
Open Signal - Android Fragmentation Visualized (2014/8)http://opensignal.com/reports/2014/android-fragmentation/
Android Developers (2015/6)http://developer.android.com/intl/zh-CN/about/dashboards/index.html
IT QA 2012 18% 2014 26% 35%
10 4 IT
51%
Capgemini - World Quality Report 2014-15https://www.capgemini.com/thought-leadership/world-quality-report-2014-15
77% of Mobile App Purchasers Have Concerns When Buying Mobile Apps -Harris (2013/8/20)http://www.businesswire.com/news/home/20130820006244/en/77-Mobile-App-Purchasers-Concerns-Buying-Mobile#.Vaz9yaad1N0
OS
AWS Device Farm Q&A
AndroidiOSFire OS
YOUR
APP+
AWS Device FarmAWS AndroidiOSFire OS
OS
AWS
AWS
AppiumJUnit, TestNG Calabash Instrumentation Android
JUnit, Espresso, Robotium uiautomator XCTest(Xcode5), KIF iOS
QA
https://github.com/appium/sample-code AndroidTest.java
https://github.com/appium/sample-code AndroidContactsTest.java
http://calaba.sh/
Fuzz Fuzz UI (seed)(seed)
Fuzz
App Exproler
Android
View
AWS Device Farm
- WiFiBluetoothGPSNFC
CPU
AWS Jenkins Jenkins CI API
Jenkins Jenkins CI AWS Device Farm
AWS Device Farm Jenkins Pluginhttps://github.com/awslabs/aws-device-farm-jenkins-plugin
Jenkins
AWS Device Farm SSL
AWS 30
WiFiBluetooth
4GB 5 WiFi
SMS
Google Play
PhoneGap, Titanium, Xamarin, Unity,
AWS Device Farm Q&A
Sier/
OS
AWS Device FarmOSAWS
AWS Device Farm
Fuzz
IHS Sphera First Report
Android/iOS
QA OS
Android 4.07000
Development/testing process
Develop SCMcommit Automated
buildtrigger Automated
UI test(simulators)
always
Ismilestone
?
no
Beta deployto
manual testers
AutomatedUI test
(Device Farm)
yes
Development/testing process
Develop SCMcommit Automated
buildtrigger Automated
UI test(simulators)
always
Ismilestone
?
no
Beta deployto
manual testers
AutomatedUI test
(Device Farm)
yes
PMease QuickBuild Java AndroidiOS HockeyAppTestFlight Appium UI AWS Device Farm
Development/testing process
Develop SCMcommit Automated
buildtrigger Automated
UI test(simulators)
always
Ismilestone
?
no
Beta deployto
manual testers
AutomatedUI test
(Device Farm)
yes
Appium
Ex: Ruby, Python, Java, JavaScript, PHP, C#
Ex: JUnit for Java, NUnit for C# UI
UIAutomation iOS UIAutomator Android
Development/testing process
Develop SCMcommit Automated
buildtrigger Automated
UI test(simulators)
always
Ismilestone
?
no
Beta deployto
manual testers
AutomatedUI test
(Device Farm)
yes
HockeyApp AndroidiOSWindows iOS
TestFlight iOS
Development/testing process
Develop SCMcommit Automated
buildtrigger Automated
UI test(simulators)
always
Ismilestone
?
no
Beta deployto
manual testers
AutomatedUI test
(Device Farm)
yes
Etsy
Etsy iOS, Android4 2 28QA
WebMobile
CI
1020 build, lint, ,
810
iOS: 50 Mac mini / MacBook Pro Android: 26 Linux containers
/
nightly build Admin
Android Espresso + Genymotion KIF(iOS) simulator/emulator1 Build Flow Plugin
AWS Device Farm
Jenkins plugin
30
EspressoAndroidKIFiOS
Google AnalyticsdevicesOS
AWS Device Farm Device Pool
4 10 30
pass/fail
Jumbotron
AWS Device Farm and Etsy> Device Farm tests results visualization
API
graphite
AWS Device Farm Q&A
- Device Minutes () 250 Device Minutes 1 Device Minute 0.17
- 250
Android
2502=$500
AWS Device Farm Q&A
Android, iOSFire OS
Fuzz
Jenkins CI
AWS Device Farm
AWS Device Farm Q&A
Thanks
Top Related