從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章
-
Upload
shuen-huei-guan -
Category
Technology
-
view
7.994 -
download
7
Transcript of 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章
![Page 1: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/1.jpg)
���������KKBOX ������ - ��
Drake (���)Sr. Principal EngineerHead of Video Content PlatformKKBOX
by Shuen-Huei Guan, KKBOX. 2015 1
![Page 2: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/2.jpg)
Who am I?
• administrator, Ptt BBS
• technical director / R&D manager, Digimax Inc.
• team player, KKBOX Technologies Ltd.
• contributor, PyCon Taiwan
• phd candidate, GINM, CISE, NTU
by Shuen-Huei Guan, KKBOX. 2015 2
![Page 3: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/3.jpg)
• VideoPass / (ongoing) AnimePass video services in Japan.
• Service up to ~1M users.
• KKBOX Video team: ~40 in one year.
• Cooperation with Japanese organizations.
• To keep growing and heading for the next!
by Shuen-Huei Guan, KKBOX. 2015 3
![Page 4: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/4.jpg)
by Shuen-Huei Guan, KKBOX. 2015 4
![Page 5: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/5.jpg)
What's a Streaming Service?
by Shuen-Huei Guan, KKBOX. 2015 5
![Page 6: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/6.jpg)
What's a Streaming Service?
by Shuen-Huei Guan, KKBOX. 2015 6
![Page 7: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/7.jpg)
Much complicated than just a data flow
• CMS for content management.
• Flexible system (we thought) for evolving business logics.
• System optimication for operation expense.
• Scalability! (we're in cloud era)
by Shuen-Huei Guan, KKBOX. 2015 7
![Page 8: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/8.jpg)
We have done that in KKBOX,and it works (really well)!
by Shuen-Huei Guan, KKBOX. 2015 8
![Page 9: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/9.jpg)
by Shuen-Huei Guan, KKBOX. 2015 9
![Page 10: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/10.jpg)
350 team players to serve10M users across 6 countries
by Shuen-Huei Guan, KKBOX. 2015 10
![Page 11: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/11.jpg)
20M songs
by Shuen-Huei Guan, KKBOX. 2015 11
![Page 12: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/12.jpg)
O2O Events
by Shuen-Huei Guan, KKBOX. 2015 12
![Page 13: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/13.jpg)
If we can make music streaming work, how
about video streaming?— KKBOX CxO
by Shuen-Huei Guan, KKBOX. 2015 13
![Page 14: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/14.jpg)
A turning point: KDDI
by Shuen-Huei Guan, KKBOX. 2015 14
![Page 15: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/15.jpg)
Inside News
• VideoPass/AnimePass switched to KKBOX.
• KDDI is a big stockholder in KKBOX.
• Japan-Taiwan is better than Japan-???.
by Shuen-Huei Guan, KKBOX. 2015 15
![Page 16: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/16.jpg)
We thought it's a peace of cake!
by Shuen-Huei Guan, KKBOX. 2015 16
![Page 17: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/17.jpg)
Let's work on a video-on-demand service
• Adaptive streaming.
• DRM protection.
• Video encoding on cloud.
• Totally cloud-based system.
by Shuen-Huei Guan, KKBOX. 2015 17
![Page 18: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/18.jpg)
Adaptive streaming
refer: https://www.wikiwand.com/en/Adaptivebitratestreaming
by Shuen-Huei Guan, KKBOX. 2015 18
![Page 19: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/19.jpg)
Adaptive streaming (cont'd)
refer: https://www.wikiwand.com/en/Adaptivebitratestreaming
by Shuen-Huei Guan, KKBOX. 2015 19
![Page 20: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/20.jpg)
Adaptive streaming benefits
• An ongoing trend for video streaming.
• YouTube, Netflix, Amazon, HbbTV, ...
• Multiple standards.
• HLS, HDS, MSS, MPEG-DASH, ...
• Matured technology.
• HTTP, HTML5, H.264, AAC, ...
by Shuen-Huei Guan, KKBOX. 2015 20
![Page 21: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/21.jpg)
Adaptive streaming benefits (cont'd)
• Friendly UX with mobile in-mind.
• 3G/4G/Wifi, Android/iOS
• Copyright done easily (and right?!).
• MS PlayReady, Google Widevine, Apple FairPlay, ...
• Scalability with good performance.
• CDN, HTTP/2 or QUIC, ...
by Shuen-Huei Guan, KKBOX. 2015 21
![Page 22: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/22.jpg)
by Shuen-Huei Guan, KKBOX. 2015 22
![Page 23: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/23.jpg)
Adaptive streaming is seizable, but DRM would give you lots of
nightmares.
by Shuen-Huei Guan, KKBOX. 2015 23
![Page 24: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/24.jpg)
ref: https://www.wikiwand.com/en/Digitalrightsmanagement
by Shuen-Huei Guan, KKBOX. 2015 24
![Page 25: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/25.jpg)
(Fortunately,) we have Common Encryption (CENC).
by Shuen-Huei Guan, KKBOX. 2015 25
![Page 26: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/26.jpg)
DRM
• Microsoft PlayReady nearly dominates DRM market right now.
• Google Widevive has huge potential because of Android/Chrome.
• Apple FairPlay stands for her own.
by Shuen-Huei Guan, KKBOX. 2015 26
![Page 27: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/27.jpg)
by Shuen-Huei Guan, KKBOX. 2015 27
![Page 28: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/28.jpg)
Video Encoding Flow
by Shuen-Huei Guan, KKBOX. 2015 28
![Page 29: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/29.jpg)
by Shuen-Huei Guan, KKBOX. 2015 29
![Page 30: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/30.jpg)
Video Encoding Flow
• Source video -> playable adaptive packages.
• Leverage all mighty open-source projects.
• All processed on cloud with capability of dynamic flow!
by Shuen-Huei Guan, KKBOX. 2015 30
![Page 31: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/31.jpg)
by Shuen-Huei Guan, KKBOX. 2015 31
![Page 32: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/32.jpg)
Job script for a workflow
Job {KKBOX Convert Video} -subtasks { Task {Source Inspection} -cmds { Cmd { emilia verify -i s3://bucket/source.mp4 } }
Task {Transcode} --parallel -subtasks { Iterate i -from 0 -to 4 -by 1 -template { Task {Transcode Audio} -cmds { Cmd { ffmpeg -i s3://bucket/source.mp4 -o /tmp/converted_$i.mp4 } } } Iterate i -from 0 -to 8 -by 1 -template { Task {Transcode Video} -cmds { Cmd { ffmpeg -i s3://bucket/source.mp4 -o /tmp/converted_$i.mp4 } } } }
Task {Adaptive} -subtasks { Task {DASH} -subtasks { } Task {HLS} -subtasks { } Task {MSS} -subtasks { } }} -sequential 1
by Shuen-Huei Guan, KKBOX. 2015 32
![Page 33: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/33.jpg)
Orchestrating the execution of workflows for media streaming
service and even morePyCon APAC 2015
by Shuen-Huei Guan, KKBOX. 2015 33
![Page 34: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/34.jpg)
System Architecture
by Shuen-Huei Guan, KKBOX. 2015 34
![Page 35: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/35.jpg)
by Shuen-Huei Guan, KKBOX. 2015 35
![Page 36: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/36.jpg)
AWS services
• EC2 / ELB / S3
• RDS / ElastiCache
• SWF (Simple Workflow)
• SQS / SNS / SES
• Lambda
• CloudWatch
by Shuen-Huei Guan, KKBOX. 2015 36
![Page 37: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/37.jpg)
System architecture
• Backend / CMS
• Laravel (PHP)
• MySQL / Elasticsearch
• Video Encoding System
• Python
by Shuen-Huei Guan, KKBOX. 2015 37
![Page 38: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/38.jpg)
AWS expense and status:
Confidential but available on stage.
by Shuen-Huei Guan, KKBOX. 2015 38
![Page 39: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/39.jpg)
Third-party services
• Pingdom
• Leanplum
• Slack
• JIRA
by Shuen-Huei Guan, KKBOX. 2015 39
![Page 40: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/40.jpg)
by Shuen-Huei Guan, KKBOX. 2015 40
![Page 41: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/41.jpg)
We love Slack.
• Server status / alarm.
• Video encoding status.
• JIRA ticket updates.
• Video quality checking status.
• ...
by Shuen-Huei Guan, KKBOX. 2015 41
![Page 42: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/42.jpg)
But we were wrong. Way more challenges
are waiting for us.
by Shuen-Huei Guan, KKBOX. 2015 42
![Page 43: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/43.jpg)
Technical challenges
• Video file size is much larger than music files.
• 2hr HD movie = 150 x “5min 192kbps music”
• Video file encoding takes much longer.
• 100x time consumption than music.
• Lots of files.
• 2hr HD movie ~= 20,000~30,000 files.
by Shuen-Huei Guan, KKBOX. 2015 43
![Page 44: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/44.jpg)
Technical challenges (cont'd)
• H.264 is not that easy.
• Each H.264-claimed devices support differently.
• Adaptive players needs effrots.
• Smart adaptation, initial lag vs smooth playback, A/V sync, ...
• Video streaming protocols are much complicated.
• HLS, MSS, DASH with DRM(s)!by Shuen-Huei Guan, KKBOX. 2015 44
![Page 45: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/45.jpg)
Business challenges
• Asian services tend to be complex than elegant, especially Japanese ones.
• Business logics changed weekly, but it's an OEM, right?
• OEM is not what KKBOX's good at.
• Programming is an art, so is communication!
by Shuen-Huei Guan, KKBOX. 2015 45
![Page 46: 從音樂走向影音服務 - KKBOX 的影音之路奮鬥史 - 序章](https://reader031.fdocument.pub/reader031/viewer/2022031610/5884893c1a28ab6d1a8b5aeb/html5/thumbnails/46.jpg)
Integration challenges
• Communication in English is ok, but Japanes is better.
• Hard problems doesn't kill developers, but communication does.
• More than 5 vendors to cooperate with.
• Docs of vendors' API? Nearly none.
by Shuen-Huei Guan, KKBOX. 2015 46