Кадры решают все, или стриминг видео, Александр Тоболь...

44
Кадры решают все, или стриминг видео в одноклассниках Александр Тоболь

description

Доклад Александра Тоболя на HighLoad++ 2014.

Transcript of Кадры решают все, или стриминг видео, Александр Тоболь...

  • 1. ,

2. 10 75 100 3. tempstoragepersistentstorageuploadserverdownloadservertransformationserverhttp://habrahabr.ru/company/odnoklassniki/blog/ 4. MPEG1/2/4, DivX, h.264, WMV7/8/9COder DECoderMP3, AAC, OGG Vorbis, DTS 5. MKV, QuickTime, FLV, 3GP, MP4, .TS 6. MP4H.264AACFLVH.263MP3WebMVP8Vorbis, MP3 7. Transformationserverpersistent storageffmpegmp4boxmp4parserMP4, H.264, AAC144, 240, 360, 480, 7201080, 1440, 2160temp storageAny container/codecWxH 8. ffmpeg -i original.mp4 -threads 3 -loglevel info -y -filter_complex [0:0]yadif=0:-1:0,split=3[split0][split1][split2];[split0]scale=640:356[out0];[split1]scale=256:142[out1];[split2]scale=426:236[out2] -vcodec libx264 -map[out0] -vb 559k -r 25 -g 60 -vprofile high -acodec libfdk_aac -refs 1 -bf 3 -x264opts b-pyramid:mixed-refs:weightb:8x8dct:deblock=2,2:no-scenecut-b_strategy 1 -wpredp 2 -qdiff 4 -flags +loop -partitions +parti8x8+parti4x4+partp8x8+partb8x8 -me_method dis -keyint_min 25 -map_metadata -1 -qmin 10 -me_range 16 -qcomp 0.6 -qmax 51 -i_qfactor 0.71 -subq 2 -af aresample=first_pts=0 -pix_fmt yuv420p -coder 1-trellis 0 -map 0:1 -ac 2 -ab 55k -ar 44100 -sn -f mp4 LOW -vcodec libx264 -map [out0] -vb 80k -r 25 -g 60 -vprofile baseline -acodeclibfdk_aac -refs 1 -bf 3 -x264opts b-pyramid:mixed-refs:weightb:deblock=2,2:no-scenecut -b_strategy 2 -wpredp 2 -qdiff 4 -flags +loop -partitions +parti4x4 -me_method full -keyint_min 25 -map_metadata -1 -qmin 10 -me_range 16 -qcomp 0.6 -qmax 51 -i_qfactor 0.71 -subq 2-af aresample=first_pts=0 -pix_fmt yuv420p -coder 1 -trellis 0 -map 0:1 -ac 2 -ab 32k -ar 44100 -sn -f mp4 MOBILE -vcodec libx264 -map[out0] -vb 250k -r 25 -g 60 -vprofile baseline -acodec libfdk_aac -refs 1 -bf 3 -x264opts b-pyramid:mixed-refs:weightb:deblock=2,2:no-scenecut-b_strategy 2 -wpredp 2 -qdiff 4 -flags +loop -partitions +parti4x4 -me_method full -keyint_min 25 -map_metadata -1 -qmin 10 -me_range 16 -qcomp 0.6 -qmax 51 -i_qfactor 0.71 -subq 2 -af aresample=first_pts=0 -pix_fmt yuv420p -coder 1 -trellis 0 -map 0:1 -ac 2 -ab48k -ar 44100 -sn -f mp4 LOWEST 9. tempstoragepersistentstorageuploadserverdownloadservertransformationserver 10. mp4 pseudo-streamingMOOV data1 data2 data mdata Nvm amFast start v0 a0 v1 a1 vN aNdata NdatamHTTPbyte-rangerequeststart timeparameter inthe requestURLMOOV data m data N 11. 12. downloadserverpersistentstorageRAM - 96GBSSD - 4TB1- 2- Segment256KbLRUFIFO 13. Zero-copydownloadserverLRU1- RAM - 96GB/dev/shm/cache.mem ssize_t sendfile(int out_fd, int in_fd, off_t *offset, size_t count);ssize_t send(int sockfd, const void *buf, size_t len, int flags);https://github.com/odnoklassniki/one-nio/ 14. downloadpersistentstorageRAM 96GbSSD - 4Tbeth4 x 10 Gbps80%20%eth10 Gbps 15. - , , 16. mp4 :- 12 - 21 17. ?Headerdata1 data2 data k data N 6 Mb 18. - 6 - 30% 19. 1. 2-3 2. 3-6 3. 4. 20. 21. HLS HDS h264,aac MP4, MPEG2-TS MPEG2-TS mpd m3u8pluginsh264,mp3,aacf4mDASHf4vdash.js native 22. wmspanel.com 23. Dynamic Adaptive Streaming over HTTPHeader data1 data2 data k data N He data1 data2 data k data N MP4MPEG-DASHMPEG2-TS 24. Hedata1 data2 data k data N He data1 data2 data k data N He data1 data2 data k data N http://example.com/manifest.mpd2160p 720pbandwidthtimeMPEG-DASH manifestHe 25. MPEG2 Transport streamhttp://example.com/video.m3u8#EXTM3U#EXT-X-STREAM-INF:BANDWIDTH=200000lowest.m3u8#EXT-X-STREAM-INF:BANDWIDTH=311111low.m3u8#EXT-X-STREAM-INF:BANDWIDTH=484444medium.m3u8#EXT-X-STREAM-INF:BANDWIDTH=737777high.m3u8http://example.com/lowest.m3u8http://example.com/low.m3u8http://example.com/medium.m3u8http://example.com/lowest.m3u8#EXTM3U#EXT-X-VERSION:3#EXT-X-TARGETDURATION:11#EXTINF:10.083333,high000.ts#EXTINF:9.958333,high001.ts#EXTINF:10.000000,high002.tshttp://example.com/high000.tshttp://example.com/high001.tshttp://example.com/high002.tsHttp Live Streaming 26. dashFFMPEGUpload FFMPEGmp4MP4BOXhls10 x 3 27. h264Upload dashpersistentstoragemp4hlsaacdownloadserverDeltatransformationserver 28. HLS1.ts 2.ts k.ts N.ts HLS MPEG2-TS 40/ 188 = 26 29. persistentstoragemp4: h264, aacDeltaHLSDASH 30. MPEG-DASH 31. Header data1 data2 data k data N He data1 data2 data k data N HeHe OffsetsMP4DASHDELTAtransformationservermp4box DASH 32. HeHeHeader data1 data2 data k data N He data1 data2 data k data N DELTAMP4DASHdownloadserver DASH 33. Header dataHeader dataHe data1 data2 data k data N He data1 data2 data k data N 720p2160pMP4fragmented MP4720p2160pHeader dataMP4Header data720p2160p DASH 34. FLASH MPEG-DASH :1. /2. UrlLoader UrlStream3. 4. , sidx5. 6. as3 dashhttps://github.com/castlabs/dashas 35. HLS 36. m3u8playlist HLSUpload FFMPEG mp4 FFMPEGdeltadelta indexpersistentstoragempeg2fragments 37. Panda diffMPEG2 MP4 DELTAINDEX 38. downloadMP4 ~50 /DASH ~40 /HLS ~20 /persistentstorageRAM 96GbSSD - 4Tbeth4 x 10 Gbps80%20%eth10 Gbps 39. 40. (MP4) /2 adaptive bitrate streamingMPEG-Dash/HLS 4 /12 1.5 4 /12 1.3 - +10% - +20% CPU 6 ! 41. , , SmartTV 42. 1. 2. DASH- FLASH HTML-53. HLS iOS- SmartTV4. 5. 6 /50 / * 4 = 200 / 43. ! ! 44.