GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov
-
Upload
amd-developer-central -
Category
Entertainment & Humor
-
view
1.761 -
download
0
description
Transcript of GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov
![Page 1: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/1.jpg)
RAPIDFIRE API FOR CLOUD GAMING BRUNO STEFANIZZI
CHRISTOPHER MAYER DMITRY KOZLOV
![Page 2: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/2.jpg)
2 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
AMD RapidFire Technology
Low Latency HD Image Quality Mul?ple Streams Virtualiza?on
Enablement
! Most cloud gaming soluJons are CPU based ! RapidFire is dedicated cloud hardware and soQware soluJon with API to simplify integraJon – Deliver more HD games streams per GPU with low latency -‐> 6 x HD 720p30 fps – Leverage AMD hardware on both server and client
![Page 3: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/3.jpg)
3 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
AMD RapidFire Technology
High resolu?on Collabora?ve Virtual Desktop Adapta?ve to
Network environment
! Design for many use cases and workflows
![Page 4: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/4.jpg)
4 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
Notebook Handheld Desktop
GCN Render Graphics
Frame Low-‐latency frame capture
Grab
VCE HD mul?-‐stream Compression
![Page 5: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/5.jpg)
5 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
AMD RapidFire Technology
Server
Client
Keyboard Mouse events
Keyboard Mouse
decode
HW H264 Encode
GPU
Frame Buffer
System Memory
DRIVER
RAPIDFIRE API
CPU
H264
Encode
Keyboard Mouse
Network
![Page 6: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/6.jpg)
6 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
RAPIDFIRE TECHNOLOGY DATA FLOW OVERVIEW
Server Client
Radeon Sky GPU
NETWORK
ENCODE 3D ENGINE
FRAME BUFFER
NETWORK GAME SERVER
DECODE UI
FRAME BUFFER
! RF API module
! external module
![Page 7: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/7.jpg)
7 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
RAPIDFIRE TECHNOLOGY SERVER SIDE DATA FLOW
Server Radeon Sky GPU
NETWORK
ENCODE
3D ENGINE GAME
INSTANCE
! Network component transfers UI events from the client to the server
EXECUTION RESOURCES
FRAME BUFFER
![Page 8: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/8.jpg)
8 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
RAPIDFIRE TECHNOLOGY SERVER SIDE DATA FLOW
Server Radeon Sky GPU
NETWORK
ENCODE
3D ENGINE GAME
INSTANCE
! Network component transfers UI events from the client to the server
! Game server sends commands to GPU to draw next frame
EXECUTION RESOURCES
FRAME BUFFER
![Page 9: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/9.jpg)
9 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
RAPIDFIRE TECHNOLOGY SERVER SIDE DATA FLOW
Server Radeon Sky GPU
NETWORK
ENCODE
3D ENGINE GAME
INSTANCE
! Network component transfers UI events from the client to the server
! Game server sends commands to GPU to draw next frame
! GPU distributes the work among execuJon resources and produces resulJng frame into the frame buffer
EXECUTION RESOURCES
FRAME BUFFER
![Page 10: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/10.jpg)
10 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
RAPIDFIRE TECHNOLOGY SERVER SIDE DATA FLOW
Server Radeon Sky GPU
NETWORK
ENCODE
3D ENGINE GAME
INSTANCE
! Network component transfers UI events from the client to the server
! Game server sends commands to GPU to draw next frame
! GPU distributes the work among execuJon resources and produces resulJng frame into the frame buffer
! VCE is performing asynchronous frame sequence encoding into H264 video stream and the data is fetched to system memory by the app
EXECUTION RESOURCES
FRAME BUFFER
![Page 11: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/11.jpg)
11 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
RAPIDFIRE TECHNOLOGY SERVER SIDE DATA FLOW
Server Radeon Sky GPU
NETWORK
ENCODE
3D ENGINE GAME
INSTANCE
! Network component transfers UI events from the client to the server
! Game server sends commands to GPU to draw next frame
! GPU distributes the work among execuJon resources and produces resulJng frame into the frame buffer
! VCE is performing asynchronous frame sequence encoding into H246 video stream and the data is fetched to system memory by the app
! SW-‐encoded audio stream is merged with the video stream and sent to the network
EXECUTION RESOURCES
FRAME BUFFER
![Page 12: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/12.jpg)
12 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
RAPIDFIRE TECHNOLOGY CLIENT SIDE DATA FLOW
Client
Radeon GPU NETWORK
DECODE
! Game client receives H264 stream from the server using RTSP protocol
FRAME BUFFER
GAME CLIENT
DECODE AUDIO
Audio device
UI
Input devices
![Page 13: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/13.jpg)
13 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
RAPIDFIRE TECHNOLOGY CLIENT SIDE DATA FLOW
Client
Radeon GPU NETWORK
H264 DECODE
! Game client receives H264 stream from the server using RTSP protocol
! The client sends the stream to Radeon GPU which performs H264 hardware decoding to the frame buffer
FRAME BUFFER
GAME CLIENT
DECODE
Audio device
UI
Input devices
![Page 14: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/14.jpg)
14 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
RAPIDFIRE TECHNOLOGY CLIENT SIDE DATA FLOW
Client
Radeon GPU NETWORK
DECODE
! Game client receives H264 stream from the server using RTSP protocol
! The client sends the stream to Radeon GPU which performs H264 hardware decoding to the frame buffer
! Sound stream is decoded using soQware audio codec and sent to audio hardware
FRAME BUFFER
GAME CLIENT
DECODE AUDIO
Audio device
UI
Input devices
![Page 15: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/15.jpg)
15 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
RAPIDFIRE TECHNOLOGY CLIENT SIDE DATA FLOW
Client
Radeon GPU NETWORK
DECODE
! Game client receives H264 stream from the server using RTSP protocol
! The client sends the stream to Radeon GPU which performs H264 hardware decoding to the frame buffer
! Sound stream is decoded using soQware audio codec and sent to audio hardware
! UI events are collected by the client and sent over the network to the server
FRAME BUFFER
GAME CLIENT
DECODE AUDIO
Audio device
UI
Input devices
![Page 16: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/16.jpg)
16 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
THE RAPIDFIRE API
" Server component The server component provides funcJons for the: ‒ Encoding of video and audio data ‒ Color space conversion ‒ Capturing of the desktop ‒ Handling of mulJple render targets ‒ Interoperability with OpenGL, D3D9 and D3D11
" Client component The client component provides funcJons for: ‒ Decoding of video and audio streams ‒ Color space conversion ‒ Interoperability with OpenGL, D3D9 and D3D11
COMPONENTS
" Network component The Network component is a sample implementaJon of video and audio streaming based on the LIVE555 Media Server.
" User Interface The UI component provide funcJons to: ‒ Capture user events on the client ‒ Send the events to the server for processing
![Page 17: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/17.jpg)
17 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
THE RAPIDFIRE API THE SERVER COMPONENT
Create RenderTarget
rfCreateEncodeSession
rfCreateEncoder
rfRegisterRenderTarget
IniJalizaJon Render Loop
rfGetEncodedFrame
rfEncodeFrame
Draw to free RenderTarget
rfRtspServerSendFrame
Done?
![Page 18: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/18.jpg)
18 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
THE RAPIDFIRE API
" rfCreateEncodeSession ‒ Creates an encoding session on the server. The sessions encapsulates the following components: ‒ Rendering context/device ‒ Compute context that is used for the color space conversion ‒ Render targets ‒ Desktop ‒ The encoder: SW, VCE or IDENTITY
‒ The following session types are supported ‒ OpenGL ‒ DX9 / DX9Ex ‒ DX11 ‒ Desktop capturing
THE SERVER COMPONENT
Create RenderTarget
rfCreateEncodeSession
rfCreateEncoder
rfRegisterRenderTarget
IniJalizaJon
![Page 19: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/19.jpg)
19 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
THE RAPIDFIRE API THE SERVER COMPONENT
Create RenderTarget
rfCreateEncodeSession
rfCreateEncoder
rfRegisterRenderTarget
IniJalizaJon " rfCreateEncodeSession ‒ CreaJng an OpenGL session that uses the VCE HW encoding
" rfRegisterRenderTargt
‒ Input: Name of the OpenGL Texture, dimension of the texture ‒ Output: The index used by RF to idenJfy this render target
RFProperties props[] = { RF_GL_GRAPHICS_CTX, (RFProperties)hGLRC, RF_GL_DEVICE_CTX, (RFProperties)hDC, RF_ENCODER, (RFProperties)RF_VCE, 0 }; // Create RapidFire encoding session RFEncodeSession session = rfCreateEncodeSession(props);
for (unsigned int i = 0; i < NUM_RENDER_TARGETS; i++) { rfRegisterRenderTarget(session, RF_RT_GL_TEXTURE, uiTexName[i], uiWidth, uiHeight, &renderTargetIdx[i])); }
![Page 20: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/20.jpg)
20 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
THE RAPIDFIRE API THE SERVER COMPONENT
Create RenderTarget
rfCreateEncodeSession
rfCreateEncoder
rfRegisterRenderTarget
IniJalizaJon " rfCreateEncoder ‒ CreaJng an Encoder using a preset configuraJon
‒ The following presets are supported: ‒ RF_ENCODE_FAST ‒ RF_ENCODE_BALANCED ‒ RF_ENCODE_QUALITY
" rfCreateEncoder2 ‒ CreaJng an encoder using properJes
rfCreateEncoder(session, uiWidth, uiHeight, RF_ENCODE_FAST)
// Create encoder using properties RFProperties props[] = { RF_ENCODER_PROFILE, (RFProperties)RF_MAIN, RF_ENCODER_LEVEL, (RFProperties)41, RF_ENCODER_BITRATE, (RFProperties)6000000, RF_ENCODER_FPS, (RFProperties)30, 0 }; rfCreateEncoder2(session, uiWidth, uiHeight, props));
![Page 21: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/21.jpg)
21 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
THE RAPIDFIRE API THE SERVER COMPONENT
Render Loop " rfEncodeFrame
‒ Non-‐blocking call to submit a frame for encoding
" rfGetEncodedFrame
‒ Check first if a frame is ready ‒ If a frame is available get the data to system memory ‒ Send the frame over the network to the client
rfGetEncodedFrame
rfEncodeFrame
Draw to free RenderTarget
rfRtspServerSendFrame
Done?
rfEncodeFrame(session, renderTargetIdx[uiCurrentRT]));
// Check if encoded frame is ready if (rfIsEncodedFrameReady(session)) { if (rfGetEncodedFrame(session, &uiBitStreamSize, (void**)&pBitStreamdata) == RF_STATUS_OK) { if (uiBitStreamSize > 0) { // Send encoded frame to Network rtspStatus = rfRtspServerSendFrame(rtsp_sn, pBitStreamdata, uiBitStreamSize, tv.tv_sec, tv.tv_usec, 1); } } }
![Page 22: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/22.jpg)
22 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
THE RAPIDFIRE API THE CLIENT COMPONENT
Create Target Textures
rfCreateDecodeSession
rfCreateVideoDecoder
rfRegisterTargetTexture
IniJalizaJon Render Loop
Display frame using Target Texture
rfDecodeFrame
Wait to receive frame
Done?
![Page 23: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/23.jpg)
23 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
THE RAPIDFIRE API THE CLIENT COMPONENT
Create Target Textures
rfCreateDecodeSession
rfCreateVideoDecoder
rfRegisterTargetTexture
IniJalizaJon " rfCreateDecodeSession ‒ Creates a decoding session on the client. The sessions encapsulates the following components: ‒ Rendering context/device ‒ Compute context that is used for the color space conversion ‒ Target Textures ‒ Decoder: SW or UVD
‒ The following session are supported ‒ OpenGL ‒ DX9 / DX9Ex ‒ DX11
![Page 24: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/24.jpg)
24 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
THE RAPIDFIRE API THE CLIENT COMPONENT
Create Target Textures
rfCreateDecodeSession
rfCreateVideoDecoder
rfRegisterTargetTexture
IniJalizaJon " rfCreateDecodeSession
‒ Creates an OpenGL session that uses the UVD decoder
" rfRegisterTargetTexture
‒ Registers an OpenGL texture ‒ The texture will be used to store the decoded frame
RFProperties props[] = { RF_GL_GRAPHICS_CTX, (RFProperties)hGLRC, RF_GL_DEVICE_CTX, (RFProperties)hDC, RF_DECODER, (RFProperties)RF_UVD, 0 }; // Create RapidFire decoding session RFDecSession session = rfCreateDecodeSession(props);
rfRegisterTargetTexture(session, uiTextureName);
![Page 25: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/25.jpg)
25 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
THE RAPIDFIRE API THE CLIENT COMPONENT
Create Target Textures
rfCreateDecodeSession
rfCreateVideoDecoder
rfRegisterTargetTexture
IniJalizaJon " rfCreateVideoDecoder
‒ Create the actual decoder
rfCreateVideoDecoder(session, uiWidth, uiHeight);
![Page 26: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/26.jpg)
26 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
THE RAPIDFIRE API THE CLIENT COMPONENT
" rfDecodeFrame
‒ Decodes a frame ‒ The decoded frame is stored in the registered texture ‒ Now the applicaJon can use the texture to display the frame
rfDecodeFrame(session);
Render Loop
Display frame using Target Texture
rfDecodeFrame
Wait to receive frame
Done?
![Page 27: GS-4139, RapidFire for Cloud Gaming, by Dmitry Kozlov](https://reader036.fdocument.pub/reader036/viewer/2022062404/554e4894b4c905f4478b556c/html5/thumbnails/27.jpg)
27 | PRESENTATION TITLE | DECEMBER 4, 2013 | CONFIDENTIAL
DISCLAIMER & ATTRIBUTION
The informaJon presented in this document is for informaJonal purposes only and may contain technical inaccuracies, omissions and typographical errors.
The informaJon contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, soQware changes, BIOS flashes, firmware upgrades, or the like. AMD assumes no obligaJon to update or otherwise correct or revise this informaJon. However, AMD reserves the right to revise this informaJon and to make changes from Jme to Jme to the content hereof without obligaJon of AMD to noJfy any person of such revisions or changes.
AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION.
AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
ATTRIBUTION
© 2013 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo and combinaJons thereof are trademarks of Advanced Micro Devices, Inc. in the United States and/or other jurisdicJons. SPEC is a registered trademark of the Standard Performance EvaluaJon CorporaJon (SPEC). Other names are for informaJonal purposes only and may be trademarks of their respecJve owners.