DIGITAL VISUAL MEDIA

TECHNOLOGY

Budianto Tandianus

ABOUT MEName : - Budianto Tandianus

- 陈 松 生 - Eon Strife

Origin : IndonesiaProgram : M.Sc.(Digital Media Technology) (Full Time)

PhD (Part Time)Interests : Gaming, Computer Graphics, MusicE-mail : BTANDIANUS@NTU.EDU.SG

eon_strife@yahoo.comSite : http://www3.ntu.edu.sg/home2008/budi0010

INTRODUCTION TO DIGITAL WORLD The ‘atom’ of computing : 0 and 1 Computer are programmed with 0 and 1

bitsFrom the earliest day :

To today’s programming : printf(“hello world\n”) ->

01110101011010010101

Source : http://en.wikipedia.org

INTRODUCTION TO DIGITAL WORLD Likewise, the processed data are also in

digital

10101101 01011011 11001101 1001110100110110 11011101 00110110 0101101100011011 11011011 11110110 10000101 …

INTRODUCTION TO DIGITAL WORLD Digital and Analog

MEDIA TECHNOLOGY

Source : http://www.travelchinaguide.com

BEFORE AFTER

Source : http://photos.pcpro.co.uk

COMMERCIALIZATION Most obvious examples : Game and

Movies

COMMERCIALIZATION Online games

Source : http://www.neoseeker.com

COMMERCIALIZATION Social Media :

Instagram I Love Coffee

Source : http://www.femalemag.com.my/

COMMERCIALIZATION Authoring tools

Autodesk 3DS MaxAdobe PhotoshopUnreal EngineCry EngineEtc.

Source : http://gamedesignschools411.com

COMMERCIALIZATION Online video streaming

YoutubeNetFlix

Online photo sharingFlickrPicasa

WHAT OUR EYES ARE SEEING Visual stimuli from the display

Source : http://irieman.com/blogSource : http://www.mobile2u.com.pk/

Source : http://3dvision-blog.com/ Source : http://www.popularmechanics.com/

VISUAL OUTPUT

10101101 01011011 11001101 1001110100110110 11011101 00110110 0101101100011011 11011011 11110110 10000101 …

DIGITAL IMAGE PROCESSING Examples of digital image processing :

Original Blurred

Gray scale Edge Detection

NoisyImage

DenoisedImage

DIGITAL IMAGE PROCESSING How to do it ?

Colour -> Grayscale Y’ = 0.212 * R + 0.7125 * G + 0.0722 * B

Image Denoising Value of the pixel is the average value of its

value and the values of its surrounding pixels

IMAGE PROCESSING IN THE INTERNET

IMAGE PROCESSING IN THE INTERNET

Source : http://pixlr.com/

VIDEO PROCESSING Video is a sequence of images in

temporal domain

Time

VIDEO PROCESSING Biggest issue in video processing is the

file size. One image size :

1920 * 1080 * 3 * 8 bits = 5.93 MB The movie is 24 Frames per second

5.93 * 24 = 142.32 MB And it’s a two-hour movie :

142.32 MB * 60 seconds * 60 minutes * 2 hours = 1 TB

And if it’s a 3D movie, then :1 TB * 2 eyes = 2 TB

VIDEO COMPRESSION Need to compress ! Common way : Discrete Cosine

Transform or Discrete Wavelength Transform

Lossy Compression Basic philosophy :

One image has a lot of information (values) Just keep the most prominent ones

VIDEO COMPRESSION

VIDEO MEDIA IN THE INTERNET

VIDEO MEDIA IN THE INTERNET

VIDEO MEDIA IN THE INTERNET Not a good solution if download the

whole video first before the user can watch

Solution : streaming based Bytes of data are transferred

one by one Advantage :

User can watch as the videois downloaded

LET’S JUMP TO 3D !

INTRODUCTION TO 3D Before the data are ‘processed’ and

shown as pixels in screen, they are still in raw format :3D Models : polygons, verticesLight sources : point light, directional light,

area light, spotlight, etc.Camera : to enable us to see scene

3D SCENE Common representation of 3D model :

3D mesh

Implicit surface, e.g. : x^2 + y^2 + z^2 = r^2

vertex

face

edge

3D SCENE Now we have the 3D mesh, now how we

can see it in display ?Answer : create a camera

Camera

ProjectionPlane

3D Model

3D SCENE But what we see is only a wireframe or

only flat color, not realistic ! Next step : add light light source(s) and

perform shading

3D SCENE Shading : calculate the amount of

illumination seen for each visible point in the scene

The most basic : Lambertian, assume the light is reflected

equally to all directions, and it’s proportional to the angle between the surface normal and direction to the light source: L . N

3D SCENE

3D SCENE We can improve more, by considering the

illumination from all directions : Global Illumination

A lot of algorithms : Distributed Ray Tracing Path Tracing Bidirectional Path Tracing Metropolis Light Transport Photon mapping Etc.

3D SCENE So far, the process from 3D scene ->

final image is called rendering But it’s boring ! Everything is not

moving ! Intermediate step : Animation

3D ANIMATION Done by performing basic

transformation :E.g. rotation

Or animating rigged character

Source : http://coreenginedev.blogspot.com

2D AND 3D Post Processing : After rendering the 3D

scene, the developer might do additional image processing on the results.

E.g. :

Source : http://udn.epicgames.com

3D GRAPHICS IN THE INTERNET

Source : http://www.awwwards.com/22-experimental-webgl-demo-examples.html

3D GRAPHICS IN THE INTERNET

Source : http://www.unrealengine.com/html5/

WEBGL All these demos are developed using

WebGL WebGL is an open standard for 3D

rendering API for web browser It’s based on OpenGL, which is much more

powerfulMany of PC games you’ve played might be

developed using OpenGL ! WebGL extends javascript Can run in a lot of web browser

Including web browser of tablets, but they are still struggling

WEBGL WebGL compatibility list :

Source : http://caniuse.com/webgl

2D AND 3D

3D Scene Data

2D Image

RenderingComputer Vision

Digital Image Processing

Geometry Processingor animation

COMPUTER VISION Given a 2D image, let the computer

interpret the 3D scene

Source : http://www.digitaltrends.com

INNOVATION IS NEEDED! I know that most of you guys here are

more interested in making money

The question here is, how to innovatively make use of these technologies, how to commercialize the project

DIGITAL MEDIA IN INDONESIA Is the digital media business very

gloomy in Indonesia’s market ?Yeah, more or lessBut there is already some interesting

development !

INFINITE SKY Downloaded by more than 500k iPhone

users Top 10 in US and UK

Source : http://www.indoboom.com

DREADOUT Horror game developed by a team in

Bandung Funded with IndieGoGo, a crowdfunding

service Going to be released soon

Source : http://www.indiegogo.com

IF YOU’RE INTERESTED FURTHER IN TODAY’S PRESENTATIONS Image Processing :

Rafael C. Gonzalez, Richard E. Woods. Digital Image Processing (3rd Edition), Prentice Hall 2007

Rendering : Donald D. Hearn, M. Pauline Baker, Warren Carithers. Computer Graphics with Open

GL (4th Edition), Prentice Hall 2010 Matt Pharr, Greg Humphreys. Physically Based Rendering (2nd Edition) : From Theory

to Implementation, Morgan Kaufmann 2010.

Animation : Rick Parent. Computer Animation, Third Edition: Algorithm and Techniques. Morgan

Kaufmann 2012.

Computer Vision : Richard O. Duda, Pattern Classification (2nd Edition), Wiley-Interscience 2004. David A. Forsyth and Jean Ponce. Computer Vision: A Modern Approach (2nd Edition),

Prentice Hall 2011.

OpenGL : Graham Sellers, Richard S. Wright, Nicholas Haemel. OpenGL SuperBible:

Comprehensive Tutorial and Reference (6th Edition), Addison-Wesley 2013. Dave Shreiner, Graham Sellers, John M. Kessenich, Bill M. Licea-Kane. OpenGL

Programming Guide: The Official Guide to Learning OpenGL Version 4.3 (8th edition), Addison-Wesley 2013.

WebGL : Kouichi Matsuda, Rodge Lea. WebGL Programming Guide: Interactive 3D Graphics

Programming with WebGL, Addison-Wesley 2013.

ONLINE RESOURCE Today’s Presentation can be found at :

http://www.slideshare.net/EonStrife OpenGL and WebGL specifications :

http://www.khronos.org/

THE END