Kim, Sung-HoSchool of Computer, Information and Communication Engineering

Sangji Univ.

증강현실의 개요ARToolKit 의 특징What does ARToolKit do?

How does ARToolKit work?

How to install & setting up it?

Setting up ARToolKit

Camera Calibration

Coordinate System

ARToolKit Tutorials

References

차 례차 례차 례차 례

가상 현실 (Virtual Reality) 의 한 분야로 실제 환경에 가상 사물이나 정보를 합성하여 원래의 환경에 존재하는 사물처럼 보이도록 하는 컴퓨터 그래픽스 기법으로 ,“ 증대 , 확대”라는 의미를 담고 있는 증강 (Augmentation) 이라는 단어와 눈앞에 펼쳐진 실제 현실 (Reality) 이라는 단어가 합쳐진 합성어 어떤 특정한 환경이나 상황을 컴퓨터로 만들어서 그것을 사용하는 사람이 마치 실제 주변 상황이나 환경과 상호작용을 하고 있는 것처럼 만들어 주는 인간과 컴퓨터 사이의 인터페이스 (Interface)

– 인공 현실 (Artificial Reality), 사이버 공간 (Cyber Space)

– 가상 세계 (Virtual Worlds), 가상 환경 (Virtual Environment)

– 합성 환경 (Synthetic Environment), 인공 환경 (Artificial Environment)

– 시뮬레이션 기술 (Simulation Technology)

가상현실 : 현실 세계를 대체하여 창조된 새로운 환경증강현실 : 현실 세계에 가상의 정보 ( 객체 ) 를 중첩함 (Mixing) 으로써 현실 세계를 보충하여 사용자에게 보여 주는 환경– 현실의 이미지와 가상의 이미지를 결합한 것– 실시간으로 Interaction 이 가능한 것– 3D 공간 안에 놓인 것

증강현실의 개요증강현실의 개요증강현실의 개요증강현실의 개요

증강현실의 요소 기술

– Marker Detection Technology

• 기준 좌표 역할을 하는 사각형의 마커를 인식하고 그 마커를

기준점으로 하는 기술

– Markerless Tracking Technology

• Feature based Tracking

• 영상 내에서 특징 점들을 추출하고 이 특징 점들을 기반으로 좌표

계를 추출

• 특징 점을 이용하기 때문에 사용자가 정의한 특정 패턴을 마커로

이용할 수도 있다 .

• 증강현실 기술에 있어서 가장 난이도 있고 점차 더 중요한 기술이 될

것으로 평가

증강현실의 개요 증강현실의 개요 (( 계속계속 ))증강현실의 개요 증강현실의 개요 (( 계속계속 ))

단일 카메라의 위치 / 방향을 추적한다

검은 사각형을 사용하는 추적코드는 간단하다

사각형의 Marker Pattern 을 사용한다

간편하게 카메라 코드를 보정 할 수 있다

빠른 실시간 어플리케이션이다

무료로 배포되는 오픈 소스이다

PC CAM( 화상카메라 ) , Visual studio 6.0 이상 , DirectX 9.0b 이상

ARToolKitARToolKit 의 특징의 특징ARToolKitARToolKit 의 특징의 특징

ARToolKitARToolKit 의 의 ArchitectureArchitectureARToolKitARToolKit 의 의 ArchitectureArchitecture

ARToolKitARToolKit 의 의 StructureStructureARToolKitARToolKit 의 의 StructureStructure

ARToolKit - Free tracking library

– Software library for building AR applications based on OpenGL

– Calculate the real camera position and orientation relative to physical markers in real time

– 3D VRML support

– Software Library for vision-based AR applications

• Open Source(C language), multi-platform(SGI IRIX, PC Linux, PC Windows)

– Overlays 3D virtual objects on real markers

• Uses single tracking marker

• Determines camera pose information (6 DOF)

– Includes utilities for marker-based interaction

– ARToolKit Website

• http://www.hitl.washington.edu/artoolkit/

• Limitations

– Monocular camera setup 을 사용하므로 3D 측정이 불가능함– Lighting condition 과 Marker 의 재질에 민감하게 반응할 수 있음

What does ARToolKitWhat does ARToolKit do ?do ?What does ARToolKitWhat does ARToolKit do ?do ?

Principle of ARToolKit

– http://www.hitl.washington.edu/artoolkit/documentation/userarwork.htm

How does ARToolKitHow does ARToolKit work?work?How does ARToolKitHow does ARToolKit work?work?

static void mainLoop(void){ static int contF = 0; ARUint8 *dataPtr; ARMarkerInfo *marker_info; int marker_num; int j, k; /* grab a vide frame */ if( (dataPtr = (ARUint8 *)arVideoGetImage()) == NULL )

{ arUtilSleep(2); return; } if( count == 0 ) arUtilTimerReset(); count++; argDrawMode2D(); ...

How does ARToolKitHow does ARToolKit work? (con’t)work? (con’t)How does ARToolKitHow does ARToolKit work? (con’t)work? (con’t)

.....

/* grab a vide frame */ if( (dataPtr = (ARUint8 *)arVideoGetImage()) == NULL )

{ arUtilSleep(2); return; } if( count == 0 ) arUtilTimerReset(); count++;

argDrawMode2D(); argDispImage( dataPtr, 0,0 );

/* detect the markers in the video frame */ if( arDetectMarker(dataPtr, thresh, &marker_info,

&marker_num) < 0 ) { cleanup(); exit(0); }

arVideoCapNext();

.....

How does ARToolKitHow does ARToolKit work? (con’t)work? (con’t)How does ARToolKitHow does ARToolKit work? (con’t)work? (con’t)

....

if( arDetectMarker(dataPtr, thresh, &marker_info, &marker_num) < 0 ) {

cleanup(); exit(0); }

arVideoCapNext();

/* check for object visibility */ k = -1; for( j = 0; j < marker_num; j++ ) { if( patt_id == marker_info[j].id ) { if( k == -1 ) k = j; else if( marker_info[k].cf < marker_info[j].cf

) k = j; } } if( k == -1 ) {

....

How does ARToolKitHow does ARToolKit work? (con’t)work? (con’t)How does ARToolKitHow does ARToolKit work? (con’t)work? (con’t)

....

/* check for object visibility */

....

/* get the transformation between the marker and the real camera */

arGetTransMat(&marker_info[k], patt_center, patt_width, patt_trans);

draw();

argSwapBuffers();}

How does ARToolKitHow does ARToolKit work? (con’t)work? (con’t)How does ARToolKitHow does ARToolKit work? (con’t)work? (con’t)

static void draw( void ){ .... /* load the camera transformation matrix */ argConvGlpara(patt_trans, gl_para); glMatrixMode(GL_MODELVIEW); glLoadMatrixd( gl_para );

glEnable(GL_LIGHTING); glEnable(GL_LIGHT0); glLightfv(GL_LIGHT0, GL_POSITION, light_position); glLightfv(GL_LIGHT0, GL_AMBIENT, ambi); glLightfv(GL_LIGHT0, GL_DIFFUSE, lightZeroColor); glMaterialfv(GL_FRONT, GL_SPECULAR, mat_flash); glMaterialfv(GL_FRONT, GL_SHININESS, mat_flash_shiny); glMaterialfv(GL_FRONT, GL_AMBIENT, mat_ambient); glMatrixMode(GL_MODELVIEW); //glTranslatef( 0.0, 0.0, 25.0 );

glTranslatef( 0, 0, 25.0 ); glutSolidCube(50.0); glDisable( GL_LIGHTING );

glDisable( GL_DEPTH_TEST );

How does ARToolKitHow does ARToolKit work? (con’t)work? (con’t)How does ARToolKitHow does ARToolKit work? (con’t)work? (con’t)

http://www.hitl.washington.edu/artoolkit/documentation/userarwork.htm

– Live video image 를 binary image 로 변환 (lighting threshold value 사용 )

– binary image 에서 모든 square region 탐색

– 각 square region 내의 pattern 을 capture 하여 pre-trained pattern

– template 들과 match 를 시도하여 실제 marker 인지를 결정

– 일단 한 marker 를 찾으면 , 알려진 square size 와 pattern orientation 을

– 사용하여 (marker 에 상대적인 ) real video camera 의 position 을 계산함

– 실제 camera 를 계산하면 , virtual object 들을 image 에 draw 할 수 있음

How does ARToolKitHow does ARToolKit work? work? (continued..)(continued..)How does ARToolKitHow does ARToolKit work? work? (continued..)(continued..)

Multiplateform library (Windows, Mac OS, Linux)

For Windows

– Download

• OpenGL

– GLUT

• DSVL(DirectShow Video Processing Library)

– Communication with the camera driver

• ARToolkit library

• OpenVRML

• Libpng / jpeg

• DXSDK

• msvcr71d.dll, msvcp71d.dll

– Building library

– Visual Studio 6 & Visual Studio .NET 2003

How to install & setting up it ?How to install & setting up it ?How to install & setting up it ?How to install & setting up it ?

http://www.hitl.washington.edu/artoolkit/documentation/usersetup.htm

http://abekdh.tistory.com/9

http://www.mironae.com/1008

http://niceyth.springnote.com/pages/409596

Compile SimpleVRML with OpenVRML

Setting up ARToolKitSetting up ARToolKitSetting up ARToolKitSetting up ARToolKit

Calibration

– 특정조건에서 측정기기 , 표준물질 , 척도 또는 측정체계 등에 의하여

결정된 값을 표준에 의하여 결정된 값 사이의 관계로 확정하는 일련의

작업을 말한다

Camera Calibration

– 카메라의 정보 ( 내부변수 [intrinsic parameter] 와 외부변수 [extrinsic

parameter]) 를 알아내는 작업으로서 , 실 세계의 장면이 카메라를 통해

픽셀로 변환되면 왜곡이 발생하는데 이러한 것들을 보정하는 작업을

말한다 .

– 카메라의 정보 : ARToolKit/bin/Data/WDM_camera_flipV.xml

http://www.hitl.washington.edu/artoolkit/documentation/usercalibration.htm

– Two Step Calibration

– One Step Calibration

Camera CalibrationCamera CalibrationCamera CalibrationCamera Calibration

http://www.hitl.washington.edu/artoolkit/documentation/cs.htm

– Computer Vision Coordinate System

– Rendering Coordinate System

Coordinate SystemsCoordinate SystemsCoordinate SystemsCoordinate Systems

Tutorial 1 : Tracking Stability

– http://www.hitl.washington.edu/artoolkit/documentation/tutorialhistory.htm

Tutorial 2 : Camera and Marker Relationships

– http://www.hitl.washington.edu/artoolkit/documentation/tutorialcamera.htm

Tutorial 3 : Multi-Marker Tracking

– http://www.hitl.washington.edu/artoolkit/documentation/tutorialmulti.htm

Camera Calibration & Marker Generation

– http://gekko.tistory.com/category/prog/AR

ARToolkit for Dummies

– http://isa.umh.es/vr2/euron06/doc/labs/ARToolkit_for_Dummies.pdf

• How to import a new VRML model

• How to make your own marker

• How to make your own AR applications

ARToolKit TutorialsARToolKit TutorialsARToolKit TutorialsARToolKit Tutorials

\ARToolkit\bin\Data\object_data_vrml for simpleVRML.exeConceptual overview of the coherence between pattern, data and image files

Expanding the S/WExpanding the S/WExpanding the S/WExpanding the S/W

The patterns to be recognized and their associated models are specified in the Data/object_data_vrml fileThis means that the VRML content associated with the pattern "patt.hiro" is specified in the file "Wrl/bud_B.dat" – #pattern 1 – VRML Wrl/bud_B.dat – Data/patt.hiro – 80.0 – 0.0 0.0

The file line specifies the actual VRML file to be loaded, while the next three lines set the translation, rotation and scale of the VRML file from the pattern original – Wrl/bud_B.wrl – 0.0 0.0 0.0 # Translation - x,y,z from center of tracking

pattern – 0.0 0.0 0.0 0.0 # Rotation angle + axis, eg 90.0 1.0 0.0 0.0 – 2.5 2.5 2.5 # Scale x,y,z

Expanding the S/W Expanding the S/W (Continued…)(Continued…)Expanding the S/W Expanding the S/W (Continued…)(Continued…)

1. Copy your_file.wrl into the Wrl directory

2. Make a your_file.dat file and associate it with your_file.wrl by

typing Wrl/your_file.wrl on the first line.

– You may take a copy of one the existing .dat files and edit it to

fit your wrl file

– Place it in the Wrl directory

3. Make a new marker or use one that is not yet in use.

4. Edit the Data/object_data_vrml file by adding a new paragraph:

– #pattern 3

– VRML Wrl/your_file.dat

– Data/patt.sample1 -- Alternatively your new marker

– 80.0

– 0.0 0.0

How to import a new VRML modelHow to import a new VRML modelHow to import a new VRML modelHow to import a new VRML model

To make it simple you can use the

blankPatt.gif located in the patterns folder of

the ARToolKit installation

Draw your new pattern inside the black square

The next thing you have to do is start the

mk_patt.exe file located in the bin folder. The

file requires an input for the camera. You can

use "data/camera_para.dat". This is the

default setting for the camera.

Position the camera right over the marker. You

should be able to see a red and green square

around the pattern. This means that ARToolKit

has found your marker. Rotate the camera until

the red corner is in the upper left

How to make your own markerHow to make your own markerHow to make your own markerHow to make your own marker

Left click your mouse inside the viewing frame. Switch to the

command window and type the name of your pattern same as

“data/patt.mysamplepattern”

To use your new pattern you have to edit object_data_vrml

It should look something like this:

– #the number of patterns to be recognized

– 1

– #pattern 1

– VRML Wrl/bud_B.dat

– Data/patt.hiro

– 80.0

– 0.0 0.0

How to make your own marker How to make your own marker (Continued…)(Continued…)How to make your own marker How to make your own marker (Continued…)(Continued…)

Explanation: 1. When you add a new set the new number of patters to be recognized: – #The number of patterns to be recognized – 2

2. Add a new pattern number: – #pattern 2

3. Insert reference to the DAT file of the VRML model to overlay the pattern: – VRML Wrl/maggie.dat

4. Insert reference to your new pattern. You can use any type of filename – Myfolder/mysamplepattern

5. Adjust the height of the overlayed object

How to make your own marker How to make your own marker (Continued…)(Continued…)How to make your own marker How to make your own marker (Continued…)(Continued…)

Explanation (Continued…): 6. X and Y coordinates relative to the center of the pattern: – 10.0 5.0

Then you should end up with something like this: – #The number of patterns to be recognized – 2

– #pattern 1 – VRML Wrl/bud_B.dat – Data/patt.hiro – 80.0 – 0.0 0.0

– #pattern 2 – VRML Wrl/maggie.dat – Myfolder/mysamplepattern – 105.0 – 10.0 5.0

How to make your own marker How to make your own marker (Continued…)(Continued…)How to make your own marker How to make your own marker (Continued…)(Continued…)

www.se.rit.edu/~jrv/research/ar/introduction.html

www.cs.unc.edu/Research/ProjectIndex/GraphicsImage.html

www.itk.ntnu.no/KIKS/demo/

www.csl.sony.co.jp/project/ar/ref.html

www.howstuffworks.com/augmented-reality.htm

Interesting LinksInteresting LinksInteresting LinksInteresting Links

http://www.hitl.washington.edu/artoolkit/documentation/

http://jhseo.tistory.com/entry/ARToolKit-VRML-Programming

http://isa.umh.es/vr2/euron06/doc/labs/ARToolkit_for_Dummies.pdf

http://jhseo.tistory.com/135

http://makerfaireri.crowdvine.com/attachments/0000/4771/ARToolkit_Blender.pdf

http://old.uvr.gist.ac.kr/wlee/web/papers/2004/KHCI04_kkim_wlee.pdf

http://www.medien.ifi.lmu.de/lehre/ss09/ar/Installation_vom_ARToolKit.pdf

http://www.hitl.washington.edu/artoolkit/Papers

http://gekko.tistory.com/45

http://www.artoolworks.com/ARToolKit_Professional.html

http://www.artoolworks.com/support/library/ARToolKit_Professional

http://www.pragprog.com/titles/cfar/augmented-reality

http://jpyun56.wordpress.com/2010/01/18/augmented-reality-application%EB%93%A4/

http://www.se.rit.edu/~jrv/research/ar/

http://makerfaireri.crowdvine.com/attachments/0000/4771/ARToolkit_Blender.pdf

http://cafe.naver.com/opencv

ReferencesReferencesReferencesReferences