Building Mobile AR Applications Using the Outdoor AR Library (Part 1)

Tutorial:Building Mobile AR Applications using

the Outdoor AR Library

Gun A. Lee Mark Billinghurst

The Human Interface Technology Laboratory New ZealandUniversity of Canterbury, New Zealand

21 Nov 2013 09:00-10:15

SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS

Welcome Mark Billinghurst

Director, HIT Lab NZ PhD, Univ. Washington AR, Interaction Design

Gun Lee Post Doc, HIT Lab NZ PhD, POSTECH AR, Android, Google Glass

Schedule

Introduction (Mark) Outdoor AR History (Mark) Design Guidelines (Mark) Example Applications (Mark) Building a Mobile Outdoor AR Application (Gun) Research Directions (Mark) Conclusions (Gun + Mark)

Introduction

Mark Billinghurst

What is Augmented Reality?

Definition [Azuma 97] Combines real and virtual images Is interactive in real-time Content registered in 3D

Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.

Outdoor AR Using mobile/wearable

systems to overlay AR content outdoors

Technical Requirements Tracking (GPS, compass,

etc) Display (handheld,

headmounted) Input devices Processing, networking

Applications

Tourism, Gaming, Architecture, Engineering, Etc

Outdoor AR History

Mark Billinghurst

Evolution of Mobile AR

Wearable AR

Handheld AR Displays

Camera phone

1995 1997 2001 2003 2004

Camera phone- Self contained AR

WearableComputers

PDAs-Thin client AR

PDAs-Self contained AR

Camera phone- Thin client AR

MIT Wearable Computing (1996)

Mobile AR: Touring Machine (1997)

University of Columbia Feiner, MacIntyre, Höllerer, Webster

Combines See through head mounted display GPS tracking Orientation sensor Backpack PC (custom) Tablet input

Feiner, S., MacIntyre, B., Höllerer, T., & Webster, A. (1997). A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment. Personal Technologies, 1(4), 208-217.

MARS View

Virtual tags overlaid on the real world “Information in place”

Backpack/Wearable AR

1997 Backpack AR Feiner’s Touring Machine AR Quake (Thomas) Tinmith (Piekarski) MCAR (Reitmayr) Bulky, HMD based

PCI 3D Graphics Board

Hard Drive

Serial

PC104 Sound Card

PC104 PCMCIA

GPSAntenna

RTK correction Antenna

HMDController

TrackerController

DC to DCConverter

Battery

WearableComputer

GPS RTK correction

Example self-built workingsolution with PCI-based 3D graphics

Columbia Touring Machine

Mobile AR - Hardware

HIT Lab NZ Wearable AR (2004) Highly accurate outdoor

AR tracking system GPS, Inertial, RTK system HMD

First prototype Laptop based Video see-through HMD 2-3 cm tracking accuracy

Image Registration

AR Stakeout Application

Wearable AR Video

Location Aware Phones (2008)

Nokia NavigatorMotorola Droid

Outdoor Information Overlay (2009)

Mobile phone based Tag real world locations

GPS + Compass input Overlay graphics data on live video

Applications Travel guide, Advertising, etc

Wikitude, Layar, Junaio, etc.. Android based, Public API released

Google Glass (2013)

View Through Google Glass

Always available peripheral information displayCombining computing, communications and content capture

Hardware CPU TI OMAP 4430 – 1.2 Ghz 16 GB SanDisk Flash,1 GB Ram 570mAh Battery

Input 5 mp camera, 720p recording,

microphone GPS, InvenSense MPU-9150 inertial

sensor Output

Bone conducting speaker 640x360 micro-projector display

Competitors Vuzix M100

$999, profession Recon Jet

$600, more sensors, sports Opinvent

500 Euro, multi-view mode Motorola Golden-i

Rugged, remote assistance

Design Guidelines

Mark Billinghurst

AR UI Design Consider your user Follow good HCI principles Adapt HCI guidelines for AR Design to device constraints Design for perception/info presentation Design for evaluation

Consider Your User Consider context of user

Physical, social, emotional, cognitive, etc Outdoor AR User

Probably Mobile One hand interaction Short application use Need to be able to multitask Use in outdoor environment Enhance interaction with real world

Adapting Existing Guidelines Mobile Phone AR

Phone HCI Guidelines Mobile HCI Guidelines

HMD Based AR 3D User Interface Guidelines VR Interface Guidelines

Clean Large Video View Large Icons Text Overlay Feedback

Applying Principles to Mobile AR

AR vs. Non AR Design

Design Guidelines Design for 3D graphics + Interaction Consider elements of physical world Support implicit interaction

Characteristics Non-AR Interfaces AR Interfaces

Object Graphics Mainly 2D Mainly 3D

Object Types Mainly virtual ob-jects

Both virtual and physical ob-jects

Object behaviors Mainly passive ob-jects

Both passive and active ob-jects

Communication Mainly simple Mainly complex

HCI methods Mainly explicit Both explicit and implicit

Maps vs. Junaio

Google Maps 2D, mouse driven, text/image heavy, exocentric

Junaio 3D, location driven, simple graphics, egocentric

Design to Device Constraints Understand the platforms and design for limitations

Hardware, software platforms Eg Outdoor AR game with handheld

Use large screen icons Consider screen reflectivity Support one-hand interaction Consider the viewing angle Do not tire users out physically Don’t require accurate tracking

AR as Perception Problem Goal of AR to fool human senses –

create illusion that real and virtual are merged

Depth Size Occlusion Shadows Relative motion Etc..

Which Object is Closest?

Depth CuesPictorial: visual cues

• Occlusion, texture, relative brightnessKinetic: motion cues

• Relative motion parallax, motion perspectivePhysiological: motion cues

• Convergence, accommodationBinocular disparity

• Two different eye images

Use the Following Depth Cues

Movement parallax Icon/Object size (for close

objects) Linear perspective

To add side perspective bar. Overlapping

Works if the objects are big enough

Shades and shadows Depends on the available

computation

Provide Perspective Cue

Eg ground plane grid

Depth Perception

Information Presentation

• Amount of information• Clutter, complexity

• Representation of information• Navigation cues, POI representation

• Placement of information• Head, body, world stabilized

• View combination• Multiple views

Twitter 360

www.twitter-360.com iPhone application See geo-located tweets in real world Twitter.com supports geo tagging

Wikitude – www.mobilizy.com

BlahBlah

Information Filtering

Outdoor AR: Limited FOV

Possible solutions Overview + Detail

spatial separation; two views

Focus + Context merges both views into one view

Zooming temporal separation

Zooming panorama, Zooming Map

Zooming Views

Mulloni, A., Dünser, A., & Schmalstieg, D. (2010, September). Zooming interfaces for augmented reality browsers. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services (pp. 161-170). ACM.

Example Applications

Mark Billinghurst

HIT Lab NZ Building Viewer Architectural Application Loads 3D models

a OBJ/MTL format Positions content in space

GPS, compass Intuitive user interface

toolkit to modify the model Targeting museum guide/outdoor site

applications

2011 Christchurch Earthquake

Christchurch Today

CityViewAR

Using AR to visualize Christchurch buildings 3D buildings, 2D images, text, panoramas AR View, Map view, List view Available on Android market, iOS App Store

User Experience

While walking in the real world people can see text, 2D images and 3D content

List View

List of all assets

Map View

Icons for buildings, viewpoints, panoramas

Building History Data

Photographic Images

Panorama Images

360 degree photo bubbles

Augmented Reality View

Building a Mobile Outdoor AR Application

Gun Lee

Research Directions

Mark Billinghurst

Looking to the Future

What’s Next?

What’s Next Key Research Problems

Wide area tracking Input Methods/Displays Collaboration

Emerging Trends AR + Social Networking AR Standards/Platforms AR + Human Computing Scaling up

Robust Outdoor Tracking

Hybrid Tracking Computer Vision, GPS, inertial

Going Out Reitmayer & Drummond (Univ.

Cambridge)

Handheld Display

Reitmayr, G., & Drummond, T. W. (2006, October). Going out: robust model-based tracking for outdoor augmented reality. In Mixed and Augmented Reality, 2006. ISMAR 2006. IEEE/ACM International Symposium on (pp. 109-118). IEEE.

Meta Gesture Interaction

Depth sensor + Stereo see-through

Meta Video

Contact Lens Display Babak Parviz

University Washington MEMS components

Transparent elements Micro-sensors

Challenges Miniaturization Assembly Eye-safe

Contact Lens Prototype

Ego-Vision Collaboration

Google Glass camera + processing + display +

connectivity

Ego-Vision Research System

How do you capture the user's environment?

How do you provide good quality of service?

Interface What visual and audio cues provide best

experience? How do you interact with the remote user?

Evaluation How do you measure the quality of

collaboration?

Massive Multiuser Handheld/Outdoor AR for the first time

allows extremely high numbers of AR users

Requires New types of applications/games New infrastructure (server/client/peer-to-peer) Content distribution…

PERSONAL VIEW

Augmented Reality 2.0 Infrastructure

Leveraging Web 2.0 Content retrieval using HTTP XML encoded meta information

KML placemarks + extensions Queries

Based on location (from GPS, image recognition) Based on situation (barcode markers)

Syndication Community servers for end-user content Tagging

AR client subscribes to data feeds

AR Standards + Markup Languages

KHARMA and Argon ARML AREL Patterns if Interest X3D+ KML vs. RDF + Multimedia Markup

Languages

AR + Human Computation Human Computation

Real people solving problems difficult for computers

Web-based, non real time Little work on AR + HC

AR attributes Shared point of view Real world overlay Location sensing

What does this say?

Human Computation Architecture

Add AR front end to typical HC platform

Scaling Up

Seeing actions of millions of users in the world

Augmentation on city/country level

AR + Smart Sensors + Social Networks

Track population at city scale (mobile networks)

Match population data to external sensor data medical, environmental, etc

Mine data to improve social services

Scaling Up

AR on a City Scale Using mobile phone as ubiquitous sensor MIT Senseable City Lab

http://senseable.mit.edu/

WikiCity Rome

http://senseable.mit.edu/wikicity/rome/

Orange Data for Development

Orange made available 2.5 billion phone records 5 months calls from Ivory Coast

> 80 sample projects using data eg: Monitoring human mobility for disease

modeling

Conclusions

Mark Billinghurst, Gun Lee

Conclusions Outdoor AR hardware available

Handhelds (GPS, compass, camera), Wearables Many possible applications HIT Lab NZ Outdoor AR platform

Easy for building AR applications Multi-view support, Client/Server interface Cross platform (handheld, Glass, etc)

Next steps More tutorials on our website

http://www.hitlabnz.org/mobileAR Tags, LocationEvents, etc.

Projects on Google Glass http://arforglass.org One week workshop in Feb 2014

Stay tuned for server components Web-based Authoring Tool closed beta (Jan 2014)

http://www.hitlabnz.org/mobileAR

http://www.arforglass.org/

More Information Website

http://www.hitlabnz.org/mobileAR http://arforglass.org

Gun Lee gun.lee@hitlabnz.org

Mark Billinghurst mark.billinghurst@hitlabnz.org

Building Mobile AR Applications Using the Outdoor AR Library (Part 1)

Technology

Transcript of Building Mobile AR Applications Using the Outdoor AR Library (Part 1)

TP700 Comfort Outdoor, TP1500 Comfort Outdoor

Fan DC Motor（AR-8 series） Indoor Unit...Fan DC Motor（AR-8 series） Outdoor Unit Applications：Air-con blower outdoor unit、Outdoor unit of Eco water heater Characteristic:High

Catalogue outdoor

Wirral Playing Pitch & Outdoor Sports Strategy & Action ... · allocations and the consideration of planning applications. Underpin the provision of an accessible and realistic network

DER OUTDOOR-GUIDE: OUTDOOR-FIRMEN IM VERGLEICH

OUTDOOR 2015

Outdoor circuit training, outdoor workout system, outdoor workout, outdoor fitness equipment, fitnes

Kit Exaustão Linha GABINETES OUTDOOR - algcom.com.br · VENTILAÇÃO Excelente sistema de circulação e troca de ar por exaustão Linha GABINETES OUTDOOR rev_01 FLEXIBILIDADE Diversos

EasyGate SPT-G Outdoor / SPT-R Outdoor

Brochure Camping Trentino Outdoor. Campeggi Trentino Outdoor su

VC3211H 10lang · sufficiently thick but not too long, or cleaner performance will be adversely affected. For outdoor applications use an extension cord suitable for outdoor use.

Outdoor Regional

Outdoor Experience

turismo outdoor

ROBERS OUTDOOR

Educatia Outdoor

Outdoor Media

CARACTERIZAÇÃO DE FUNGOS DE AR INDOOR E AR OUTDOOR …repositorio.roca.utfpr.edu.br/jspui/bitstream/1/5075/1/CM_COEAM... · 1 universidade tecnolÓgica federal do paranÁ departamento

OUTDOOR EDUCATION

DuroSite LED Linear Fixture - UL / CSA - World Leading … · · 2018-03-28DuroSite® LED Linear Fixture - UL / CSA for Indoor and Outdoor Industrial Applications