GROSS ANATOMY AUGMENTED REALITY (GAAR MOBILE … · kasar manusia dan membantu guru atau pendidik...
Transcript of GROSS ANATOMY AUGMENTED REALITY (GAAR MOBILE … · kasar manusia dan membantu guru atau pendidik...
GROSS ANATOMY AUGMENTED REALITY
(GAAR MOBILE APPLICATION)
MOHD. AIRIL SOLEHAN BIN JOHARI
BACHELOR OF INFORMATION TECHNOLOGY
(INFORMATICS MEDIA) WITH HONOURS
UNIVERSITI SULTAN ZAINAL ABIDIN
2019
MO
HD
.AIR
IL S
OL
EH
AN
BIN
JOH
AR
I B
AC
HE
LO
R O
F IN
FO
RM
AT
ION
TE
CH
NO
LO
GY
2
01
9
i
GROSS ANATOMY AUGMENTED REALITY
(GAAR MOBILE APPLICATION)
MOHD. AIRIL SOLEHAN BIN JOHARI
043911
BACHELOR OF INFORMATION TECHNOLOGY
(INFORMATICS MEDIA) WITH HONOURS
FACULTY OF INFORMATICS AND COMPUTING
UNIVERSITY OF SULTAN ZAINAL ABIDIN
2019
ii
DECLARATION
I hereby declare that this report is based on my original work except for quotations and
citations, which have been duly acknowledge. I also declare that it has not been
previously or concurrently submitted for any degree at University Sultan Zainal Abidin
or any other institutions
Name : Mohd Airil Solehan Bin Johari
Date :…………………….
iii
VALIDATION
This is to confirm that:
The research conducted and the writing of this report was under my supervision, Dr.
Wan Aezwani Binti Wan Abu Bakar
Name : Dr. Wan Aezwani Binti Wan Abu Bakar
Date :…………………….
iv
APPRECIATION
First, I am grateful to Allah for giving me an opportunity and strength to
complete this proposal. I hereby also express my gratitude to Dr. Wan Aezwani Binti
Wan Abu Bakar, in particular as my project supervisor who has shared ideas, advised
and constantly monitored me to participate in this dissertation as well as providing
insights and assistance in providing this project.
Many of the challenges and tests that I have come to ensure that I understand
the needs and interests of this project title will make it easier for me to achieve this. I
dedicate one hundred percent commitment to the preparation of the required documents
and the experience I have gained from this project gives me an awareness of the
importance of understanding and practicality to ensure that the skills in this field can be
donated to universities and outside the community.
Finally, special thanks also to all those involved directly or indirectly assisting
the success of this system development project. Ironically, this project is a very good
field for me to improve my skills in this field of computing.
Thank you.
v
ABSTRACT
Gross Anatomy Augmented Reality (GAAR) developed to assist the educators
especially teachers and students to learn human anatomy in much more fun and
interactive way as to be more efficiently. Existing learning methods changed with AR
technology and uses the smartphone. This application may be able to show the students
the actual form of human gross anatomy and assist the teachers or educators’ in
explaining the sciences behind human body in more interactive and interesting.
Furthermore, this application uses a 3-dimensional object, video on paper and
interactive info so that students are interested in using this application. Methodology
Agile model was used to develop this application. Agile method is divided into five
main sections, namely the start of analysis requirements, designing, development,
testing and implementation. The use of this agile method can ensure the development
of the system running smoothly and can be change during each phase if required.
vi
ABSTRAK
Anatomi Kasar Augmented Reality (GAAR) yang dibangunkan untuk
membantu pendidik terutama guru dan pelajar untuk mempelajari anatomi manusia
dalam cara yang lebih menyeronokkan dan interaktif untuk menjadi lebih cekap.
Kaedah pembelajaran sedia ada berubah dengan teknologi AR dan menggunakan
telefon pintar. Aplikasi ini dapat menunjukkan kepada pelajar bentuk sebenar anatomi
kasar manusia dan membantu guru atau pendidik dalam menjelaskan sains di sebalik
tubuh manusia lebih interaktif dan menarik. Selain itu, aplikasi ini menggunakan objek
3-dimensi, video di atas kertas dan maklumat interaktif supaya pelajar berminat
menggunakan aplikasi ini. Kaedah Model Agile digunakan untuk membangunkan
aplikasi ini. Kaedah tangkas dibahagikan kepada lima bahagian utama, iaitu
permulaan keperluan analisis, perancangan, pembangunan, pengujian dan
pelaksanaan. Penggunaan kaedah tangkas ini dapat memastikan perkembangan sistem
berjalan dengan lancar dan boleh berubah pada setiap fasa jika diperlukan.
vii
CONTENT
ITEMS PAGE
TITLE PAGE
i
DECLARATION
ii
VALIDATION
iii
APPRECIATION
iv
ABSTRACT
v
ABSTRAK
vi
CONTENT
vii
LIST OF TABLES
ix
LIST OF FIGURES
x
CHAPTER 1 INTRODUCTION 1
1.1 Project Introduction 2
1.2 Problem Statements 2
1.3 Objectives 2
1.4 Scopes 3
1.5 Expect Result 3
1.6 Limitation of Work 3
1.7 Gantt Chart 4
1.8 Report Structure 5
CHAPTER 2 LITERATURE REVIEW 6
2.1 Introduction 6
2.2 Research on Different Techniques for AR 7
viii
2.3 Research on Platform Used for AR 8
2.4 Research on Development Kits Used for AR 10
2.5 Research on Augmented Reality SDK for Mobile Apps 12
2.6 Review of Existing AR Application 15
2.7 Review of Thesis/Article related to AR 16
2.8 Discussion 17
CHAPTER 3 METHODOLOGY 18
3.1 Introduction 18
3.2 Suitable Methodology 19
3.3 Agile Methodology 19
3.4 Agile Model vs Traditional SDLC Model 33
3.5 Discussion 34
CHAPTER 4 IMPLEMENTATION & tESTING 29
4.1 Introduction 35
4.2 GAAR Interface 36
4.3 Testing 48
4.4 Discussion 48
CHAPTER 5 CONCLUSION
5.1 Introduction 49
5.2 Project Contribution 50
5.3 Future Work 50
5.4 Project Planning 51
5.5 Summary 51
REFERENCES 52
ix
LIST OF TABLES
TABLE TITLE PAGE
1.1 Gantt chart for making GAAR in chapter 1 4
2.1 Augmented reality system development kit comparison 14
table in Chapter 2
2.2 Review of Existing AR Application Table in Chapter 2 15
2.3 Review of Thesis/Article related to AR Table 16
2.4 Agile Model vs Traditional SDLC Model 33
x
LIST OF FIGURES
FIGURE TITLE PAGE
3.1 Agile Model figure in chapter 3 19
3.2 Content of GAAR Mobile Application in chapter 3
21
3.3 Analysis Phase in chapter 3
21
3.4 Conceptual Model of GAAR mobile Application in 23
Chapter 3
3.5 Flowchart for GAAR Mobile Application in Chapter 3
24
3.6 The storyboard of the introduction screen in Chapter 3
25
3.7 The storyboard of the main menu in chapter 3
25
3.8 The storyboard of SCAN menu exercise in chapter 3
26
3.9 The storyboard of 3d model menu in chapter 3
26
3.10 The storyboard of Library menu in chapter 3
27
3.11 The storyboard of exercise menu in chapter 3
27
3.12 Choosing Wikitude SDK for Unity in chapter 3
28
3.13 Modelling 3d human brain in chapter 3
29
3.14 Texturing the lung model in chapter 3
30
3.15 Rigging process for heart animation in chapter 3
30
3.16 3D Human Heart in Unity3D in chapter 3
31
4.1 Home Menu for GAAR in Chapter 4
36
xi
4.2 Guide button interface in Chapter 4 37
4.3 Book Section button in Chapter 4 38
4.4 Anatomy AR in Chapter 4 39
4.5 Anatomy AR Interfaces 40
4.6 Guides in Anatomy AR 41
4.7 3D model of skeleton system 42
4.8 Quiz Game 43
4.9 Final result for quiz game 44
4.10 About app button 45
4.11 Setting/Option Button 46
4.12 Exit button 47
1
CHAPTER 1
INTRODUCTION
1.1 Project Background
Recently, augmented reality is becoming more popular in Western and Europe country
and have started to enter Malaysia’s market. More importantly, augmented reality is
very interactive and enjoyable compared to traditional methods. When analyse the
situation today, technology have improved very quickly and have given multiple effect
toward the society which included education, medical, games, architecture, furniture,
geography, history, travel, safety and decoration.
High professional jobs such as doctors, architectures, lawyers, politician, and
lecturers are now becoming a big issue since Malaysia is lack with these people. Society
now starting to downgrade themselves due to poor mindset. This mindset needs to be
change and ultimately make the society lifestyle more intellectual and love the
knowledge as part of their life.
Hence, Gross Anatomy Augmented Reality (GAAR) is being developed as a
touch-up for the combination of gross anatomy study and augmented reality for students
and teachers at early age. This application is to enhance teacher’s ability to educate
students, make students understands and help them visualize more when it comes to
science and human anatomy. This also provide new environment to current education
when it comes to smartphone device and application which will need the students to
carry out smartphone or make a device specially for GAAR mobile application to be
use at all school.
2
Conclusively, this application will ultimately change the way society think
about education and anatomy study by using augmented reality (AR) to amplify
everything in the future. All the contents, information, animation, video, text, pictures,
3D modelling and so on will be put up together to make GAAR mobile application and
promote new interactive and enjoyable education in Malaysia.
1.2 Problem Statements
The problem involved in GA: AR Project as follows:
1.2.1. The traditional method on education does not make students comfortable
due to lack of technology and equipment to make learning environment
more interactive.
1.2.2. Teachers facing hardship to make students understands due to lack of
good explanation or only visual by slide or video or using science
courseware provided.
1.2.3. Students feel that they do not know something inside their body because
they lack of visualization that make the knowledge is unworthy to learn.
1.3 Objectives
This project has several objectives that need to achieved which are:
1.3.1. To develop a special mark for GAAR to function along with 3D
modelling, animation, video, text, pictures and augmented reality
elements to make the education more interactive.
3
1.3.2. To delivered the GAAR information using mobile technology by
scanning special mark places such as book, paper or 3D model
1.3.3 To design an application that can make ease on teachers on educating
student about science in human anatomy.
1.4 Scopes
The scope that is highlighted:
1.4.1. User – Students/Teachers
User need to use smartphone for scanning special mark provided. Then
a main menu will appear on their smartphone. They can touch the screen
to zoom in or zoom out or to select between 3D models, watch videos
and animations.
1.5 Expect Result
User will be able to use their smartphone to scan and see the anatomy show by
GAAR
1.6 Limitation of Work
This application has its own purpose and uses. There are several limitations
which is:
1.6.1 Hardware
1.6.1.1 Size of smartphone’s screen may affect the app to show the whole
contents.
4
1.6.2 Software
1.6.2.1 Android smartphone with at least using Android lollipop and
above.
1.6.3 Other
1.6.3.1 Only use for education purpose. Not suitable for high level
hospital to use it.
1.7 Gantt Chart
Table 1.1: Gantt chart for making GAAR mobile application.
Task Name September October November December
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Topic Discussion
Project Title Proposal
Proposal writing
Proposal writing –Literature Review
Proposal Progress Presentation &
Evaluation
Discussion
Correction Proposal
Proposed Solution Methodology
Proof of Concept
Drafting Report of the Proposal
Submit draft of report to supervisor
Seminar Presentation
Correction Report
Final Report Submission
5
1.8 Report Structure
This report consists of five chapter. Chapter 1 is introduction which contain project
background, problem statement, objectives, scope, expect result, limitation of work and
report structures. Chapter 2 is about literature reviews consisting of introduction,
existing AR application, technique and theory that had been carry out by other
researchers. Chapter 3 is methodology which discuss about method or technique used
in this project, design, modelling and also specifying in detail the design of development
project. Chapter 4 is implementation and result from the project and the report of testing
that had been carried out to the developed application. Lastly Chapter 5, which conclude
the overall project followed by references.
6
CHAPTER 2
LITERATURE REVIEW
2.1 Introduction
This chapter provides the literature review for the augmented reality app that will be
developed. The purpose of this review is to provide an overview of how augmented
reality help students and teachers to improve their knowledge on human gross anatomy.
Besides, in this chapter, also provide the study of technique and approach. The related
journal and application about augmented reality (AR) was analyse to find out the best
implementation technique that can be used in this GAAR. The comparison between the
different applications and journals is done to get better understanding on each strength
and weakness on the differences which can make contribution on.
The concept of augmented reality is design to ensure that all the messages in it
will be delivered successfully by teachers and to students or any user involved. The
traditional concept of studying science human anatomy usually involves books, slides,
courseware and 3D model of human anatomy. Now with the evolvement of technology
of smartphones, people now can make use of it to study and gain knowledge from
anywhere at any time. In some occasions, teachers having problems in teaching their
students about human body and science behind it. Thus, the students cannot catch all
the knowledge in time due to lack of technology at home and lack of visualization on
how the actual human body work from inside. This will cause the students to show no
interest in science or anything that complicated them. Thus, by developing this
7
augmented reality called GROSS ANATOMY AUGMENTED REALITY (GAAR)
might be able to improve the way of study for students and also teachers in all school.
2.2 Research on Different Techniques for AR
Several researches on different techniques about augmented reality application aim to
compare which technique that suit well with the complexity of application depends on
the problem stated. On this project, there is a few researches with different techniques
have been reviewed (Mykola V, Gleb B, 2018)
2.2.1 Marker-based technique
Marker-based technique is a technique that recognise a pattern or a code when
a camera is pointed at it that is used to trigger the AR content. Most common
example of marker-based AR implementations use Quick Response (QR) codes.
If the pattern is unique and not repeated pattern, that marker can be used
successfully and AR content will be smooth-in-play.
2.2.2 Marker less AR technologies
Marker less AR technologies is generally opposite from the above technique
which does not need any pattern or code that overlay 3D content into the scene
or user’s environments and hold it to a fixed point in space. For example, Project
Tango which being adoption of its platform – which uses camera to scan the
environment in real-time, using the data to bring up virtual environment around
user, including 3D object like furniture with virtual layer.
8
2.3 Research on Platform Used for AR
2.3.1 Android
The android platform is a mobile device platform that uses a modified Linux
kernel. Introduced by Open Handset Alliance in November 2007. Most
application that run in this platform are written in the Java programming
language.
Android is an open development platform but it is not open in the sense
that all can contribute while a version in development. This is all done behind
closed door at Google. After it is finalized, only then Android start released its
source code and anyone who is interested can take the code and alter it.
To create any application for this platform, a developer needs the
Android SDK, which includes tools and APIs. Developer usually integrates
SDK into graphical user IDEs (Integrated Development Environments).
Beginners also can make use of App Inventor, which an application to create
Android app that can be accessed online.
2.3.2 iOS
This type of platform is a mobile OS for Apple-manufactured device such as
iPhone, iPad, iPod Touch and Apple TV. Best known for allowing user to
interact with their phone using gestures such as swiping, tapping and pinching.
These actions performed on multitouch capacities touch screen display that
provide fast response and accept multiple finger as input.
9
iOS is a product of Mac OS X and is a Unix-like OS. Within iOS, there
are four abstraction layers which is:
• Core OS layer: provide low level features as well as framework for
security and interaction with external hardware.
• Core Service Layer: provide service required by upper layers.
• Media Layer: provide the needed technologies for graphic, audio and
video
• Coca Touch Layer: used when creating an application.
Developers need iOS SDK to create any application for Apple mobile
devices. It includes tools and interfaces for developing, installing, running and
testing the apps.
10
2.4 Research on Development Kits Used for AR
Developing an app require developer to choose the tools suitable for the
platform that is going to be implemented. Thus, they need to consider some
development kits that allow them to build the apps (Mykola V, Gleb B, 2018).
2.4.1 Image Recognition
A feature that allow camera to detect or to identify objects, places and
images. Smartphone use machine vision together with camera and artificial
intelligence software to track image that can be later overlay with animation,
HTML contents, sound and etc.
2.4.2 3D Recognition and Tracking
Due to tracking, an app can recognise and enhance large space around
the user inside of large building. Commonly used in mobile games and e-
commerce.
2.4.3 Unity Support
Unity is one of the most powerful game engines worldwide and can
utilized in the making of AR app with powerful effect. Multipurpose tool like
Unity allow developers to implement both cutting-edge experience and extend
more traditional idea with new techniques.
2.4.4 OpenSceneGraph Support
An open source 3D graphic toolkit (application programming interface).
Used by developers in computer games, augmented and virtual reality, scientific
visualization and modelling.
11
2.4.5 GPS support
Main used for location-based AR application for geolocation AR. Can
be used both in AR games and apps made to overlay at on some nearby
locations.
2.4.6 SLAM Support
Simultaneous Localization and Mapping (SLAM), an algorithm that
maps the environment that user located and track all their movement. This
feature also can remember the position of physical object within some
environment and position virtual object accordingly to their movement and
position. SLAM has a very big potential and can be used in many kinds of apps,
not only AR type. Main advantage is the ability to be used indoors while GPS
for outdoors.
12
2.5 Research on Augmented Reality SDK for Mobile Apps
Tools that available on the market for AR application development:
(Andrii Zhuravlov-Galchenko, 2018):
2.5.1 Vuforia
A leading portal for AR application development that has a broad set of
features. Vuforia AR SDK contains:
• Recognizes multi-objects including cylinders, boxes and 3D objects as
well as images.
• Able to recognize text including about 100,000 words or a custom
vocabulary.
• Able to create customized VuMarks, which look higher quality than
typical QR-code
• Allow creating a 3D geometric map of any environment using Smart
terrain feature
• Turn static image into full motion video
• Provides Unity Plugin
• Support cloud and local storage.
The platform supported is iOS, Android, Universal Window Pattern and Unity.
As for the price, it has several versions which is free version, classic version -
$499 one time, cloud - $99 per month and Pro version for commercial use.
13
2.5.2 ARToolKit
An open-source tool to create AR application. Even if it is free library, it
provides rich set of features for tracking, which include:
• Unity3D and OpenSceneGraph support.
• Support both single and dual camera
• For location-based AR apps – GPS and compasses support
• Possible to create real-time AR Application
• Integration with smart glasses
• Multiple language supported
• Automatic camera calibration.
Supported platforms are Android, iOS, Linux, Window, Mac OS and Smart
Glasses and since it is an open source, it is free to use.
2.5.3 Wikitude
Wikitude has been introduced with its SDK7 that includes for simultaneous
localization and mapping. The features are:
• 3D recognition and tracking
• Image recognition and tracking
• Cloud Recognition
• Location-based services
• Smart glasses integration
• Integration with external plugins include Unity
Supported platform are Android, iOS, smart glasses. The price is different for
each version. Pro version is €2490 for each app per year, Pro3D is €2990 per
14
app per year, Cloud is €4490 per app per year and the last one Enterprise
version.
2.5.4 AR SDK Comparison Table (Andrii Zhuravlov-Galchenko, 2018)
Table 2.1: Augmented reality system development kit comparison table
AR SDK Best For Support Platform Pricing
Vuforia
Marker-based
apps
• iOS
• Android
• Universal Window
Platform
• Unity
• Free version
• Classic version-
$499 one time
• Cloud - $99 per
month
• Pro version for
commercial use
ARToolKit
Location-
based apps
• Android
• iOS
• Linux
• Windows
• Mac OS
• Smart Glasses
Free
Wikitude
Marker-less
apps
• Android
• iOS
• Smart Glasses
(currently Google
Glass, The Epson
Moverio BT-200, and
the Vuzix M100)
• Pro version -
€2490 per year per
app
• Pro3D - €2990 per
year per app
• Cloud - €4490 per
year per app
• Enterprise version
15
2.6 Review of Existing AR Application
Table 2.2 Review of Existing AR Application Table
Developer/Year Application Name
Technique Strengths Weakness
Nicola Casucci
(Smartech group)/
9 July 2014
Human
Anatomy
AR
Marker-
based app
→Able to detect
marker to display
3D model of human
and its organ
→Provide audio in
the model
→3D model is too
big for normal
smartphone’s screen.
→ Hard to hear
clearly the audio with
no text or subtitle.
→User cannot
control the rotation of
3D model
→If not hit the
marker or the app
move the camera
away from marker,
the 3D model will
disappear.
Roar Interactive /
13 March 2016
ROAR
Augmented
Reality
App
Marker-
based app
→Able to detect
marker and be able
to move around
outside the marker
→Have its own
website that store
the marker.
→High quality of
target marker and
texture of 3D modelling
→Lack of sound and animation
→Hard time on
applying hand
gesture to the app
→No information
related to actual
product
→Unable to make
people understand
the concept of AR
Graphic ViZion/
1 September 2016
Visual
Anatomy
3D |
Human
3D-
Modelling
based
→Able to use hand
gesture to
manipulate the
modelling
→Able to select
bone and show
name of it
→Smooth
presentation
→High quality texture of skeleton
→ Not applying the
concept of AR
→Only human
skeleton be display,
the rest of the content
need payment.
→Only name appear.
Not much other info
can get.
16
2.7 Review of Thesis/Article related to AR
Table 2.3: Review of Thesis/Article related to AR Table
Author/Year Journal Name Technique Description
Sebastian
Weib/ 25 September 2016
A Mobile Augmented
Reality Application to
Improve Patient
Education in Urology
Marker-based
app
The thesis about using AR
to simplify the knowledge
transfer between
physician and patient
about urology or
treatment of prostate,
details about pathologies
and 3D print of a prostate
by using AR in tablet
computer.
Shah Ahmad
Chowdhury/
September
2013
A Mobile Augmented
Reality and Multimedia
Applications for Mobile
Learning
Marker-based
app
This article is about
applying AR into learning
process that involve
marker detection and
tracking, sound and voice
over, video playback and
menus & options.
Wee Sim Khor/
23 December
2016
Augmented and virtual
reality in surgery—the
digital surgical
environment:
applications, limitations
and legal pitfalls
Marker less based app
+
See through
Concept
This review article on
innovations and
technology in surgery is
about analyse the whole
concept of AR and VR in
digital surgical
environment. This review
shows the application, its
limitation and legal pitfall.
In this article, it is more
suitable to use marker less
AR rather than VR that
allow user to see what
they need to see rather
than see in new virtual
environment during
surgery.
17
2.8 Discussion
This chapter is all about explaining about fact and statement that will support the
objectives of GAAR. In this chapter, analysis about the technology that will be use and
current understanding of Augmented Reality (AR) to be used in making the project. The
purpose is to decide the compare the current apps and come out with new solutions that
will make improvement for the future. The development of this project is in parallel
with the current computer technology.
18
CHAPTER 3
METHODOLOGY
3.1 Introduction
Methodology is a series of phase, procedure, rule of law, technique, tools used,
documentations, management and practices that are proposed in developing this
application. According to Merriam-Webster, methodology also can be defined as
approach, fashion, form, how, recipe strategy and ways to make up something. A
methodology used a set of techniques to carry out specific activities. There are several
type of methodology developments that can be used to develop GAAR. Every approach
needs to be analysed thoroughly and suitable with the project that is going to be develop.
For GAAR, methodology that suit to it is Software Development Life Cycle (SDLC).
SDLC is a complete process that conclude the key activities during project system
development. There are five stages of SDL which is planning, design, develop, testing
and maintain the product. There are various of SDLC model, each have unique pattern
of steps, in order to ensure excellent accomplishment in the process of software
development. According to a case study written by Mihaleva and Elena, agile SDLC
focuses on quick delivery of functioning software product by adaptable process and
constant user feedback. The agile model was the most excellent approach because it can
deliver functioning software faster, receives user feedback constantly and it gives
freedom for the development process.
19
3.2 Suitable Methodology
For this project, Agile Model is chosen for this project for several reasons. Agile SDLC
focuses more on quick delivery of functioning software product by adapter-able process
and constants user feedback. In Figure 3.1, Agile Model is illustrating the series of steps
during the agile development process. According to a case study written by Mihaleva
and Elena, agile SDLC focuses on quick delivery of functioning software product by
adaptable process and constant user feedback. The Agile Model was the most excellent
approach because it can deliver functioning software faster, receives user feedback
constantly and it gives freedom for the development process.
3.3 Agile Methodology
According to a case study written by Nataliia Peterheria, Figure 3.1 illustrates the series
of steps during the agile development process.
Figure 3.1: Agile Model
20
No matter what type of the models has been chosen, each of them has basic
stages which are used by every software development company. Let’s explore those
stages as this is important for the understanding of the each of SDLC models and the
differences between them.
3.3.1 Planning/ Analysis Requirement Phase
At this stage, planning is very important part that will determine how the end
result of the product or project should be. It is also need to identify the necessity that
need to develop mobile AR application. At this phase involves are requirement analysis,
task analysis and instructional analysis. The activities needed is to identify the problem
statement, the goals and objectives of the mobile AR application to be developed, the
end-user’s need, existing app review and characteristics and also the content of mobile
AR application. The content of mobile AR application can be referred in the Figure 3.
The mobile AR application is developed focusing on human anatomy learning content
specifically for student aged 10 – 16 years old. The analysis also carried out by doing
review on existing courseware or mobile AR applications especially in human anatomy.
Through the analysis, the weakness of the existing application related with human
anatomy can be identify. The comparison of existing application can be observed in
Chapter 2. All the information gathered were used to develop goals and objectives of
the project. The software and hardware requirement are also determined in this phase.
The flow of analysis phase is shown in Figure 4. (Roslinda Ramli, 2016)
21
SCAN
Exercise
Library
Figures 3.2: Content of GAAR Mobile Application
Figures 3.3: Analysis Phase
Marker-less Human Figure
GA:AR
Mobile AR
Application:
“GA:AR”
Related to
content
3D Human
Anatomy
Did You Know?
22
3.3.2 Designing Project Architecture
In design phase, conceptual model, flowchart and storyboard were developed.
The design phase involves the development of the conceptual model for the mobile AR
application which involves the learning theory, the medium and AR technology to be
used in the application; the teaching and learning approach to be implemented and the
components involve in this mobile GAAR application. (Roslinda Ramli, 2016).
3.3.2.1 Conceptual Model
The conceptual model is shown in Figure 5. The framework list
the following components as the important part to be included included
in the development of GAARmobile application which is:
• Teaching and learning approach
• Learning theory
• Teaching and learning approach
• Prototype development
23
Physical area
Physical and digital area
Digital area
- Multimedia
Fusion
Library Prototype
Learning
Theory
Interface
VR multidimension
environment
Text, audio,
animation, 2D
graphic, 3D object
Exercise SCAN
Constructivism Cognitivism
TAR
Interface
Graphical
Interface
Intuitive: Hold, Interactivity zoom in, zoom
Visual
in real time out, Rotate literacy
Teaching
and
Learning
Figures 3.4: Conceptual Model Of GAAR mobile Application
3.3.2.2 Flow Chart
The flow chart for this project is shown in Figure 6. The
application starts by providing user to choose from a selection of main
menu. They can choose either to scan button, exercise button or library
button. For this project, the scan button is most important feature that
allow user to scan the special mark place on the human anatomy picture.
This button automatically turn on the camera on user’s mobile phone.
After scan the special mark, a 3D model of human body appear on the
mobile screen. At this point, the technology used is augmented reality
that combine reality and virtual environment together to make a learning
process become more interesting. The exercise button bring users to do
24
some exercise related with the knowledge from the 3D model scan from
the mobile’ camera. A few questions are created to test the user
understanding of human anatomy. The library bring user to the fun fact
about human body that can give more knowledge for user especially for
students to learning. (Roslinda Ramli, 2016)
Figures 3.5: Flowchart for GAAR Mobile Application
Start
No
Yes Yes
SCAN Choose
No
Yes
Exercise
No Yes
Library
No
Exit
Yes
End
Exercise
Fun Facts
3D Human Body parts
Human Body
Main menu
25
Code 01
Title Start Page
Code 02
Title Home Page
3.3.2.3 Storyboard
Storyboard also play a big part of multimedia design process. The
storyboard shows the look and feel of the application that will be developed.
Some of the storyboard are shown in Figure 7,8,9 & 10 below.
Figure 3.6 The storyboard of the Figure 3.7 The storyboard of the
introduction screen main menu
Description: The main menu for
user to interact
A: Button - Exercise
B: Button - Scan
C: Button – Library
Description: The first page that
user will see
26
A C B
D
E
F
G
A C B
D
E
F
G
Figure 3.8 The storyboard of Figure 3.9 The storyboard of
SCAN menu 3d model menu
Description:
Phone camera on, scan the
marker and display the AR model
on screen.
A: Button - Exercise
B: Button - Scan
C: Button – Library
D: Scan area
2D/3D- allow user to switch
platform 2d or 3d
Description:
Phone camera on, scan the
marker and display the AR model
on screen.
A: Button - Exercise
B: Button - Scan
C: Button – Library
D: Scan area
Code 03
Title Scan Page
Code 04
Title 3d model Page
27
A C B
D
A C B
D
Fun fact
Figure 3.10 The storyboard of Library Figure 3.11 The storyboard of
Menu exercise menu
Code 06
Title Exercise Page
Code 05
Title Library Page
Description: Display the quizzes
for students to answer
A: Button - Exercise
B: Button - Scan
C: Button – Library
E: Option button
Description: Fun fact about
human body
A: Button - Exercise
B: Button - Scan
C: Button – Library
E: Option button
28
3.3.3 Development Phase
Development is the third phase of life cycle process. At this phase, the
development require material, software, hardware and the ability to produce the project.
The development phase is the actual production and assembly of the materials, hardware
and software that were discuss before in the design phase. Development phase is also
mean that the project is making to be test.
3.3.3.1 Marker-less
Figure 3.12 show that the first step for this project is to choose the correct marker-
less Wikitude SDK for Unity. Wikitude SDK that we downloaded will provide a sample
app, documentations and tutorials that will help developer to get started. Keep in mind
that the sample app given is just a sample with trial mark when we use it in actual
environments. The sample app will give a basic understanding on how to developed the
GAAR and developer can modified it depends on the developer’s capability.
Figure 3.12: Choosing Wikitude SDK for Unity
29
3.3.3.2 Modelling
Figure 3.13 show the developer making the process of modelling a 3d human
brain using Autodesk Maya software by using the brain picture as references
Figure 3.13: Modelling 3d human brain
3.3.3.3 Texturing
Figure 3.14 now showing the process of texturing the model in Autodesk Maya
software. Texturing process is to make the model’s surface look like it does in the
concept art or to match its real-world counterpart. This will make the model that is
created become realistic and can be understandable. The texturing can be created or can
30
be used from existing materials. The process of texturing need photoshop software to
create or to combine the uv-shell with texture before wrap it all up to a model.
Figure 3.14: Texturing the lung model
3.3.3.4 Rigging
Figure 3.15 show the rigging process on a character. Rigging is a process of
taking a static mesh, creating an internal digital skeleton, creating a relationship
between the mesh and the skeleton (known as skinning, enveloping or binding) and
adding a set of controls that the animator can use to push and pull the character around
Figure 3.15: Rigging process for heart animation
31
3.3.3.5 Building Augmented Reality Features
In this process, 3D model of the human body will be imported into Unity3D
software. With the Wikitude SDK that we implanted into Unity3D, we can now fully
focus on Unity3D with the process of animating the character, inserting the info about
the part of human anatomy and make the interaction between the part of model and user
experiences.
Figure 3.16 show the human body is placed in Unity3D software. The developer
needs to keep in mind that there are some tools that need to be set up first. For example,
Android Studio which is need for Android System Development Kit (SDK) to connect
the app made by Unity3D into another platform which is Android. Wikitude SDK also
needed to enhance the process of augmented reality (AR) in Unity3D. In Unity3D, the
platform to make the controller for 3D animator and AR camera setting which need
the coding platform which is called Visual Studio Code. The language used in this
coding platform is C Sharp also known as C#.
Figure 3.16: 3D Human Heart in Unity3D
32
3.3.4 Testing Phase
The testing phase refer to exercising the project at prototype level before actual
delivery of the instruction. The purpose of this phase is to make a series of checking
that allow developer to make improvement along with promoting the users
understanding of the project and to ensure users to gain some knowledge from the
prototype project.
3.3.4.1 Test Run (Alpha Test)
According to guru99.com, Alpha testing is a type of acceptance testing which
performed to identify all possible issues/bugs before realising the product to end users
or public. At this alpha test, the testing will be conducted by close friends and
supervisor. The feedback gain from this test will be used to improve the weakness of
this project.
3.3.4.2 User Test Run (Beta test)
Beta testing is performed by “real users” of the software application in a “real
environment” and can be considered as a form of external User Acceptance testing. Beta
version of the software is released to a limited number of end-users of the product to
obtain feedback on the product quality. Beta testing reduces product failure risks and
provides increased quality of the product through customer validation.
It is the final test before shipping a product to the customers. Direct feedback
from customers is a major advantage of Beta Testing. This testing helps to tests the
product in real time environment. All this are followed as www.guru99.com/alpha-beta-
testing-demystified.html
33
3.3.5 Evaluate Phase/ Review
At this phase, the measurement of effectiveness and efficiency of the project
will be taken place. Evaluation should be occurred throughout the entire phase because
agile methodology require developer to always check the project is working smoothly
and be ready for any changes in the process. If there some problems, the solution needs
to be come out to settle it. At the end of this phase, the final product is ready to be
presented and launches for public use.
3.4 Agile Model vs Traditional SDLC Model
Customer interaction is the backbone of this Agile methodology and open-
minded communication with minimum documentation are typical features of agile
development environment. Each phase will be consulted with end-user to give the best
experience for them.
Table 2.4: Agile Model vs Traditional SDLC Model
Difference Between
Agile Model Traditional SDLC Model
• Based on adaptive approach
• No detailed planning
• There is clarity on future tasks only in
respect of what features need to be
developed
• Driven person or team to adapt to be
changing product requirements
dynamically.
• Tested frequently
• Minimize risk of major failures in
future
• Based on predictive approach,
• detailed planning and complete
forecast on exact tasks
• Depends entirely on deep analysis
and planning
• Any change needs a strict change
control management and
prioritization
• Tested only at the end of SDLC
• Higher risk of major failures in future
34
3.5 Discussion
The conclusion can be made from this chapter is that the methodology used for
the project need to be clear and understandable for the developer to come out with a
project that can deliver the quality, effectiveness and efficiency of the mobile
application to the public. The whole process of the agile methodology needs to be point
out so it can be understandable for the other who read this.
35
CHAPTER 4
IMPLEMENTATION AND RESULT
4.1 Introduction
This chapter describes the implementation of GAAR mobile application. It consists of
three main features which crucial for GAAR implementation which are anatomy AR,
book anatomy and quiz game about anatomy. GAAR allow user to get a better
understanding about the structure of human skeleton and several main organs including
their main function which suitable for young students to understands more about human
science anatomy by providing 3d model which resemble to the actual organ and
skeleton.
36
4.2 Interfaces
4.2.1 Home Menu / Front Interface
Figure 4.1 shows the home menu of the GAAR mobile application. When user
first using the application, they will go to this home menu which consists seven button
for user to explore.
Figure 4.1: Home Menu for GAAR
37
4.2.2 Guide Button
In Figure 4.2 show the guide button feature. When user click on this button, they
show the guide on what do each button state. As such, user can see which button that
they need to know and be notify in early stage. These button are guide button with the
symbol of boy reading manual, book section with the book symbol, quiz game with the
symbol star.The middle button which is called Anatomy AR button which contain eight
model of anatomy. The next three button is about application, option button and exit
button.
Figure 4.2: Guide button interface
38
4.2.3 Book Section Button
Figure 4.3 show the book section which contains seven buttons that indicated
the several information about heart, lung, kidney, stomach, brain, liver and skeleton.
Some model such as heart and lung have been provided with sound and animations. All
the button give user the 3D model of anatomy and produce 4 main information about
each button that suitable for young students and children. On the top, we have two
button which are home button and Anatomy AR button.
Figure 4.3: Book Section button
39
4.2.4 Anatomy AR button
For the main part of GAAR mobile application is Anatomy AR button.
Figure 4.4 is now showing the interface of Anatomy AR. In this figure, the
interface provided info start-up button which is used to explain the step that user
need to know before using Anatomy AR.
Figure 4.4: Anatomy AR
40
4.2.5 Anatomy AR Interface
After user click Initialize button below, user will be show with eight
different icons which represent the 3d model organ. GAAR contains eight
different 3d models which are heart with animation, lung with animation,
skeleton system, liver, kidney, brain, stomach and intestines.
Figure 4.5: Anatomy AR Interfaces
41
4.2.6 Guides
User also be notice with guides when using Anatomy AR. This small
guide inform them on what user should do if the want to zoom, move, and rotate
the 3d object. If the screen is full of 3d models, they can easily reset it all by
pressing Reset button above go to Home page.
Figure 4.6: Guides in Anatomy AR
42
4.2.7 3D Model AR
In Figure 4.7, a 3D skeleton system model is display on the screen
without any marker. The model also displays the name of bones and description
about it. For each model, user have privileges to rotate using arrow provided or
zoom by using two finger or move the model to another location.
Figure 4.7: 3D model of skeleton system
43
4.2.8 Quiz Game button
Next is Quiz game. Just as Figure 4.6 has shown, GAAR provided simple
questions which can test user memory based on the information that book
section. Based on the figure above, we provided nine multiple answer questions.
Each question has its own countdown with test user capabilities to answer
questions with at short timing. Next button act as submit answer after user pick
an answer or not that allow user and audience know either the answer is true or
false.
Figure 4.8: Quiz Game
44
4.2.9 End Quiz Game
After finish answering those nine questions, the game give user
their score and notify them the highest score of the previous challenge. If user
want to try again, user can do this by pressing play again button or go to
homepage at the home button below high score.
Figure 4.9: Final result for quiz game
45
4.2.10 About App Button
In Figure 4.8, about app button show the picture of developer and credit
for supervisor, who have help giving advise and make this project possible to be
developed.
Figure 4.10: About app button
46
4.2.11 Option button
Figure 4.9 show the graphic and game volume for GAAR mobile
application. As for the graphic, it provided user low, medium and high resolution
of pictures in the app. Game volume can be adjust either high or low depends
on user. Hit apply button to set the options or hit back button to go to home
menu.
Figure 4.11: Setting/Option Button
47
4.2.12 Exit button
Lasly, the exit button used for user to check out from GAAR mobile
application. In the button, it gives user the privilege for user to exit from using
the app or stay using it.
Figure 4.12: Exit button
48
4.3 Testing
Testing is one of important phase in the development of the project. This phase
works to test the system or application whether it fully function or vice versa. In this
phase there are three type of testing to test the application which are unit testing,
integrate testing and system testing. Unit testing is carry out to verify the functionality
of specific section code. Integration testing is to exposed defect in the GAAR
application interfaces and interaction between module.
4.4 Summary
This chapter briefly discussed about the implementation of coding, functionality
of the mobile application and testing GAAR mobile application. To make GAAR
mobile application working well and functioning properly, all the test case is tested
carefully to make sure it achieved the intended result. The mobile application had been
tested and evaluated.
49
CHAPTER 5
CONCLUSION
5.1 Introduction
Augmented reality now has become one of the important technologies
nowadays. The augmented reality can be use by using mobile phones and can be used
for many purposes such as education, medical, games, architecture, furniture,
geography, history, travel, safety and decoration. Many applications such as IKEA,
Pokémon Go and Holo show that how importance augmented reality for human daily
life. Thus, a mobile application called Gross Anatomy using Augmented Reality
(GAAR) which help students and teacher understand more during education sessions.
The technique of implementing the augmented reality in mobile phone is called marker-
less which allow user to display 3D model at any flat surface. This allow user to capture
the information provided after scanning the flat surface. The information about human
gross anatomy allow them to explore more and create a new environment of education
which allow students to be more interactive and help teacher more in explaining the
scientific term in much more understanding. The comparative study about augmented
reality and anatomy had been conducted and the data show that the current application
doesn’t have augmented reality feature that can interact with students about education
especially in human anatomy. Therefore, Gross Anatomy Augmented Reality (GAAR)
is developed to solve the problem face by the students and teacher. By implementing
GAAR, these students and educators can be use for interactive study and help improving
education environment to a new level.
50
5.2 Project Contribution
Gross Anatomy Augmented Reality developed based on Agile method is mainly aim to
give the user to experience the new way to explore the human body by using 3D model and
also to provide user with the information of the human anatomy in this application.
Meanwhile, the others aim is to give the user exposure and education about the AR
technology. Furthermore, it is very convenience to use because user only just need to be
download on the playstore and can use them without any internet connection and require any
special target or marker.
5.2 Future Work
There are a few suggestions that can be used to upgrade GAAR mobile
application to be more efficient in future work such as:
5.2.1 The app needs to use a paid version of Wikitude SDK to get rid of the
trial watermark in AR camera.
5.2.2 Add more sound to give impact to user when using the application when
studying.
5.2.3 Give a consistent layout of each scene in GAAR mobile app to make it
more appeal to user experience
5.2.4 Add more game or interesting activities that can increase user
understanding and allow them to memorize the info through game and
fun activity.
5.2.5 Provided more unusual info which need to be know for young student to
learn more about human anatomy.
51
5.3 Project Planning
The project started on week 1 and finished on week 15. The detail project
schedule is based on Appendix (Gantt Chart).
5.4 Summary
GAAR is a mobile application that is useful for young generation of student and
teacher which give more interactive ways to explore human body by using smartphone
in real environment. Student does not need to understand something about human
anatomy since they can now use the current textbook and explore it using GAAR. This
not only help students increase their knowledge but also help teacher explain more about
human anatomy in their science subject. It is anticipated that by using GAAR, young
student can achieve more knowledge and see much more than just using textbook only.
Thus, GAAR mobile application is useful for young students and teachers
52
REFERENCES
a) Book
1. YM Dr. Tengku Fadilah Bt. Tengku Kamalden, 2013, Akademik
Kejurulatihan Kebangsaan,
Nota Sains Sukan Tahap 1, Skim Persijilan Kejurulatihan Kebangsaan
(SPKK)
2. Julie Carmigniani & Borko Furht, July 2011,
Chapter 1 Augmented Reality: An Overview
b) Thesis
1. Mohd Safarez bin Tifin, 2017, Universiti Malaysia Terengganu
Ship Profiling Using Augmented Reality
2. Sebastian Weiß, 206291, 25th October 2016, Universität Magdeburg, A
Mobile Augmented Reality Application to Improve Patient Education
in Urology,
3. Christoph Nol, Bernhard Häussermann, Ute von Jan, Ulrike Raap, Urs-Vito
Albrecht, Peter L. Reichertz Institute for Medical Informatics of the
Braunschweig Institute of Technology and Hannover Medical School
Demo: Mobile Augmented Reality in Medical Education: An
Application for Dermatology,
4. G. Dini, M. Dalle Mura, 2015, Department of Civil and Industrial
Engineering, University of Pisa,
Application of Augmented Reality Techniques in Through-life
Engineering Services,
53
5. Roslinda Ramli, 2016, Kolej Universiti Islam Antarabangsa
Methodology Design Of Mobile Augmented Reality (Ar) Education
Application On Learning Jawi: “Belajar Jawi Dengan Ar”
c) Articles
1. Andreas Duenser, Raphael Grasset & Mark Billinghurst, 2008,
A Survey of Evaluation Techniques Used in Augmented Reality
Studies
2. Shahan Ahmad Chowdhury, September 2013,
A Mobile Augmented Reality and Multimedia Application for
Mobile Learning
d) Web page
1. Christian Nordqvist, Tue 7 March 2017, Reviewed by University of Illinois-
Chicago, School of Medicine
Anatomy: A brief introduction,
2. Nataliia Peterheria, https://producttribe.com/project-management/agile-
sdlc-guide#article-header1, accessed on October 2018,
Why Agile SDLC Model Is the Best for Your Startup
3. Mykola V, Gleb B. https://rubygarage.org/blog/best-tools-for-building-
augmented-reality-mobile-apps, accessed on October 2018,
Best Tools for Building Augmented Reality Mobile Apps Best Tools for
Building Augmented Reality Mobile Apps
4. Andrii Zhuravlov-Galchenko, May 14, 2018, reviewed by Sergio De
Simone, Six Top Tools to Build Augmented Reality Mobile Apps
54
e) Google Play Store
1. Nicola Casucci (Smartech group), 9 July 2014, Human Anatomy AR
2. Roar Interactive, 13 March 2016, ROAR Augmented Reality App
3. Graphic ViZion, 1 September 2016, Visual Anatomy 3D | Human