Hierarchy Viewer Internals

Hierarchy Viewer Internals

Kyungmin Lee

Software Platform Lab., LG Electronics [email protected]

What-How-Why-Why not?

The 9th Kandroid Conference

http://sangminpark.wordpress.com/2011/09/02/what-how-why-why-not/

What?

How?

Why? Why not?

When? Who?

Past Present Future










Goal


Android와 같은 S/W 플랫폼이 성장하고 널리 확산되는데 있어 SDK(Software Development Kit)의 역할은 매우 큽니다. 아무리 좋은 기능과 성능을 제공하는 플랫폼이라 할지라도 그 위에서 동작하는 앱을 만드는 게 어렵다면 그 플랫폼은 개발자들로부터 외면 받게 됩니다. 이는 해당 플랫폼에서 이용 가능한 앱의 개수가 줄고 품질이 저하되는 결과를 만들게 되어 그 플랫폼은 사용자들에게도 외면 받게 됩니다. 이로 인해 개발자들이 다시금 그 플랫폼을 외면하게 만드는 악순환이 형성됩니다. 이러한 악순환을 끊기 위한 첫 걸음은 바로 개발 생산성을 높여주는 SDK를 잘 만들어 제공하는 것입니다. 따라서 Android 기술 분야에서 경쟁력을 확보하고 주도권을 갖기 위해서는 Android 플랫폼을 제대로 이해하는 것 뿐만 아니라 Android SDK를 제대로 분석하고 개선할 수 있어야 합니다.

이번 세션에서는 "Android SDK 기술의 이해"라는 큰 주제의 첫 꼭지로써 Android UI(User Interface)를 디버깅하고 프로파일링하는데 사용하는 Hierarchy Viewer를 분석하고자 합니다. 먼저, Hierarchy Viewer를 분석하는데 필요한 몇몇 배경 지식을 설명한 후 Hierarchy Viewer의 내부 구조와 동작 방식을 살펴보겠습니다.

From the Conference Program Overview

Bad

SDK

SDK (Software Development Kit)


A software development kit (SDK or "devkit") is typically a set of software development tools that allows for the creation of applications for a certain software package, software framework, hardware platform, computer system, video game console, operating system, or similar platform. It may be something as simple as an application programming interface (API) in the form of some files to interface to a particular programming language or include sophisticated hardware to communicate with a certain embedded system. Common tools include debugging aids and other utilities often presented in an integrated development environment (IDE). SDKs also frequently include sample code and supporting technical notes or other supporting documentation to help clarify points from the primary reference material.

Small No. of

Low Quality

Apps

Ignored by

Users

Ignored by

Developers

VS.

From Wikipedia

Good

SDK

Large No. of

Hig Quality

Apps

Attract

Users

Attract

Developers

Android SDK & Hierarchy Viewer


SDK Tools contains tools for debugging and testing

your application and other utility tools.

SDK Platform-tools contains platform-dependent tools for

developing and debugging your application.

Each SDK platform component includes a fully compliant • Android library, • system image (API Level <= 13),

• sample code (API Level <= 6), and • emulator skins.

Documentation contains a local copy of the latest multi-version documentation for the Android framework API.

Samples contains the sample code and apps available for each Android development platform.

Hierarchy Viewer • Related to graphics

• Complex enough to get insights on how to develop a tool for Android

http://developer.android.com/guide/developing/tools/index.html

http://developer.android.com/guide/developing/tools/index.html

Hierarchy Viewer


Hierarchy of Views in an Window

Properties of a View

List of Windows

Layout of Views

Hierarchy Viewer Architecture


Host

Hierarchy

Viewer

ADB

Server

Target|Emulator

ADB

Daemon

USB|TCP

system server user app.

user app.

user app.

Window Window

Window …

View

… View View

View

… View View

View

… View View

4939 4939

4940

Another Target or Emulator

…

Port Forwarding

adb forward tcp:4939 tcp:4939

4939

Roadmap


How to get information?

How to deliver information?

List of Windows

Between host and target

Hierarchy of Views in Window

Properties of View

Between processes in target

Window & View System

Java Reflection

Java Annotations

Android Debug Bridge

Binder IPC

1

2

4

3

5

Hierarchy Viewer

Internals

6

Questions Answers

In computer science, reflection is

the process by which a computer

program can observe and

modify its own structure and

behavior at runtime.

Reflection & Annotation in General


Base Level

Meta Level

Base Object

Application

Meta Object

Metadata

Reflection

Introspection

1 2

1

Intercession

2

Annotation

Add New

Metadata

(Type)

Set/Get

Metadata

(Value)

Type, interface, class, methods, attributes,

variables, functions, control structures, etc.

In programming, annotations are used mainly for

the purpose of expanding code documentation

and comments. … as a special form of syntactic

metadata in the source code.

From Wikipedia

From Wikipedia

Using Java Reflection


• B byte • C char • D double • F float

• I int • J long • S short • Z boolean

• Lclass-name class-name

Using Java Annotations


Using Java Annotations


Built-in Annotations


ADB Server

cmd-line adb (in platform-tools)

DDMS (in tools)

Hierarchy Viewer (in tools)

ADB Clients

Act as proxy between clients & daemons (multiplexing loop)

adbd

adbd

adbd

Emulator

Target

(TCP mode)

Target

(USB mode)

USB Connection

TCP Connection

ADB Daemons


ADB Architecture


Host

ADB Server

5037

adbd

ADB client

ADB client

…

Service Service

Service

Target

Guest (10.0.2.15)

5555

5554

console port

Emulator

5555

adb port

USB|TCP Service Service

Service

Service

Service

…

adbd

fork

Network Address Description

10.0.2.1 Router/gateway address

10.0.2.2 Special alias to your host loopback interface

(i.e., 127.0.0.1 on your development machine)

10.0.2.3 First DNS server

10.0.2.4 / 10.0.2.5 / 10.0.2.6 Optional second, third and fourth DNS server (if any)

10.0.2.15 The emulated device's own network/ethernet interface

127.0.0.1 The emulated device's own loopback interface

Connection to Emulator


Network Address in Emulator

ADB Server

Emulator

ADB Services


host:version

host:kill

host:devices

host:track-devices

host:emulator:<port> host:transport:<serial>

host:transport-usb

host:transport-any

host-serial:<serial>

host-usb

host

host:get-serialno

host:get-state

forward:<local>;<remote> shell:command arg1 arg2 …

shell:

remount:

dev:<path> tcp:<port>

tcp:<port>:<server-name> local:<path>

host:connect

host:disconnect

localreserved:<path>

localabstract:<path>

localfilesystem:<path>

log:<name>

framebuffer

dns:<server-name>

recover:<size>

jdwp:<pid>

track-jdwp

sync:

host-local

host:transport-local

killforward:<local>;<remote>

wait-for-usb|local|any

dns

reboot:

root:

backup:

restore:

tcpip:

usb:

Host Services

<host-prefix>

Local Services

Providing “Simple” Host Service


Host

ADB client

5037

ADB Server

3

asocket [local]

alistener

[smart]

asocket

[smart]

listen 2

create

accept

4

5

6

7 “000Chost:version”

read

“OKAY0004001d” 10

enqueue 9

connect

<Request> := <Length><Payload>

• <Length> := Payload length, as 4-byte

hexadecimal string in ASCII

• <Payload> := Request data

<Response> :=

• “OKAY”, for success

• “FAIL”<Length><Reason>, for failure

• “OKAY”<Length><Version>, for version query

• …

struct alistener { alistener *next; alistener *prev; fdevent fde; int fd; const char *local_name; const char *connect_to; atransport *transport; adisconnect disconnect; };

main()

install_listener 1

8

Phase 1: ADB Server Startup

Phase 2: Connection Establishment

Phase 3: Service Request/Response

A listener is an entity which

• binds and listens to a local port,

• creates an asocket upon receiving a

connection on that port, and

• connect the new local connection to a

specific service.

create

fdevent loop

Providing “Not-so-simple” Host Service


Host

ADB client

ADB Server

read

asocket [local] asocket

[local]

service thread

asocket [smart]

write

create create 4

create

5

6

read

12

enqueue

14 write

“OKAY”

7

13

close

2

1

8

10

11 <response>

15 <response>

Phase 3: Service Request

Phase 4: Service Response

* Phase 1 & 2 Omitted

struct asocket { asocket *next; asocket *prev; unsigned id; int closing; asocket *peer; fdevent fde; int fd; apacket *pkt_first; apacket *pkt_last; int (*enqueue)(asocket *s, apacket *pkt); void (*ready)(asocket *s); void (*close)(asocket *s); void *extra; atransport *transport; };

An asocket represents one half of a connection

between local and remote entity.

• A local asocket is bound to a file descriptor.

• A remote asocket is bound to an atransport.

<request>

enqueue 3

9

Providing “Complex” Local Service


Target

asocket [local]

Host

ADB client

ADB Server

read

asocket [local]

asocket

[remote]

asocket [smart]

enqueue

send

create

4

21 write

“OKAY”

close

<request>

2

1

6

22 <response>

atransport

adbd

atransport service

thread

read

write

create

<response>

asocket [remote]

<A_OPEN> 5

8 9

10

11

create

create

<A_OKAY> 12

13

14

15 16 17

enqueue

18

19 <A_WRTE>

create send

20 enqueue

12

#define A_SYNC 0x434e5953 #define A_CNXN 0x4e584e43 #define A_OPEN 0x4e45504f #define A_OKAY 0x59414b4f #define A_CLSE 0x45534c43 #define A_WRTE 0x45545257

struct amessage { unsigned command; unsigned arg0; unsigned arg1; unsigned data_length; unsigned data_check; unsigned magic; };

struct apacket { apacket *next; unsigned len; unsigned char *ptr; amessage msg; unsigned char data[MAX_PAYLOAD]; };

Phase 3: Service Request

Phase 4: Service Response

• Phase 1 & 2 Omitted

3

7

ADB Source Files


adb.c

fdevent.c

transort.c

transport_local.c

tansport_usb.c

services.c

sockets.c

utils.c

console.c

file_sync_service.c

jdwp_service.c

usb_linux_clients.c

remount_service.c

framebuffer_service.c

log_service.c

backup_service.c adb_client.c

file_sync_client.c

usb_vendors.c

usb_linux.c

get_my_path_linux.c

$ANDROID_TOP/system/core/adb

usb_macos.c

get_my_path_darwin.c

usb_windows.c

get_my_path_freebsd.c

usb_libusb.c

get_my_path_windows.c

Sources for adb (client & server)

Sources for adbd

commandline.c

Changing Transport Mode: USB TCP


1

2

2

$setprop service.adb.tcp.port 5555 $stop adbd $start adbd

3

4

5

…

6

Changing Transport Mode: TCP USB


$setprop service.adb.tcp.port -1 $stop adbd $start adbd

2

1

3

* Protocol fault error if USB not connected

* USB need to be re-connected

Using Service Manager


ActivityThread Binder Service

Activity

Manager

Service

LINUX KERNEL

Shared Memory Driver Binder (IPC) Driver

L I B R A R I E S

RUNTIME

Dalvik Virtual

Machine

Core Libraries

HelloAndroid

Activity

Looper

Message Queue

Service

Receiver

Provider

View

H

Handle Message()

Handle Message()

main()

Custom Impl.

LIBRARIES

System

Server

Service

Manager

LIBRARIES

Binder

Service Binder

Context

Manager

User Application Process

4

RUNTIME

Dalvik Virtual

Machine

Core Libraries

1

0x0:3:activity:0x0000a000 0x0:3:activity:0x1 2

3 6

7 8

9

10

0x0000a000 Permission

Unique Name

From the 8th Kandroid Conference

0x0:1:activity

activity:0x1

0x0:1:activity

activity:0x1

5

ViewRoot

Impl

Invoking Service


ActivityThread Binder Service

LINUX KERNEL

Shared Memory Driver Binder (IPC) Driver

L I B R A R I E S

RUNTIME

Dalvik Virtual

Machine

Core Libraries

HelloAndroid

Activity

Looper

Message Queue

Service

Receiver

Provider

View

main()

Custom Impl.

LIBRARIES

System

Server Service

Manager

LIBRARIES

Binder

Service Binder

Context

Manager


RUNTIME

Dalvik Virtual

Machine

Core Libraries

ActivityManager

Service

IActivityManager

3 : startActivity

…

0x1:3:[parcel]

0x0000a000

Binder

Thread

Pool

0x0000a000:3:[parcel]

BT #1

…

[parcel]

1

2

3

4

Permission Unique Name


activity:0x1

H

Handle Message()

Handle Message()

ViewRoot

Impl

activity:0x1

Launching Activity



Activity

Thread

Looper

Message

Queue

H

Application

Thread

System Server

Activity

Manager

Service

IActivityManager

IApplicationThread

IApplicationThread. scheduleLaunchActivity()

Zygote

IActivityManager. startActivity()

1

2 fork request

3 fork

9

sendMessage()

handleMessage()

&addr handle

10

4

LAUNCH_ACTIVITY


IApplicationThread. bindApplication()

IActivityManager. attachApplication()

6

7

5

8


Creating Window


Activity

Thread

Looper

Message

Queue

H

handleMessage()

1

LAUNCH_ACTIVITY

Activity

PhoneWindow

onCreate()

PhoneWindow

$DecorView

(FrameLayout)

TextView

(id/title)

onStart()

onResume()

LinearLayout

LinearLayout

FrameLayout FrameLayout

(id/content)

TextView

create

create 2

3

5

create

4

6

7

ViewRootImpl

8

create

ViewRootImpl

$W

9 create

handleMessage()

Adding Window 10

Adding Window


User Application Process System Server

Window

Manager

Service

IWindow

IWindowManager. openSession( )

Activity

Thread

Looper

Message

Queue

ViewRootImpl

ViewRootImpl

$W

handleMessage() IWindowSession. add( )

Session

WindowState

&addr handle

1

3

IWindowManager

2 create

4 create

5

sendMessage()

6

7

8

(Per Window)

(Per User Application)


Activity

Thread

Looper

Message

Queue

System

Server

Activity

Manager

Service

Window

Manager

Service

User Application Process Internal


Activity • onCreate() • onStart() • onResume() • onPause() • onStop() • onDestory() • onRestart()

Service • onCreate() • onStartCommand() • onBind() • onUnbind() • onDesstory()

BroadcastReceiver • onReceive()

ContentProvider • onCreate()

View • onMeasure() • onLayout() • onDraw()

H handleMessage()

handleMessage()

sendMessage()

sendMessage()

ActivityThread

$Application

Thread

Binder IPC

Thread Pool

BT #1

BT #N

ViewRootImpl

ViewRootImpl

$W

IApplicationThread

IWindow

APPLICATION_MEDIA_SUBLAYER (-2)

APPLICATION_MEDIA_OVERLAY_SUBLAYER (-1)

APPLICATION_LAYER (2)

WALLPAPER_LAYER (2)

PHONE_LAYER (3)

SEARCH_BAR_LAYER (4)

SYSTEM_DIALOG_LAYER (5)

TOAST_LAYER (6)

PRIORITY_PHONE_LAYER (7)

SYSTEM_ALERT_LAYER (8)

INPUT_METHOD_LAYER (9)

INPUT_METHOD_DIALOG_LAYER (10)

KEYGUARD_LAYER (11)

KEYGUARD_DIALOG_LAYER (12)

STATUS_BAR_SUB_PANEL_LAYER (13)

STATUS_BAR_LAYER (14)

STATUS_BAR_PANEL_LAYER (15)

VOLUME_OVERLAY_LAYER (16)

SYSTEM_OVERLAY_LAYER (17)

NAVIGATION_BAR_LAYER (18)

SYSTEM_ERROR_LAYER (19)

DRAG_LAYER (20)

SECURE_SYSTEM_OVERLAY_LAYER (21)

BOOT_PROGRESS_LAYER (22)

POINTER_LAYER (23)

Layering Windows (Surfaces)


HIDDEN_NAV_CONSUMER_LAYER (24)

APPLICATION_PANEL_SUBLAYER (+1)

APPLICATION_SUB_PANEL_SUBLAYER (+2)

Application Windows Including • TYPE_BASE_APPLICATION • TYPE_APPLICATION • TYPE_APLICATION_STARTING

Sub Windows • TYPE_APPLICATION_PANEL • TYPE_APPLICATION_ATTACHED_DIALOG

System Windows

Hierarchy Viewer Architecture


Host

Hierarchy

Viewer

ADB

Server

Target|Emulator

ADB

Daemon

USB|TCP

system server user app.

user app.

user app.

Window Window

Window …

View

… View View

View

… View View

View

… View View

4939 4939

4940

Another Target or Emulator

…

Port Forwarding

adb forward tcp:4939 tcp:4939

4939

Hierarchy Viewer Commands


boolean startViewServer(int port)

ViewServer Control

boolean stopViewServer()

boolean isViewServerRunning ()

(In android.view.IWindowManager)

REMOTE_COMMAND_CAPTURE = “CAPTURE”

REMOTE_COMMAND_DUMP = “DUMP”

REMOTE_COMMAND_INVALIDATE = “INVALIDATE”

REMOTE_COMMAND_REQUEST_LAYOUT = “REQUEST_LAYOUT”

REMOTE_PROFILE = “PROFILE”

REMOTE_COMMAND_CAPTURE_LAYERS = “CAPTURE_LAYERS”

REMOTE_COMMAND_OUTPUT_DISPLAYLIST = “OUTPUT_DISPLAYLIST”

View Information/Control

(In android.view.ViewDebug)

(In com.android.server.wm.ViewServer)

Version Information

COMMAND_PROTOCOL_VERSION = “PROTOCOL”

COMMAND_SERVER_VERSION = “SERVER”

COMMAND_WINDOW_MANAGER_LIST = “LIST”

COMMAND_WINDOW_MANAGER_AUTOLIST = “AUTOLIST”

COMMAND_WINDOW_MANAGER_GET_FOCUS = “GET_FOCUS”

Window Information

(In com.android.server.wm.ViewServer)

Controlling View Server


Host

Hierarchy

Viewer

adbd

Target

4939

system server

/system/bin/

sh

ADB

Server

WindowManagerService

4939

1 “host:transport:<serialno>”

USB|TCP

2 “shell:service call window 1 i32 4939”

3 “shell:service call window 1 i32 4939”

[fork] /system/bin/sh –c service call window 1 i32 4939 4

IWindowManager.startViewServer(4939) 6

/system/bin/

service

[fork] service call window 1 i32 4939

Thread Pool

create & start

Remote View Server [port=4939]

listen

create Thread

5

ViewServer

7

8 create 9

10

T T …

shell:service call window 1 i32 <port>

shell:service call window 2

shell:service call window 3

ViewServer Multi-Threading


Querying Version Information


Hierarchy

Viewer

Target

4939

system server


1

Thread Pool

accept

Thread

ViewServer

3

submit 4

2

T T …

connect

$View

Server Worker

Host

• ADB (Server & Daemon) Omitted

run()

run()

create

5

Phase 1: Connection Establishment

Phase 2: Command/Result

6 read

7

① “SERVER” ② “PROTOCOL”

① “4” ② “4”

Querying Windows


Hierarchy

Viewer

Target

system server


Thread Pool

ViewServer

T T … $View Server Worker

Host


• Phase 1 Omitted

run()

1


2 read

4

3 invoke

① “413931a8 StatusBar” and “411945c1 com.android.browser/com.android.browser.BrowserActivity“ and … “DONE.”

② “411945c1 com.android.browser/com.android.browser.BrowserActivity” ③ “LIST UPDATE” and/or

“FOCUS UPDATE”

<command>

<result>

① “LIST” ② “GET_FOCUS” ③ “AUTOLIST”

<command> <result>

① viewServerListWindows() ② viewServerGetFocusedWindow() ③ addWindowChangeListener()


Hierarchy

Viewer

Target

system server


Thread Pool

ViewServer

T T … $View Server Worker

Host


• Phase 1 Omitted

run()

1


2 read

3

viewServerWindowCommand()

user application

$W

ViewRootImpl

ViewDebug

5 IWindow.executeCommand()

dispatchCommand()

6

View

… View View

7 invoke

8

① “DUMP 411945c1” ② “CAPTURE_LAYERS 411945c1” ③ “CAPTURE 411945c1 android.widget.Button@405185a8” ④ “OUTPUT_DISPLAYLIST android.widget.Button@405185a8” ⑤ “INVALIDATE android.widget.Button@405185a8” ⑥ “REQUEST_LAYOUT android.widget.Button@405185a8” ⑦ “PROFILE android.widget.Button@405185a8”

<command>

<result> findWindow()

4

<command>

Querying/Controlling Views

Dumping View Hierarchy


Dumping View Properties


Selecting View Properties


Cache

Annotating View Properties


Summary


How to get information?

How to deliver information?

List of Windows

Between host and target

Hierarchy of Views in Window

Properties of View

Between processes in target

Window & View System

Java Reflection

Java Annotations


Binder IPC

1

2

4

3

5

Hierarchy Viewer

Internals

6

Questions Answers

References • General

– Wikipedia (http://en.wikipedia.org/wiki/)

– Java Documentation (http://docs.oracle.com/javase/6/docs/)

– Android Documentation (http://developer.android.com/index.html)

– Android Source Code (http://source.android.com/)

• Java Reflection – http://docs.oracle.com/javase/tutorial/reflect/index.html

– http://www.slideshare.net/CiaranMcHale/java-reflection-explained-simply

• Java Annotations – http://docs.oracle.com/javase/tutorial/java/javaOO/annotations.html

– http://www.ibm.com/developerworks/kr/library/j-annotate1/index.html

– http://www.ibm.com/developerworks/kr/library/j-annotate2.html

– http://www.slideshare.net/jennapederson/decorating-code-7252638

• Android Debug Bridge – http://www.slideshare.net/tetsu.koba/adbandroid-debug-bridge-how-it-works

– http://developer.android.com/guide/developing/tools/adb.html

– http://developer.android.com/guide/developing/tools/emulator.html

• Binder IPC – http://www.kandroid.org/board/board.php?board=conference&command=body&no=76

• Window and View System – http://www.kandroid.org/board/board.php?board=conference&command=body&no=19

• Hierarchy Viewer – http://developer.android.com/guide/developing/debugging/debugging-ui.html


http://en.wikipedia.org/wiki/

http://en.wikipedia.org/wiki/

http://docs.oracle.com/javase/6/docs/

http://docs.oracle.com/javase/6/docs/

http://developer.android.com/index.html

http://developer.android.com/index.html

http://source.android.com/

http://source.android.com/

http://docs.oracle.com/javase/tutorial/reflect/index.html

http://docs.oracle.com/javase/tutorial/reflect/index.html

http://www.slideshare.net/CiaranMcHale/java-reflection-explained-simply








http://docs.oracle.com/javase/tutorial/java/javaOO/annotations.html



http://www.ibm.com/developerworks/kr/library/j-annotate1/index.html




http://www.ibm.com/developerworks/kr/library/j-annotate2.html





http://www.slideshare.net/jennapederson/decorating-code-7252638





http://www.slideshare.net/tetsu.koba/adbandroid-debug-bridge-how-it-works












http://developer.android.com/guide/developing/tools/adb.html

http://developer.android.com/guide/developing/tools/emulator.html

http://www.kandroid.org/board/board.php?board=conference&command=body&no=76




http://developer.android.com/guide/developing/debugging/debugging-ui.html



Hierarchy Viewer Internals

Technology

Transcript of Hierarchy Viewer Internals