Exploring State Diagram-based Web Browser Programming

Yuka Obu Graduate school of Science and Engeneering,

Ibaraki University 07nd301x@hcs.ibaraki.ac.jp

Mizuaki Yamamoto Power & Industrial Systems

Hitachi, Ltd.

Tatsuhiro Yonekura Ibaraki University

yone@mx.ibaraki.ac.jp

Masaru Kamada

Ibaraki University m.kamada@mx.ibaraki.ac.jp

Shusuke Okamoto Seikei University

okam@st.seikei.ac.jp

Abstract

Over the last few years a great deal of interest has focused on a major paradigm shift in World Wide Web-based services referred to as Web 2.0. In marked contrast to the earlier Web, in Web 2.0 sites become sources of information and functionality that enable users to create new content of their own.

Consequently, users are now looking for more versatile browsers that will let them edit and display content based on their own preferences. This motivated us to develop a state diagram-based Web browser programming scheme that supports the close interaction between the end-user and Web content. Using state diagrams to represent browser behavior is easier for end-users with little or no programming experience to grasp than text-based programming systems. 1. Introduction

Over the last few years a great deal of interest has focused on a major paradigm shift in World Wide Web-based services referred to as Web 2.0. Based on a new concept that began to emerge around 2004, Web 2.0 is a comprehensive term involving a range of Web-related technologies and Web sites and services [1]. One salient feature of Web 2.0 is the concept of the Web as a subjective participation platform. In marked contrast to the earlier Web where information was unilaterally delivered to users from isolated sites, in Web 2.0 sites become sources of information and functionality that enable users to create new content of their own [1].

These capabilities have placed new demands on browsers: rather than just displaying the information as it is provided from Web sites, users are now looking for more versatile browsers that will let them edit and display content based on their own creative concepts

and preferences. This motivated our current project to develop a state diagram-based Web browser programming scheme that supports the participatory Web and achieves close interaction between the end-user and Web content. Our aim is to customize the content of the Web sites on the user side. Customize means to define the action of the browser freely so the user can browse the site as he like. For example, the user can filter sites that have bad influence to their children. And the user can also let the browser cache the content of the link before browsing, or search highlighted key words and cache results on background. The most important point of our scheme is to actualize these functions on the user side web browser. 2. Web Browser Programming

Our objective was to develop Web browser programming capabilities that would enable ordinary, non-technically-oriented end-users to freely customize the behavior of their browsers so they can edit and display the Web content according to their own preferences.

2.1. Visual Programming

Use of ordinary language-based programming

requires a degree of expertise in a programming language and other specialized knowledge, but the vast majority of people using the Web today do not have these special programming skills. In other words, the programming methods that are available today are poorly suited as tools for enabling ordinary end-users to customize Web browsers that are kind of application software.

This led us to pursue an alternative visual programming approach that is not based on language. Rather than words and language, visual programming

2007 International Conference on Cyberworlds

Unrecognized Copyright InformationDOI 10.1109/CW.2007.35

121

employs a system of diagrams. Compared to language-based programming, this visual approach is far easier to learn (the grammatical rules are more intuitive), involves much less abstract thinking, and requires relatively little use of the keyboard, so it is well suited to the needs of ordinary end-users. 2.2. State Diagram Model

Visual programming naturally requires a visual programming language, and here we have turned to state diagrams to implement a visual language. State diagrams have been in common use by the software design and development communities for a long time as a method of representing systems. Basically, state diagrams consist of two kinds of symbols -circles representing states and arrows representing transitions-. And the grammar is represented by transition connecting source and target states with arrows. This approach has the advantage of being intuitively easy to understand and easy for end-users to learn [2][3].

Here we have exploited these advantages to define a visual language model based on state diagrams. The model further defines three types of widget for different uses.

(1) GUI Widget

This is a set of widgets for creating the graphical user interface (GUI) of Web browsers. For example, the set includes a “return” button for going back to the previous Web page and a scroll bar for scrolling within a window. (2) Content Widget

A set of widgets for representing Web page content - that is, HTML tags. A range of HTML elements are included such as for representing links and image tags. (3) Selection Widget

A set of widgets created through selection operations of end-users. Since actions in the selection domain affect Web content, these widgets are equivalent to content widgets in the selection domain.

By using our model provided with these three types

of widget that have different functional attributes as defined above, it does not only clarify exactly what is to be customized but also help the end-user understand what is going on. Combining the widgets in various ways enables a wide range of customization options.

In our programming scheme, the state of the each item in the widget corresponds to the state of the state diagram, and the action of the user corresponds to the transition. By using this model, user can define the behavior of the web browser.

3. Related Researches

The research about the state diagram interface has been researched for a long term [4][5][6]. And the UML [7], the object oriented software design and development scheme, is also widely used. The UML is regarded as one of the state diagram schemes. In the UML-based approach, a line called the link represents the relationship between the objects. The state diagram in the UML is used for the representation about the each object behavior. And an interactive animation authoring tool base on the state diagram has proposed [2][3]. By the authoring tool, the animation of the characters can be defined by non-programmers.

On the other hand, Greasemonkey [8] is a Mozilla Firefox [9] extension that allows users to customize Web pages. Greasemonkey scripts are persistent so that user-defined changes to Web pages (called user JavaScripts) are executed every time the Web page is opened. User JavaScripts can utilize a full range of functions that are provided by Greasemonkey in the form of a library, and thus permit a wide range of customization options: changing the layout of a Web page, adding functions to a Web page, appropriating information from another Web site, and so on. Today, there is a great many user JavaScripts that have been made available on many Web sites [10][11]. For example, one Greasemonkey user JavaScript displays images retrieved using a Google image search as layers [12]. Figure 1 shows an example of what happens when this user JavaScript is executed. The Firefox logo at the upper right in Figure 1 is displayed directly on the Google search results page. Other scripts are available that remove most ads from the Google page [13], that add a window for searching Tokyo libraries [14] to the Amazon products page [15], and so on Scripts with similar capabilities called user scripts are available for the Opera browser that is distributed by Opera Software [16].

Figure 1: An example of JavaScript execution of

Greasemonkey

122

While these scripting techniques support advanced customization, they require highly programming skills and specialized knowledge of the Web that are far beyond the abilities of most ordinary end-users.

Platypus [17] is another Firefox extension that lets you modify a Web page from browser (GUI) and save the changes as a Greasemonkey user JavaScript so that they will be repeated the next time the user visits the page. This GUI enables ordinary end-users to easily customize Web pages. The main drawback of the Platypus GUI is that it was designed to customize Web page layouts, so is useless for customizing the behavior of Web pages. Figure 2 shows an example of what happens when a Platypus script is executed. One can see in the figure that the layout of the website (http://www.google.co.jp/) has been altered, and the layout information is entered in the spaces provided in the dialog box in the center of the screen.

Figure 2: An execution example of Platypus

4. GUEST 4.1. Overview

In this work we have developed a prototype Web programming system called GUEST (Graphical User inter face Editor by STate diagram) that is used as a Mozilla Firefox [9] browser extension.

Firefox is a Web browser developed by the Mozilla Foundation that was designed and built to support plugins and add-on program packages called extensions. Firefox extensions mainly consist of JavaScript and parts written in XUL [18], an XML language: algorithms are written in JavaScript and GUIs are written in XUL. Because the majority of Firefox operations can be controlled from extensions, the process of designing and developing software can

be reduced to a minimum and original Web browsers created.

Figure 3 shows a schematic overview of how GUEST works. As illustrated in the figure, (1) the user first describes desired Web browser behavior using the state diagram editor. The GUEST system then (2) automatically converts the state diagrams to JavaScript, and (3) the newly defined behavior is reflected to the web content by the Web browser. Note that (4) the state diagrams can be saved so they are loaded and executed every time the page is opened.

Figure 3: Schematic overview of GUEST

4.2. Functions GUEST consists of three basic elements: a state

diagram editor to generate state diagrams, a conversion function to convert state diagrams to JavaScript, and a file function to save and reload the diagrams every time the page is opened.

(1) State Diagram Editor

Figure 4 shows a typical screenshot of the state transition editor. And figure 5 is a screenshot focused on the state diagram on the figure4. States are represented by the little squares and the transitions by arrows. The little circle in the upper left corner of the screen in figure 5 signifies the initial state of the state diagram. The labels in the state boxes are the state names, and the labels shown alongside transition arrows represent transition conditions. In this case, the state name is the anchor tag. And the transition condition means mouse-over.

GUEST (Firefox extension)

Firefox

(1) Edit state diagram

(2) Convert to JavaScript

(3) Execute JavaScript

(4) Save and load the state diagram

123

Figure 4: A screenshot of the state transition editor

Figure 5: A screen shot focused on the state Diagram

(2) JavaScript Conversion Function

In the transition from state diagram to JavaScript program, states are converted to functions (one to one), and the transitions are converted to the function pointers. This simplifies the automatic conversion of state diagrams to JavaScript. Figure 6 shows a part of the script that is converted from the state diagram of figure 5.

(3) State Diagram Save and Load Functions

GUEST uses the RDF [19] as the format for state diagram permanent files. RDF (Resource Description Framework) is an XML language to describe metadata of the resources. So we can describe information such as headlined of articles and other kinds of metadata. Using RDF is advantageous because it reduces the implementation burden and provides a generic permanent file format Specific data stored in the permanent files includes:

* State-related data

State attributes (types of tags, actions, etc.), coordinates of state image in diagrams, names, and so on.

* Transition-related data

Transition attribute (types of events, etc.), coordinates of transition images in diagrams, control points (Bézier curves), names, and so on.

* State diagram-related data

List of state transitions included in state diagrams, URLs of pages for which diagrams have been generated, names, and so on.

Figure 6: A part of JavaScript converted from the

state diagram

4.3. Implementation Example Figure 7 shows a screenshot of an actual Web

Browser created using GUEST. As one can see, the browser has been customized so the previous page is transparently superimposed on the page currently displayed in the browser [20]. The two windows superimposed in Figure 7 are essentially see-through windows revealing pages that were visited earlier.

function gst_dia0() { gst_initTag("A"); gst_startDia("gst_dia0", gst_st00);

} function gst_st00(act) {

if (act) { return;

} if (gst_tagObjectList["A"].mouseover) {

gst_statePointer["gst_dia0"] = gst_st01; return;

} } function gst_st01(act) {

if (act) { gst_tagObjectList["A"].miniWindow(); gst_tagObjectList["A"].setSuccess(); return;

} {

gst_statePointer["gst_dia0"] = gst_st00; return;

} } function gst_startup() {

gst_dia0(); gst_startGuestScheduler(); return;

}

124

Figure 7: A screenshot of an actual Web Browser

created using GUEST 5. Conclusions

This paper presented a novel state diagram-based Web browser programming scheme, and described the actual prototyping of the scheme called GUEST (Graphical User interface Editor by State diagram). It is confirmed that using state diagrams to represent browser behavior is intuitively easier for novices and end-users with little programming experience to grasp, than text-based programming systems. Our efforts now will turn to system implementation of our approach and evaluating GUEST including the range of supporting Web service’s handling. Careful assessment of GUEST is particularly important, and we are currently studying a usability testing procedure that will involve end-users as test subjects. After using GUEST, the subjects will be asked to provide feedback regarding ease of use, relative difficulty of learning the state diagrams, and so on. Moving forward with the assessment, we plan to complete the actual implementation work as quickly as possible and proceed with assessment trials. 6. References [1] Tim O’Reilly, What Is Web 2.0, http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html [2] S.Okamoto, M.Kamada, T.Nakao, Proposal of an Interactive Animation Authoring Tool Based on State-transition Diagram, Information Processing Society of Japan, Vol.46, No.SIG 1(PRO24), pp19-27, Jan. 2005, in Japanese [3] S.Okamoto, T.Shimomura, M.Kamada, T.Yonekura, Programming with Islay, an Interactive Animation Authoring tool, Information Processing Society of Japan, Vol.47, No.SIG 6(PRO29), May 2006, in Japanese

[4] R.J.K.Jacob, Using Formal Specifications in the Design of a Human-Computer Interface, Communications of the ACM, vol.26, no.4, pp.259-264 1983 [5] R.J.K.Jacob, A Specification Language for Direct Manipulation User Interfaces, ACM Transactions on Graphics, vol.5, no.4, pp.283-317 1986 [6] D.Harel, Statecharts: A Visual Formalism for Complex Systems, Sci. Compt. Programming, (8), pp.231-274 1987 [7] Object Management Group, OMG’s Unified Modeling Language, http:www.omg.org/ [8] Greasemonkey, http://greasemonkey.mozdev.org/ [9] Mozilla Firefox, http://www.mozilla.or.jp/products/firefox/ [10] Userscripts.org, http://userscripts.org [11] GreaseMonkeyUserScripts,

http://dunck.us/collab/GreaseMonkeyUserScripts

[12] http://www.eonet.ne.jp/~wdf/s6ftware/ fx_g0_g66g3e50age3ayer.html [13] http://diveintomark.org/projects/butler/ [14] http://www.yasuhisa.com/could/entries/ 000754.php [15] Amazon, http://www.amazon.com/ [16] Opera UserJavaScripts,

http://jp.opera.com/support/tutorials/userjs/ [17] Platypus, http://platypus.mozdev.org/ [18] Mozilla Foundation, “XUL”,

http://www.mozilla.org/projects/xul/ [19] RDF, http://www.w3.org/RDF/ [20] T.Yanagida, T.Yonekura, Kaleidscope, proc. of Interaction2005, Information Processing Society of Japan, Vol.2005, No.4, D-411, May, 2005, in Japanese

125