bachelor thesis-peter gerhat
TRANSCRIPT
Czech Technical University in Prague
Faculty of Electrical Engineering
Bachelor Project
UIProtocol Client for Apple iPhone
Peter Gerhat
Bachelor Project Supervisor: Ing. Vaclav Slovacek
Study Programme: Software Engineering and Management
Specialisation: Software Engineering
Prague, May 28, 2010
2
3
Acknowledgements
It was a great honor to work on this project under Ing. Vaclav Slovacek’s supervision. His support and advises helped me to achieve this successful result. Under his guidance I discovered IntelliJ IDEA, a very powerful IDE, which was used throughout the development. The materials, which he offered, helped me a lot and I would like to praise his work on the documentation of UIProtocol, which was very self explanatory.
I owe my gratitude to my family and my friends, which were very supportive and showed a great deal of patience during the time I had to dedicate to work on this project. Their everyday support made the work engaging and I would like to thank them for that.
4
5
Statement
I declare, that I have elaborated the project independently and used only the sources included in the attached bibliography list. I have no compelling reason against the use of this school work within the meaning of §60 of Act No. 121/2000 Coll., on copyright, rights related to copyright and amending some laws (Copyright Act.)
In Prague, 5/21/2010 .............................................................
6
7
8
Abstrakt
Cílem projektu bylo vyvinout UIProtocol klienta pro Apple iPhone, který je schopen komunikovat se serverem pomocí UIProtocolu. UIProtocol je standardizovaný jazyk pro popis uživatelského rozhraní. Takový klient doposud nemohl být vyvinut pro mobilní zařízení, protože jsou schopny spouštět pouze aplikace napsané v běžných programovacích jazycích. Nedávno Adobe rozšířili svůj produkt Adobe Flash pro mobilní platformy, což umožnilo vývoj UIProtocol klienta pro Apple iPhone, jedné z prvních Flash aplikací pro iPhone a většinu jiných mobilních zařízení.
Abstract
The objective of the project was to develop UIProtocol client for Apple iPhone, able to communicate with UIProtocol server using UIProtocol, a standardized language for describing user interfaces. Such client has not yet been developed for mobile devices, because they are unable to run applications written in commonly used programming languages. Recently, Adobe extended its product Adobe Flash to mobile platforms, which allowed development of UIProtocol Client for Apple iPhone, one of the first Flash applications for iPhone and most other mobile devices.
9
Table of Contents
1. Introduction 14
1.1 Purpose of Project 14
1.2 Overview of the Document 14
1.3 About UIProtocol 15
1.4 About iPhone OS 15
1.5 Context of the Project 15
2. Platform Selection 16
2.1 List of Today’s Mobile Platforms 16
2.2 Market Share Point of View 17
2.3 Development Point of View 18
2.4 Development for iPhone 18
2.5 Selected Platform 18
2.6 Flash Platform 19
2.7 Flash or Flex Question 19
2.8 Other Considerations 20
2.8.1 Java 20
2.8.2 IPhone and Objective C 20
2.9 Recent Developments 20
2.9.1 Flex Mobile Framework 20
2.9.2 Apple’s New Policy 20
3. Analysis 22
3.1 Requirements 22
3.1.1 Overall Description 22
3.1.2 System Environment 22
3.2 Functional Requirements Summary 22
3.3 Non-functional Requirements Summary 23
3.4 UIProtocol Specification 24
3.5 User Interface Specification 24
3.6 State Diagram 24
10
3.7 List of Actors 26
3.8 Use Case Diagram 26
3.9 Actions Diagram 27
3.9.1 Immediate Requests for Definitions 28
3.9.2 Requesting Definitions When They are First Used 28
4. Design 30
4.1 Deployment Diagram 30
4.2 UIProtocol Client for Apple iPhone Architecture 31
4.2.1 UIProtocol Client Application Layers 31
4.2.2 Class Architecture 32
4.3 Special Classes 33
4.4 User Interface Design 33
4.5 Behaviors Design 34
4.6 Modifiers design 34
4.7 Events Design 35
4.7.1 Events Sent in Standard Communication 35
4.7.2 Special Events 35
4.8 Property Containers Design 35
4.9 Binding Design 36
4.10 Configuration 37
4.11 Hotkeys 38
5. Implementation 39
5.1 Specifics of the Implementation 39
5.2 XML Handling 39
5.3 Connection Layer 40
5.3.1 XML Validation in Connection Layer 40
5.3.2 Policy XML 40
5.4 Client Layer 41
5.4.1 Data Binding 41
5.4.2 Models and Model Updates 42
5.4.3 Data Storage 42
5.5 User Interface Layer 43
11
5.6 Application Performance 43
5.7 Platform Issues 43
5.7.1 Creation of an MXML wrapper 43
5.7.2 Class Instantiation 44
5.7.3 Problem With Loading .FLV Videos 44
5.8 iPhone Specific Issues 44
5.8.1 Default Layout Should Respect Apple Layout Guidelines[4] 44
5.8.2 Problems With Touch Interface 45
5.9 Coding Standard 45
6. Testing 46
6.1 Test Schedule 46
6.2 Future Test Plan 46
6.3 UIProtocol Testing Server 47
7. Conclusion 48
7.1 Project Overview 48
7.2 Summary 48
7.3 UIProtocol Client for iPhone Implemented Features 49
7.4 Future Development 50
Bibliography 51
A. Glossary 52
B. Screenshots 53
C. User Manual 54
C.1 Compilation Manual 54
C.2 Installation Manual 55
C.3 Normal System Usage 55
C.4 Unusual Events 55
C.5 Error Messages and Error Recovery 55
C.6 Customizing 56
D. CD Contents 58
12
Table of Figures
13
Chapter 1
Introduction
Development of applications for mobile devices always used to be connected with rebuilding them, when they had to be extended to other mobile platforms. Adobe recently started its effort to make cross-platform development possible. UIProtocol is a language for describing user interfaces (UIs), which is also intended to be ran on any platform.
1.1 Purpose of Project
With the use of Adobe Flash technology, the outcome of the project is an application, which can be currently ran on any iPhone OS device and several other mobile platforms. The application was intended to be downloadable via Apple’s App Store until Apple put a ban on distribution of Flash applications via its store.
The reason for extending UIProtocol to be used on mobile devices is that it is a promising technology, which needs to be cross platform. There are several other similar languages, which can be used for describing user interfaces, the simplest are used in web applications. UIProtocol is also a simple, but yet powerful solution. Because of its success on desktop platforms, it is expected to have the same success on today’s mobile devices.
1.2 Overview of the Document
Some parts of the document were taken from my previous work,[2] in which I compared existing mobile platforms and their suitability for the project. Important conclusions will be briefly summarized in the Platform Selection chapter.
Throughout the document will be references to the UIProtocol specification, which provides a comprehensive source of all information about UIProtocol and they will not be listed directly.
Rest of the work will be aimed mostly on providing information from software engineering point-of-view on the details of the implementation. Only the most important parts will be addressed with code samples and simple diagrams included. Users will find an attached user manual.
14
1.3 About UIProtocol
UI protocol provides a simple transfer method of user interfaces over the internet, similar to HTML/CSS/JavaScript/AJAX. The protocol is used in various projects, one of them is I2Home, which is aimed on providing control over household appliances using phones, computers and other devices as remote controllers, based on industry standards.
Advantage of UI protocol over other technologies is the use of XML language. The technology has several important advantages, even though a faster binary (UIProtocol-B), compressed (UIProtocol-Z) and Fast Infoset (UIProtocol-FI) versions are available. The advantages are:
• It is human readable - The documents can be easily created by human or computer, even though most of them are computer generated, it saves time for checking and creation for humans.
• UI documents can be parsed using standard SAX or DOM parser - The parsers are available for most of today’s platforms and therefore it can be easily extended to new platforms.
• Faster development of clients - The developer can focus on work with the document as a tree, instead of having to process binary data.
The protocol uses a client-server architecture and thus client must be connected to the server in order to work properly. It uses the server to fetch new interfaces definitions and perform most actions.
1.4 About iPhone OS
IPhone OS is the operating system, which runs on devices developed by Apple. As of June 29th, 2007, three types of devices with iPhone OS were introduced, the iPhone, iPad and iPod Touch.
The operating system has a lot in common with the Mac OSX. It has four abstraction layers. The top layer is the Cocoa Touch layer, which has been derived from the Cocoa layer used in computer versions. The graphic performance is very smooth, which gives possibility to create advanced UIs.
Thanks to its advanced features and user interface, the platform does not have its support only among Mac users. Also, it has the biggest App Store. The iPhone users are the most likely to use advanced smartphone features, such as internet, camera, email, blogging etc.
1.5 Context of the Project
In present, some other implementations of UIProtocol clients are already developed and being used. The protocol is still being revised and new functionality is being added in every revision. The implementation of the UIProtocol Client for Apple iPhone was done according to UIProtocol specification draft 8, [1] which describes UIProtocol 1.0, the currently highest available version.
No running server was available during the implementation, so the functionality could not be fully tested in real world environment. Instead of that, a simple UIProtocol server was being developed in Java together with the main project to provide at least some testing environment.
The server was not able to simulate real conditions, as the number of UI documents it could send was limited. Therefore, the client application should be fully tested before being deployed.
15
Chapter 2
Platform Selection
For comparison purpose a list of platforms was made with the basic criteria in mind. The list contains only the platforms, that are used today by major phone manufacturers. Smaller manufacturers may be using use other platforms, but their share is not relevant. Also platforms, which are now outdated are not being compared as the focus is on platforms with future potential.
2.1 List of Today’s Mobile Platforms
The list was last updated in February 2010. The prices may have changed, for instance, with the release of Adobe CS5 in May 2010. Also, it is expected, that some platforms will become open source and therefore it is strongly suggested to do some research about the current situation.
No. Name Programming Language License Development Costs
1 Android Java Open Source Free
2 iPhone OS Objective C ProprietaryD eve l o p m e n t - Fr e e, Releasing Software - 99$/year for membership
3 BlackBerry Java ProprietaryD eve l o p m e n t - Fr e e, Releasing Software - $20 for Java code signing keys
4W i n d o w s Mobile C++ Proprietary
Development IDE - $190, Releasing Software - $99/year for membership
5 Java ME Java Open Source May Require Signature (cost varies)
6 Java FX JavaFX Script P r o p r i e t a r y ( w i l l become open source)
Free
7 Symbian Symbian C++P r o p r i e t a r y ( w i l l become open source)
May require signature (cost varies), Some development tools are commercial
16
8 Adobe FlashActionScript (currently 3.0) Proprietary
Requires purchase of Adobe Flash (minimum $699)
9 Palm OS HTML 5, JavaScript, CSSPropr ie tar y ( some o p e n s o u r c e components)
Free
Figure 2.1: List of Platforms Used in Mobile Development
2.2 Market Share Point of View
The market situation changes dramatically. For better overview about its changes, there are two charts, one is for Q3 2009, second for Q3 2008. A number of platforms came out just recently, and their shares are growing speedily. Also, there is a number of outdated platforms on the top ranks, which sales are now dropping. Recently the Google Android platform was introduced, therefore it cannot be seen in the older chart in Figure 2.2.[13]
Figure 2.2.:Sales Diagram for Q3 2009
The current sales situation does not exactly map the devices share among users, because some users may own legacy devices from time, when the market was dominated by Symbian OS. Also, the statistics are mapping the worldwide trends, which might differ from those in Czech Republic.
The phone market is not just split between the phone companies, but also between different types of devices, in terms of quality. While some of them are low-end or obsolete, others are brand new and their capability of running modern applications is higher. Functions such as WiFi and GPS are part only of the more sophisticated devices. For instance, only 75% of them support WiFi.
No survey can satisfactorily answer the question how many devices would be able to support the software platforms, such as Java and Adobe Flash. While Java runs on most platforms, Flash used to run only on Symbian OS. It is due to competitiveness on the market and fast development.
Symbian RIM Apple Microsoft Google Others
Symbian46%
RIM21%
Apple18%
Microsoft9%
Google4%Others3%
Sales in Q3 2009
Symbian47%
RIM15%
Apple17%
Microsoft14%
Google0%Others7%
Sales in Q3 2008
17
2.3 Development Point of View
From a developer point of view it is important to be able to learn a new language quickly, use a modern IDE and reuse as many code as possible. Here, platforms, that use Java and scripting languages, are in advantage. Java code can be written in IDEs, such as Netbeans and Eclipse, which provide advanced functions, such as debugging and spell-checking, which save time for development.
Most of the UIProtocol client’s code was already written in Java and it would require minimum work to make a functional application and the development could focus immediately on implementing its more advanced features. Scripting languages provide even more simplicity. There is, however, no ready code for them. Typically they have lower performance, because they have to be interpreted.
The development support is surprisingly better for the newly introduced platforms. While the Windows Mobile and Symbian OS platforms have unsatisfactory documentation and support, Google Android, Adobe Flash, Java and iPhone OS documentation is much more available in both its content and ease of looking it up.
Battery life is not a key factor. The platforms perform very similarly here, although the Symbian OS platform showed its power efficiency, because of special programming techniques used.
2.4 Development for iPhone
One of the platforms, which was not targeted yet, is the iPhone OS. The platform is relatively new on the market and its share was growing rapidly in recent years. The primary goal of this project is to target iPhone, because of the platform’s advanced multimedia features, that might be used by UIProtocol. Currently, there are three possible ways of developing iPhone content:
1. Using Objective C and the Cocoa Touch framework - Cocoa Touch is the API used for creation of user interfaces on the iPhone. It provides the best functionality, and it is designed to work flawlessly with the multi-touch interface, which all iPhone OS devices provide.
2. Using WebKit[14] - An open source project started by Apple, adopted by most of today’s mobile web browsers on various phones. However, its primary function is to display web content and it would not be possible to create any larger application.
3. Using Adobe Flash Packager for iPhone - The Adobe Flash Packager for iPhone came out with the new Adobe CS5 and it offers deployment of applications on the iPhone and other iPhone OS devices. Flash is a promising platform for development of rich internet content, since Adobe has plans to target most of today mobile platforms.[15] The packager recompiles the otherwise interpreted code into a native iPhone application.
2.5 Selected Platform
The decision was based on objective facts, which were collected and analyzed and then subjectively evaluated. All tested platforms had the required API for making the UIProtocol client. However, as previously mentioned, some of them did not have built-in SAX. The criteria for evaluation were:
18
1. Application should be able to run on most of today’s phones with good outlooks to run even on future mobile devices.
2. Programming language should be easy to grasp and development should be straightforward without having to write any redundant code.
3. Possibility of reusing code, which was already written in Java.
4. Advanced IDE, low costs for both development and distribution of applications.
5. Platform suitability for making rich user interfaces.
6. Developer support should be sufficient, developer pages should have enough appropriate content to help solve any issues during development.
7. Platform performance.
The main goal was to find a platform, developing for which would target most of today’s devices and primarily those running iPhone OS. Adobe announced, it will expand its platform on the majority of mobile devices this year, while most mobile operating system platforms support only one brand and its set of devices. Flash uses the same set of APIs for every device and the application could be extended to other platform without even having to be re-compiled and therefore it was selected.
2.6 Flash Platform
Todayʼs most common multimedia platform is Adobe Flash. It is still under proprietary license, despite some efforts to make it open source. It uses a scripting language named ActionScript, which meets the ECMA guidelines. It is used to create interactive animations and user interfaces, which require less space than videos. It is a compiled language, which can be decompiled with relative ease.
With the use of Adobe Flash Packager for iPhone it is possible to create applications for iPhone. Additionally, it is going to be expanded on majority of mobile devices this year, while most mobile operating system platforms will still be targeted only on one brand and its set of devices. Flash has outstanding options for creating advanced UIs and all the required APIs. Performance should be sufficient on upcoming devices.
2.7 Flash or Flex Question
There are two commonly used frameworks for building UI of Flash applications. Those are Adobe Flex and Adobe Flash. Flex uses a different API, which is an extension to Flash API. It is not recommended to combine them, as each of them has its own purpose. Flash is more suitable for designers and creating simple, but graphically rich applications or UI components, while Flex is intended for more advanced application development.
The components developed in either Flash or Flex are interchangeable, but usually in compiled .SWF (Flash movie format), than uncompiled ActionScript format. They can be loaded by the default Loader class at runtime.
19
Even though Flash is very suitable for creating user interfaces, it would not provide good environment for programming the application logic. At time, when the development started, there was no development tool for Flash, except Adobe Flash Professional CS4, which did not provide some features needed for development. Currently, there is the new Flash Builder CS5, which was not available, when this project started. Furthermore, Adobe recently announced the new Flex Mobile API, which will make Flex framework the best choice for mobile development in Flash.
2.8 Other Considerations
2.8.1 Java
Java platform is often used to target mobile devices, but it has lower capabilities for creating advanced UIs, than Adobe Flash. It is used mainly by Google Android platform and it cannot be ran on any iPhone OS device. However, Android devices will be also able to run Flash soon without sacrificing much on performance.
2.8.2 IPhone and Objective C
Another considered platform was iPhone OS with its native Objective C programming language, which would offer native application look and feel on iPhone OS devices. However, the solution would lack portability to other platforms and it would be hardly extensible, because Objective C is not a very common programming language to most programmers.
2.9 Recent Developments
When the project started, the iPhone Packager by Adobe had yet to be released. However, some Flash applications for the iPhone already existed and even a private beta version of the packager was available. Later, Adobe denied making it public and therefore it came out to the public just in late April together with Adobe CS5. Since the start of the project a few changes were announced by both Apple and Adobe.
2.9.1 Flex Mobile Framework
Adobe announced a new framework targeted for mobile devices, which are using touch interface. It is called Flex Mobile[16] and it will appear by the end of 2010. It will be a lightweight version of Flex framework. It should be compatible with the existing framework and therefore no changes will have to be made to existing applications, which are using Adobe Flex.
2.9.2 Apple’s New Policy
The major change during the project was the announcement, that Apple will not support distribution of Flash applications compiled with iPhone packager via its app store on its devices. Apple claimed it wants to protect its platform from sub-standard applications, which would be performance ineffective and not meet the company’s UI standards.
20
2.9.3 IPhone Packager Discontinuation
Adobe claimed it will discontinue development of the iPhone packager and will not upgrade its functionality in future products. Instead of that, it will focus on Google Android and other mobile platforms. It did not directly affect the development of this project, because the packager application is available in Adobe Flash CS5 and as a console application, but it will have affect on its future.
The iPhone Packager supports applications written in ActionScript 3.0 programming language and Flex framework, but it is likely, that with their new versions support on Apple devices will be lost. The application may, however, still be deployed locally for usability testing on iPhone OS devices.
21
Chapter 3
Analysis
After selecting the Flash platform a deeper analysis was conducted. The application has to communicate with the user and server. It should be capable of performing basic application logic. Following sections describe the actions and provide better visualizations of typical workflows.
3.1 Requirements
3.1.1 Overall Description
The implementation is expected to perform well in aspects of performance and extensibility. It should be extensible on other mobile platforms. The solution should be robust enough for simply adding new elements, because the specification of UIProtocol is often a subject to change.
3.1.2 System Environment
The application should be capable of running on every iPhone, iPad or iPod touch (devices with iPhone OS installed). Even though the Flash application cannot use the Cocoa Touch API, it should have similar look and feel.
The UIProtocol client never runs without being connected to a server. It is because UIProtocol client is intended to be a thin client and therefore most of the application logic has be performed on the server side.
Each time, when the application connects to server, it fetches the interfaces, which are needed to be rendered. It must be able to read UIProtocol documents containing models, interfaces and behaviors and render them correctly. The UIProtocol client data are discarded after every restart of the connection.
3.2 Functional Requirements Summary
The functional requirements are based on the specification of UIProtocol client for Apple iPhone. The support for these requirements should be legitimate. Not all elements must be supported yet, but the solution should provide a simple way of adding them into the application in future.
22
List of functional requirements:
• Rendering UIProtocol user interfaces - Except for standard interfaces the UIProtocol Client for Apple iPhone should support also nested interfaces.
• Support for basic-level user interface components - Components such as checkboxes, buttons, images should be supported by default.
• Support for actions (client and server side) - Support of actions including those, which only invoke model update on the client side.
• Support for models and model updates - Models are a common practice in UIProtocol to bind values, which are used or changed often to a single model, which has to be updated just once.
• Support for modifiers - Modifiers alter the values when binding component, with their use a single value can be used in multiple representations.
• Requests for missing actions and interfaces - Sometimes application has to request a missing interface or action definition, this helps to avoid sending duplicate definitions to the client.
• Support for binding and model-wide binding - Binding is used to bind any property to a model. It is often used for positions and styles, which are bound to model, that can be easily updated.
• Support for undefined element - Any unrecognized element is identified as an undefined element and not displayed. Once its definition arrives, its class is recognized.
• Support for default container - The container with default absolute layout must be supported.
• Basic checking of XML validity - UIProtocol client should be able to distinguish whether an XML is an UIProtocol document.
3.3 Non-functional Requirements Summary
Non-functional requirements are specific for the iPhone. They differ because of the platform’s hardware and operating system differences. For instance, behaviors are very specific. More on that will be in Design and Implementation sections.
List of non-functional requirements:
• Adding New Interfaces and Other Elements - Some UI components, which have not yet been implemented. The implementation must be robust to make it possible to add them in future.
• Adding New Functionality - The application will be extended in the future. Comprehensive documentation must be provided. Functionality should be added without big changes to the code.
• Extending to Other Platforms - It should be possible to run the application on similar platforms by doing minimum changes to the code, or by just modifying the configuration file.
• User interfaces should have native look for the given platform (Adobe Flash) - The native look is based on the Apple’s guidelines for human interfaces.[4]
23
• Satisfactory performance on iPhone with loading time below 0,5 seconds in fast networks
• Compatibility with UIProtocol standards, including default values and application behavior[1]
• Application should be stable, it should be able to display valid content and hide invalid
• Client should not do any processes when idle
• Application should be easy to debug and test against a stub server
3.4 UIProtocol Specification
In order to specify the functionality, client classes have been created. Client of each level should provide all functionality of a given class, in order to become a client of that class level.
The UIProtocol Client for Apple iPhone should provide at least basic functionality needed to handle data from server rightly and send feedback. The client is constantly evolving and therefore it is not possible to specify the its class exactly. Also, it is specific, because of the platform’s limitations.
An advanced client of Class 2 or Class 3 would offer more complex features. For instance, advanced clients can perform some actions without notifying the server. Also, one of the features, which can be seen only in the higher client classes are interpolations.
3.5 User Interface Specification
UIProtocol does not does not declare strict default values for UI components. Instead of that, every implementation uses its own UI components, which match its native look and feel. Java uses SWING, for instance. However, restrictions are on attributes. Each component can have only certain set of attributes, which can be divided into position, style and component properties. Developers are recommended to use target platform’s native look, while following the UIProtocol specification.
UIProtocol specifies behaviors, which may be different from programming languages. At least the most common mouse triggers should be supported due to user input restrictions on iPhone.
3.6 State Diagram
The application has five defined states, which all depend on the initialization and connection status. For instance, if the application received a disconnect event from the server, it will not attempt to reconnect again. However, if the application is connected to the server and the server is not responding to its requests, it may attempt to send the request to the server again before closing the connection.
24
Figure 3.1: State Diagram
Figure 3.2 contains descriptions of the application states. They are based on the UIProtocol’s connection initialization and termination guidelines. More on handshakes and client-server communication can be found in UIProtocol specification.
State Description
INITIALIZING The application is initializing its UI. The UI is displayed only, when the iPhoneMode is set to true. Otherwise, only a mx.containers.Canvas is displayed.
LOADING CONFIGURATION
In this part of the initialization the application loads all values from the configuration files, including default models and default XMLs. The application cannot advance to next state without the files being fully loaded.
CONNECTING Application is attempting to connect to the server by sending the public.connection.connect event and information about itself to the server. No other UIProtocol data are transferred, except the initial communication between client and server. The client must receive data from the server in order to advance to the next state.
CONNECTED This state is used throughout the communication between the client and the server. The client does not have any timeout set and only way to advance to the disconnected state is through receiving a public.connection.disconnect event from the server.
DISCONNECTED T h e s e r v e r m u s t a l w a y s s e n d t h e public.connection.disconnect event in order to terminate the connection properly. However, the application must be prepared for the alternative, that it does not receive the event and still be able to end properly. After reaching this state the application remains open, but it does not attempt to connect anymore.
Figure 3.2: Application States Descriptions
25
3.7 List of Actors
Two sides will interact with the application, the user and server. The server is already functional and it typically serves multiple devices at once. The UIProtocol client will have to respond to the interactions from both user and server.
Actor Description
User User is any person interacting with the application.
Server Server is a remote computer, which is responsible for most of the application logic and it will be sending or receiving updates from the client.
Figure 3.3: Actors Descriptions
3.8 Use Case Diagram
The use cases in the Figure 3.4 provide a visualization of the application’s typical usage. Especially the application’s communication with server is very compelling. The server only sends model updates, interfaces and actions, while the client can send only events, which often represent the user’s interactions or different kinds of requests.
Figure 3.4: Use Case Diagram
Use Case Description
Open application User opens the application, which starts connecting to server.
Interact with UI User performs interactions with UI, the application responds accordingly.
26
Use Case Description
Send updates Any time when the application is connected, the server may send updates, which may cause the client to update its data or even re-render the interface.
Disconnect If the connection is no longer needed to be open, the server may disconnect from the client. According to the UIProtocol specification is must notify the client about its intent.
Close application User closes the application window, which causes the application to exit. In that case the connection with server is closed and a disconnect event should be sent to the server.
Figure 3.5: Descriptions of Use Cases
3.9 Actions Diagram
One of the most commonly used action sequences in the application is the initial handshake, which has to be supported and is important for further communication between the server and the client.
Figure 3.6: Initial Handshake
27
In UIProtocol it is very common, that the server does not send all actions, interfaces and models together with their definitions, but the UIProtocol client has to request for them. It is because the client often has the definition and sending it twice would just increase the amount of data, which have to be transferred. There are two different approaches to client-side requests:
3.9.1 Immediate Requests for Definitions
The UIProtocol client requests their definitions immediately, when their name was not resolved. This approach is preferred, because the definitions are already loaded, when they have to be used. It makes the application appear more responsive to the user. On the other hand, it may consume more memory and require more data to be transferred, because often not all definitions are used. The figure below shows, how this process is applied to elements.
Figure 3.7: Immediate Request for Definition
3.9.2 Requesting Definitions When They are First Used
In the second approach the definitions are loaded, when they are first used. Due to slow network speed on mobile devices it is not effective to be used in the UIProtocol Client for Apple iPhone, as it would slow down the application and lower the user experience. It is expected, that the iPhone has sufficient memory and computation power to load definitions using the first method.
UIProtocol also has support for actions, which are divided into client and server. The UIProtocol Client for Apple iPhone supports both types of actions. In Figure 3.9 the execution process of an action can be seen.
28
Figure 3.8: Delayed Request for Definition
Figure 3.9: Action Execution Process
29
Chapter 4
Design
When designing the application, important considerations had to be made. It is the application’s design, which makes it extensible and ready for future development. The specifics of the Flash platform led to designing some aspects differently, than in other versions. During the design process, priorities were to make it not very different from existing implementations and extendible.
Interesting parts of this chapter for future developers and experienced users are those about the design of UI components, modifiers, actions, etc. They are intended to provide better understanding of how to add custom components without knowing the application logic. The last part contains informations about the structure of the configuration files and gives an introduction to configuring.
4.1 Deployment Diagram
The application uses the typical client-server architecture. It is expected, that both sides run at the same time. As seen in Figure 4.1 each UIProtocol client relies on the server and performs only basic application logic. In this case the application is a thin client, because it relies on the server heavily.
Figure 4.1: Deployment Diagram
30
The UIProtocol client and UIProtocol server must be able to communicate with each other independently on their software platform. It has to be achieved by strictly following the UIProtocol specification on both sides.
• UIProtocol server - The server has all the definitions, which are sent to UIProtocol client. It has to perform most of application logic and be able to serve multiple clients.
• UIProtocol client - The client in case of UIProtocol can send only events and also fires actions and updates models, which are stored in the client only during the session.
4.2 UIProtocol Client for Apple iPhone Architecture
4.2.1 UIProtocol Client Application Layers
The client application uses an architecture, which is similar to other UIProtocol implementations. The data are transformed into XML in the connection layer and those are turned into UIDocument objects in the client layer. The UIDocument objects are then finally rendered as their UI component representations.
Figure 4.2: Layer Architecture
There are 3 visible layers in the UIProtocol client. The data layer is missing, because the client never stores any data permanently and just holds them for the length of the session. The data in form of UIDocument objects are also part of the client layer.
Layer Name Description
User Interface Layer Layer, in which the UI components reside and the only visible layer to the user. It is used for displaying user interfaces.
Client Layer The Client Layer provides a link between the User Interface Layer and Connection Layer. It stores data for the length of the session and is responsible for reading and writing UIDocuments.
31
Layer Name Description
Connection Layer The bottom layer, which is used only for data communication with server. It gets data in form of XML from server and passes them to the Client Layer. It also does basic XML checking for validity, but does not validate against UIProtocol XML schema.
Figure 4.3: Descriptions of Application’s Layers
4.2.2 Class Architecture
Classes, which process the data in UIProtocol client contain only static methods and therefore do not have to be instantiated. Class UIDocument, which stores data is singleton. It is for performance reasons and the fact, that the classes do not have any instance specific methods or variables.
Each class performs a function, which it was designed for. If a class needs other class instance, it calls the appropriate factory, which returns the instance, that is already set by a setter class.
Class Name Description
Display Extends the main display container and has the functionality to update itself and render interfaces from UIDocument. Usually it is singleton, but it can contain embedded instances of itself (nested interfaces).
UIClientConnection The class, which opens and maintains the connection with the server. When it receives XML from the server, it does short XML validation and forwards the received document to UIDocumentReader class, which reads it.
UIDocumentReader UIDocument reader is responsible for reading UIDocument XMLs and it delegates reading to other classes within the property reading module. Each class in the module is specialized for one kind of UIDocument object. For instance, Event, Model Update, Property and so on.
UIDocumentWriter UIDocument writer is responsible for writing UIDocument XMLs and it delegates writing to other classes within the property writing module.
UIDocument UIDocument class provides temporary storage for the data received from the server in form of UIDocument objects. The objects are easier to maintain than XMLs and they can be approached as objects with defined methods.
UIClient UIClient is the main class of the whole application, which is responsible for the initialization in the beginning and setting up the environment.
32
Class Name Description
ConfigurationReader Configuration reader and the configuration itself is an important part of the application, because dynamically loaded configuration allows to make changes without recompiling.
Figure 4.4: Important Classes Descriptions
4.3 Special Classes
Some classes were designed to provide tools for XML validation and UIProtocol properties resolving. A short list of the most important ones is provided in Figure 4.5.
Class Name Description
UIProtocolPropertyResolver Checks if the property is of specified property type.
UIProtocolToActionScript Translates property names from UIProtocol to ActionScript.
UIProtocolValidator Validates position and style tags and removes properties, which are not allowed in them.
XMLTools Class for basic XML validation.
Figure 4.5: List of Special Classes
4.4 User Interface Design
The key functionality of the UIProtocol client is to render rich UIs, which use different UI components by using UI components used in Adobe Flash. They often have different parameters, layout and sometimes behavior. Because many UIProtocol UI components cannot be found in Adobe Flash a better solution than creating a separate class for every component had to be found.
A wrapper class was created to fix the problem. The class allows to set properties of Flash UI components by using their names specified in UIProtocol. The wrapper class is shown in Figure 4.6.
Figure 4.6: Wrapper Class for Display Objects
33
The class contains a Flash DisplayObject, which is the common abstract class in Flash representing any UI component. When some UI component property has to be updated, the class looks up in the configuration, which property has to be changed on the DisplayObject using its UIProtocol name and updates it using its correct Flash name. The developer is responsible for specifying property mapping in the configuration file or, for advanced UIProtocol UI components in a custom class.
4.5 Behaviors Design
Behaviors are used to attach events to UI components. UIProtocol Client for Apple iPhone supports two ways of their execution. They can be either executed by the client, sent to the server or both.
ActionScript supports event handling, in which object (subclass of EventDispatcher) dispatches events, that are handled by event handlers for each type of event. The pattern was used for behaviors.
Behaviors are attached to the corresponding DisplayObjects as event handlers. Each handler is specific for one event type. Its function is to add implicit properties of the action to the properties specified in the behavior, if not specified otherwise, and forward them to the UIDocumentWriter class, which fires the action and/or sends an event with the behavior properties to the server.
The Handler class is abstract and therefore never instantiated. It has the method getHandler(), which returns the event handler. Other handlers inherit from the class and and have their handler functions for specific events. The handler functions are then returned by getHandler() method.
4.6 Modifiers design
Modifiers are used to alter values of properties, when the update is propagated to bound UI components. In UIProtocol the modifiers may be specified in the value tag of each property and are separated by a colon. There is no limit on the number of modifiers each property may have.
The design of modifiers in the UIProtocol Client for Apple iPhone allows multiple modifiers to be attached to a single property. The modifiers are then chained and one sends the value to another. It is done using the chain of responsibility design pattern.
Figure 4.7: Modifiers in UIProtocol Client for iPhone
34
Modifiers can be chained in any order and the current design also allows creation of custom modifiers. A custom modifier must implement the IUpdatable interface shown in Figure 4.7.
4.7 Events Design
The UIProtocol Client for Apple iPhone was designed to support sending data back to the server in form of events. Those are the only data, which it sends and they are used to notify the server about client’s activity. Events are divided into events sent in standard communication and special events.
4.7.1 Events Sent in Standard Communication
They contain values specified in the behavior tag of each element, including implicit properties of an event, which triggered them. Those are created dynamically during runtime by the EventWriter class, which inherits from PropertyWriter.
4.7.2 Special Events
These events are used only in specific situations and special rules apply to them. They cannot be found in any behavior tag and are used only in special situations and connecting or disconnecting. They must be strictly written in UIProtocol version 1.0. The version is specified in the UIDocument tag of every UIDocument XML. Additionally, they may contain information about the client.
They are not created dynamically, because they are different from other events and not often used. Every time when they have to be sent, they are loaded from the filesystem. Currently, the UIProtocol Client for Apple iPhone sends only the public.connection.connect event. It is expected to add support for more special events in future to notify server about errors and other events.
4.8 Property Containers Design
Design of property containers may be different from other implementations. The element properties are not directly attached to element, but they are in Property container for easier access.
Figure 4.8: Types of Property Containers
35
The Property container is the default container, which is used to hold element properties. For special types of properties, such as style, position, behavior or model, specialized containers are used.
4.9 Binding Design
Binding is a powerful mechanism, which allows to bind an element, style, position or any other object configurable by properties to a single model. Binding both saves time to the UI developer and lowers the amount of data needed to be transferred through the network. It can be also specified for each individual property, which has higher priority than model wide binding.
The client should be able to support this feature by ability to create a model. Every element can be then bound to the model specified in its model attribute. The element properties are then overridden by values of corresponding properties in the referenced model. Property binding works similar way.
Because certain properties can be used only in the style, position, or other tags in the UIProtocol document, it is necessary to determine their type. Every property has a type assigned to it, so that only property of the appropriate type can be bound to the specific tag. Types are listed in Figure 4.9.
Property Can Be Bound To Description
PropertyType.ELEMENT element properties, which belong to element tag
PropertyType.STYLE style properties, which belong to style tag
PropertyType.POSITION position properties, which belong to position tag
PropertyType.MODEL - property type has to be resolved
PropertyType.UNKNOWN - property cannot be used in binding
Figure 4.9: Property Types
Model properties were designed to support binding. They share the same interface (IProperty) with other properties, which can be found inside other UIProtocol XML tags. Properties are updated using the IUpdatable interface, which is implemented by every Modifier and DisplayObjectWrapper.
Figure 4.10: Binding in UIProtocol Client for iPhone
36
Name and type of the property often have to be changed to match the UIProtocol property name and type of the referencing property. It is done by implicitly adding a PropertyModifier into the modifier chain. It acts just as any other modifier, but alters property attributes instead of value.
4.10 Configuration
UIProtocol is a subject to many changes, especially new components and events are being added to meet new requirements, while the UIProtocol document syntax remains same. Therefore, the configuration has to be loaded at runtime from an XML file, which reflects those changes. Also, some values, such as port numbers, are held in the configuration rather than directly in the code.
There are several configuration files, which can be accessed only from computer. When the application is compiled into an .IPA file for iPhone, they become part of that file. Each file contains specific configuration and it can be validated against XML schema, which is included with the project. The list of configuration files with their default paths can be seen in Figure 4.11.
Configuration Type Configuration Path Description
Main configuration configuration/files/mainconfig.xml Contains informations about UI and policy server hosts and port numbers, paths to other configuration files.
Default models configuration/files/defaultmodels.xml Contains paths to default models, which are loaded at application startup.
Default XMLs configuration/files/defaultxmls.xml Contains paths to default XMLs, which are loaded at application startup.
UI components configuration/files/uicomponents.xml Contains informations about corresponding Flash classes to UIProtocol classes and settings for overriding.
UI modifiers configuration/files/uimodifiers.xml Contains informations about corresponding Flash classes to UIProtocol modifiers.
UI triggers configuration/files/uitriggers.xml Contains informations about corresponding Flash events to UIProtocol event triggers.
UI properties configuration/files/uiproperties.xml Specifies properties, which can b e u s e d o n ly i n c e r t a i n UIProtocol document tags.
Figure 4.11: Configuration Files and Their Paths
37
The configuration reader typically loads the configuration only once during startup and stores it into the Configuration class, which is used for storing configuration and quick access to its data.
4.11 Hotkeys
Hotkeys may not always be used on mobile devices the way they are used on computers and devices with full keyboard. There are different phone categories, some phones do not have full hardware keyboard, instead of which they have software keyboard, external keyboard or reduced keyboard. It also depends on the used API and the specific platform, whether it allows access to all hotkeys.
The iPhone OS platform has no support for hotkeys and therefore the feature was not considered in UIProtocol Client for Apple iPhone. It is possible to add it in later versions of the implementation, because the current Flash version is expected to be ran on many other devices, which have hotkeys.
38
Chapter 5
Implementation
ActionScript has some specific traits, which give new possibilities of implementing common routines. This part gives important answers and it should be read by future developers in order to understand the current implementation. Some hindrances had to be resolved as they occurred and therefore the solutions do not always have to be the best possible.
Also, the best common practices for ActionScript are were not always the most effective ones in terms of performance and memory efficiency for the UIProtocol Client for Apple iPhone because of specifics of development for mobile platforms. For instance, the ActionScript commonly used event bubbling model can be performance expensive, because the event is often sent to objects, which it was not intended for. More on optimizing for iPhone will be found later in this chapter.
5.1 Specifics of the Implementation
This implementation is the first implementation of the UIProtocol ever done in ActionScript and one of the first implementations ever done in ActionScript for iPhone. ActionScript is the common language for writing Flash applications. However, MXML language can also be used. ActionScript has proven to be very suitable for building of UIProtocol client, because it allows higher level of abstraction than other commonly used programming languages.
5.2 XML Handling
The default Flash API already contains a class, which represents XML. The class is called XML and it is a DOM model of the XML created from a string. The XML documents are parsed using E4X (ECMAScript for XML). The document is pre-loaded and a DOM is created in the memory. Therefore XML can be directly retrieved from received data.
The Flash platform is one of the few platforms, which do not have a native SAX parser and therefore it relies only on DOM parsing. During the implementation the default DOM parser had to be used, which was provided by the default XML class. The advantages were faster application development at low performance cost, as can be seen in chapter with platform comparison.
In future the parser can be switched to SAX by modifying the classes responsible for XML writing and reading. XMLs are already processed using an event based parsing model.
39
5.3 Connection Layer
The connection layer contains only one class, which establishes connection to the remote server and handles received data. It is a standard ActionScript implementation, which opens a client socket and attaches event handlers to it. The handlers convert the data to XML and do basic XML validation.
The above code snippet shows the initialization of connection to a server. The values host and port(4) are obtained from the configuration. After establishing connection, client is ready to receive data, which are passed as events to dataHandler(11), which does also validation(12).
5.3.1 XML Validation in Connection Layer
The class, which does the initial validation is XMLTools. It validates every data, which are received before converting them to XML. It also checks the XML header.
The validation consists of checking if:
• XML header is present
• XML header contains information about encoding
• root tag name is UIProtocol
• root tag contains version attribute
5.3.2 Policy XML
Another consideration was whether to perform retrieving a list of trusted domains from policy server. ActionScript typically uses the sandbox security model, in which the server during initial communication with the client specifies trusted domains by sending a policy file in XML format to the client. The following code snippet is example of a basic policy file, which allows client to accept data from any domain(5).
01 private static function connect(host:String, port:Number):void { 02 clientSocket = new XMLSocket(); 03 configureListeners(clientSocket); 04 clientSocket.connect(host, port); 05 } 06 07 private static function configureListeners(dispatcher:IEventDispatcher):void { 08 dispatcher.addEventListener(DataEvent.DATA, dataHandler); 09 } 10 11 private static function dataHandler(event:DataEvent):void { 12 xmlIn = XMLTools.validateXML(xmlIn, event.data); 13 if (xmlIn != null) { 14 UIClient.debug("XML is valid"); 15 UIDocumentReader.processDocument(xmlIn); 16 } else { 17 UIClient.debug("XML is not valid"); 18 } 19 }
40
The security model is supported by the client, but it is turned of by default due to performance reasons. The client used to perform slower with the security checking on and therefore if was turned off in the compiler settings. To turn it back on, the compiler flag -use-network must be set to false and the following line in class UIClientConnection must be uncommented:
5.4 Client Layer
The layer consists of multiple classes, which refer to each other, when performing their tasks. The classes can be divided into three groups according to their functionality:
• Writing and reading UIProtocol documents
• Storing data from UIProtocol documents
• Creating instances of UIProtocol document objects
The layer is also responsible for performing model updates and setting binding, which is done by multiple classes. In future those features should be separated into one module.
5.4.1 Data Binding
A property in UIProtocol can be either specified as a constant value, bound to a model or both. UIProtocol distinguishes between two kinds of binding. Either a property can be bound to a property of a referenced model, or a whole property container can be bound to a whole model.
UIProtocol client for iPhone supports both these approaches and allows referencing models up to one level in depth. The values may be modified by modifiers.
Two different kinds of properties are used in the implementation of UIProtocol Client for Apple iPhone. They both share the same IProperty interface.
• BindablePropertyObject - Is used for properties, which are only part of models and are bound to other properties. They fire an event each time their value is changed and notify their listeners.
• PropertyObject - Is used for simple properties, which are default part of any UIProtocol style, element or other tag containing properties.
01 <?xml version="1.0"?> 02 <!DOCTYPE cross-domain-policy 03 SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd"> 04 <cross-domain-policy> 05 <allow-access-from domain="*" /> 06 </cross-domain-policy>
01 //connect(ConfigurationReader.policyServerHost, ConfigurationReader.policyServerPort); //connect to policy server
41
The function update() is passed as a parameter of the setBinding() method of the model, which holds the property. Function update() is then passed as a listener to the properties of the model, which are going to be bound. If modifiers are specified, then they are added as a chain between the referenced and referencing property. If the property is changed in the model, a BindingUpdateEvent.UPDATE is fired, containing the new value of the property.
5.4.2 Models and Model Updates
Models are a commonly used practice in UIProtocol to use same properties on multiple objects and easily updating them. A model can be updated using one of two ways. A full model update alters all properties in the model with new ones. However, a partial update only changes the values of properties specified in the model update and leaves other properties intact. Partial updates have to be supported by every UIProtocol client.
The application currently supports partial model updates, because full model updates would require some bindings to be discarded and the client does not allow restoring original values yet.
Updates are handled by the ModelUpdate class, which represents the model in its object representation. It alters the property values inside the model. The updated properties send update events to their listeners.
5.4.3 Data Storage
Data are stored in the UIDocument class, which is used as a storage for interfaces, actions and models. Their object names are similar to those in UIProtocol document.
The UIDocument class provides searching for UIProtocol document objects and if no object is found, for instance an interface, it can request for it and call a display update, when its definition is found. The example code below shows, how it was implemented.
One of the features, which was implemented and is important for the client to work properly is handling missing definitions, which were received after the interface was displayed.
The above code snippet shows how the feature was implemented. If array of displayed interfaces contains the new interface definition(1), the interface is first added to the interface store(5) and display is updated(6).
01 if (displayedInterfaces.contains(iface.getId)) { 02 03 //reloads display with the missing interface 04 requestedInterfaces.removeAll(); 05 interfaces.push(iface); 06 UIComponentsContainer.updateDisplay(getInstance().findInterface(getRootInterface())); 07 return; 08 }
42
5.5 User Interface Layer
The top layer of the application is the layer, which displays UIs. The main class, which contains UI components as its children is UIComponentsContainer, which extends the default Flex container Canvas. It can contain any number of nested containers, which are called nested interfaces in UIProtocol. The tool allows custom interfaces creation and their easy reuse.
An interesting feature, which can be found in the User Interface Layer is UI component instantiation from the component class name in UIProtocol. The example below shows, how it was implemented.
The code snippet above is an example of how object instantiation works for some objects in UIProtocol Client for Apple iPhone by the previously mentioned technique. The class definition is obtained from its name, which is stored in the configuration file, as a class object(3), and it can be directly instantiated(4).
5.6 Application Performance
The application was implemented with performance as one of the priorities. It meets the requirement on time for loading interfaces under 0,5 seconds. It was achieved by effective communication with server, pre-fetching interfaces and actions and using event-based model, in which application is idle, when not accepting any interactions from user or server.
5.7 Platform Issues
During the implementation no serious platform issues have appeared. The platform required some special practices, which cannot be found in other implementations.
5.7.1 Creation of an MXML wrapper
In order to use classes from the mx.* namespace, which contains important parts of the Flex API, that are used as UI components, a wrapper written in MXML had to be created. The wrapper is also an entry point to the application.
01 private function createDisplayObject():DisplayObject { 02 03 var DisplayObjectClass:Class = getDefinitionByName(classConfiguration.getClassPath()) as Class; 04 return new DisplayObjectClass(); 05 06 }
01 <?xml version="1.0" encoding="utf-8"?> 02 <mx:Application name="App" xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute" applicationComplete="EntryPoint.main()"> 03 </mx:Application>
43
When the wrapper application is loaded, it calls the main() method of class EntryPoint(2), written in ActionScript, which is an entry point to the rest of the application.
5.7.2 Class Instantiation
A problem was experienced when calling the getDefinitionByName() function. It is a compiler error, in which the method cannot get the class name if the requested class constructor is not called elsewhere in the class. An unused method containing the constructor calls(2-5) had to be created.
5.7.3 Problem With Loading .FLV Videos
The application supports video playback using ExtendedVideoDisplay UI component, but the component does not allow to input relative path in the filesystem. Therefore, absolute path must be always inputed. Other UI components (images, etc.) do do not have this limitation.
5.8 iPhone Specific Issues
Adobe published special guidelines for the creation of iPhone apps in Flash.[3] When developing content for iPhone some practices have to be done differently, because the UI on iPhone is different from desktop platforms. The target was to create UIs, which have the same look and feel as the ones on iPhone. To achieve that following guidelines were considered during implementation.
5.8.1 Default Layout Should Respect Apple Layout Guidelines[4]
UIProtocol allows developers to choose default values of UI component style and size properties. If the property value is not specified in UIProtocol document a default value, respecting the iPhone guidelines, is set. The UIProtocol Client for Apple iPhone uses global default style.
Style Property Name Default Value Description
font.size 20 size of the font
font.color 0x4C566C color of the font
font.bold bold style of the font
font.name Helvetica name of the font
font.family Helvetica name of the font family
Figure 5.1: Default Style Values
01 public static function unused():void { 02 new Image(); 03 new Button(); 04 new TextArea(); 05 ... 06 }
44
5.8.2 Problems With Touch Interface
The application’s UI behavior was not yet tested on the iPhone, but the Flex UI components were not originally designed for any iPhone or other mobile device using the touch interface. Therefore, the behavior of such components will differ from the behavior of standard iPhone UI components on the iPhone.
It is a goal for future developers to make improvements to the existing components used in the application. Improvement of the UI components should be achieved by switching to the Flex Mobile framework when it will be released by the end of 2010.[16]
5.9 Coding Standard
Many developers developing Flash applications are not following any coding standard and therefore it makes the code more difficult to understand their code. It is caused by many changes to the programming language and the developers not being familiar about the coding standards.
The UIProtocol client is following the coding conventions for Flex defined by Adobe.[5] The same standard should be used, when continuing this project. However, some parts of the code may differ in details.
45
Chapter 6
Testing
An important part of every application’s development cycle is testing. The tests could not have been performed due to time restrictions. This part of the document will only give suggestions for future testing and give an overview about debugging the application.
Application has yet to be tested for internal errors using the white box testing method. It was tested only against the UIProtocol testing server, which could pass only certain interfaces to the application, which did not test all possible inputs.
6.1 Test Schedule
The application testing started at an early stage, when it had to be connected to a UIProtocol server. Because currently no server was available a server written in Java was developed as part of this project. It sends automated reply to every request from the client and therefore it is not intended for any other use than testing.
Throughout the development the testing server was being developed alongside with the client and the set of testing XMLs grew. The same server can be used to preform validation tests and other tests, which test the whole application.
6.2 Future Test Plan
Before the application can be released, it should be tested thoroughly. The tests can be done with use of the FlexUnit testing framework.
The parts, which will need the most testing are all factory classes, which create and set the properties of objects created from XML. It must be considered, that the data in XML may be missing or corrupt, because they are received form an external server. Also binding and model updates should be a subject to deeper testing, because they are important for the application to function properly and their implementation was very complex.
46
6.3 UIProtocol Testing Server
The server can only open connection to one UIProtocol client and communicate using predefined reply XMLs. When it receives an XML from the client, it looks for patterns specified in every reply XML and when it finds a matching pattern it sends the reply back to client.
It runs on two ports. Port 3332 is used for UIProtocol server by default and port 3333 is used to send the Flash policy file to client. The policy file is, however, no longer needed, because the client no longer requests for it.
For UI testing the Selenium SE tool can be used, which can be used to test the Flash UI components. For fully automated tests it is recommended to use an automated script in Selenium to test the UI and the UIProtocol testing server to test communication with server. The server provides good logging functionality:
The log contains important information for debugging. Every event is logged and currently the output is displayed only on console. It contains:
• Time of the event
• Name of the class, which logged the event
• Client’s local port number
• Additional informations
Advantages of the testing server are, that it can be configured dynamically during runtime and its overall simplicity. The testing server is expected to be further developed to evaluate data sent in events and offer sets of tests, which will be used to test the functionality of any UIProtocol client.
[19:25:42] PolicyServer (3333)listening [19:25:42] UIServer (3332)listening [19:25:46] UIServer (3332)client connection from 49879 [19:25:46] UIServerConnection (49879)init [19:25:46] UIServerConnection (49879)Sending:application.uip.xml, Description:Application model [19:25:46] UIServerConnection (49879)Data: <UIProtocol version="1.0">
47
Chapter 7
Conclusion
At this stage of development the initial requirements on the application are met and an implementation of UIProtocol client is now available and can be uploaded to any iPhone OS device. Now it is expected, that the application will bring improvements to user interface testing and other areas, where UIProtocol is used and it will continue evolving.
7.1 Project Overview
The project was a successful attempt to use the latest technology to create mobile applications using Adobe Flash. The Flash platform allowed to implement some of the advanced features of UIProtocol during time designated for the project.
At the time of the initial requirements the iPhone Packager for iPhone was yet to be released. Adobe released the iPhone Packager and its product Adobe CS5 just one month ago and the UIProtocol Client for Apple iPhone became available just shortly after that event.
Currently, there are only few hundreds applications written in Adobe Flash for the iPhone and the implementation is very innovative. Opposed to native apps for iPhone OS devices, the Flash applications can be deployed on almost any other platform supporting Adobe Flash.
Therefore, the project targeted not just iPhone, but secondarily most of current devices, including phones of different brands and made implementations of UIProtocol client available for them.
Also, at the time before the project started there was no tool for testing the client. The UIProtocol testing server was an important side product. It can be used now as a simple server for testing any UIProtocol client. It can be ran locally on the computer used for development of the client and it has great potential for debugging and testing.
7.2 Summary
The current implementation of UIProtocol client in Adobe Flash for the iPhone has all the functionality needed to be used as a fully functional client. It supports most of the advanced features, which UIProtocol brings. However, its main feature is its promising potential for future extension.
48
The UIProtocol Client for Apple iPhone will find its usage in many projects, which are required to run on various platforms without having to target each platform separately. There are many of such projects currently running, which rely on the use of UIProtocol.
It was expected, that the Flash implementation will be primarily for iPhone, but because of the recent change in Apple’s policy it might be used to target any other platform. The change in their policy does not have to be interpreted as a setback for the project, because most of other phone manufacturers have already opened their platforms to Flash.
Furthermore, the application can still be deployed locally using the ad-hoc method to any iPhone as it was intended to and it met also that criterium. It is now the first available implementation of UIProtocol for both iPhone OS and other mobile devices.
7.3 UIProtocol Client for Apple iPhone Implemented Features
The client was expected to achieve high performance and display basic interfaces with support for data binding and interaction with server. The current implementation can display almost any interface, including custom interfaces. Here is an overview of the client features:
• Rendering user interfaces, have similar look to native iPhone UI components - The interfaces, which are rendered have default iPhone properties, such as font color, font size set by default. Properties can be declared in the UIProtocol document in various ways.
• Effective communication with server - Application is able to communicate with the server using XML files in format, which is specified in the UIProtocol XML schema.
• Support for basic level user interfaces and possibility to flexibly add new components - Most of the basic UI components are already supported and new components can be added without recompiling. Also custom interfaces can be created by combining basic elements.
• Support for actions (client and server side) - Client supports action execution on both server and client side, multiple behaviors can be attached to an element, action can trigger multiple model updates.
• Support for models and model updates - Client supports models, which are used in UIProtocol to describe frequently repeated sets of properties and can be bound to multiple elements.
• Support for modifiers and possibility to flexibly add new modifiers - Modifier support is not required, but it allows to modify the value specified in model. New modifiers can be simply added.
• Reading configuration from file - All configuration and environment properties are read from configuration files, including definitions of UI components
• Requests for missing actions and interfaces - Client sends requests for missing actions or interfaces each time they are not found. When the definition is received it is able to display the missing interface or fire the missing action.
• Support for binding and model-wide binding - Properties defined in elements can be bound to properties of a model, or even the whole model. This allows properties of UI components to be updated dynamically when their model is updated.
49
• Support for undefined element - In situations, when no corresponding UI component is found, no UI component is displayed. However, the client first tries to find a similar component from the same namespace.
• Support for default container - Layouts other than absolute layout were not implemented, because their implementation would require a more complex client. However, the default container is displayed correctly
• Debugging - In the debugging mode developers can see the console output directly on the screen and in developer mode the configuration is updated without having to restart the application.
7.4 Future Development
The development so far focused mainly on providing the key functionality of the client, while some features were implemented less throughly or require to be improved, because they were not in the scope. The time for implementation did not allow some higher level features. These are the main areas, on which the future development should focus:
• Adding new UI components - The developers have a solid base, which allows them to quickly add new components. Therefore, it should not be a problem to add even the advanced UI components.
• Extending to other platforms - For instance, Google Android is one of the major platforms, which supports Adobe’s efforts to make Adobe Flash cross-platform.
• Improving the UIProtocol testing server - Some of the areas, in which it could be improved are displaying the data sent in events in console, logging into log files and providing sets of tests.
• Improving behaviors - Behaviors may be improved by adding functionality to bind them to a model, also configuration file for behavior triggers should include their implicit properties.
• Reloading the display - A better method of reloading the display should be used, when missing interface comes, than reloading it whole.
• Adding support for full model update and interpolations - Those are some of the advanced features of UIProtocol.
• Adding support for other layouts - Currently only the default absolute layout is supported.
Developers are encouraged to continue improving the UIProtocol Client for Apple iPhone in any of the ways mentioned in the beginning and throughout this document. It is considered very likely that soon the number of supported UI components will increase and the application used to display UIProtocol interfaces on most mobile devices.
50
Bibliography
[1] Slovacek, Vaclav. UIProtocol Specification draft 8
[2] Gerhat, Peter. Comparison of Platforms Used for Mobile Development
[3] Building ACTIONSCRIPT® 3.0 Applications with the Packager for iPhone Preview (http://help.adobe.com/en_US/as3/iphone/air_deviphoneapps.pdf)
[4] iPhone Human Interface Guidelines for Web Applications (http://developer.apple.com/safari/library/documentation/InternetWeb/Conceptual/iPhoneWebAppHIG/iPhoneWebAppHIG.pdf)
[5] Action Script coding standard (http://opensource.adobe.com/wiki/display/flexsdk/Coding+Conventions)
[6] Thoughts on Flash (http://www.apple.com/hotnews/thoughts-on-flash/)
[7] Loading external code to ActionScript (http://www.astahost.com/info.php/Loading-External-Actionscript_t14023.html)
[8] How to use string as constructor (http://www.ultrashock.com/forums/actionscript/as3-how-to-use-string-as-constructor-90756.html)
[9] Harold, ER, Processing XML with Java, 2002
[10] XML Parsers: DOM and SAX Put to the Test (http://www.devx.com/xml/Article/16922)
[11] The Flash Guys Blog: Compiling Flash to iPhone from the command line (http://www.theflashguy.co.uk/blog/?p=3)
[12] How to use Adobe’s iPhone Packager without an Apple Developer Subscription (http://www.instructables.com/id/How-to-use-Adobes-iPhone-Packager-without-an-Appl/)
[13] Sales statistics by Canalys for Q3 2009 (http://www.mobilephonedevelopment.com/archives/922)
[14] WebKit wiki (http://trac.webkit.org/wiki)
[15] Adobe Flash Player for mobile devices and PCs (http://www.adobe.com/aboutadobe/ pressroom/pressreleases/200910/100509AFPforMobileDevicesandPCs.html)
[16] Slider: Flex Mobile Framework (http://labs.adobe.com/technologies/flex/mobile/)
51
Attachment A
Glossary
ActionScript - Is a scripting language created by Adobe, which meets the ECMA script specification.
Binding - Often practiced in computer science, creating reference to another object or property, which changes frequently.
Cocoa Touch API - Is one of the APIs created by Apple, which is used for building user interfaces on Apple’s devices running the iPhone OS.
DOM parser (Document Object Model parser) - Is a commonly used API, which is used in applications to interpret XML, the XML model is created in computer memory.
ECMA Script - Is a commonly used, standardized scripting language standardized in ECMA-262 specification and ISO/IEC 16262.
Flex Framework - An SDK released by Adobe for development of rich internet applications, which can be used on any platform supporting Adobe Flash Player.
Flex Mobile Framework - Is a new lightweight version of the Flex framework, which was announced by Adobe, that will be targeted on mobile devices to meet their user interface standards.
Google Android - Is an open source operating system from Google, which runs on Linux kernel.
IPhone OS - Is the operating system used on Apple’s iPods and iPhones.
MXML (Macromedia XML) - Is a programming language based on XML, which was announced by Macromedia in 2004. Macromedia was later taken over by Adobe.
SAX parser (Simple Api for XML parser) - Is a common API used in applications to interpret XML, uses event model.
Thin Client - Is the type of a client program, which requires to be connected to the server to fulfill most of it tasks.
UI (User Interface) - Is the visible part of an application, which the user interacts with.
XML (eXtensible Markup Language) - Is a general-purpose specification for creating custom markup languages.
52
Attachment B
Screenshots
53
Figure B.1: Interface With Custom Styles - Each UI
component can be styled, styles affect its color, font and other
visual attributes.
Figure B.2: Interface Containing Nested Interfaces - Nested interfaces provide an option
for creating custom interfaces built from existing UI components.
Figure B.3: Interface With Video UI Component - The media.video component is an example of more complex components, which can be also
added in future.
Attachment C
User Manual
This is a brief reference for the user, who gets to use the application and developer, who wants to only to extend the application by adding new UI components, modifiers or other customizations without changing anything in the application code. The topics addressed here are installation, usage, customization and error recovery.
C.1 Compilation Manual
Application first has to be compiled to the .IPA format, which is used for native iPhone apps. It can be done by running the included script compile_for_iphone.sh. The packager for iPhone (pfi.jar) must be located in the same directory as the script. It can be find in the same folder as Adobe Flash CS5 in the PFI directory.
The example of the script above contains parameters, which can be set. Table x describes them.
Parameter Description
java -jar pfi.jar starts the iPhone packager
-package -target ipa-test type of the build (can be also ipa-app-store, ipa-ad-hoc, etc.)
-provisioning-profile ... path to provisioning profile
-storetype pkcs12 encryption type used
-keystore ... path to Apple Developer Certificate exported from keychain
UIClientForIPhone.ipa name of the iPhone app
test-app.xml path to app descriptor XML file
-C ... path to folder where the .SWF resides
_MXML.swf name of the .SWF
Figure C.1: Compilation for iPhone Parameters
#!/bin/bash java -jar pfi.jar -package -target ipa-test -provisioning-profile \ ... -storetype pkcs12 -keystore ... -storepass \ 1234 UIClientForIPhone.ipa test-app.xml -C ... _UIProtocolClientForIPhone.swf exit
54
C.2 Installation Manual
The application cannot be downloaded from Apple’s App Store and therefore it can be uploaded to the iPhone OS device only using iTunes. The method described[11] was not tested yet with UIProtocol for iPhone yet and it is just for reference.
1. Copy the compiled application into iTunes
2. Plug in iPhone or iPod Touch
3. ITunes will open or open it manually
4. In the left menu of iTunes select Applications
5. Copy the application to the connected device by dragging it
6. The application is now installed and can be opened on iPhone or iPod Touch
C.3 Normal System Usage
The application can be opened or closed as any other app for iPhone or iPod Touch, the usage is not defined and user may click on any UI element, which may result in action depending on the behavior defined for the UI component.
C.4 Unusual Events
Event Description
missing/incorrect icon The application has no icon defined, it can be defined in the app descriptor XML file.
strange UI behaior The UI components have different behavior from native iPhone UI components, the application uses Adobe Flash UI components.
Figure C.2: Unusual Events
C.5 Error Messages and Error Recovery
The error messages are displayed in the console or in the debug window in debugging mode. For debugging it is recommended to turn the debugging mode on. The mode cannot be accessed from iPhone.
Event Description
I/O error Application is unable to connect to server, check connection, if the server is running and the configuration file.
55
Event Description
XML is not valid Received XML is malformed, error should be in header or in root tag.
Undefined element The element defined in the received UIDocument is not supported and cannot be altered by any of the supported elements.
Figure C.3: Error Messages
C.6 Customizing
• UI components - are added in uicomponents.xml, which also specifies property mapping
Tag Attributes
class name - name in UIProtocol
path - path in ActionScript namespace, if no corresponding class exists a custom class has to be created
property uiName - name in UIProtocol
asName - property name in Acrionscript, “()” after the name means property is set by a function
defaultValue - value set by default
Figure C.4: UI Components Configuration File
• Modifiers - are added in uimodifiers.xml
Tag Attributes
class name - name in UIProtocol
path - path in ActionScript namespace, the class has to be created
Figure C.5:Modifiers Configuration File
• Events - are added into uitriggers.xml, for sending implicit properties a custom handler class must be created, because they have not been fully integrated into configuration.
Tag Attributes
trigger name - name in UIProtocol
path - name of the event in ActionScript
Figure C.6: Triggers Configuration File
56
• Position and style properties - are set in uiproperties.xml
Tag Attributes
class name - type of property (style|position)
path - empty
property uiName - name in UIProtocol
asName - name in ActionScript
defaultValue - value set by default
Figure C.7: Properties Configuration File
57
Attachment D
CD Contents
58
- compiled version- this document in .doc, .pages and .pdf- iPhone version and compilation script- application source!