newton technology journal: volume 2, number 4

7/28/2019 Newton Technology Journal: Volume 2, Number 4

1/24

Inside This Issue

continued on page 3 continued on page6

Volume II, Number 4 August 1996

gygyNewton TechnoloJ O U R N A L

NewtonScript Techniques Real World Newton

NewtonScript TechniquesslimPicker: A Slimmer listPicker Proto 1

Real World NewtonPracticum/powerPen: Four-Year Student

Software Teamsfor Newton 1

Desktop CommunicationTechniquesMini-Meta Data: Another wayto getinformation to your PC 7

Newton CommunicationsNewton Programming:CommunicationsOverview 14

Communications TechnologyNewton Programming:DILsOverview 19

Practicum/powerPen: Four-YearStudent Software

TeamsforNewtonby Jeffrey C. Schlimmer,Washington State Uni versity

New computer science graduatesfrom mostuniversitiesare not readyto go to work in

modern software companies. Assoftware

professionalsare quick to point out, these

studentshave onlylearned how to write 300-

line programsbythemselvesfrom scratch.

These programswere classassignmentsthat

focused on elegance rather than efficiencyor

maintainability. The specificationscame out of

thin air, the code waspoorlytested, the

interface wasa simple command line, and

there wasnever anyusersmanual. The

studentsdid not learn how to reuse existing

code or how to work in teams(that would be

cheating). Theydid not learn how to trade-off

featuresand deadlineswith customers.

Enough battering of the current university

system. That type of education providesbasicknowledge of data structuresand algorithms

essential for engineering software. Our idea is

to supplement the traditional, fundamental

coursework with a four-yearpractical

curriculum modeling the activitiesof a

software company. From the universitys

point of view, entering freshmen are enrolled

in a four-year, software engineering course

calledTeam-Oriented Software Practicum.

slimPicker: ASlimmer listPickerProtoby Jeremy Wyld and Maurice Sharp,Apple Computer, Inc.

The Newton 2.0 OS providesmanynewprototypesfor developersto use. One of the

popular onesisprotoListPicker. It wasdesigned to provide a generalized framework

and interface for presenting listsof choicesto

the user. Unfortunately, protoListPickeralso triesto do everything for the developer.

The result isa complex API, and overhead in

space and time that isnot necessaryin a lot

of cases. Thisarticle presentsa slimmer

picker.

THEDATAISTHEPICKER(LISTPICKER)Most of the overhead from listPicker isdue to

the waythat it handlesdata. The purpose of

listPicker isto displaylistsof data that the

user can select itemsfrom. The data itemscan

be elementsin an array, soup entries, or a mix

of both. To make the developers li fe easier,

listPicker requiressome understanding of

what the data isand how it isformatted. This

iswhere the pickerDef and nameRef structurescome from.

A nameRef isa genericdata wrapper. It can

be used to wrap an arrayelement or a soup

entry. All of the relevant data slotsare part of

the top level frame of the nameRef so that

listPicker doesnot need to modifythe actual

data referenced bythe nameRef. The pickerDef

isthe code object responsible for creating

and managing nameRefs.


2/24

August 1996 Newton Technology Journal

2

Theysaythat all good thingsmusteventuallycome to an end. I saythat all

good thingsmust eventuallyevolve into

better things. And so it iswith the

Newton Technology Journal. Like

everything and everyone at Apple

Computer, we are in the processof

evaluating programs, projects, and

organizationsand designing them to be

better, more efficient and, above all else,

significant in contributing to the success

of Appleskeytechnologiesand partners.

Thisisgreat newsfor Applesdeveloper

communityand the projects, programs

and publicationsthat feed the successof

thiscommunity. It isespeciallygood news

for Newton developers, who are finding

renewed commitment and excitement

around the Newton platform asa

technologycritical to Applesfuture

success. It isaswe have alwaysknown it

should be.

So, in the vein of making good things

better, the Newton SystemsGroup is

dedicating more resourcesto developer

information, education, and support. Along

with thiscomesnewpeople with fresh ideas

about improvement and expansion. While

we continuallyreceive feedback that the

Newton Technical Journalisa valuable

piece of your developer support portfolio,

we are looking to improve it and growit

into a publication that servesyour specific

needs. With thisgoal in mind, it iswith

great excitement and expectation for such

continued growth that I passthe editorialreinsover to a new managing editor, who

will bring to the publication fresh ideas,

new enthusiasm and a close tie to Newton

training and educational programs. All

these thingscombined will undoubtedly

result in a better, more efficient suite of

developer education products, including

an expandedNTJ. So, without further ado,

let me welcome Jennifer Dunvan asthe

Managing Editor of theNewton Technology

Journal.

Published by Apple Computer,Inc.

Jennifer D unvan Managing Editor

GerryKane Coordinating Editor,Technical Content

Gabriel Acosta-Lopez Coordinating Editor, DTS and

Training Content

Technical Peer Review Board

J. Christopher Bell,Bob Ebert, D avid Fedor,

Ryan Robertson,Jim Schram, M aurice Sharp,

Bruce Thompson

Contributors

Ryan Robertson, JeffreyC . Schlimmer, JeremyW yld

and Maurice Sharp,

Produced by Xplain Corporation

N eil T icktin Publisher

John Kawakami Editorial Assistant

M att Neuburg Editorial Assistant

Judith Chaplin Senior Art Director

1996 Apple Computer,Inc.,1 Infinite Loop,C upertino,C A95014,408-996-1010. A ll rightsreserved.

Apple, the Apple logo, APD A, AppleDesign, AppleLink,AppleShare, Apple SuperDrive, AppleTalk, HyperCard,LaserWriter,Light Bulb Logo,Mac,M acApp,Macintosh,MacintoshQ uadra, MPW, N ewton, N ewton Toolkit, N ewtonScript,Performa, Q uickT ime, StyleW riter and WorldScript aretrademarks of Apple C omputer, Inc., registered in the U.S. and

other countries. AO CE, AppleScript, AppleSearch, ColorSync,develop, eWorld, Finder, O penD oc, Power M acintosh,Q uickD raw, SNA ps, StarCore, and Sound Manager aretrademarks,and AC O T isa service mark of Apple Computer, Inc.Motorola and Marco are registered trademarksof Motorola, Inc.N uBus is a trademark of Texas Instruments. PowerPC is atrademark of International BusinessM achinesCorporation, usedunder license therefrom. W indowsis a trademark of M icrosoftCorporation and SoftWindowsisa trademark used under licenseby Insignia from Microsoft Corporation. UN IX is a registeredtrademark of UN IX System Laboratories, Inc. CompuServe,Pocket Q uicken byIntuit,C IS Retriever byBlackLabs,PowerFormsbySestra, Inc.,ACT! bySymantec,Berlitz,and all other trademarksare the propertyof their respective owners.

Mention of products in this publication is for informationalpurposes only and constitutes neither an endorsement nor arecommendation.A ll product specificationsand descriptionswere

supplied by the respective vendor or supplier. Apple assumesnoresponsibility with regard to the selection, performance, or use ofthe products listed in this publication. A ll understandings,agreements,or warrantiestake place directlybetween the vendorsand prospective users. Limitation of liability: Apple makes nowarrantieswith respect to the contentsof productslisted in thispublication or of the completenessor accuracyof thispublication.Apple specifically disclaims all warranties, express or implied,including, but not limited to, the implied warranties ofmerchantabilityand fitnessfor a particular purpose.

gygyNewton TechnoloJ O U R N A L

Volume II, Number 4 August 1996 Letter From the Editorby Lee Dorsey and Jenni fer Dunvan

Editors Note

continued on page 22


3/24

Newton Technology Journal August 1996

3

The pickerDef and nameRef structure providesflexibility, but data that

isonlyarraybased or onlysoup based paysthe overhead for

representing the mixed array/soup data. Everyline of data displayed inthe listPicker requiresa nameRef structure which requiresheap space.

Each nameRef created requiresseveral levelsof function callsto the

pickerDef object. Each accessto a nameRef requiresseveral levelsof

functionscalls. Thisistrue even for a simple accesssuch ascomparison.

Caching the data in the nameRef can sometimesavoid the calling

overhead, but it costsheap space.

The pickerDef/nameRef abstraction doesprovide some benefits. In

addition to the obviousmixing of arrayand soup items, it also enables

listPicker to render the individual data displaylineswith little developer

intervention. It also makesfiling easy. On the downside, the

representation isquite brittle. The seeminglysimple task of using an

icon asthe first item in the rendered line of data isactuallyverydifficult

to implement.

A hidden cost isthe complexityof learning to use listPicker. To

create a simple picker that displaysdeveloper data requires

understanding protoListPicker, the pickerDef object ( such as

protoNameRefDataDef) and the nameRef wrapper. It also requires

learning how these three entitiesinteract. There isno clear delineation

of data manipulation and displaycharacteristics. A good example of

thisissingle selection, which isa characteristic of the pickerDef not the

listPicker.

Another hidden cost isthe cursor used bythe listPicker. In order

to handle both array-and soup-based data, the listPicker must

implement a pseudo-cursor that can wrap both typesof data.

Unfortunately, the detailsof the implementation are hidden. Thismeans

that i t isverydifficult to use listPicker on data that isrepresented

acrossmultiple soups.

If you are displaying listsof names, or if your data can be both array

and soup based, listPicker providesa nice proto for you to use.

However, for most developers, all theywant isto displaysoup-based

data in a list. Enter slimPicker...

YOUDATA, MEPICKER(SLIMPICKER)When we set out to design slimPicker, we set four goals(well five, see

below):

1. Provide the listPicker look and feel.

2. Minimize space and time costs.

3. Make it easyfor the developer to understand and use.4. Allowthe developer to customize it with minimal effort.

Originally, we had a fifth goal of keeping the same API aslistPicker. We

hoped that a developer could just substitute slimPicker for listPicker

and everything would work. However, we dropped that requirement

after finding that supporting the API required verysimilar overhead to

listPicker. Most of the overhead came from an iterator that could work

with both soup cursor and arrays.

Once we dropped the requirement for supporting the API, we also

dropped the pickerDef and nameRef structures. Thiscontributesto all

four goals. Most of the overhead of listPicker isin the pickerDef/nameRef

call chains, asismost of the complexityand brittlenessof

implementation.The rest of thissection givessome examplesof how the goals

effected the design and implementation.

continued from page 1

slimPicker: A Slimmer listPicker Proto


4/24


4

Look and Feel

The look and feel waseasyto do. We examined listPicker to see

which individual elementswere used in itsconstruction. We had the

source code, but you could do a similar thing using DV in the

inspector. We used the same elementsin slimPicker with onlyone

notable exception: instead of using aprotoStaticTextfor theselection counter, we draw it. Thissavesa bit of time and space

(though not much).The main part of slimPicker isimplemented using a protoOverview

because it can be used with either array-or soup-based data. It

requiresa cursor (or cursor-like) object for iterating over the data. The

other nice feature of protoOverview isthat it handlesthe selection

check boxes.

The downside isthat there isno supported wayto cache the

shapesused for each data displayline. Thisresultsin a time penalty

everytime the displaylist of data linesisrebuilt. Unfortunately, this

occursanytime you update the visual displaylist ( that is, scrolling and

the RefreshPicker call) . Luckily, slimPicker doesthisfaster than listPicker.

If slimPicker were built into ROM, we could overcome thisdifficultyby

using undocumented features. However, we did not use these features

because theyare likelyto change in future Newton OS devices.

Space and Time

The main performance gainscome from making the developers

responsible for their data. There isno pickerDef or nameRef structure

to provide a generic data wrapper. Instead there isa well defined

interface between slimPicker and the data. The developersare

responsible for rendering the shape that isa line of data. Theyare also

responsible for providing the cursor structure used byprotoOverview,

handling verification, creating new items(including anyediting slips)

and maintaining a list of the current selections.

Note that eliminating the pickerDef also eliminatesthe popup and

validation overhead. In listPicker, the onlywayto determine if a given

item in a column requiresa popup character isto build the popup. That

meanseach line of data built bylistPicker requiresa call to MakePopup in

the pickerDef. In slimPicker, if a popup isrequired, the developer renders

the popchar into the displayline for that data. Theyalso need to detect

if a hit to that line isin the popable item, pop the correct picker and

handle the result.

Although it seemslike we have added more work for the developer, it

isactuallyeasier to implement a simple soup-based slimPicker than an

equivalent listPicker.

In essence, we speeded up slimPicker and reduced the space by

removing the data abstraction layer. Thisdoesincrease the work a

developer must do, but it also removesmost of the overhead aswell as

reducing the complexityof the proto.

Easy to Understand and Use

In addition to eliminating the pickerDef/nameRef/listPicker

interactions, we also reduced the overall number of slotsand methods

that a developer needsto learn. Asan example, all of the listPicker

suppress settingsare now in one bit field called

visibleChildrenFlags.Another good example isadding new items. In listPicker you had to

enable the New button and then provide several methodsin your

pickerDef. In addition, the callbacksfor adding the new item are sent to

the pickerDef context, not to the listPicker. In slimPicker, you enable the

New button and provide one method calledCreateNewItem. On thedownside, slimPicker will not do anywork such asbringing up a slip or

calling back when the slip isdismissed. Instead, CreateNewItem iscalled when the New button ispushed. Everything else isup to you.

The change that providesthe most flexibilityisletting you render a

line of data. You provide anAbstractmethod that isgiven the data

item and a bounding box and returnsa shape that representsthe dataitem. That meansyou can put anything in that shape that isrequired,

including iconsand columns. In listPicker, adding an icon wasdifficult

at best. In slimPicker, just add it to the shape for your data line.

slimPicker also providesanAlphaCharactercall that letsyoureturn the character used for sorting a particular item of data. In

listPicker you would have to provide at least one method (possibly

two) and set up the column description. If the first displayed column of

data wasnot the one used for sorting, there are additional methods

and slotsthat must be provided. Thismeansthingslike displaying

iconsin the first column isverydifficult. For slimPicker, you can display

what you wish, and control the order with AlphaCharacter.A downside of slimPicker isthat the developer isresponsible for

tracking selection. The developer needsto provide theIsSelectedandSelectItemmethodsthat do the right thing. Each change inselected state will require an update in the visual displayof the data

lines, which meansredoing the children of the overview (that is, a call

toRefreshPicker ).Single select isrelativelyeasyto implement, use a single slot to

represent the selected item and refresh the picker. Your SelectItemmethod will replace the value of the slot and your IsSelectedmethodwill onlyreturn true if the entrypassed in matchesthe one in the slot.

The rest ishandled byslimPicker.

Easy to Customize

Even though single selection isprovided bylistPicker (through the

pickerDef!) , it providesa nice example of howslimPicker iseasyto

customize. It also pointsout that, once again, eliminating the

pickerDef/nameRef representation wasa good decision.

Perhapsthe biggest area of customization in slimPicker isthe ability

to change the data typesand representation on the fly. In listPicker it is

not possible to change the pickerDef or modifythe cursor that access

the data. The onlywayto do that isto close and open the listPicker.

With slimPicker you have complete control over how the data is

accessed (cursor) and how it isrepresented (Abstract). All that isrequired isa call toRefreshPicker, and everything will update.

Another example isfiling. To add filing you just add the support you

normallywould in an application. That is, activate the folder tab (set

the appropriate flag invisibleChildrenFlags) , then provide thestandard system API for filing (appAll, NewFilingFilter, etc.) . Whenthe filter changes, you can change your cursor and call

RefreshPicker.

SIXOFONE,ADOZENOFANOTHERThissection givesyou some comparisonsbetween listPicker and

slimPicker. To be candid, the testsare stacked in favor of slimPicker.

Theyare based on simple soup-based viewing.


5/24


5

All the testswere performed on the same MessagePad. MP 120,

running Newton OS 2.0 and having 79 entriesin Names. The listPicker-

based FAX picker wasopened from faxing a note and choosing Other

Names from the Name picker. Heap space wasmeasured before and

after the picker wasopened using HeapShow in the accurate setting

with no timed updates. Timing started when the picker including

Other Names wasclosed and stopped when the FAX picker was

opened.For the listPicker based People picker, we used the PeoplePicker-1

sample from Newton Developer Technical Support. Heap usage was

measured using HeapShow. Timing started after the pen wasreleased

on the icon in the extrasdrawer and ended when the people picker

appeared and wasreadyfor input.

The slimPicker measurementswere done on the protoSlimFaxPicker-1

and protoSlimPeoplePicker-1 code samples. These will be on Newton

Developer CD # 10 and on the web at

http://dev.info.apple.com/newton/techinfo/slimPicker.html.

listPicker slimPicker

Heap Used FAX picker 9300 3856

in bytes People picker 8496 2740

Time to Open FAX picker 5 3

in seconds People picker 3 2

Asyou can see from the table, slimPicker ismore efficient than

listPicker in all cases. For heap usage thisisnot reallysurprising:

listPicker hasthe overhead of nameRefs, a more complex selection

tracking, extra cursorsand a pickerDef. And slimPicker also hasminimal

extra information.

Time to open isa bit different. Both listPicker and slimPicker use

protoOverview, but listPicker also hasto construct the soup/array

iterator and the nameRefs. In addition, each data accesshasthe

overhead of calling through the pickerDef object. slimPicker can just

iterate over the visible data and call Abstracton each entry. There isverylittle housekeeping overhead.

Unfortunately, slimPicker isstill fairlyslow to launch. A quick profile

of slimPicker showsthat about 10% of the opening time isspent in the

Abstractmethod of protoOverview. The other major piece isprobablysoup access. Thismeansthere isno effective wayto speed up the

code. The usual idea of caching isnot useful since the slownessispart

of the opening process. If the slownesswasin scrolling, caching might

makessense, but of course that would increase the memoryfootprint.

One measure that isharder to estimate isthe size of the object. We

can get a good measure of slimPicker byeither looking at the size of the

package on the desktop, or byusing TrueSize on the MessagePad.

There isno similar wayto find the size of listPicker.

APIThissection presentsthe API to slimPicker. Of course, you will have all

of the source code so you could modifyanything. But just in case this

code showsup in ROM someday, stick to the API.

Slots

cursor

Thisslot isrequired.

The iterator for the data displayed bythe slimPicker. Can be either a

soup cursor or a developer-defined object that implementsthe

methodsrequired bythe cursor slot of protoOverview. See the Newton

ProgrammersGuide or Newton Developer Technical Support Q&A for

more information on the protoOverview cursor structure.

folderTabTitle

The text to put into the protoFolderTab. Thisisused to identifythe

slimPicker that isopen. The default isNIL (i.e., no title).

You can use SetValue to change the value at runtime.

reviewSelections

If true, the slimPicker will onlydisplaythe selected items. If NIL, all items

will be displayed. Correspondsto the Selected Only checkbox in the

user interface of the slimPicker. The default isNIL.

You can use SetValue to change the value at runtime.

viewLineSpacing

An integer representing the height of each line of data in the slimPicker.

Thisvalue must be at least the height of the checkbox. The default is

14.

visibleChildrenFlags

Bit flagsidentifying which child viewsare to be visible. The valuesare:

Constant Value Shows/HidesvNewButton (1< < 0) New button for adding new dataitems

vScrollers (1< < 1) Scrollersfor scrolling the listvAZTabs (1< < 2) AZTabsfor alphabetical navigationvFolderTab (1< < 3) Folder tab for filingvSelectionOnly (1< < 4) Selection OnlycheckboxvCloseBox (1< < 5) Close box for closing the slimPickervCounter (1< < 6) Count of selected items

The default isall viewsvisible.

Methods

slimPicker:Abstract(entry, bounds)

Thismethod isrequired.

Returnsthe shape that representsthe given entryin the slimPicker. The

shape must not be larger than bounds. Thisiswhere the developer

rendersan individual line of data. The returned shape must be one that

DrawShape can use.

continued on page 22


6/24

From the studentspoint of view, theyhave joined a software company

calledpowerPen. The combination of fundamental and practical works

well because studentsbecome highlymotivated. For example, theywantto learn in their English course because theywant to write effective

marketing material or a clear usersmanual. Theywant to understand

compilersbecause theywant to add an authoring language to their

application.

To be more specific, Practicum isroughlydivided into two stages. In

the first two years, student teamsstudyactivitiesthat surround

programming in a software company, i.e., marketing, testing, project

management, user documentation, and technical support. Freshmen

and sophomorescan tackle these topicsbefore theybecome effective

programmers. Each semester the team of studentsstudiesa text or two

on the topic and completesa related project for a regional software

company, sometimesunder nondisclosure. In return, the company

provideswritten feedback on the student work. ( We have found that

studentstake constructive criticism from future employersmuch more

seriouslythan from professors.) The goal of thisfirst stage isto develop

basic skillsin varioussoftware development tasks. We want them to

walk in the shoes of othersthat theywill be working with when they

become software developers.

In the second stage of Practicum, studentsare reorganized into

product groupsto build and support commercial-qualityapplications.

Studentsin these product groupsdesign, program, market, and

support either a new version of an existing application or an initial

version of a new application. Thisprocessincludesthe following

steps: a user survey, a marketing requirementsdocument (often done

in conjunction with a freshman team), a design specification, a

program, testing (with a freshman team), a usersmanual (with a

sophomore team), and a product release. The goal of thissecond stage

isto build experience while developing complete, high-quality

applications. We want them to see the whole software processin

action, and occasionally, make mistakesbefore jobsare riding on their

decisions.

After a yearsexperience, we found that studentshad trouble applying

themselvesto their industryprojectsin the first two yearsbecause we did

not have a common technologybase. Some of the studentsknewonly

Windowsprogramming (and had PCs), some onlyMac. To remedythis, we

adopted Newton asour common technologywith the help of Apple in

early1994. Apple donated a MessagePad for each of our studentsand

helped usset up a senior-level course in mobile computing that focuseson Newton programming. The universitypurchased several Macsto host

the Newton Toolkit and allocated space for a development lab. Students

in Practicum/powerPen take the mobile computing course and gain

common, essential skillsfor building modern, graphical user-interface

applications. Newton hasturned out to be an excellent choice because

of the huge potential in the hand-held market and the ease of developing

NewtonScript applications. It hasalso helped that the Newton developer

communityissmall and cooperative. Our freshmen and sophomore teams

have been able to do Newton marketing, testing, and documentation

projectsfor Newton companies. Theyhave also volunteered at Newton

conferencesand had summer internshipswith Newton companies.

Our first team started in 1992 with 10 freshmen. A second team

followed the next year, and a third in 1995. Currently17 studentsareinvolved in Practicum/powerPen: 5 seniors, 6 juniors, and 6 freshmen.

Like manycompanies, we experience regular turnover, losing about 50%

of the new studentsin the first year and another 25% of the overall

group upon graduation. The studentsare currentlysupporting six

Newton applicationsand developing a seventh. Each application is

available in at least one of six foreign languages. Theyhave over 500

registered usersin 25 countries.

Our most significant marketing channel isthe Internet, and in specific

the World-Wide Web. We host a seriesof pageswith the most current

version of each application. We distribute applicationsfor free because

we want to have asmanyusersaspossible. These pageslist known

defects, feature requests, source code, functional specs, and other

development information. Wemake our development information and

source freelyavailable because we want to help othersunderstand how

we are doing development including other studentsnot at our

university. For applicationswith a significant content market, the pages

also list content we and othershave developed (e.g., plug-in dictionaries

for hangMan or decksof cardsfor flashCard). Where possible, the pages

list related applicationscommerciallyavailable from Newton companies.

Practicum/powerPen also providesan excellent opportunityto study

and refine the software development process. Two areasin which we

hope to contribute are localization and design methods. In

localization, we have devised techniquesfor managing the user-

interface stringsof a Newton application so it can be easilyconverted

from one human language to another. Our latest effortsmake it

possible for a non-programmer (but language expert) to specify strings

byfilling out a Web form, submitting it, and receiving a compiled

Newton application correctlylocalized. Thisrepresentsa dramatic

savingscompared to Applesmarketing estimatesof US$10-25K to

convert a Newton application into each new language. We are inviting

our international usersto test thissystem. It will leverage their

expertise and provide additional international accessto Newton

applications.

In design methodswe have adapted a functional specification

technique for event-driven programming in general and Newton

programming in specific. Based on a table structure, thisspecification

type iseasyto code against and providesa ready-made test plan. Our

initial attemptswith thismethodologyare also available on our Webpages( under flashCard).

Asan educational project, Practicum/powerPen relieson industry

support in three major ways. First, Newton companiescan help by

making a tax-deductible donation to purchase new hardware. With

Appleshelp we have recentlyupgraded most of the studentsto Newton

2.0 OS but we will fall short when newfreshmen join Practicum/powerPen

next fall. Second, companiescan hire our studentsfor internshipsor

upon graduation. Besidesthe benefit of personal experience and

growth, returning studentsshare their expertise. Third, companiescan

propose and evaluate marketing, testing, or user documentation


Practicum/powerPen: Four-Year Student Software Teamsfor Newton

NTJ


6


7/24


7

Thisarticle describesthe development of a pair of applicationsthatexport data from a Newton device to a desktop computer. There are

two applications: one that runson a Newton device and one that runs

on a desktop computer. Theyare designed to allow a developer to

register data definitionsso that soup information can be transferred

between the Newton device and the desktop computer.

(Mini-MetaData isa Newton DTS sample that should be available by

presstime. You can find the Mini-MetaData source code on AppleLink

and the Newton WWWSite,

http://dev.info.apple.com/newton/newtondev.html. The next Newton

Developer CD will also contain thissample.)

YOUARETHECONNECTIONWith all the additionsto the Newton 2.0 OS, it mayseem like exporting

data to your desktop computer hasbeen overlooked: i t hasnot: the

intention isfor application developersto incorporate the Desktop

Integration Libraries(DI Ls) into their existing applications. This

approach will allow a user to directlyconnect to their Newton device

using their favorite desktop application. Before the DILsbecame

available, the user wasrequired to use a second application called

Newton Connection Kit to transfer their Newton devicesdata to a

more generic format that could then be used bydesktop

applications.

The Desktop Integration Librariesare a set of platform-independent

C librariesand APIsthat can be easilyincorporated into an existing

application. Theyprovide all the necessarysupport to connect to your

Newton and to transfer data between a Newton device and a desktop

computer.

There are currentlytwo typesof DILs: the communication DILs(CDILs),

and the frame DILs(FDILs). The communication DILsare used to open a

connection with the Newton and to read and write bytesof data. The

frame DILslet you read and write other Newton data types, such asframes

and arrays.

The two largest advantagesto using the DILsare:

1) The DILsabstract all underlying transport detailsinto an easytouse API.

2) Your code will run on Mac OS and Windows with veryli ttle

modification.

The implementation of the Newton application described in this

article isintended to be asgeneric and extensible aspossible and

allowsa developer to register information about how to format data

before it issent to the desktop computer. Bydoing this, the Newton

application can export data from manydifferent soupsusing many

different formats(thisformat will be explained below in further detail) .

For instance, you will be able to export your Namesfile to a tab-

delimited format that could be read into a database or a spreadsheet.

The desktop application will take the incoming data and dump it

into a text file. It should also be designed so that it will easilyport to

other platforms. Thismeansthat the user interface code will be

separated from the implementation code. Thisimplementation only

dealswith text data, so the FDI Lsare not needed.

Ill begin thisdiscussion bydescribing the protocol used for

transferring data between the Newton device and the desktop

computer. After that, Ill go into more detail of some of the major design

decisionsfor both the Newton application and the desktop

application.

YAKITYYAK, DOTALKBACKThe protocol used for sending and receiving data isfairlysimple. First I

will discussthe Newton side of the protocol.

At varioustimesduring the connection, the Newton will send one of

three things: a command code, a string length, or a string. The

command codeshave been purposefullyselected aslarge numbersso

asto avoid conflict with the string length. Table 1 summarizesthe

command codessent from the Newton during the protocol.

Table 1. Command Codes Used by the Newton Protocol

Command code name Command code value Description

kNewtonCancelled 0x0FFF Sent when the userpressesthe Cancel

button on the Newton.

kNewtonFinished 0x0FFE Sent when all of thedata

hasbeen transferred to

the desktop application.

Command codesare onlysent when the user iscanceling the

operation or the export hascompleted. The rest of the protocol on the

Newton side consistsof sending stringsto the desktop computer. Inour protocol, the length of the string issent first to let the desktop

application know how large the receiving buffer needsto be. Byusing

thistechnique, we guarantee that there will be no ambiguityasto

whether the received data isa command code or a string length.

Here isthe C function that readsdata from the CDIL pipe on the

desktop. It returnsa value indicating whether the read wasa success,

a failure, a cancel, or whether the Newton isreadyto disconnect.

long ReadBuffer( LPSTR bufferPtr, long* length ){

Boolean eom;CommErr anErr;

Mini-Meta Data: Another wayto get information to your PCby Ryan Robertson, Apple Computer, Inc.

Desktop Communication Techniques


8/24


8

long command;

// read the first four bytes, this will either be acommand code or astringlength*length = 4;anErr = CDPipeRead( gOurPipe, &command, length, &eom, 0, 0,

kPipeTimeout, 0, 0 );if (anErr) {

return kReadError;}

// interpret the command code and act on it. If the datawas not acommand code,

// then it is astringlength, so read in the stringif (command == kNewtonCancelled)

return kNewtonCancelled;else if (command == kNewtonFinished)

return kNewtonFinished;else if (command) {

*length = command;

// resize the buffer to the size of the stringplus one for the null character.if ( realloc( bufferPtr, command+1 ) ) {

anErr = CDPipeRead( gOurPipe, bufferPtr, length, &eom,0, 0, kPipeTimeout, 0, 0 );

if (anErr)return kReadError;

bufferPtr[(*length)] = (char)0;// Null terminate the stringreturn kReadSuccess;

}}

return kReadError;} // ReadBuffer

The desktop PC side of the protocol consistsof four command

codeswhich are summarized in Table 2.

Table 2. Command Codes Used by the Desktop PC Protocol

Command Command Descriptioncode name code valuekHelloCommand 0x0FFD Sent when the connectionhas

been established. Thistellsthe

Newton application to start the

protocol.

kGoCommand 0x0FFC Sent when the desktopapplication isreadyto start

receiving the export data.

kAckCommand 0x0FFB Sent after the desktop hassuccessfullyreceived a line of

data.

kErrorCommand 0x0FF9 Sent if there isan errorduring

the connection.

Because most of the data transfer consistsof the Newton device

sending data to the desktop machine, the Newton application uses

onlyone input specification for the entire protocol.

{form: 'number,

InputScript: func( ep, data, termination, options ) beginif data = kHelloCommand then begin // Hello command was received,

// start the protocol.ep:DoEvent( 'StartProtocol, nil );end else if data = kGoCommand then begin

// go command was received.// Initialize and output the first line

ep._parent.fStatusView:StopBarber();

local numEntries := ep.fCursor:CountEntries();ep:Parent().fStatusView:GoGoGadgetGauge(

numEntries, kSendingDataString );

ep:DoEvent( 'OutputData, nil );end else if data = kAckCommand then

// ack command was received,//output the next line

ep:DoEvent( 'OutputData, nil );else begin // There was an error, so disconnect

GetRoot():Notify(kNotifyAlert, kAppName, kProtocolErrorString );

ep:DoEvent( 'Cancel, nil );ep:DoEvent( 'Disconnect, nil );

end;

nil;end,

CompletionScript: func( ep, options, result ) beginep:DoEvent( 'Disconnect, nil );

end,}

If an unknown command code isreceived on the Newton device, the

Newton application signalsa cancel and disconnects. An unknown

command code islikelyto be caused bya communicationserror. If the

protocol were more robust, the Newton could tryto resync with the

desktop machine and start sending data again

PROTOCOLOFTHEWILDOnce the connection hasbeen established, kHelloCommand issentfrom the desktop PC to the Newton device. Seeing the

kHelloCommand, the Newton application will send the name of theapplication for verification purposes. Thisname ischecked on the

desktop PC to make sure the connection iswith the Mini-MetaData

application and not with the Newtonsbuilt-in Connection application

or with the Toolkit App.

The next step isfor the Newton application to send the name of the

file that data will be exported to. Once the desktop PC receivesthis

name, the standard save dialog will be opened with that file name asthe

default.

When the user finishesselecting the target file, the desktop

application will sendkGoCommand indicating it isreadyto beginreceiving data.

At thispoint, the Newton application beginssending data in the

following pairs: a string length followed bythe string. When the

desktop successfullyreceivesand writesthisstring to the file, it will

send ankAckCommand to the Newton to signal that i t isreadyformore data.

Finally, the Newton application sendskNewtonFinished when ithasfinished transferring data. It then disconnects.

If the desktop encountered an error during the protocol, it will sendkErrorCommand to the Newton and disconnect.

Here isan example of the protocol in action:


9/24


9

Figure 1. Newton-Desktop Communication Protocol

Now that you understand the protocol, letsdive into the code on

the Newton.

NEWTONSIDEUPTo extend the mini-meta data application, you will add a format frame to

a global registry. The format frame includessuch information aswhich

soup to send data from, what the queryspecification is, and how to

create a formatted string from a soup entry. Thisregistrywill be

discussed in more detail below.

The Newton application handlesthe format information and

providesa simple interface for selecting which format to use. To keep

the implementation asgeneric aspossible, a form of meta data was

created. Using thismeta data, a developer can have a maximum amount

of control over the format of outgoing information without explicitly

having to know much information about the Newton storage or

communicationssystems.

Here isa screen shot of what the Newton interface lookslike.

Figure 2. The Mini -MetaData User Interface

The NTK Project for the Newton application consistsof 10 files, 4 of

which are layout files.

Figure 3. The NTK Project Window for the Mini-MetaData Application

The important filesto look at are: Endpoint.t, StatusView.t,

ProtocolFSM, and Main.t.

The hierarchyof the Newton application isillustrated in Figure 4.

Figure 4. Hierarchy of the MiniMetaData Application

GOGOGADGETSTATUSVIEWIt isveryimportant to give the user feedback during the connection.

Newton 2.0 OS providesa terrificproto,protoStatusTemplate, for

conveying statusinformation to a user. StatusView.t containsthetemplate for the statusview that isused during the connection. One of

the beautiesof usingprotoStatusView isthat it hasmultiplepersonalities. Among other things, a view based on protoStatusView

can be a single line of text, a barber pole, or a gauge. During our

connection we will use all three of these.

The barber pole element isused during the connection phase. The

barber pole waschosen because the time it takesto connect isnot a

known value, and a simple line of text doesnt necessarilygive the user

the impression that a lengthyoperation istaking place. During the

connection phase, the user mayforget to signal a wait for connection


10/24


10

event on the desktop which leavesthe Newton waiting until the

connect request timesout.

The gauge element isused while data isbeing sent. Because we

know the number of itemsthat will be sent, a deterministic interface

element isa more appropriate choice here.

The simple statusview isused for disconnecting. A barber pole was

not used because the disconnect operation isusuallyveryfast. The

disconnect operation will also complete successfullyregardlessofwhether the desktop computer isdisconnecting.

The statusview template hasthree main methodsof interest. They

are: GoGoGadgetBarberPole, GoGoGadgetGauge, andGoGoGadgetSimpleStatus. Each of these methodswill set up thestatustemplate with the correct information and open it if necessary.

There are also some additional methodsfor updating the text, the

gauge, and the barber pole once the view hasalreadybeen opened.

BACKTOTHEBASICSThe mini-meta data application usesprotoBasicEndpoint astheprototype for the connection endpoint. UsingprotoEndpoint isnot recommended, and isactuallyimpossible to use in a 2.0 only

application. Thisnew endpoint proto ismuch more reliable and

functional thanprotoEndpoint.Endpoint.t containsthe template for our endpoint. In addition to

the standard endpoint methods, there isone other method of interest:

OutputLine. OutputLinecallsa helper function to format a soupentryinto an output string (thismethod will be discussed in more

detail later). It then outputsthat string and updatesthe statusview.

Here isthe definition of OutputLine:

func() beginlocal entry := fCursor:Entry();

// if there is an entry, then output the next line of data. Otherwise,// output kNewtonFinished and disconnect.

if entry then beginfData := :CreateStringFromEntry( entry, fMetaDataFrame );fCursor:Next();

// Output the length of the datathen output the data. If either// output fails then post acancel event.:Output( StrLen(fData), nil,

{async: true,form: 'number,CompletionScript: func( ep, options, result)begin

if NOT result thenep:Output( ep.fData, nil,

{async: true,form: 'string,CompletionScript: func(

ep, options, result)begin

if NOT result then begin

ep._parent.fStatusView:UpdateGauge();ep.fData := "";end elseep:DoEvent( 'Cancel, nil );

end,} );

elseep:DoEvent( 'Cancel, nil );

end,} );

end else begin// Output kNewtonFinished command and disconnect when the Output completes.:Output( kNewtonFinished, nil, {async: true,

form: 'number,CompletionScript: func(

ep, options, result )

beginep._parent.fStatusView:FinishGauge();ep:DoEvent( 'Disconnect, nil );

end;} );

end;

end

HOLYFINITESTATEMACHINESBATMAN!Using a deterministic finite-state machine for communicationswas

covered in depth in the April 1996 issue of NTJ(volume II, issue 2). This

application leveragesoff of the sample code produced for that article.

The file of interest isProtocolFSM which hasthe layout of all the states

and eventsneeded for our application.

There are three eventsworth pointing out. The first event isthe

Create event in the Genesisstate. Thisevent setsup the endpoint, the

statusview, and registersa power off function. Anyinitializations

needed for the connection should be done here.

Next we have the Connect Success event in the Connect state.

Thisevent setsthe input specification for our protocol, and also has

the definition of our input specification in thefInputSpecificationinstance variable. Thisevent isperformed once there hasbeen a

successful connection with the desktop computer.

Finally, we have the OutputData event in the Connected state.

Thisevent simplycallsthe endpointsOutputLinemethod describedabove. So whyisthisevent of interest to us? Another possible

implementation for outputting data would have been to call the

OutputLinemethod directlyfrom the input specification. Doing thiswould remove an event from the state machine, and make the code more

centralized. However, byplacing theOutputLinemethod in an event,canceling functionalityisprovided for free. When the finite state

machine receivesa cancel event, all posted communicationsrequests

will be canceled, including the input specification.

Byusing the finite state machine sample, the code ismore

understandable, and more modular. Thistype of modularityprovidesan

almost complete separation between the interface code and the

communicationscode. Having thisseparation will make future revisions

easier.

All communicationscode on the Newton side isasynchronous. This

decision wasmade because synchronouscommsare generallyevil.

When you post a synchronouscommsrequest on the Newton, an

additional task iscreated thatsNewton lingo for a new thread. This

addsneedlessoverhead to the system, and can potentiallyreveal some

interesting problems. For instance, you maybe outputting lotsof data

in a loop using synchronousoutput requests. Each time through the

loop a new task will be created, which isa rather expensive operation.

The newtask will take up system memory, and will not release controluntil i t returnsto the main event loop (which doesnot happen until

you are finished with your output loop) . Asa consequence, the

Newton will eventuallyrun out of system memoryand come crashing to

itsknees. Another drawback of using synchronouscommsisthat the

user losescontrol of their Newton while the commsrequest iswaiting

to complete.

GRANDCENTRALOur main layout file ismain.t. Thisfile containsthe code for selecting a

format, and creating an output string from a soup entry.


11/24

The important function to look at isCreateStringFromEntry.Thismethod iscalled repeatedlyduring the protocol. It ispassed a

soup entryand will return a string representation of that entry by

using the format frame. It iteratesover the field arrayin the format

frame, building a string from the elementsof that array.

func( entry, metaFrame )

beginlocal line, lineItem, result;

line := foreach lineItem in metaFrame.fields collect begin// build the itemstringfromthe metadataframe.

// if lineItemis apath expression, the resolve it and return the valueif ClassOf( lineItem ) = 'pathExpr ORClassOf( lineItem ) = 'symbol then begin

if entry.(lineItem) thenentry.(lineItem) & metaFrame.itemSeparator;

elsemetaFrame.emptySpace & metaFrame.itemSeparator;

end else if IsFunction(lineItem.format) ANDHasSlot( lineItem, 'pathExpr ) then begin

// if we have aformat function then pass in the value found using// the pathExpr slot to the function.result := call lineItem.format with (

entry.(lineItem.pathExpr) );

if result thenresult & metaFrame.itemSeparator;else

metaFrame.emptySpace & metaFrame.itemSeparator;end else if lineItem.format = 'quotedString AND

HasSlot( lineItem, 'pathExpr ) then begin// if format is quotedString, then quote the value found using// the pathExpr slot.result := result;if result then

$" & result & $" & metaFrame.itemSeparator;else

metaFrame.emptySpace & metaFrame.itemSeparator;end else if lineItem.format = 'quoteIfExists AND

HasSlot( lineItem, 'pathExpr ) ANDentry.(lineItem.pathExpr) then begin

// if format is quoteIfExists then quote if the value found using// the pathExpr slot exists$" & entry.(lineItem.pathExpr) & $" &

metaFrame.itemSeparator;end else

metaFrame.emptySpace & metaFrame.itemSeparator;end;

// return astringwith the proper line separatorreturn Stringer( line ) & metaFrame.lineSeparator;

end

DONTFORGETTHEDESKTOP

Asdiscussed earlier, the desktop application usesthe DILsto transfer

data between the Newton device and the output file. The requirements

of thisapplication were simple enough that onlythe CDILswere

needed.

To help in the effort to create crossplatform code, the project isbroken into two C files. There isa file for the main OS event handling

code and a file for the protocol code. Theyare Interface.c and

Protocol.c. The event code and the dialog code isnot crossplatform

because much of that code isspecific to either platform. The protocol

code iscrossplatform and consistsof the code to open the

connection with the Newton, handle the protocol, and close the

connection.

There are four functionsin Interface.c that are not used for handling

OS events. TheyareCreateNOpenFile, WriteToFile,UpdateNCloseFile, andInitializePipe. The first three are not in

Protocol.c because theycontain file accessroutinesthat are specific to

one platform. WhyInitializePipe isnot in Protocol.c isnot asobvious: the underlying transport optionsare specified slightly

differentlydepending on whether you are running on MacOS or on

Windows.

OS EVENTHANDLING

The interface code isin the Interface.c file if you are using MacOS and isin the INTERFAC.C file if you are using Windows. These filescontain all

the standard event handling code and should probabylook pretty

familiar. In addition to the above mentioned functions

(CreateNOpenFile, WriteToFile, UpdateNCloseFile, andInitializePipe) , the MacOS code containsone other function ofinterest: SetupPortMenu. Thisfunction correctlycreatesa list of theportsavailable on the given machine. For instance, most Macintoshs

have a printer and a modem port. However, if the user isrunning on a

Duo there isone printer/modem port.

PROTOCOL.CThisfile containsall the code necessaryto handle the protocol and the

variousstatesof the connection. It also containsthe code to handle

error reporting to the user. Most of the functionsand proceduresin

thisfile are easyto understand. However, there are a couple of areas

that warrant further discussion.

The procedure that handlesmost of the protocol isDoProtocol(),and isdefined asfollows:


11


12/24


12

void DoProtocol(){

StandardFileReply fileReply;short fileRef = 0;long length;char *bufferPtr = NULL;long fBufferResult;long anErr;

// preallocate abuffer that we think will be large enough for most data.// This buffer will be resized as datais received.

if ( !(bufferPtr = malloc( 256 )) ) {ConductErrorDialog( kNoMemoryString );return;

}

// Send kHelloCommand to the NewtonfBufferResult = WriteCommand( kHelloCommand );if (fBufferResult == kWriteError)

Fail(FailWrite);

// Make sure the we have connected to the Mini-MetaDataapp on the NewtonfBufferResult = ReadBuffer( bufferPtr, &length );

switch (fBufferResult) {case kReadSuccess:

if ( strcmp( kHeloResponse, bufferPtr ) ) {Fail(FailWrongApp);

}break;

case kNewtonCancelled:

Fail(NewtonCancelled);case kReadError:Fail(FailRead);

}

// Read the filename to save the incomingdatatofBufferResult = ReadBuffer( bufferPtr, &length );

switch (fBufferResult) {case kNewtonCancelled:

Fail(NewtonCancelled)case kReadError:Fail(FailRead);

}

// create and open the file, then start dumpingdatainto it.anErr = CreateNOpenFile( bufferPtr, &fileReply, &fileRef );if ( anErr == noErr ) {

fBufferResult = WriteCommand( kGoCommand );

if (fBufferResult == kWriteError) {UpdateNCloseFile( fileRef, &fileReply );Fail(FailWrite);

}

// Loop until there is either an error, or until the Newton sends acancel// command or afinished commandwhile( true ) {

CDIdle( gOurPipe );fBufferResult = ReadBuffer( bufferPtr, &length );

switch (fBufferResult) {case kReadSuccess:

anErr = WriteToFile( fileRef, &length, bufferPtr );

// if there was an error writingto the file, close the file, display// an error and return.if (anErr) {

Fail(FailWriteFile);

}// send an kAckCommand, if there was an error then handle it.fBufferResult = WriteCommand( kAckCommand );if (fBufferResult == kWriteError) {

UpdateNCloseFile( fileRef, &fileReply );Fail(FailWrite);

}break;

case kNewtonCancelled:UpdateNCloseFile( fileRef, &fileReply );Fail(NewtonCancelled);

case kNewtonFinished:ConductErrorDialog( kDownloadWasSuccessful );

UpdateNCloseFile( fileRef, &fileReply );free( bufferPtr );

bufferPtr = NULL;return;case kReadError:

UpdateNCloseFile( fileRef, &fileReply );Fail(FailRead);

}}

}

WriteCommand( kErrorCommand );free( bufferPtr );bufferPtr = NULL;return;

// These are the Goto locations that are jumped to usingthe Fail() macro.FailWrite:

WriteCommand( kErrorCommand );ConductErrorDialog( kBufferWriteErrorString );free( bufferPtr );bufferPtr = NULL;return;

FailRead:WriteCommand( kErrorCommand );

ConductErrorDialog( kBufferReadErrorString );free( bufferPtr );bufferPtr = NULL;return;

NewtonCancelled:

ConductErrorDialog( kNewtonCancelledString );free( bufferPtr );bufferPtr = NULL;return;

FailWriteFile:ConductErrorDialog( kFileWriteErrorString );WriteCommand( kErrorCommand );UpdateNCloseFile( fileRef, &fileReply );free( bufferPtr );

bufferPtr = NULL;return;

FailWrongApp:ConductErrorDialog( kWrongAppString );free( bufferPtr );

bufferPtr = NULL;return;

} // HandleProtocol

ReadBufferandWriteCommandare helper functionsused to readto and write from the CDIL pipe.

The return value ischecked to make sure there were no errorsin the

protocol. If an error occurred, it isassumed that there isa problem with

the connection, and the connection isaborted. In an effort to retain

some amount of synchronicity,kErrorCommand issent after an error hasoccurred. There isa good chance that thekErrorCommandmaynot besent because the original error wasa communicationserror. A more

robust protocol would examine the error value and take appropriate

action based on that value. It maybe possible to recover from the error

and continue receiving data.

REGISTERINGTHEMETADATATo extend the mini-meta data application, you will add a format frame to a

global registry. The symbol for thisregistryis

'|MiniMetaDataRegistry:DTS|. Here isan example of how you adda format frame:

local registry;if GlobalVarExists( '|MiniMetaDataRegistry:DTS| ) then

registry = GetGlobalVar( '|MiniMetaDataRegistry:DTS| );else

registry := DefGlobalVar(


13/24


13

EnsureInternal('|MiniMetaDatRegistry:DTS|),EnsureInternal([]) );

AddArraySlot( registry, myFormatFrame );

Here ishow you would remove your format from the registry:

if GlobalVarExists( '|MiniMetaDataRegistry:DTS| ) then begin

local registry = GetGlobalVar( '|MiniMetaDataRegistry:DTS| );local pos :=LSearch( registry, myFormatSym, 0, '|=|, 'symbol );

if pos thenRemoveSlot( registry, pos );

end;

Here are some examplesof what a format frame might look like:

{title: "Names File - First, Last",symbol: '|Format1:DTS|,soupName: "Names",lineSeparator: unicodeCR,itemSeparator: ",",emptySpace: " ",fields: ['name.first, 'name.last]}

{title: "Names File - First, Last, Address, Phone",symbol: '|Format2:DTS|,soupName: "Names",fileName: "Names Export",fields: ['name.first, 'name.last, {format: func(s)

if s then CapitalizeWords(s) else nil,pathexpr: 'address},

[pathexpr: 'phones, 0]]}

{title: "Names File - First, Last, City",symbol: '|Format3:DTS|,soupName: "Names",

itemSeparator: ",",lineSeparator: unicodeCR,emptySpace: " ",fields: ['name.first, 'name.last,

{format: 'quotedString, pathexpr: 'city}]}

Each MetaData frame must have the following slots: title, symbol,

fields, either GetSoupName or soupName, and either GetQuerySpec or

querySpec. It mayoptionallyhave a lineSeparator slot, emptySpace slot,and an itemSeparator slot.

Here isa more in-depth description of each slot:

titleThe name of thisparticular meta data frame. Thisisthe name that will

appear to the user in the list of installed meta data frames.

SymbolThisisa unique symbol used to identifythisparticular meta data frame.

If you register two meta data frameswith the same symbol, the second

one that isinstalled will overwrite the first one. Append your developer

signature to the symbol.

soupNameThe name of the soup to export data from, or nil.

GetSoupNameIf you cannot know the name of the soup at compile time, specifythis

slot instead of the soupName slot. GetSoupName will hold a function

NTJ

If you have an idea for an article

youd like to write

for Newton TechnologyJournal,

send it via Internet to: NEWTONDEV@ applelink.apple.com

or AppleLink: NEWTONDEV


14/24


14

The Newton operating system isdesigned with communicationsasanintegral part of the system. The pervasive approach isthat whatever

you can see in a Newton application, you can send. Part of this

approach isthe notion that, asmuch aspossible, the user will have a

verysimilar experience in sending data regardlessof the medium used

to send it.

From a programming point of view thingsare not quite so simple

but the architecture isdesigned in a layered wayso that little

programming isrequired unlessthe situation isfairlyunusual. In other

words, there isa great deal of built-in communicationssoftware in the

Newton which can be used to provide basic communications

functionalityfor almost anyprogram.

Figure 1 showsthe variouslayerscomprising the Newton

communicationssystem and the programming interfacesused to

accessthese elements. The rest of thisarticle givesbrief descriptionsof

these APIsaswell asproviding referencesto more detailsabout them.

Figure 1: Newton Communi cations Layers

The following isa brief description of the itemsshown in Figure 1

and their APIs.

ANewtonScript applicationusing the Routing API isthe simplestwayfor application programmersto provide communicationssupport

from a Newton device. Anyapplication written in NewtonScript can use

communicationsmoduleswhich have been installed asTransports. In

Newton 2.0 OS, thisincludesthe built-in transportsfor beaming, faxing,

mailing, and printing. To use the available transports, the Routing

application programming interface (API) isused to specifywhich data is

being routed, what form it takesand how it should appear ( for

example, print format) . The Newton 2.0 OS usesa store-and-forward

model for thiskind of communicationsand the In/Out boxesare where

incoming or outgoing data isstored in the routing model.

In/Out Box Application and Transport API. Built into the system isthe In/Out Box application which managesthe soupsused to store

incoming and outgoing data. Thisapplication communicateswith one

of several Transports, which consist of code provided to move the data

to or from the appropriate destination or source. While there are

several built-in transportsin the system, NewtonScript programmers

maywrite their own transport to provide system-wide data

management.

Endpoint API and Endpoint System. The Endpoint API isaNewtonScript interface for performing direct communicationswith the

outside world. Applicationsprogrammersmayadd endpoint code to

their programsto communicate directlywith an external source or

destination. An example of thismight be endpoint code which

communicatesdirectlywith a GPS device on demand. Transportshave

endpoint code to move the data theyreceive out to an external

destination or to receive data from an external source prior to passing

it to the In/Out Box Application.

Low-level Communication Tools. These toolsare implemented inC+ + and actuallycommunicate with the C+ + interfacesto the

hardware drivers. While these interfacesare not yet available, theywill

be published in the future.

Each of the APIswill be described in more detail in the remainder of

thisarticle.

ROUTING

The Newton OS isprovidesa store-and-forward model for

communicationsand usesthe In/Out Boxesasthe place where target

data isstored. The termstore and forwardmeansthat messagesarerouted (directed) to a distant communicationsdevice or from such a

device to a Newton application through an intermediate holding area

(the In/Out Boxes). Target data isanypiece of information which isbeing

routed in or out of the Newton.

The In/Out Boxesare actuallya single application which provides

the storage in the form of a soup, the functionalityof sending and

Newton Programming: CommunicationsOverviewNewton Developer Training

Newton Communications


15/24


15

receiving messages, and the interface that letsthe user look at pending

message and dispose of them ashe or she desires. Figure 2 shows

what the In/Out Boxesmight look like when there are messages

pending. Note that at anytime the user can switch from In Box to Out

Box and vice versa via the radio buttonsat the top of the view.

Figure 2. The In/Out Box User Interface

Routing isusuallytriggered byusing the Action button that is

displayed in the view from which something will be routed. The Action

button isdisplayed in the view asa pop-up which showsavailable user

actionsasillustrated in Figure 3. Some applicationswill have one

Action button in the statusbar, otherswill have one in each of several

views. The Namesapplication isan example of a single Action button

because normallyonlyone name at a time isviewed. The Notes

application hasan Action button attached to each note since there may

be manynoteson the screen at anygiven time.

The Action button iscreated on screen byadding the prototypeprotoActionButtonto the desired view.

Figure 3. The Action Button Popup

Each target object which isrouted must have a meaningful class.

For frames, thismeansthat the frame must have a classslot which

identifiesthe type of data associated with thiskind of object. Normally,

each application will supplyitsown classof data for routing, such as

'|myData:MYSIG|. Thisclassisused bythe system to look up in theData View Registrythe list of routing frameswhich maybe used to

route data of a specific class. From these routing frames, a list of

transportsor communication methods(for example, faxing, printing,

beaming) which can route the target data are supplied to the Action

button. The net result of thisisthat when the user tapson the Action

button a list of the destinationsappear which are appropriate for the

target data.

Figure 4 showshow thisisall interconnected. An application,

usuallyin itsInstallScript, will put one or more framesnamed for the

classesof data which it will route into the Data View Registry. These

Frameswill consist of one or more Routing Frameswhich describe what

format the target data can take when it isrouted. The system usesthis

to search the list of installed transportsand, when it findsa transport

which supportsone of the routingTypes, addsthe transport name to

the list to be displayed in the Action button.

So in the example shown in Figure 4, the application hasinstalled a

frame|forms:MYSIG| in the View Definition Registrywhich supports

dataTypesof 'view, 'frame and 'text. These are used to choose thetransportsfor printing, faxing, beaming and mailing so these appear in

the Action button the user haspressed. Note that it doesnt pick up

the compresstransport whose dataType is'binary.

When the user selectsa transport from the Action button, an

appropriate routing slip isdisplayed and all formatsthat in which the

data can be displayed are displayed in the format picker asshown in

Figure 5. Formatsdescribe how the target data should be organized

before sending it onward to the appropriate destination. For example,

when printing, there might be several formatssuch asletter, memo,

two-column, and so on, which describe how the target data will be


16/24


17/24

Transportsusuallywork with an application via the In/Out Box

application. When an application routesdata out to a transport, the

transport providesa routing slip and isnotified when the routed data

reachesthe Out Box. When itemsare sent, the transport will get the

data from the Out Box and do whatever isnecessaryto send the data,

setting statusinformation at appropriate stagesduring the transfer.

In the case of a request to receive data, the sequence isjust slightly

more complicated. In the simplest case, when the user selectsatransport from the Receive button list in the In Box, the selected

transport issent a request to receive data. The transport will then

connect to the remote source, get anypending data, and add it to the

In Box list.

There isalso an option for a transport to get information about the

data being routed from the remote source and post thisinformation

into the In Box without actuallygetting the data. Thisisuseful in a

situation such asa mail transport where the user often wantsto simply

get the titlesof pending messagesso he or she maychoose which

messagestheywant to download to the Newton device.

The main proto used to create a transport isprotoTransport.The powerful thing aboutprotoTransport isthat in manycases,surprisinglylittle code other than the actual endpoint code must be

written. Thisisbecause the transport defaultstypicallydo the right

thing to provide an interface and behavior for the transport. Only

those featuresspecific to the transport (for example, archive name for

an archive transport) must be added to the standard interface.

Transportswhich can send data also have their own routing slips

based on the prototypeprotoFullRouteSlip. Thisallowsuserstoprovide transport-specific optionssuch asaddressesin a particular

format, and so on.

In particular, such thingsasdisplaying the statusof a routing

request, logging of routed items, error handling, power-off handling,

and general user interfacesare handled well bythe defaultsif the

transport simplysetsor updatesa few slotswhen appropriate. Onlythe actual service code (such ascommunications) will differ from one

transport to another.

ENDPOINTS

Endpointsare the primaryNewtonScript API for programming

communicationson the Newton device. Theyprovide a virtual

pipeline for all communications. Theyare designed to hide most of

the specificsof a particular communicationsmedia and, once

connected, endpoint input and output code isusuallythe same

regardlessof the media being used.

Endpoint code to receive data from an AppleTalk network can be

identical to code to receive data through a modem, which can be

identical to code to receive data over a serial line, and so on. Such

thingsaspacketization which occursin anynetwork protocol are

hidden from the endpoint user during sending and receiving, asare

operationssuch asflow control, error recovery, and so on.

The onlyexceptionsto thisrule occur when there are specific

hardware limitationsthat push through the endpoint API . For example,

IR beaming isa half-duplex protocol ( it can onlybe in send mode or

receive mode, not both at the same time) while serial, AppleTalk, or

modem communicationsare all full-duplex (theycan be in send and

receive mode at the same time).

Of course, while sending and receiving are purposefullymedia-

independent, the connection processisnecessarilytied to the media

being used. So, for example, with AppleTalk it isnecessaryto specify

network addresses; for modem communications, a phone number; forserial communications, speed, parity, stop bits; and so on.

Figure 6 showsthe life cycle of an endpoint. An endpoint isinitially

defined asa frame protoed fromprotoBasicEndpoint. The framehasseveral slotsdescribing the settingsof the endpoint and methods

that maybe called bythe system during the course of itsexistence.

However, such a frame isnot an endpoint. That is, it describeswhat

an endpoint might look like, but it isnot a NewtonScript object. To

create such an object, it must fi rst be instantiated. Note that since

most objectsin the Newton OS are views, and since the view system

automaticallyinstantiatesa view object when it isopened, we usually


17


18/24


18

dont see thisstep. But since an endpoint isindependent of the view

system, we must explicitlyinstantiate it to create an endpoint object.

Figure 6. Life Cycle of an Endpoint

Once instantiated, an endpoint isopened bysending theOpen()message to it. Thistiesthe endpoint to a low-level communications

tool in the system and spawnsa new task at that lower level.

Once the endpoint isconnected, it mayneed to be bound to a

particular media-dependent address, node, and so on. An AppleTalk

endpoint, for example, isbound to a node on the network. Thisis

done bysending the endpoint theBind()message. Note that someprotocols(such asserial communications) do not have a required

binding phase but it isstill necessaryto call Bind() (and later,Unbind()).

After binding the endpoint, theConnect()message issent toconnect to the particular media being used. For a remote service that

isaccessed through a modem endpoint, the endpoint would dial the

service and establish the physical connection. Note that the endpoint

doesnot handle protocol itemssuch aslogging on, supplying

passwords, and so on; these are part of an ongoing dialog that the

application and the service must engage in once connection is

established.The endpoint methodListen()maybe used to establish a

connection instead of theConnect()method if the endpoint isinstantiated and readyto listen to an offer bythe remote source.

Based on the particular situation with the communicationsmedia, an

application mayeither reject the connection bysending the

Disconnect()message to the endpoint, or accept it with theAccept()message. (Note that since infrared connectionshave oneside sending and the other side receiving, in thiscase the passive side

connectsbycallingListen() instead ofConnect().)After connecting, the endpoint isreadyto send and receive data.

Sending isfairlystraightforward and isdone byusing the method

Output(). When sending data, information about the form of the data(such asthat it isa string, a NewtonScript frame, and so on) isusually

sent. Thisgivesthe system a description of how the data should be

formatted asit isbeing sent.

Output maybe made either synchronouslyor asynchronouslywith

asynchronouscallsrequiring that a callback method be specified.

Receiving data isa little more complex. Incoming data isbuffered by

the system below the application endpoint level. An application must

set up a description of when it wantsto get incoming data. This

description isin the form of aninputSpec. For example, aninputSpec could be created which looked for the string login:, or itcould be set to trigger when 200 characterswere received. To some

extent, it can be set to notifythe endpoint of incoming data after aNTJ

To send comments

or to make requestsfor articlesin Newton TechnologyJournal,

send mail via the Internet to:

NEWTONDEV@ applelink.apple.com


19/24


19

An important new addition to Newton communicationsprogrammingisa set of librariescalled the Desktop Integration Librariesor DILs. The

first and most important thing to understand about DILsisthat they

have nothing to do with programming communications on the Newton

device. Instead, theyare used to create a communicationslink between

Newton devicesanddesktopmachines. In other words, theyare an aid

for writing code for a desktop machine which will communicate with a

Newton.

Figure 1 showsthisrelationship. Essentially, endpoint code on the

Newton, whether part of an application or in a transport, transfersdata

between a Newton device and a desktop machine. On the desktop

machine, an application which usesa DIL sendsand receivesdata

which ishandled bya desktop application. CurrentlyDILsare available

for the MacOS and for Windows-based machines.

Figure 1: DILs and Newton Devices

The main advantage that DILsprovide isthat theymake it easier to

write code for a desktop machine which communicateswith a Newton

device. In particular, theyabstract the Newton connection to a virtual

pipe for bytesand provide control over such thingsasASCI I-to-

Unicode conversionsand Newton data structuresand typessuch as

framesand 30-bit integers.

Figure 2: Hierarchical Structure of DILs

Asshown in Figure 2 there are three DILswhich build on one another:

CDIL, FDIL and PDIL. The CommunicationsDIL (CDIL) providesbasic

connectivityto a Newton device and must be used to establish a

connection before you can use the FDIL and PDIL. The FramesDIL (FDIL)

providesa relativelysimple wayto map NewtonScript framesto C

structuresand also providesa mechanism to handle data which was

added dynamicallyto the frame. The Protocol DIL (PDIL) providesan easy

mechanism for synchronizing data between a Newton application and a

desktop application. At the time of writing, the PDIL isnot yet available

but will be available in the future.

All of the DILsare librarieswritten originallyin C+ + but called using

a C-like syntax with a magic cookie object token passed into the calls.

On the MacOS side, there are MPWand Metrowerkslibraries. On the

Windowsside, DILsare implemented asDLLsand so should be

independent of particular C language implementations.

CDILThe CDIL essentiallyhasthe following phases: initialization,

connecting, reading or writing, and disconnecting. Thisis

purposefullyverysimilar to the normal endpoint life cycle. The idea is

to create and open a virtual pipe to the Newton device and then

communicate using some user-defined protocol bysending and

receiving messagesor data down the pipe. Figure 3 showsthe normal

order of callsin using the CDIL.

CDInitCDIL()CDSetAppplication() // Windows onlyCDCreateCDILObject()CDPipeInit()CDPipeListen()CDPipeRead()/CDPipeWrite()CDPipeDisconnet()CDDisposeDILObject()CDDisposeCDIL()

Figure 3: CDIL Calls

TheCDInitCDIL() routine must be called before anything elsecan be done with the CDIL. On Windowsmachinesthe routine

CDSetAppplication()must be called next. There isno equivalentto thiscall on the MacOS.

Next, the routineCDCreateCDILObject()iscalled to create aCDIL pipe. It returnsa pointer to a pipe which must be used for all

subsequent callswhich involve that pipe.

CDPipeInit() initializesthe pipe so that it isopen for business.In particular, it setsthe communicationsoptionsincluding the media

detailssuch asmedia type (for example, serial, AppleTalk, and so on),

and relevant media options(for example, speed of connection, data

bits, modem type, and so on).

Next, the pipe usestheCDPipeListen()call to wait for a

Newton Programming: DILsOverviewNewton Developer Training

Communications Technology


20/24


20

connection from the Newton device. When the Newton device contacts

the desktop machine, the application using the CDIL mayaccept the

connection onceCDPipeListen()) returnsbycallingCDPipeAccept(). Thisallowsa connection to be canceled if, forexample, the application decidesthat the actual connect rate wastoo

slow. At anytime in thisprocess, the desktop application can cancel an

attempted connection bycallingCDPipeAbort().

Once a connection isestablished and working, streamsof bytescanbe sent and received using the routinesCDPipeRead()andCDPipeWrite(). Aswith most CDIL routines, these callsmayeither bemade synchronouslyor asynchronouslywith a callback routine.

From thispoint on, the desktop application and the Newton

application can engage in an application-specific protocol where there

will be an predictable exchange of messagesand data via the CDILs

virtual pipeline.

When the decision ismade to terminate the connection, the routine

CDPipeDisconnect()should be called. Once thisroutine hascompleted, connection hasbeen broken and both sidesmust re-

establish the connection before data can again be sent or received.

Finally, when the desktop application iscompletelyfinished with the

pipe, it must call the routinesCDDisposeDILObject() to tear downthe pseudo-object andCDDisposeCDIL() to close the CDILenvironment. On the MacOS, CDDisposeCDIL()closestheCommunicationsToolbox tool which wasopened, and on a Windows

machine it closesthe appropriate driver.

There are several other additional CDIL callswhich maybe of use to

the desktop programmer. These fall into four categories: encryption,

utilities, status, and miscellaneous.

There are two encryption routines: CDEncryptFunction()andCDDecryptFunction(). These passcallback routinesused to theCDIL. These callback routinesare called bythe CDIL libraryat the

appropriate time to encrypt or decrypt data passing through the pipe.

There isno attempt bythe CDIL to packetize data and so, if the

programmer isusing a unit-oriented encryption scheme (for example,

cipher block chaining), it isup to the application to buffer the

incoming or outgoing data until there isenough data to encrypt or

decrypt the block.

The utilityroutinesinclude such thingsasCDFlush()which isused to flush the contentsof the pipe in a given direction, CDIdle()which isused to call completion routinespassed into asynchronous

callsaswell aschecking on and updating the statusof the pipe, and

CDPipeAbort()which abortsanytransactionsin a given directionwhich are pending.

The statusroutinesreturn information about the pipe.

CDBytesInPipe()returnsthe number of bytescurrentlywaiting in a

given direction in a particular pipe. CDConnectionName()returnsthe name set byCDPipeInit(), CDGetConfigStr()returnsthemedia configuration string passed intoCDPipeInit(), andCDGetPortStr()which returnsthe name of the port the pipe isconnected to (for example, COM2 or Modem) .

CDGetPipeState() andCDSetPipeState() get and return thecurrent state of the pipe. Figure 3 showsa list of the possible statesof a

pipe.

kCDIL_UnitializedkCDIL_InvalidConnection

kCDIL_StartupkCDIL_ListeningkCDIL_ConnectPendingkCDIL_ConnectedkCDIL_BusykCDIL_AbortingkCDIL_DisconnectedkCDIL_Userstate

Figure 3: CDIL Pipe States


21/24


22/24


2222

NTJ

Wow, what an introduction! It will no doubt be an enormouschallenge to continue the tradition of service and technical excellencethat Lee hasestablished with the production of theNewton Technical

Journal. So with both enthusiasm and trepidation I accept the

challenge of managing and growing theNewton Technical Journalto

be an outstanding informational and technical reference for Newton

developers.

Asfor me, I have spent the past year in the Newton SystemsGroup

managing Newton developer training. Mycurrent major projectsare

the development of an on-line communication course (excerptsof

which are included in thisissue), the conversion of the Newton

EssentialsCourse to a self-paced on-line version, and the preparation

of new technologiestraining. Prior to joining the Newton Systems

Group, I wasat Borland International asan engineering manager in theC+ + developer support group. I am committed to providing Newton

developersall the information theyneed to write great applications,

whether it be in the form of training, technical journals, documentation,

on-line information, or support.

Id like to start byinviting you, the Newton developer, to

communicate your ideas, interests, and requestsvia email. Several

monthsbefore each issue ofNewton Technical Journalispublished, a

core team of representativesfrom DTS, engineering, marketing,

solutionsrelations, and training meet and discusspossible topicsto

slimPicker:AlphaCharacter(entry)

Thismethod isrequired if the AZTabsare visible.

Returnsa character representing the index value for the given entry.

The character will be used to set the appropriate tab in the AZTabs.

slimPicker:CreateNewItem()

Thismethod isrequired if the New button isvisible.

The method iscalled when the user tapsthe New button. The

developer isresponsible for anywork that needsto be done to add the

new entry. Thisincludescreating and opening anyediting or data

entryslip, adding the data to the cursor, selecting the new item, and

refreshing the slimPicker ( see RefreshPicker below).

slimPicker:GetHiliteShape(xcoord, bounds)

Thismethod iscalled to get the hilite box for a list item. The developer

should provide thismethod if theywish to create a multiple column

picker. Thismethod should return something suitable for DrawShape.

xcoord isthe current x coordinate of the pen normalized to bounds.

bounds isthe bounding box for the item being hilited.

slimPicker:GetNumSelected()

Thismethod isrequired if the counter isvisible or

UpdateSelectedText iscalled.

Returnsthe number of selected items.


Letter From the Editor


slimPicker: A Slimmer listPicker Proto

NTJ


23/24

InNTJVolume II Number 2, in the article Apple AnnouncesNew

MessagePad 130 with Newton 2.0!, there wasan error in the

description of the backlight API . On page 21, a function called

BackLightPresent isdocumented. Thisfunction should not beused. Although it existsin the MessagePad 130, it will not be

present in future hardware that hasa backlight. The correct wayto

test for the presence of a backlight isto use theGestalt functionasfollows:

// define this somewhere in your project until// platformfile defines itconstant kGestalt_BackLight :=

'[0x02000007, [struct,boolean], 1] ;

local isBacklight := Gestalt(kGestalt_BackLight) ;

if isBacklight AND isBacklight[0] then// has abacklight

else// has not got one

Correction

Correction

NTJ

To request information on or an application for

ApplesNewton developer programs,

contact ApplesDeveloper Support Center

at 408-974-4897

or Applelink: NEWTONDEV

or Internet: NEWTONDEV@ applelink.apple.com


23


24/24

newton technology journal: volume 2, number 4

Documents