A Multiwindow System for Smooth Window Operations by theCombination of Drawing-Period Generation Method and

Display-Period Generation Method

Yoichiro Sato,1 Tokumi Yokohira,2 Hiroto Kagotani,2 Takuji Okamoto,3 and Isao Kayano1

1Faculty of Computer Science and System Engineering, Okayama Prefectural University, Soja, 719-1197 Japan

2Faculty of Engineering, Okayama University, Okayama, 700-8530 Japan

3Faculty of Engineering, Okayama University of Science, Okayama, 700-0005 Japan

SUMMARY

In this paper, the authors propose a multiwindowsystem that can smoothly execute window operations suchas moving, resizing, or scrolling by using a combination ofa drawing-time composition method and display-time com-position method. Memory for storing the image of thewindow for which the window operation is to be performed(operation target window) and memory for storing a com-posite image of all other windows generated by a drawing-time composition method are both established in thissystem. The image that is displayed on the actual screen isobtained by using a display-time composition method tocombine images that were read in parallel from both ofthese memories to generate the multiwindow image. Theauthors performed a trial experiment using a 640-pixel by480-pixel display device to verify that no flicker appearedon the display screen when updating the operation targetwindow. From the results of this trial experiment, theysuggested that smooth window operations probably canalso be executed for a 1280-pixel by 1024-pixel displaydevice. © 2004 Wiley Periodicals, Inc. Electron Comm JpnPt 2, 88(1): 12–23, 2005; Published online in Wiley Inter-Science (www.interscience.wiley.com). DOI 10.1002/ecjb.20141

Key words: multiwindow system; user interface;high-density display; smooth window operations.

1. Introduction

Although window moving, resizing, and scrolling arebasic operations for manipulating multiwindow images ona display screen (these operations are referred to simply aswindow operations below), we will refer to the manipula-tion of windows smoothly by following the movement of apointing device, in particular, as smooth operations.

Recently, high-resolution display devices are ofteninstalled in special-purpose systems such as CAD systemsor presentation systems, and smooth operations are fre-quently requested from the standpoint of improving workefficiency or visual effects. Smooth operations are alsofrequently requested for general-purpose workstations orpersonal computers depending on usage objectives or userpreferences.

In current systems that are used for the purposesmentioned above, smooth operations are normally imple-mented accompanying an increase in the speed of thesystem itself by some means such as using a parallel proc-essing architecture [1, 2]. Also, a multiwindow systembased on a drawing-time composition method [3, 4], which

© 2004 Wiley Periodicals, Inc.

Electronics and Communications in Japan, Part 2, Vol. 88, No. 1, 2005Translated from Denshi Joho Tsushin Gakkai Ronbunshi, Vol. J86-D-I, No. 9, September 2003, pp. 650–660

12

is typified by the UNIX operating system’s X WindowSystem or the Windows operating system, is used in the vastmajority of commercial workstations or personal comput-ers, and when high-resolution display devices are used,smooth operations are made possible by introducing high-performance graphics accelerators [2, 5]. However, to thebest of our knowledge, in both cases, these speed increasesare largely due to an increase in the drawing speed and havepractically no connection to a decrease in the drawingmagnitude itself. Therefore, to achieve further increases inthe resolution of display devices along these lines, eventu-ally, we cannot help but demand further increases in thespeed of processor-based hardware.

On the other hand, various systems that reduce thedrawing magnitude accompanying the display of a multi-window image have been proposed. One is a system thatsequentially reads and displays in synchronization with theraster scan only those pixels that should be displayed in themultiwindow image from among multiple images that werestored in one image memory [6, 7]. Another is a system thatestablishes a pair consisting of an independent image mem-ory and an address generator circuit for each window andselects and displays pixels having the highest display pri-ority from among the pixels that were simultaneously readfrom them [8]. In this paper, we refer to the multiwindowcomposition method used in these systems as a display-timecomposition method. When a multiwindow system basedon a display-time composition method is compared with amultiwindow system based on the drawing-time composi-tion method described above, achieving smooth operationsfor a high-resolution display device is considered to beremarkably simple. However, since the operating speed ofthe above-mentioned pixel selection and display mecha-nism drops as the number of windows increases, such asystem has not been made practical so far because thenumber of windows that can be displayed is limited and theamount of required memory increases.

In this paper, we propose a multiwindow system thatactively utilizes the special features of the drawing-timecomposition method and the special features of the display-time composition method, which were mentioned above, toeasily implement smooth operations regardless of the num-ber of windows, even for a high-resolution display device.As the drawing-time composition method and display-timecomposition method on which this system is based, we usedthe methods described in Refs. 3 and 8, respectively. Theimages of all windows other than the window that is subjectto the window operations are combined as a multiwindowbased on the principles of the drawing-time compositionmethod and stored in one image memory. This multiwin-dow, which is referred to below as the nontarget multiwin-dow, is abbreviated as NTMW. In contrast, the image of thewindow that is subject to the window operations is storedin another image memory. This window, which is referred

to below as the target window, is abbreviated as TW. TheTW image and the NTMW image are combined accordingto the principles of the display-time composition methodand displayed on the display device as a multiwindow. Atthis time, the TW can be inserted according to a specifica-tion between any two windows having adjacent displaypriorities in the NTMW. According to this method, thesystem has the advantages that the required amount ofimage memory is limited to the amount equivalent to twoframes, and smooth operations can be easily implementedwithout any consideration for an increase in processorspeed. However, when the TW is switched, the contents ofthe two image memories must be changed before executionof the window operations begins. As a result, in this paper,we also describe a processing procedure for making thefullest possible use of the TW and NTMW images beforethe change to generate new TW and NTMW images whilecompletely suppressing flicker on the display screen duringthe change. We also verify the effect of this procedurethrough a trial experiment.

The rest of this paper is organized as follows. InSection 2, we show the concepts of multiwindow imagegeneration based on the proposed method. In Sections 3 and4, we describe the TW switching procedure and the methodof implementing that procedure, respectively. In Section 5,from the results of a trial experiment performed using alow-resolution display device, we investigate the effective-ness of the proposed method, centered on the TW switchingprocedure, concerning performance when this method isapplied to a high-resolution display device.

2. Concepts of Multiwindow ImageGeneration

Let W1 to Wn denote n windows that are displayed ina multiwindow image, and let a larger subscript valuecorrespond to a higher priority. To execute a window opera-tion, the client specifies any one of windows W1 to Wn asthe TW and instructs that a window operation be executedfor it. The image of the NTMW (hereafter referred to as theNTMW image) consists of images of the remaining n – 1windows.

The image that is obtained by combining all of thewindows W1 to Wn as a multiwindow is called the multi-window image. Figure 1(a) shows an example of a multi-window image when n = 4. The region in which the imageis displayed in each window in the multiwindow image iscalled the visible region of that window. The visible regionof W1 in Fig. 1(a) is the region in Fig. 1(b). Also, the regionin which the image is displayed in each window in theNTMW image is specifically called the virtual visible re-gion to distinguish it from the visible region in the multi-

13

window image. When Fig. 1(c) is the NTMW image, Fig.1(d) is the virtual visible region of W1. In addition, theimage of a window when it is not hidden by another windowis called the full image of that window, and the regioncorresponding to a full image in a multiwindow image iscalled the full region of that window. The full image of W1

in Fig. 1(a) and the full region corresponding to it are theimage in Fig. 1(e) and the region in Fig. 1(f), respectively.

As long as there is no risk of misunderstanding, theterms related to the various images and regions describedabove will also be used without further notification not onlyfor images and regions on the display screen but also forimages and regions in the corresponding memory.

Figure 2 shows the concept of multiwindow imagegeneration used in this paper. W2 is the TW and W1, W3,and W4 constitute the NTMW. BMTW and BMNTMW are theimage memories for storing the full image of the TW andthe NTMW image, respectively. Both of these imagememories have the address space corresponding to onescreen of the display device and the same depth as the imagecolor. Also, FMTW and FMNTMW are the memories for stor-ing the full region of the TW and the union of the virtualvisible regions of the windows (W3 and W4) having higherpriorities than the TW among the windows constituting theNTMW, respectively. This union is called the priority dis-play region below. These memories have the same addressspace as the image memories and a 1-bit depth, and a logical1 is written in the full region and priority display region (alogical 0 is written elsewhere). Hereafter, FMTW andFMNTMW will be referred to as region memories, and theinformation stored in them will be referred to as regioninformation.

The NTMW image in BMNTMW is read as the serialsignal iNTMW in synchronization with the scan of the displaydevice in ascending order from address 0 (corresponding tothe upper-left corner of the screen) for each frame period of

the display device. Similarly, the region information inFMNTMW is read as the serial signal fNTMW starting fromaddress 0 in parallel with iNTMW. Although the full image inBMTW and region information in FMTW are also read asserial signals iTW and fTW, respectively, the read startingaddresses are moved from the upper-left corner positionaccording to the change in the window display position onthe display screen accompanying the window operation.Note that the read starting addresses of BMTW and FMTW

do not necessarily match as described later.MUX, which is a 2-1 multiplexer, displays the mul-

tiwindow image on the display device by selecting andoutputting iTW only when fTW = 1 and fNTMW = 0 and byselecting and outputting iNTMW otherwise. In other words,MUX displays the TW as though it were inserted into theNTMW.

As in the display-time composition method, the TWis managed based on five types of information consistingof the full region, display position (relative position of theupper-left corner of the window to the upper-left corner ofthe display screen), BMTW read starting address, FMTW readstarting address, and display priority. The BMTW full imageand FMTW full region are generated and stored based onthese types of information. Also, each window constitutingthe NTMW is managed according to four types of informa-tion consisting of the virtual visible region added to thethree types of information remaining when the read startingaddresses are excluded from the five types of informationdescribed above. The union of the NTMW image inBMNTMW and the virtual visible region in FMNTMW is gen-erated and stored based on these types of information.Hereafter, the information for managing the TW andNTMW will be referred to as management information.

Fig. 1. Examples of images and regions.

Fig. 2. Concept of multiwindow image generation.

14

A single moving, resizing, or scrolling action per-formed on the TW can be executed, as in the display-timecomposition method [8], by the procedure briefly summa-rized below. The operation magnitudes (amount of move-ment, amount of resizing, or amount of scrolling) in thehorizontal and vertical scan directions, which are specifiedfrom the pointing device, are denoted by x and y, respec-tively.

[Moving]

After x and y are added to the current read startingrow addresses and read starting column addresses, respec-tively, of BMTW and FMTW, the start of the window opera-tion is invoked.

[Resizing]

Pixels are appended to or written in BMTW so that thenew pixels are displayed after resizing, the full region of theresized TW is stored in FMTW, and after the read startingaddresses of BMTW and FMTW are set, if necessary, in asimilar manner as described for [Moving], the start of thewindow operation is invoked.

[Scrolling]

Pixels are appended to or written in BMTW so that thenew pixels are displayed accompanying scrolling, and afterthe read starting address of only BMTW is set in a similarmanner as described for [Moving], the start of the windowoperation is invoked.

As is apparent from the above, in a multiwindowsystem using the proposed method, smooth operations areachieved for the TW by using the same principles as thedisplay-time composition method when images and regioninformation are set in two image memories and two regionmemories. Moreover, even if the number of windows in-creases, only the number of windows constituting theNTMW image increases, and the speed of the Ref. 8-likedisplay mechanism does not decrease and the requiredmemory does not change. However, when the TW is to beswitched, processing is required for rewriting the BMTW

and BMNTMW images and the FMTW and FMNTMW regioninformation, as suggested earlier. Hereafter, this processingwill be referred to as target window (TW) switching. Sincethe quality of this switching processing determines theperformance of the proposed method, it is explained indetail in the next section.

3. Target Window Switching Procedure

3.1. Switching policy

First, we define the following symbols.

IC(h): full image of Wh

IV(h): image in visible region of Wh

IB(h): image in virtual visible region of Wh before TWswitching

IA(h): image in virtual visible region of Wh after TWswitching

RC(h): full region of Wh

RV(h): visible region of Wh

RB(h): virtual visible region of Wh before TW switchingRA(h): virtual visible region of Wh after TW switching

Let the TWs before and after switching be denotedby Wi and Wj, respectively. From the time TW switching isperformed until immediately before execution of the win-dow operation begins, the display screen should be exactlythe same as it was before switching. However, each mem-ory’s contents change accompanying the switch. The con-tents of each memory at this time are regulated as shown inTable 1. The symbol ∪α < h < β X denotes the union of theimages or regions X for which α < h < β is satisfied. Thecontents of BMTW and FMTW change completely accompa-nying TW switching due to their nature. In contrast, thecontents of BMNTMW may change at most only in the unionof regions RA(i) and RB(j) accompanying TW switching.Also, the contents of FMNTMW change only in the union ofthe regions RB(h)(i < h ≤ j) when i < j or in the union of theregions RA(h)(j < h ≤ i) when i > j. For example, when theTW in Fig. 2 is switched from W2 to W3, each memory’scontents change as shown from (a) to (b) in Fig. 3.

When the TW is switched, the required time for theCPU to rewrite BMTW, FMTW, BMNTMW, and FMNTMW in-creases and the start of execution of the first windowoperation in the smooth operations ends up being delayed.Also, the image corruption that occurs on the screen accom-panying TW switching is displeasing. Therefore, in thispaper, we decided to take these points into considerationand perform TW switching based on the following policy.

When the TW is switched, the image that should bewritten in BMTW is IC(j). However, the image of the regioncorresponding to RB(j) within this image is stored inBMNTMW as IB(j) at a time prior to switching. Similarly, the

Table 1. Memory contents before and after TWswitching

15

range that should change in the image in BMNTMW is limitedto at most the union of regions RA(i) and RB(j). However, allof the images IA(i) that should be written in RA(i) within thisrange are stored in BMTW as parts of IC(i) at a time prior toswitching. Therefore, it seems that the CPU load can bereduced by using DMA transfers to transfer the imagescorresponding to IB(j) in BMNTMW and IA(i) in BMTW toBMTW and BMNTMW, respectively. It also seems that screencorruption can be efficiently prevented when switching theTW by following a procedure that first temporarily storesor displays the multiwindow image that is present immedi-ately before switching (performs control so that the MUXselects only one alternative) in one image memory, thenchanges the contents of the remaining memories (otherimage memory, FMTW, and FMNTMW) to the contents thatare present after TW switching, and then again returns thedisplay (MUX control) to its original state. Therefore, inthis paper, we decided to design the TW switching mecha-nism from this point of view.

Next, we will describe the image memory selectionmethod for temporarily storing or displaying the multiwin-dow image. Table 2 shows the relationships between theimage memory to be used and the image memory region forwhich the CPU should update the image. The BMTW andBMNTMW in the top row of the table indicate the options forthe image memory for temporarily storing the multiwindowimage when performing TW switching. Also, the four rowsbelow the top row indicate the BMTW or BMNTMW regionthat the CPU should update during the TW switchingprocessing. When i > j, using BMNTMW for temporarilystoring the multiwindow image is advantageous from thestandpoint of reducing the amount of the image to berewritten by the CPU since RB(j) ⊇ RV(j), and when i < j,using BMTW is clearly advantageous. For the TW switchingshown in Fig. 3, i < j. When BMNTMW and BMTW are used

for storing the multiwindow image, the pixels that the CPUshould update are indicated by the hatched portions in Figs.4(a) and 4(b), respectively. Using BMTW is advantageoussince not only should fewer pixels be updated, but also thereare fewer windows for which redrawing should be executed[in Fig. 4(b), redrawing need not be executed for W2]. Fromthe above, we decided to use BMTW when i < j and BMNTMW

when i j.

3.2. Switching procedure

First, we will discuss the case when i < j. To describethis situation clearly, we will use a raster scan device as thedisplay device. The switching procedure is organized asfollows based on the policy described in Section 3.1. TheCPU executes the processing of images or region informa-tion except where DMA transfers are explicitly mentioned.

(1) Delete RC(i) − RA(i) from the region information inFMTW in the frame scan period (hereafter referred to as thescan period) immediately after the switching request wasgenerated.

(2) Use DMA transfers to execute (2-1) and (2-2) inthe scan period immediately after step (1) ended.

(2-1) Among the pixels that were read from BMNTMW,overwrite each non-RV(i) pixel in the corresponding regionin BMTW.

Fig. 3. Example of memory contents before and afterTW switching.

Table 2. Image memory regions for which the CPUshould update the images

Fig. 4. Example of image memory regions for whichthe CPU should update the images when i < j.

16

(2-2) Among the pixels that were read from BMTW,overwrite each pixel of the part corresponding to RA(i) in thecorresponding region in BMNTMW.

(3) Fix the MUX control so that the MUX selectsiTW in the vertical blanking interval immediately after step(2) ended.

(4) Execute (4-1) to (4-3) in any order after step (3)ended.

(4-1) After deleting all region information (RA(i)) inFMTW, overwrite RC(j).

(4-2) Delete ∪i<h≤jRB(h) from the region information

in FMNTMW.(4-3) Overwrite the pixels of RA(h) ∩ (RB(j) − RA(i))

among the pixels of IA(h)(1 ≤ h < j; h ≠ i) and the pixels ofRB(j) − RA(i) − ∪1≤h<j;h≠iR

A(h) among the pixels of the back-ground image in the corresponding regions in BMNTMW,where X ∩ Y indicates the intersection of X and Y.

(5) Return the MUX control to normal control [beforeexecution of step (3)] in the vertical blanking intervalimmediately after step (4) ended.

(6) Overwrite the pixels of RC(j) − RB(j) in the corre-sponding region in BMTW after step (5) ended.

The specific method of executing pixel overwriting accord-ing to a DMA transfer is described in Section 4.1. Also, notethat the union of the regions in BMNTMW for which pixelsare overwritten in step (4-3) is RB(j) − RA(i) (see Table 2).[This is also true for step (6′) described later.]

When steps (1), (2), (4), and (6) are executed, eachmemory’s contents change as shown in Fig. 5(b), (c), (d),and (e), respectively. The arrows indicate that the contentswere unchanged. Since the contents of BMTW are the mul-tiwindow image when step (2-1) is completed, this multi-window image is displayed on the display screen when step(3) is completed and thereafter. Also, when step (2-2) iscompleted, IA(i) is overwritten in BMNTMW. When step (5)is completed, although the image or region information thatis present after TW switching is stored in the three memo-ries other than BMTW, there is a possibility that an imageother that IC(j) is included in BMTW. However, since non-IV(j) pixels are masked by the region information in FMTW

after step (5) is completed, this is not inconsistent with theprinciples described in Section 2.

The time required for this sequence of processes isthe sum of the time for one scan period, which is requiredin step (2), and the times for deleting or overwriting imagesor region information in memory in steps (1), (4), and (6).However, the amount of overwriting in steps (4-3) and (6)is reduced by executing step (2), and the screen is no longercorrupted.

Also, when i > j, the above-mentioned step (1) shouldbe deleted, and steps (2), (3), (4-2), (4-3), and (6) should bereplaced by the following steps (2′), (3′), (4-2′), (4-3′), and(6′), respectively.

(2′) Use DMA transfers to execute (2′-1) and (2′-2)in the scan period immediately after the TW switchingrequest was generated.

(2′-1) Overwrite all pixels that were read fromBMNTMW (NTMW image and background image) directlyin BMTW.

(2′-2) Overwrite the IV(i) pixels among the IC(i) pixelsthat were read from BMTW in the corresponding region inBMNTMW.

(3′) Fix the MUX control so that the MUX selectsiNTMW in the vertical blanking interval immediately afterstep (2′) ended.

(4-2′) Overwrite the region information ∪j<h≤iRA(h) in

FMNTMW.(4-3′) Overwrite the pixels of RC(j) − RB(j) in the cor-

responding region in BMTW.(6′) After step (5) ends, overwrite the pixels of

RA(h) ∩ (RB(j) − RA(i)) among the pixels of IA(h)(1 ≤ h < j)and the pixels of RB(j) − RA(i) − ∪1≤h<jR

A(h) among the pixelsof the background image in the corresponding regions inBMNTMW.

4. Processing Mechanism for Switching

4.1. Hardware

The main functions that should be implemented byhardware in the procedure described in Section 3.2 are the

Fig. 5. Example of memory contents during the TWswitching process.

17

DMA transfer function [steps (2) and (2′)] and the functionfor fixing MUX control [steps (3), (3′), and (5)]. In thispaper, we implemented these functions by using the twoserial ports possessed by image memory and the MUXcontrol signal, respectively. Figure 6 shows an example ofthis hardware architecture. The combinational logic circuitsconsisting of groups of logic gates other than AND gate Gare circuits for implementing the two functions mentionedabove. The address generator for each memory and controlsignal generator are omitted. D1, which is a 1-bit registerfor maintaining the TW switching instruction for the hard-ware, is set in the vertical blanking interval immediatelybefore execution of step (2) or (2′) described in Section 3.2and is reset in the vertical blanking interval immediatelyafter step (2) or (2′) ended. D2, which is also a 1-bit register,is set when i < j or is reset when i > j in the vertical blankinginterval immediately before execution of step (2) or (2′).D3, which is a 1-bit register for fixing the MUX function,is set in the vertical blanking interval immediately after step(2) or (2′) ended and is reset in the vertical blanking intervalimmediately after step (4) ended. Each memory’s SIP is aninput-only serial port with an internal line buffer (hereafterdenoted by LBSIP) equivalent in size to one row of thememory-cell array, and SOP is an output-only serial portwith a similar internal line buffer (hereafter denoted byLBSOP). SWE is the signal that enables writing to LBSIP

only when it is 1.To display a multiwindow, since the D1 and D3 out-

puts are 0 and since 0 is impressed at SWE and S (1 onlywhen fTW = 1 and fNTMW = 0) is impressed at C, the multi-window image is generated based on the composition prin-ciples described in Section 2.

On the other hand, steps (2), (3), and (5) in the TWswitching processing described in Section 3.2 are executedby hardware as follows. However, to make the descriptioneasier to understand, we decided to make the read starting

addresses of BMTW, FMTW, BMNTMW, and FMNTMW all bethe same (address 0).

First, the outputs of D1 and D2 are set to 1 in thevertical blanking interval immediately after step (1) ended.Then, in the following horizontal blanking interval imme-diately before scanning starts, the first one row of pixels andone row of region information in the corresponding mem-ory-cell arrays are read into LBSIP and LBSOP of BMTW andinto LBSOP of FMTW, respectively, and one row of pixelsand one row of region information are similarly read intoLBSIP and LBSOP of BMNTMW and into LBSOP of FMTW,respectively. When the horizontal blanking interval occurs,in addition to one row of pixels and one row of regioninformation being simultaneously and serially read fromfour memory SOPs, the outputs of the BMNTMW and BMTW

SOPs are impressed on the BMTW and BMNTMW SIPs,respectively, in this interval. However, since the S negationand fTW are impressed on the BMTW and BMNTMW SWEs,respectively, only the pixels not in the visible region(RV(i)) of the TW are written to LBSIP of BMTW, and onlythe pixels of the virtual visible region (RA(i)) after the TWwas switched is written to LBSIP of BMNTMW. In the nexthorizontal blanking interval, the LBSIP pixels in each imagememory are written into the cell array row that was read,and then the next row of pixels of the cell array in eachimage memory is read into LBSIP and LBSOP, and the nextrow of region information in each region memory is readinto LBSOP. The above operations are repeated each time ahorizontal blanking interval arrives, and steps (2-1) and(2-2) are executed.

Also, since the D3 output becomes 1 in the verticalblanking interval immediately after step (2) is completed,1 is always impressed on the MUX input C, input A (iTW)is always selected, and step (3) is executed.

For the procedure when i > j, only the functions ofBMTW and BMNTMW are reversed, and the procedure isexecuted in practically the same way.

4.2. Software

The multiwindow system server in this paper consistsof a window manipulation function group for manipulatingwindows, a switching function group for performing TWswitching, and a set of data used as management informa-tion. Among these, the window manipulation functiongroup is the same as the window manipulation functiongroup in the multiwindow system in Ref. 8 except for thememories to be rewritten always being BMTW and FMTW.Also, the set of data is similar to the set of server data in adrawing-time composition method multiwindow systemexcept for the virtual visible region being used instead ofthe visible region of each window constituting the NTMW,as described in Section 2.

Fig. 6. Example of hardware architecture for TWswitching.

18

In contrast, the switching function group is a functiongroup specific to the multiwindow system in this paper.This function group consists of function (A) for updatingmanagement information accompanying TW switching,function (B) for executing steps (1), (4), (4-2′), (4-3′), (6),and (6′), which were described in Section 3.2, according tosoftware [including a function for setting bits in D1 to D3

in steps (2), (2′), (3), (3′), and (5)], and function (C) forgenerating the various kinds of information required forswitching other than management information. Function(A) is the same as the function for calculating the visibleregion in the drawing-time composition method except fortargeting only the windows that constitute the NTMW.Function (B) is a function that simply accesses memory ora register. Function (C) generates the various kinds ofinformation described below, which are required only whenthe TW is switched.

(a) RA(i) in step (1) (unnecessary when i > j)(b) ∪i<h≤jR

B(h) in step (4-2)(c) ∪j<h≤iR

A(h) in step (4-2′)(d) RA(h) ∩ (RB(j) − RA(i)) and RB(j) − RA(i) −

∪1≤h<j;h≠iRA(h) in step (4-3)

(e) RA(h) ∩ (RB(j) − RA(i)) and RB(j) − RA(i) −∪1≤h<jR

A(h) in step (6′)(f) RC(j) − RB(j) in steps (4-3′) and (6)

These various kinds of information can be calculatedfrom the management information (full region, displayposition, virtual visible region, and display priority of eachwindow) in a similar manner as in the calculation methodsfor each type of region in the drawing-time compositionmethod.

5. System Evaluation

5.1. Trial system configuration

We created trial hardware [including a host computerinterface circuit, synchronous signal generation circuit, andaddress generator and control signal (such as RAS andCAS) generation circuit for each memory] for the multi-window system proposed in this paper according to theconfiguration shown in Fig. 6. We used the dual-port mem-ory HM534252ZP-10 to configure FMTW and FMNTMW.Also, we simulated an implementation of the BMTW andBMNTMW functions by duplexing two dual-port memoryHM534252ZP-10 units as shown in Fig. 7. The SIP, SOP,and SWE in Fig. 7 correspond to the SIP, SOP, and SWE inthe BMTW or BMNTMW in Fig. 6, respectively. SP is amemory serial input/output port, and SI/SO functions as aninput port or output port when SI/SO is 0 or 1, respectively.Input to SP is enabled only when SE = 1 and SI/SO = 0. Dis a 1-bit register for switching the SP function. Its output

is inverted each time execution of TW switching is com-pleted. Therefore, the functions of the two memories aremutually reversed each time TW switching is executed.Also, B1 and B2 are three-stage buffers. We used TTL74-series integrated circuits to configure the parts other thanthe HM534252ZP-10 units.

The host computer and display device used to createthe trial multiwindow system were a Sun Microsystemsworkstation SPARC Classic and 640 × 480 color display(refresh frequency 70.1 Hz and dot clock frequency 25.175MHz), respectively, and we used the standard keyword andmouse provided with the workstation for input devices. Thehardware described above was connected to the host com-puter’s extended port (SBus).

We constructed the trial multiwindow system serveron the host computer by modifying the sample server of XWindow System Version 11 Release 5. Figure 8 shows itsstructure.

Xlib and DD are the low-level programming libraryand device drivers, respectively. Also, the portion inside thedotted lines is the server, and MF, WF, DF, RF, TWF, andDS are the main function, window manipulation functiongroup, drawing function group, region function group, TWswitching function group, and data structure, respectively.Among these, TWF is a newly established function group,and the others were created by modifying the function,function group, or data structure having the same name in

Fig. 7. Implementation of BMTW and BMNTMW by theuse of two dual-port memories.

Fig. 8. Server structure.

19

the sample server mentioned above. In particular, to enablean existing client program to be executed directly, partsrelated to the X protocol were not changed. The mainmodifications are as follows.

• The TW switching function group, which consistsof the management information updating routineaccompanying TW switching [part of function(A)], function (C), and function (B), was estab-lished as TWF.

• The virtual visible region of each window consti-tuting the NTMW was added to the data structureand the function group for calculating the virtualvisible region [part of function (A)] was added tothe region function group to establish RF.

• Statements for calling the TW switching functiongroup were added to the main function to establishMF.

• Among the functions of the window manipulationfunction group, the functions for executing mov-ing, resizing, and scrolling operations were re-placed by functions for executing display-timecomposition method window operations to estab-lish WF.

The server was written in the C language, and thenumber of source program lines increased from approxi-mately 40,000 lines to approximately 52,200 lines when themodifications were made.

When a request related to the execution of a windowoperation arrives at the server from Xlib, if TW switchingis required, first MF calls TWF functions and those func-tions call RF functions to generate the various kinds ofinformation required during TW switching [(a) to (f) inSection 4.2] in DS. Next, DD is called via DF to performthe TW switching processing described in Section 3.2.Then, appropriate functions of the window manipulationfunction group are called, and those functions call functionsof the region function group to update management infor-mation of each window in DS. Finally, DD is called via DFto reflect the window manipulation execution results on thedisplay screen. If no TW switching is required, the process-ing is the same except that the TWF functions are not called.

5.2. Evaluation results

The trial hardware is broadly divided into memoryand its peripheral logic circuits. The capacities of the graph-ics memories (BMTW and BMNTMW) and region memories(FMTW and FMNTMW) are equivalent to 4 frames (with depthequal to the color depth of the image) and 2 frames (withdepth 1), respectively, and the hardware size of the periph-eral logic circuits is approximately 18,000 gates. A break-down of the 18,000 gates includes approximately 400 gatesin circuits for TW switching, approximately 4600 gates in

address generators, approximately 10,000 gates in controlsignal generator circuits for memory, and approximately3000 gates in the host computer interface circuit. If triple-port memories with two serial ports are used for the graph-ics memories [9–11], a memory capacity equivalent to 2frames (with depth equal to the color depth of the image) isenough.

Next, we will discuss the window operation execu-tion time. The size of the root window that was used formeasurements was set to 640 pixels (W) × 480 pixels (H),and the sizes of the other windows were set to be less thanor equal to this size. Also, the letter “A” with a size of 10pixels (W) × 10 pixels (H) was drawn consecutively withno intervening gaps in all windows.

The number of windows other than the root windowwas set to 4 or 6, and the sizes of these windows weregenerated randomly. The TW was randomly selected, andmoving, resizing, and scrolling operations were each exe-cuted 100 times. The results verified that all of these opera-tions were executed smoothly following the mouse. Theminimum resizing magnitude and scrolling magnitude perframe interval (approximately 16 ms) were equivalent toapproximately 5 pixels and approximately 10 pixels, re-spectively. At least 90% of the execution time during theseoperations was the required time for memory access. Next,we performed switching for windows of the same sizesbased on the same window generation conditions, and noflicker at all was noticed on the screen. Figure 9 shows anexample of the required switching times for this experi-ment. These results were obtained when measurementswere made 1000 times using six windows. It is apparent thatas the number of pixels in the TW increases, the majorportion of the execution time is the required time for steps

Fig. 9. TW switching time.

20

(4) and (6) in the switching procedure or, in other words,the required time for memory access.

5.3. Discussion

First, we examine the case when the system proposedin this paper is applied to a high-resolution display device.From the experimental results described above, the windowoperation execution time (excluding the moving operation)and the TW switching time are both strongly governed bythe image rewriting magnitude. Image rewriting is imple-mented in the trial system by partitioning the image to berewritten into multiple pixel groups and then performingeach of the following processes for each pixel group: (a)issue a request, (b) store the request in a buffer, (c) acceptand process the request, (d) activate a channel, (e) transferdata by using the channel. Therefore, even for a high-reso-lution display device, like with the trial system, we decidedto take as a prerequisite that the X Window System sampleserver’s request processing algorithm and the UNIX inter-rupt dispatch routine were used and that the execution timesof (a) to (d) were proportional to the clock pulse period Tc

of the CPU. We also decided that the required time for (e),which is performed by a device connected to an externalbus in a similar manner as in the trial system, could berepresented in the form a(bTb + Tm) where Tb, Tm, a, and bare the clock pulse period of the bus, access time of the mainstorage unit, number of transfer bytes per request, andnumber of required clock pulses per transfer byte, respec-tively.

Therefore, the required time for image rewriting Te

for a high-resolution display device can be represented asthe number of requests (r) multiplied by the sum of d × Tc

[which is the execution time of (a) to (d), where d is aconstant] and a(bTb + Tm) [which is the execution time of(e)]. When the experimental results are used to obtain d, b,Tb, and Tm according to the least-squares method, Te is givenby

If we assume that the same trial system other than the CPUis used with a 1280-pixel (W) × 1024-pixel (H) displaydevice and we set Te = 16.67 ms, Tc = 0.58 ns (clock pulsefrequency 1.7 GHz), and a = 3000, then from Eq. (1), r =96.67 and approximately 290,000 pixels per frame (16.67ms) can be drawn. This means that a 1000 × 800-pixelwindow can be resized to a 1280 × 1024-pixel window inone frame, and if we consider that the window operationnormally having the longest execution time is resizing, thisspeed enables sufficiently smooth operations to beachieved.

Also, if we increase the TW size proportional to theresolution, then from Eq. (1), the TW switching time is

approximately 73 ms, which is less than 5 frames if therefresh period of the display device is 60 Hz. Although thistime cannot be considered sufficiently short, we believe thatit can be considerably shortened if a is set to a larger value.We also believe that making TW switching begin when themouse cursor enters the visible region of a window otherthan the TW would contribute to shortening the TW switch-ing time.

Next, we examine child windows. This paper consid-ered the root window to be a parent window and dealt onlywith windows having no child windows (grandchild win-dows relative to the root window). However, in an actualenvironment, Widget is often used to generate an objectsuch as a scroll bar as a child window, and this object isoften clicked or dragged to issue a window operation exe-cution instruction. Therefore, if this kind of child windowgeneration were permitted in the multiwindow system pro-posed in this paper, a TW switching request would end upbeing generated each time a window operation was per-formed. We believe that a means of avoiding this is toslightly modify Widget and Xlib and establish an identifierfor distinguishing whether or not the child window is anobject in order to determine whether or not to perform TWswitching in the server.

6. Conclusions

In this paper, we proposed a multiwindow system thatcombined the drawing-time composition method and dis-play-time composition method and verified through trialexperiments that smooth operations can be achieved byusing only image memory having a size equivalent to twoframes regardless of the number of windows that can bedisplayed. We also presented a method of using DMAtransfers between image memories to change the contentsof image memories accompanying TW switching morequickly without any screen flicker. In addition, we showedfrom the results of trial experiments that smooth operationsprobably can be achieved even for a high-resolution displaydevice by only increasing the CPU speed to approximately1.7 GHz.

We expect that the multiwindow system proposed inthis paper can be used as a multiwindow display acceleratorin a CAD system or presentation system. Also, if themultiwindow system proposed in this paper were used in amultidisplay environment like the one in Ref. 1, we believethat the drawing magnitude accompanying window opera-tions could be reduced. We also believe that if the means ofshortening the required TW switching time (screen refreshtime) were implemented and the system was restructuredso that child windows could be generated, the multiwindowsystem proposed in this paper would become even morepractical.

(1)

21

REFERENCES

1. Ogino H, Haraishi H, Tsumori O, Yajima S. Devel-opment of a multi-computer multi-screen graphicsworkstation. IPSJ J 1986;27:970–978.

2. http://WWW-6.ibm.com/jp/pc/option/obi/nob13/9503dg1/9503dg1a.html

3. Brunhoff T. Pleasing the eye. UNIX Rev 1990;7:65–72.

4. Myers BA. A complete and efficient implementationof covered Windows. IEEE Comp 1986;19:57–67.

5. http://WWW.apple.co.jp/macosx/jaguar/quartzextreme.html

6. Nakamura T, Sugawara S, Maejima T. A multi-win-dow display device. Trans IEICE 1988;J71-D:2069–2077.

7. Maejima T, Nakamura T, Sugawara S. Increasingperformance of virtualization functions in a multi-window system. Trans IEICE 1989;J72-D-I:642–651.

8. Maeda T, Sato Y, Yokohira T, Okamoto T. An imple-mentation method for a multi-window systemequipped with high speed manipulation functions fornon-rectangular windows. Trans IEICE 1995;J78-D-I:509–518.

9. µPD42232 Tripleport graphic buffer Data Book. ID-7254, 1987.

10. Sumita K. Image memory, Patent Application Num-ber (1993) 221334.

11. Tozaki K. Image memory. Patent Application Num-ber (1999) 78098.

AUTHORS (from left to right)

Yoichiro Sato (member) earned his B.E. degree in electronics at Okayama University in 1982, completed his M.E. coursein 1984, and joined Toshiba Corporation. In 1987, he enrolled in the doctoral course at Okayama University and also becamean assistant in the Department of Information Sciences. Currently, he is an associate professor in the Faculty of ComputerScience and System Engineering at Okayama Prefectural University. He is engaged in research related to computer hardware.He holds a D.Eng. degree, and is a member of the Information Processing Society of Japan.

Tokumi Yokohira (member) earned his B.E. degree in computer and information science from Osaka University in 1984and Ph.D. degree in 1989. Currently, he is an associate professor in the Department of Communication Network Engineeringat Okayama University. He is engaged in research on performance evaluation and reliability of communication systems andcomputer systems. He is a member of the Information Processing Society of Japan.

Hiroto Kagotani (member) earned his B.E. degree in information science from Tokyo Institute of Technology in 1988and completed his doctoral course there in 1994. Currently, he is a lecturer in communication science at Okayama University.He is engaged in research on asynchronous processor design techniques, image processing hardware, and parallel processing.He holds a D.Eng. degree, and is a member of the Institute of Electrical and Electronics Engineers Computer Society.

Takuji Okamoto (member) earned his B.E. degree in communication science from Osaka University in 1958 and joinedKawasaki Heavy Industries, Ltd. In 1960, he moved to Mitsui Engineering and Shipbuilding Co., Ltd., and in 1967, he joinedthe Faculty of Engineering at Okayama University. Currently, he is professor of electronics at Okayama University of Science.He is engaged in research mainly on computer hardware focused on logic circuits. He holds a D.Eng. degree, and is a memberof the Information Processing Society of Japan, IEEJ, and IEEE.

22

AUTHORS (continued)

Isao Kayano (student member) earned his B.E. degree in information systems from Okayama Prefectural University in1999, completed his M.E. course in 2001, and enrolled in the doctoral course there. He is engaged in research related to humaninterfaces and design techniques for large-scale logic circuits.

23