journal of neuroengineering and rehabilitation biomed centralcognitive and higher level-cognitive...

BioMed Central

Journal of NeuroEngineering and Rehabilitation

ss
Open AcceReviewVideo capture virtual reality as a flexible and effective rehabilitation toolPatrice L Weiss*1, Debbie Rand1, Noomi Katz2 and Rachel Kizony1,2,3
Address: 1Dept. of Occupational Therapy, University of Haifa, Israel, 2School of Occupational Therapy, Hadassah-Hebrew University, Israel and 3Dept. of Occupational Therapy, Chaim Sheba Medical Center, Israel

Email: Patrice L Weiss* - [email protected]; Debbie Rand - [email protected]; Noomi Katz - [email protected]; Rachel Kizony - [email protected]

* Corresponding author

AbstractVideo capture virtual reality (VR) uses a video camera and software to track movement in a singleplane without the need to place markers on specific bodily locations. The user's image is therebyembedded within a simulated environment such that it is possible to interact with animatedgraphics in a completely natural manner. Although this technology first became available more than25 years ago, it is only within the past five years that it has been applied in rehabilitation. Theobjective of this article is to describe the way this technology works, to review its assets relativeto other VR platforms, and to provide an overview of some of the major studies that have evaluatedthe use of video capture technologies for rehabilitation.

IntroductionTwo major goals of rehabilitation are the enhancement offunctional ability and the realization of greater participa-tion in community life. These goals are achieved by inten-sive intervention aimed at improving sensory, motor,cognitive and higher level-cognitive functions on the onehand, and practice in everyday activities and occupationsto increase participation on the other hand [1,2]. Inter-vention is based primarily on the performance of roteexercises and/or of different types of purposeful activitiesand occupations [3,4]. The client's cognitive and motorabilities are assessed throughout the intervention periodso that therapy may be continually adjusted to the client'sneeds. For many injuries and disabilities, the rehabilita-tion process is long and arduous, and clinicians face thechallenge of identifying a variety of appealing, meaning-ful and motivating intervention tasks that may be adaptedand graded to facilitate this process. Clinicians alsorequire outcomes that may be measured accurately. Vir-

tual reality-based therapy, one of the most innovative andpromising recent developments in rehabilitation technol-ogy, appears to provide an answer to this challenge.Indeed, it is anticipated that virtual reality (VR) will havea considerable impact on rehabilitation over the next tenyears [5].

Virtual reality typically refers to the use of interactive sim-ulations created with computer hardware and software topresent users with opportunities to engage in environ-ments that appear to be and feel similar to real worldobjects and events [6-8]. Users interact with displayedimages, move and manipulate virtual objects, and per-form other actions in a way that attempts to "immerse"them within the simulated environment thereby engen-dering a feeling of presence in the virtual world [9,10].

The objective of this article is to briefly describe the use ofVR in rehabilitation, and then emphasize the unique

Published: 20 December 2004

Journal of NeuroEngineering and Rehabilitation 2004, 1:12 doi:10.1186/1743-0003-1-12

Received: 29 November 2004Accepted: 20 December 2004

This article is available from: http://www.jneuroengrehab.com/content/1/1/12

© 2004 Weiss et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

of 12(page number not for citation purposes)

http://www.jneuroengrehab.com/content/1/1/12

http://creativecommons.org/licenses/by/2.0

http://www.biomedcentral.com/

http://www.biomedcentral.com/info/about/charter/

Journal of NeuroEngineering and Rehabilitation 2004, 1:12 http://www.jneuroengrehab.com/content/1/1/12

attributes of the video capture VR to rehabilitation,including an overview of some of the major studies thathave evaluated the use of this technology forrehabilitation.

Virtual reality applied to rehabilitationVirtual reality has a number of well-known assets, whichmake it highly suitable as a rehabilitation interventiontool [11]. These assets include the opportunity for experi-ential, active learning and the ability to objectively meas-ure behavior in challenging but safe and ecologically-validenvironments while maintaining strict experimental con-trol over stimulus delivery and measurement. VR also pro-vides the capacity to individualize treatment needs, whilegradually increasing the complexity of tasks and decreas-ing the support provided by the clinician [5,12].

During the mid to late 1990s, virtual reality technologiesfirst began to be developed and studied as potential toolsfor rehabilitation assessment and treatment intervention[7]. The list of applications is long and diverse, and onlyseveral examples are provided here. VR has been used as amedium for the assessment and rehabilitation of cogni-tive and metacognitive processes, such as visual percep-tion, attention, memory, sequencing and executivefunctioning [13]. Rizzo and colleagues [14,15] developeda Virtual Classroom for the assessment and training ofattention in children with Attention Deficits HyperactiveDisorder. Piron, et al. [16] used a virtual environment totrain reaching movements, Broeren, et al. [17] used a hap-tic device for the assessment and training of motor coor-dination, and Jack et al. [18] and Merians, et al. [19] havedeveloped a force feedback glove to improve handstrength and a joint position glove to improve the rangeof motion and speed of hand movement. The studies citedabove share a common goal of using virtual reality to con-struct a simulated environment that aimed to facilitate theclient's motor, cognitive or metacognitive abilities inorder to improve functional ability. In some cases, theapplications take advantage of the ability to adapt virtualenvironment to simulate real life activities such as mealpreparation [20] or crossing a street [21-25]. The ultimategoal of such applications is to enable clients to becomeable to participate in their own real environments in amore independent manner. Attempting to achieve similarresults via conventional therapy when clinicians and cli-ents must deal with real world settings (e.g., a visit to a realsupermarket) is fraught with difficulty. In contrast, virtualenvironments may be adapted with relative ease to theneeds and characteristics of the clients under care.

Given the variety of VR platforms and the diverse clinicalpopulations that may benefit from VR-based intervention,it is helpful to view the VR experience as a multidimen-sional model that appears to be influenced by many

parameters. A conceptual model was developed withinthe context of terminology established by the Interna-tional Classification of Functioning, Disability and Health(ICF) [2] and the rehabilitation process [25,26]. Thismodel helps to identify the clinical rationale underlyingthe use of virtual reality as an intervention tool in rehabil-itation as well as to design research to investigate its effi-cacy for achieving improved performance in the realworld. The process of using VR in rehabilitation is mod-eled via three nested circles, the inner "Interaction Space",the intermediate "Transfer Phase" and the outer "RealWorld".

The "Interaction Space" denotes the interaction thatoccurs when the client performs within the virtual envi-ronment, experiencing functional or game-like tasks ofvarying levels of difficulty, i.e., the activity componentaccording to the ICF terminology. This interaction is influ-enced by user characteristics, which include personal fac-tors (e.g. age, gender, cultural background), bodyfunctions (e.g. cognitive, sensory, motor abilities) andstructures (e.g., the parts of the body activated during thetask). It is also influenced by the characteristics of VR plat-form and its underlying technology (e.g. point of view,encumbrance) that presents the virtual environment andthe nature and demands of the task to be performedwithin the virtual environment.

It is during the interaction process that sensations and per-ceptions related to the virtual experience take place; herethe user's sense of presence is established, and the processof assigning meaning to the virtual experience as well asthe actual performance of virtual tasks or activities occurs.The sense of presence enables the client to focus on thevirtual task, separating himself temporarily from the realworld environment. This is an important requirementwhen motor and, especially, cognitive abilities and skillsare trained or restored. The concept of meaning is alsothought to be an essential factor that enhances task per-formance and skills in rehabilitation in general [1,3], andthus also in the VR-based rehabilitation [27]. Environ-mental factors within the virtual environment may con-tribute information about issues that facilitate or hinderthe client's performance, and may serve as facilitators ofperformance in the virtual environment leading toimproved performance in the real world.

Two outer circles, the "Transfer Phase" and the "RealWorld" denote the goal of transferring skills and abilitiesacquired within the "Interaction Space" and eliminatingenvironmental barriers in order to increase participationin the real world (i.e., participation in the natural environ-ment according to the ICF terminology). The "TransferPhase" may be very rapid and accomplished entirely bythe client or may take time and need considerable



guidance and mediation from the clinician. The entireprocess is facilitated by the clinician whose expertise helpsto actualize the potential of VR as a rehabilitation tool.

Virtual reality platformsVirtual environments are experienced with the aid of spe-cial hardware and software for input (transfer of informa-tion from the user to the system) and output (transfer ofinformation from the system to the user). The selection ofappropriate hardware is important since its characteristicsmay greatly influence what is taking place in the Interac-tion Space, i.e., the way users respond (e.g. sense of pres-ence, performance) to a virtual environment [28]. Theoutput to the user generates different levels of immersion,which may be enhanced by different modalities includingvisual, auditory, haptic, vestibular and olfactory stimuli,although, to date, most VR platforms deliver primarily vis-ual and auditory feedback. Visual information is com-monly displayed by head mounted displays (HMD),projection systems, or flat screen, desktop systems of var-ying size. Input to a virtual environment enables the userto navigate and manipulate objects within it. Input maybe achieved via direct methods such as inertial orientationtracker or by video sensing which tracks user movement.Input may also be achieved via activation of computerkeyboard keys, a mouse or a joystick or even virtual but-tons appearing as part of the environment.

In addition to specialized hardware, application softwareis also necessary. In recent years, off-the-shelf, ready-for-clinical-use VR software has become available for pur-chase. However, more frequently, special software devel-opment tools are required in order to design and code aninteractive simulated environment that will achieve adesired rehabilitation goal. In many cases, innovativeintervention ideas may entail customized programmingto construct a virtual environment from scratch, using tra-ditional programming languages.

Video capture VRVideo capture VR consists of a family of camera-based,motion capture platforms that differ substantially fromthe HMD and desktop platforms in wider use. When usinga video-capture VR platform, users stand or sit in a demar-cated area viewing a large video screen that displays one ofa series of simulated environments. Users see themselveson the screen, in the virtual environment, and their ownnatural movements entirely direct the progression of thetask, i.e., the user's movement is the input. The result is acomplete engagement of the user in the simulated task. Asingle video camera converts the video signal of the user'smovements wherein the participant's image is processedon the same plane as screen animation, text, graphics, andsound, which respond in real-time. This process is referredto as "video gesture", i.e., the initiation of changes in a vir-

tual reality environment through video contact. The user'slive, on-screen video image responds at exactly the sametime to movements, lending an intensified degree of real-ism to the virtual reality experience. Video capture pro-vides both visual and auditory feedback with the visualcues being most predominant.

Myron Krueger [29] was the first to investigate the poten-tial of video capture technology in the 1970s with hisinnovative Videoplace installation. This was one of thefirst platforms that enabled users to interact with graphicobjects via movements of their limbs and body, and wasused to explore a variety of virtual art forms. The qualityof the video image in these applications was relativelyprimitive, consisting of silhouetted figures. Nevertheless,the immediate response of the virtual environment inreal-time to the user's movements presented compellingevidence of the possibility of using this technique forinteractive simulation.

The next major development occurred with the release ofVividGroup's Mandala Gesture Extreme (GX) platformhttp://www.gesturetekhealth.com in 1996, together witha suite of interactive, game-type environments. This plat-form makes use of a chroma key-based setup so that theexisting background is subtracted and replaced by a simu-lated background. GX VR has enjoyed considerable suc-cess around the world in numerous entertainment andeducational facilities including science museums andentertainment parks. During the past five years it has alsobegun to be adapted for use in rehabilitation and has gen-erated great interest in clinical settings (see below). GX VRcurrently offers a wide variety of gaming applicationsincluding, Birds & Balls, wherein a user is required totouch balls of different colors; if the touch is "gentle", theballs turn into doves whereas an abrupt touch causesthem to burst. In another application, a soccer game, theuser sees himself as the goalkeeper whose task it is to pre-vent balls from entering the goal area (see Figure 1).

In the late 1990s two other commercial companies devel-oped video-capture gaming platforms, Reality Fusion'sGameCam and Intel's Me2Cam Virtual Game System[30]. Both of these platforms aimed for the low-cost, gen-eral market, relying on inexpensive web camera installa-tions that did not entail the use of the chroma keytechnique. For reasons that are not clear, Reality Fusionand Intel discontinued their products within the past twoyears.

Somewhat later, Sony developed its very popular EyeToyapplication designed to be used with the PlayStation IIplatform http://www.EyeToy.com. This is an off-the-shelf,low-cost gaming application, which provides the oppor-tunity to interact with virtual objects that can be displayed


http://www.gesturetekhealth.com

http://www.EyeToy.com


on a standard TV monitor [31]. As with the VividGroup'sGX platform, the EyeToy displays real-time images of theuser. However, it does not require a chroma key blue/green backdrop behind the user nor bright ambient light-ing (see Figure 2). This makes for an easier setup of theplatform in any location but, on the other hand, it meansthat the user sees himself manipulating virtual objectswithin a video image of his own physical surroundingrather than within different virtual environments. Anadditional difference between the cheaper EyeToy plat-form and the more expensive GX platform is that theformer is capable of recognizing users or objects onlywhen they are in motion. A user who remains stationerydoes not exist for EyeToy applications. In contrast, the GXVR is responsive to users whether they are in motion ornot.

The EyeToy application includes many motivating andcompetitive environments which may be played by oneuser or more than one user sequentially in a tournamentfashion. With GX VR, two users can compete togethersimultaneously (e.g., boxing, spinning plates) as well ascombine their efforts to create different visual effects with-out a competitive component (e.g., painting a rainbow,mirror image distortions and popping bubbles).

The potential of these platforms for rehabilitation wasreadily apparent despite the fact that they were originallydeveloped for entertainment and gaming purposes.Indeed, VividGroups's GX platform was first applied with-out adaptations within a clinical setting by Cunninghamand Krishack [32] who used it to treat elderly patients whowere unstable and at high risk for falling. Unfortunately,

Individual with a stroke performing within the Soccer environment using the VividGroup GX systemFigure 1Individual with a stroke performing within the Soccer environment using the VividGroup GX system.



the inability to grade these platforms to levels suited topatients with severe cognitive or motor impairments ini-tially limited the application of these environments inclinical settings. In order to broaden the potential clinicalapplications of the platforms, our research group adaptedthe GX VR platform [33,34]. VividGroup developed, andnow also markets, a version of the GX platform, known asIREX (Interactive Rehabilitation EXercise) platform http://www.irexonline.com which enables therapists to adaptlevels of difficulty and record performance outcomes [35].

Characteristics of the Video-Capture PlatformsVideo-capture VR differs from other platforms in anumber of ways that have great relevance for its use as atool for rehabilitation evaluation and intervention. Someof these characteristics appear to be advantageous whereasothers may limit the utility of video-capture VR.

Point of ViewVideo-capture VR provides users with a mirror image viewof themselves actively participating within the environ-ment. This contrasts with other VR platforms such as theHMD which provides users with a "first person" point ofview, or many desktop platforms in which the user is rep-resented by an avatar. The use of the user's own image hasbeen suggested to add to the realism of the environmentand to the sense of presence [10]. It also provides feedbackabout a client's body posture and quality of movement,comparable to the use of video feedback in conventionalrehabilitation during the treatment of certain conditionssuch as unilateral spatial neglect [36].

Freedom from encumbranceThe user in video-capture VR does not have to wear or sup-port extraneous devices such as an HMD, glove or markers

Individual with a stroke performing the Wishy Washy application using the Sony EyeToy systemFigure 2Individual with a stroke performing the Wishy Washy application using the Sony EyeToy system.


http://www.irexonline.com

http://www.irexonline.com


in order to achieve a substantial intensity of immersionwithin the virtual environment. This eliminates a sourceof encumbrance that would likely hinder the motorresponse of patients with neurological or orthopedic def-icits. Although the newer HMDs and stereoscopic glassesare considerably less cumbersome than previous models,little information is available regarding their use by indi-viduals undergoing cognitive or motor rehabilitation.

Interaction and ControlThis characteristic relates to how the user controls objectswithin the virtual environment. As indicated above, ratherthan relying on a pointing device or tracker, interactionwithin video-capture based environments is accom-plished in a completely intuitive manner via naturalmotion of the head, trunk and limbs. Not only is the con-trol of movement more natural, but, in the case of thechroma key GX VR, a "red glove" option (or any objectwith a distinct color) may be used to restrict systemresponse to one or more body parts as deemed suitable forthe attainment of specified therapeutic goals. For exam-ple, when it is appropriate to have the interventiondirected in a more precise manner, a client may berequired to repel projected balls via a specific body part(e.g., by the hand when wearing a red glove or by the headwhen wearing a red hat). Or, when intervention is moreglobal, the client will not use the red glove option andthus be able to respond with any part of the body. Theability to direct a client's motor response to be either spe-cific or global makes it possible to train diverse motorabilities such as the range of motion of different limbs andwhole body balance training.

FeedbackA limitation of currently available video capture platformsis the reliance on visual and auditory feedback and theabsence of a haptic interface that would provide partici-pants with real-time indications of contact with the virtualstimuli. Such feedback could serve as an important addi-tion when used in therapy since the balls, for example,could be rendered to appear to have progressively greatermass, making the task more or less difficult. It would alsoadd an additional element of realism to the gaming expe-rience, and ensure that feedback to participants was morerealistic. This could be accomplished to some degree via aquasi-haptic effect that might use vibration to simulate atrue haptic interface (A.A. Rizzo, personal communica-tion). For example, small buzzers may be affixed to thetips of the digits. Touching a virtual ball in the Vivid GXBirds & Balls application would generate a low amplitude,high frequency "buzz". In contrast, repelling a larger ballin the Soccer application would generate a high ampli-tude, low frequency "buzz".

User positionVideo-capture VR may be implemented while users stand,sit, or even walk on a treadmill. For example, the sameenvironment may thus be suitable for training standingbalance of a patient who had a stroke, sitting balance ofan individual with an incomplete quadriplegic spinal cordinjury, and balance during treadmill locomotion of anindividual with a paraplegic spinal cord injury.

Multiple usersMoreover, one or more users may participate within thesame environment. In some applications, the ability tohave two "rival" users interact simultaneously within thesame game or task adds an element of competitivenessthat may be motivating. Of greater importance is the abil-ity of the therapist to support a client or use handling tech-niques in order to facilitate active movement while theclient interacts with the virtual stimuli. The therapist canbe concealed behind the client in order not to be seen inthe VE, or can join the client within the virtualenvironment.

Two-dimensional motion planeAnother limitation of the currently available video cap-ture VR platforms is that they may be operated with onlyone camera. This means that all tasks must be performedwithin a single plane. In the case of the typical coronalplane setup where the camera is positioned to face theuser, any functional movement that takes place in the sag-ittal or transverse planes is disregarded. Virtual scenariosmust therefore be carefully designed such that a meaning-ful task can be performed despite the restriction to unipla-nar movement. Moreover, care must be taken whenanalyzing the kinematic trajectories since any out-of-plane motion will not be recorded. It is encouraging tonote that three dimensional, functional environmentswill likely soon become available (I. Cohen and A.A.Rizzo, personal communication).

Applications of video-capture VR in rehabilitationAlthough video-capture platforms have only begun to beused for rehabilitation applications within the last fiveyears, there are already results from a number of researchgroups who have studied its utility with different patientpopulations. In this section we highlight the major studiesthat provide evidence that this technology appears to besuitable for use in rehabilitation. The evidence concerningparticipant sense of presence, enjoyment, usability andperformance are summarized as reported by studies ofsingle platforms and by studies that compared differentVR platforms



Side effectsNone of the studies carried out to date have reported anysignificant occurrence of cybersickness-type side effectswhen using video-capture VR. Rand et al. [28] explicitlyexamined the incidence of side effect of a group of 89healthy participants who experienced the GX platform.The occurrence of the side effects was very low, and noparticipants requested to terminate their participation inthe study. To date, evidence from a fewer number ofpatient subjects with spinal cord injury (SCI) or strokeindicates that they also are not disturbed by side effectswhen using video-capture VR [25,34].

Presence and enjoymentSeveral studies examined the influence of video captureplatform of the user's sense of presence and level of enjoy-ment. Rand et al. [28] in their study of 40 healthy youngadult participants, compared two different VR platforms,the GX-monitor and a combination of GX environmentsviewed via an HMD. They found that the participants'sense of presence was significantly higher when using theGX monitor platform than when using the GX-HMD. In acompanion study, which compared the GX-monitor withan HMD with two age groups, 33 young adults and 16 eld-erly participants, the older group felt a significantly highersense of presence and enjoyment than did the youngergroup using the HMD. Lott et al. [37] used the IREX videocapture platform and an HMD and found that the levelsof presence reported by the young adult participants didnot differ significantly for the two virtual realityconditions.

The results of these studies showed that a high sense ofpresence and level of enjoyment can be achieved in avideo capture VR platform. They also demonstrate thatuser characteristics such as age influence the sense ofpresence.

In another study, Rand et al. [38] compared the sense ofpresence, performance and perceived exertion experi-enced by 30 healthy young participants when theyengaged in two games performed within video-projectedvirtual environments that differed in their level of struc-ture and spontaneity. The non-structured application wasapplied using VividGroup's Gesture Xtreme (GX) VR plat-form, and the structured application was applied usingthe IREX platform, a rehabilitation-oriented applicationof GX, developed to train a specific movement (e.g.,shoulder abduction) in order to increase range of motionor endurance. No main effect or interaction effect wasfound for the sense of presence (assessed using Witmer &Singer's [39] Presence Questionnaire (PQ) although sig-nificant differences were found for several of the PQ sub-scales. It was concluded that it is possible to provide userswith a satisfactory level of presence and enjoyment using

both structured and non-structured paradigms. Therefore,both movement options, structured and non-structured,enhance the therapist's repertoire of VR intervention toolsin order to maximize rehabilitation.

Rand at al. [40] reported the results of another study, inwhich two different video-capture platforms, GX and Eye-Toy, were compared to determine their effect on users'sense of presence, level of enjoyment, perceived exertionand side effects. In this study, 18 healthy young adultsexperienced two games in each platform (Birds & Ballsand Soccer in GX and Kung-Foo and Wishy-Washy in Eye-Toy) in a counter-balanced order. There was no significantdifference in the sense of presence between the two plat-forms. However, the EyeToy Kung-Foo game, whichencourages participants to eliminate successive invadingwarriors by hitting at them, was found to be significantlymore enjoyable than the other games. In a continuationof this study, Rand et al. [40] examined the feasibility ofusing the EyeToy with healthy elderly users. Ten healthyelderly participants, aged 59 to 80 years, found this plat-form easy to operate and enjoyable. The results forpatients with stroke at a chronic stage (1–5 years poststroke) were similar to the healthy elderly. They thoughtthat it could contribute to their rehabilitation process, andwere able to operate the platform independently. Theresponses of a third group of users, patients with stroke atan acute stage (1–3 months post stroke), were somewhatdifferent. They also reported that they enjoyed the experi-ence; however, they became frustrated while performingthe EyeToy games, even when played at the easiest levels.This latter observation highlights a major limitation of theclosed architecture of the EyeToy; to date, Sony has beenunwilling to adapt the games to include a greater range oflevels of difficulty, nor to provide tools to external pro-grammers to do so (R. Marks, personal communication).It also emphasized the effect that user characteristics, inthis case, time post onset of stroke, have on the sense ofpresence.

The GX VR platform has consistently generated high levelsof presence and enjoyment across a wide range of clinicalpopulations and ages including adults with paraplegicspinal cord injury [34], stroke [25,33], and young adultswith cerebral palsy and intellectual impairment [41]. Apilot study using the GX platform to determine its suita-bility for leisure time activities among older stroke survi-vors was carried out. These participants enjoyed theexperience, and perceived it to be therapeutic [42].

Performance outcomes and sensitivity of video capture VRThe measures of performance used by video-capture VRstudies to date include response times to presented virtualstimuli, percent success with which a given game is per-formed (e.g., how many balls are repelled by the user in



the role of soccer game goal keeper), a subjective report onhow much effort the user has felt while in the environ-ment. The chroma key video capture platforms such as GXand IREX also provide a relatively gross measure of limbkinematics. Whether these data have sufficient precisionand resolution to warrant their inclusion in a researchstudy remains to be investigated (F. MacDougal, personalcommunication).

Sveistrup, McComas and colleagues have used the IREXplatform for balance retraining. Following six weeks oftraining at an intensity of three sessions per week,improvement was found for all 14 participants in both theVR and control groups [35]. However, the VR groupreported more confidence in their ability to "not fall" andto "not shuffle while walking". The same research grouphas also demonstrated that an exercise program deliveredvia video capture VR can improve balance and mobility inadults with traumatic brain injury [43] and the elderly[44].

Kizony et al. [34] performed a feasibility study of the GX-VR platform to train balance of people who had a paraple-gic SCI. The study included 13 adult participants who hadparaplegia. Results from the patient group were comparedto data from a parallel study of a group of 12 healthy adultparticipants who performed a similar protocol, while sit-ting on a chair with hands supported. The results showedthat the participants with SCI who had better balancefunction performed higher within the virtual environ-ments and the healthy participants performed signifi-cantly better than the participants with paraplegia. Thisplatform appeared to be suitable for use with people whohave paraplegia and it was able to differentiate betweenparticipants with different levels of balance function.

In a second study Kizony et al. [25] examined the relation-ships between cognitive and motor ability and perform-ance within the GX-virtual environments with peoplewho have had a stroke. Thirteen older adult patients withstroke participated in the full study. Significant moderatepositive correlations were found between VR performanceand cognitive abilities suggesting that higher cognitiveabilities relate to higher performance within the VR. Incontrast, almost no positive correlations were found withthe motor abilities. Indeed, as pointed out by theseauthors, perhaps motor performance demands and theircharacteristics should not be expected to be identicalwithin the real and the virtual worlds. It may be that dif-ferences in presence, motivation, or other factors influ-ence the movement patterns differently in virtual versusnatural environments. This result is in accordance withLott et al.'s [38] findings which showed significant differ-ences between functional lateral reach performed in a realversus virtual environment. They reported that the partic-

ipants reached significantly further when virtual objectswere presented within the virtual environment using avideo capture VR platform than when they were asked totouch a person hand standing on their side. They suggestthat embedding the reaching task in a game shifts the per-son's attention from the possibility of losing his balancethereby enabling him to achieve greater function.

Rand et al. [28] used a virtual office environment whichwas developed by Rizzo et al., [15] and was displayedboth via an HMD and via the GX-monitor platform. Inthis case, participants stood in front of the GX monitorand visually scanned the Virtual Office. Performance byboth age groups was significantly higher when using theGX-monitor platform than when using an HMD, whereasthe younger group's visual scan ability was better than theelderly on both platforms. The results also demonstratedthe effect that different user characteristics, such as ageand gender, have on the VR experience and thus should betaken into consideration when considering which VR plat-form to use in rehabilitation.

Weiss at al. [41], in a study of five young male adults withphysical and intellectual disabilities, explored ways inwhich virtual reality could provide positive and enjoyableleisure experiences during physical interactions with dif-ferent game-like virtual environments and potentiallylead to increased self-esteem and a sense of self-empower-ment. The results of this study showed that the GX-VRplatform was feasible for use with this population. Theparticipants were able to use the platform and expressedtheir considerable enjoyment from the virtual games.However, the authors raised several concerns, especiallythat some of the participants displayed involuntary move-ment synergies, increased reflexes and maladaptive pos-tures due to the too difficult levels of the games that wereused in study. Thus, a more controlled study with thesame population is currently in progress in order to exam-ine more thoroughly the potential of the platform as amean for providing leisure opportunities to thispopulation.

Performance within two games (Kung-Foo and Wishy-Washy) was measured while three different groups, youngadult participants, healthy senior participants and indi-viduals who were several years post-stroke, used several ofthe EyeToy games [40]. Performance was scored for eachgame in terms of how much of a given activity (e.g., howmany windows washed, how many warriers eliminated)was accomplished within a preset time limit. Higherscores were achieved when clients were able to performthese activities faster and/or more accurately. There weresignificant differences in performance between the youngand stroke groups, with the young adults having greater



success in both games than the stroke group. The olderadult group performed as well as the younger group.

The performance results described above highlight theinterplay between the user and VR platform characteris-tics, and emphasize the importance of taking these char-acteristics into consideration while using VR inrehabilitation. Moreover, they demonstrate the sensitivityof the VR performance measures in their capacity to differ-entiate between levels of participant ability.

Due to the motivating nature of the game-like environ-ments, it is important to determine how much efforthealthy subjects and those with disabilities expend whileengaged in these tasks. In a study of healthy young adults,the participants using the GX platform perceived the high-est level of exertion while playing Soccer, less for Birds &Balls and still less for a third game, Snowboard where onlyweight transfer was needed [28]. When differencesbetween the age groups were assessed, the younger groupperceived higher levels of exertion in comparison to theolder group. There were also differences in the perceivedlevel of exertion of the Birds & Balls game in GX as com-pared to comparable games in the EyeToy [40]. Overall,the level of perceived exertion was rated as "somewhat dif-ficult" which is an ideal level to use in therapy.

Initial comparisons of VR-based intervention to conventional therapyUsing the IREX platform, Sveistrup et al. [35] performedtwo studies designed to compare VR-delivered therapy toconventional therapy. In their first study, patients suffer-ing from frozen shoulder received exercise either via IREXapplications or via conventional physiotherapy. In bothcases, therapy was directed at improving the quality ofthree specific shoulder joint movements. In the secondstudy, individuals who suffered from post-traumatic braininjury were assigned to either VR-based (applications suchas the virtual soccer game were used where patients wereencouraged to reach towards the virtual stimulus inaddition to weight transfer) or conventional therapy (e.g.,stepping, picking up objects, reaching) for balance train-ing for a total of 24 sessions. In their report on prelimi-nary data from 14 patients, the authors concluded thatboth exercise programs resulted in improvement ofpatients' balance. However, additional benefits were iden-tified for the VR group, including greater enthusiasm forthe VR-delivered therapy program, increased enjoymentwhile doing the exercises, improved confidence whilewalking and fewer incidents of falling.

Cunningham & Krishack [32] presented VR as it was usedin occupational therapy to improve balance and dynamicstanding tolerance with geriatric patients. They reportedgreater improvement in dynamic standing tolerance in a

small group of older adults following a VR therapy than ina small group following a standard occupational therapy.More recently, Bisson, et al. [44] demonstrated significantimprovements in balance and functional mobility incommunity-living older adults following a VR exerciseprogram delivered with the IREX platform. The compari-son group completed a biofeedback exercise program andalso demonstrated significant balance improvement.

Analysis of conventional and video capture VR treatmentfor SCI by specialists in rehabilitation highlighted severalkey differences between the two methods of intervention[34]. First, control over delivery of the stimuli via the VRplatform enabled the therapist to intervene more effec-tively, especially in terms of physical guidance and sup-port. In addition, the VR platform allowed precise controlover delivery of the number of stimuli simultaneouslypresented to the patient as well as their speed and direc-tion. These features appeared to increase the number oftimes a desired balance-recovery movement was per-formed by patients. Finally, the ease with which this plat-form elicited dynamic equilibrium recovery responses, anessential component in balance training and encouragedweight transfer movements was remarkable. In contrast,the static presentation of stimuli during conventionaltherapy restricts intervention to focus almost exclusivelyon weight transfer.

Towards functional video-capture environmentsOne of the newest developments in video-capture VR isthe simulation of more functional environments. Rand etal. [45] have created a Virtual Mall (VMall), using the GXplatform. It has been designed to support intervention ofpatients following a stroke who have motor and/or execu-tive functions deficits that restrict their everyday activities.This environment enables participants to engage in tasksbased on typical daily activities such as shopping in asupermarket. In the initial application, shown in Figure 3,the user moves from aisle to aisle by activating iconslocated on a large monitor around thereby encouragingactive movement, transfer of weight from side to side, andbalance reactions. Virtual food items are manipulated(e.g., selected from a shelf and placed in a supermarketcart in accordance with a shopping list selected inadvance. The performance of the task provides multipleopportunities to make decisions, plan strategies and mul-titask, all in a relatively intuitive manner. Output meas-ures include a record how well the user accomplishes thetask (e.g., how many correct items selected) will berecorded and saved thus giving an option to monitorimprovement over time. Initial performance measuresand user feedback has been recorded from six patientswho had a stroke more than two years since onset and suf-fer from residual motor and cognitive deficits. The resultssuggest that the VMall provides a motivating task that



requires active movement as well as the ability to plan andproblem solve.

Sony's EyeToy Wishy Washy application involves thecleaning of successive dirty windows via wiping move-ments of the hand and arms. Most recently, VividGrouphas developed a laundry application (V.J. Vincent, per-sonal communication). These moves towards more func-tional applications are encouraging.

ConclusionsEvidence from the literature has demonstrated the feasi-bility, usability and flexibility of video-capture VR, andthere is little doubt that this technology provides a usefultool for rehabilitation intervention. The results ofpresence questionnaires, reports of user satisfaction, andthe sensitivity to differences in user ability as functions ofage, gender and disability are all strong indicators of thesuitability of this tool. A short video-clip, taken from alocal news report of applications of video-capture VR forstroke, illustrates the extremely positive response of oneuser to the use of this technology (see Video 1).

To date, as indicated by the studies reviewed above, videocapture VR shows great promise for a variety of therapeu-tic goals including intervention for cognitive and motorrehabilitation, functional activities and leisure opportuni-ties. The general assets of virtual reality summarized abovecombined with several assets that are unique to video-cap-ture VR, are compelling arguments for the inclusion ofthis technology in the repertoire of tools available in clin-ical settings.

Market demand, user interest and improvements in tech-nology have led to the availability of a number of differentvideo-capture platforms. There is no doubt that these plat-forms are valuable as intervention tools during the reha-

bilitation of patients with neurological andmusculoskeletal disorders. Motivated patients would beencouraged to practice movements in a repetitive mannerthereby improving their condition, an achievement that isnot easy to attain via conventional therapy [46]. Cur-rently, the two main contenders for the rehabilitationmarket are VividGroup's GX and IREX platforms andSony's PlayStation II's EyeToy. Both use large monitors todisplay real-time images of users interacting with virtualobjects in a simulated environment. The VividGroup plat-forms are considerably more expensive and require amore elaborate setup including a chroma key blue/greenbackdrop behind the user and bright, ambient lighting.Sony's EyeToy is an off-the-shelf, low-cost gaming appli-cation that may be run under almost any ambientconditions.

Studies comparing these two platforms have shown thatpresence, enjoyment, usability and performance wereequivalent under many conditions and for diverse users.Thus, despite the EyeToy's limitations, its low cost, user-friendly interface and simple setup requirements makes ithighly attractive to therapists. It may be readily acquiredfor use in any clinical setting, and even be purchased foruse at home to provide regular, intensive therapy after dis-charge from hospital.

Nevertheless, it is clear that the EyeToy is not suited foruse with the most severely impaired users. The currentlyavailable games seem to have a broad appeal for users ofdifferent ages but an open architecture that permits adap-tations of existing applications and development of newenvironments appears to be a basic requirement to makethis platform truly functional as a clinical tool. A systemfor generating an outcomes report comparable to the IREXplatform would also be of great benefit for clinicians.Additional low-cost video-capture platforms are currently

Screen shots of the VMall showing clients with stroke selecting a shopping aisle (left panel), a food item (middle panel) and ver-ifying the contents of the shopping cart (right panel)Figure 3Screen shots of the VMall showing clients with stroke selecting a shopping aisle (left panel), a food item (middle panel) and ver-ifying the contents of the shopping cart (right panel).



under development (M. Shahar, personal communica-tion). Moreover, video-capture platforms that will providethree dimensional, functional environments will likelysoon become available (Cohen and Rizzo, personalcommunication).

In contrast to the EyeToy's closed architecture, Viv-idGroup's IREX platform provides a user-friendly interfacethat a therapist may use to specify a much greater range oflevels of difficulty. Their SDK (Software Development Kit)provides programmers with the ability to further adaptexisting applications such as the standard set of games[33] and to design and implement novel applicationssuch as the virtual mall described above [45]. The popularpress has been generating a considerable amount of pub-licity in the EyeToy platform [31], and it is clear that low-cost video-capture systems such as these are poised tomake VR available to a wide range of users. We anticipatethat future developments in technology, such as low-costvirtual environments that are more functional will enableclinicians to take advantage of the considerable benefitsthat VR has for rehabilitation.

Additional material

AcknowledgementsWe gratefully acknowledge programming support by Meir Shahar and Yuval Naveh. Development of our video-capture research has been supported by the Baruch Foundation, the Koniver Foundation and the University of Haifa Development Fund.

References1. American Occupational Therapy Association: Occupational ther-

apy practice framework: domain and process. Am J Occup Ther2002, 56:609-639.

2. World Health Organization: International classification of func-tioning disability and health (ICF). Geneva: World HealthOrganization; 2001.

3. Trombly CA: Occupation: purposefulness and meaningfulnessas therapeutic mechanisms – 1995 Eleanor Clark SlagleLecture. Am J Occup Ther 1995, 49:960-972.

4. Katz N: Cognition and occupation across the life span: modelsfor intervention in occupational therapy. Bethesda MD: Amer-ican Occupational Therapy Association. in press

5. Schultheis MT, Rizzo AA: The application of virtual reality tech-nology for rehabilitation. Rehabilitation Psychology 2001,46:296-311.

6. Sheridan TB: Musings on telepresence and virtual presence.Presence 1992, 1:120-125.

7. Rizzo AA, Buckwalter JG, Neumann U: Virtual reality and cogni-tive rehabilitation: a brief review of the future. J Head TraumaRehabil 1997, 12:1-15.

8. Weiss PL, Jessel AS: Virtual reality applications to work. WORK:A Journal of Prevention, Assessment and Rehabilitation 1998, 11:277-293.

9. Slater M: Measuring presence: A response to the Witmer andSinger Questionnaire. Presence: Teleoperators and VirtualEnvironments 1999, 8:560-566.

10. Nash EB, Edwards GW, Thompson JA, Barfield W: A review ofpresence and performance in virtual environments. J Human-Comp Interact 2000, 12:1-41.

11. Rizzo AA, Kim GJ: A SWOT analysis of the field of VR rehabil-itation and therapy. Presence: Teleoperators and Virtual Environments. in press

12. Rizzo AA, Schultheis MT, Kerns K, Mateer C: Analysis of assets forvirtual reality in neuropsychology. NeuropsychologicalRehabilitation 14:207-239.

13. Pugnetti L, Mendozzi L, Attree EA, Barbieri E, Brooks BM, CazzulloCL, Motta A, Rose FD: Probing memory and executive func-tions with virtual reality: past and present studies. CyberpsycholBehav 1998, 1:151-162.

14. Rizzo AA, Buckwalter JG, Bowerly T, van der Zaag C, Humphrey L,Neumann U, Chua C, Kyriakakis C, van Rooyen A, Sisemore D: Thevirtual classroom: a virtual environment for the assessmentand rehabilitation of attention deficits. Cyberpsychol Behav 2000,3:483-500.

15. Rizzo AA, Bowerly T, Buckwalter JG, Schultheis MT, Matheis R, Sha-habi C, Neumann U, Kim L, Sharifzadeh M: Virtual environmentfor the assessment of attention and memory processes: thevirtual classroom and office. Proceeding of the 4th International Con-ference on Disability, Virtual Reality and Associated Technology: Universityof Reading: Vresprem, Hungary 2002:3-11.

16. Piron L, Cenni F, Tonin P, Dam M: Virtual reality as an assess-ment tool for arm motor deficits after brain lesions. StudiesHealth Technology Information 2001, 81:386-392.

17. Broeren J, Bjorkdahl A, Pascher R, Rydmark M: Virtual reality andhaptics as an assessment devise in the postacute phase afterstroke. Cyberpsychol Behav 2002, 5:207-211.

18. Jack D, Boian R, Merians A, Tremaine M, Burdea GC, Adamovich SV,Recce M, Poizner H: Virtual reality-enhanced stroke rehabilita-tion. IEEE Transactions of Neural Systems and Rehabilitation Engineering2001, 9:308-318.

19. Merians A, Jack D, Boian R, Tremaine M, Burdea GC, Adamovich SV,Recce M, Poizner H: Virtual reality-augmented rehabilitationfor patients following stroke. Phys Ther 2002, 82:898-915.

20. Zhang L, Abreu BC, Masel B, Scheibel RS, Christiansen CH, Huddles-ton N, Ottenbacher KJ: Virtual reality in the assessment ofselected cognitive function after brain injury. Am J Phys MedRehabil 2001, 80:597-604.

21. Weiss PL, Naveh Y, Katz N: Design and testing of a virtual envi-ronment to train CVA patients with unilateral spatialneglect to cross a street safely. Occup Ther Int 2003, 10:39-55.

22. Katz N, Ring H, Naveh Y, Kizony R, Feintuch U, Weiss PL: Interac-tive virtual environment training for safe street crossing ofright hemisphere stroke patients with unilateral spatialneglect. Proceedings of the 5th International Conference on Disability, Vir-tual Reality and Associated technologies: Oxford, U.K 2004.

23. McComas J, MacKay M, Pivik J: Effectiveness of virtual reality forteaching pedestrian safety. Cyberpsychol Behav 2002, 3:185-190.

24. McFadyen BJ, Malouin F, Fung J, Comeau F, Chapdelaine S, BeaudoinC, Lamontagne A, Laurendeau D, Richards C: Development ofcomplex virtual environments for locomotor training follow-ing stroke. Proceedings of the 15th Congress of the International Societyof Electrophysiology and Kinesiology, Boston . June 18–21, 2004

25. Kizony R, Katz N, Weiss PL: A model of VR-based interventionin rehabilitation: relationship between motor and cognitiveabilities and performance within virtual environments forpatients with stroke. Proceedings of the Fifth International Conferenceon Disability, Virtual Reality and Associated Technologies, Oxford, UK 2004.

26. Weiss PL, Kizony K, Feintuch U, Katz N: Virtual Reality in Neu-rorehabilitation. In Textbook of Neural Repair and NeurorehabilitationEdited by: Selzer ME, Cohen L, Gage FH, Clarke S, Duncan PW. Cam-bridge University Press. in press

27. Hoffman HG, Prothero J, Wells M, Groen J: Virtual chess: the roleof meaning in the sensation of presence. International Journal ofHuman-Computer Interaction 1998, 10:251-263.

28. Rand D, Kizony R, Feintuch U, Katz N, Josman N, Rizzo AA, WeissPL: Comparison of two VR platforms for rehabilitation: video

Additional File 1Video 1: This video clip shows a patient who had a stroke using the Viv-idGroup VR system for cognitive and motor rehabilitation.Click here for file[http://www.biomedcentral.com/content/supplementary/1743-0003-1-12-S1.WMV]


http://www.biomedcentral.com/content/supplementary/1743-0003-1-12-S1.WMV

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=12458855
















Publish with BioMed Central and every scientist can read your work free of charge

"BioMed Central will be the most significant development for disseminating the results of biomedical research in our lifetime."

Sir Paul Nurse, Cancer Research UK

Your research papers will be:

available free of charge to the entire biomedical community

peer reviewed and published immediately upon acceptance

cited in PubMed and archived on PubMed Central

yours — you keep the copyright

Submit your manuscript here:http://www.biomedcentral.com/info/publishing_adv.asp

BioMedcentral

capture versus HMD. Presence, Teleoperators and VirtualEnvironments . in press

29. Krueger MW: Artificial Reality II Reading: Addison-Wesley; 1991. 30. Warren J: Unencumbered full body interaction in video

games. Master's thesis [http://a.parsons.edu/~jonah/thesis/jonah_thesis.pdf]. Sept. 5, 2004

31. Kushner D: Computing gets physical. Technology Review 2004,July/August:57-61.

32. Cunningham D, Krishack M: Virtual reality: a wholistic approachto rehabilitation. Stud Health Technol Inform 1999, 62:90-3.

33. Kizony R, Katz N, Weiss PL: Adapting an immersive virtual real-ity system for rehabilitation. Journal of Visualization and ComputerAnimation 2003, 14:261-268.

34. Kizony R, Raz L, Katz N, Weingarden H, Weiss PL: Using a videoprojected VR system for patients with spinal cord injury. InProceedings of the 2nd Internationnal. Workshop o Virtual RehabilitationRutgers University: New Jersey, USA; 2003.

35. Sveistrup H, McComas J, Thornton M, Marshal S, Finestone H,McCormick A, Babulic K, Mayhew A: Experimental studies of vir-tual reality-delivered compared to conventional exerciseprograms for rehabilitation. Cyberpsychol Behav 2003, 6:245-249.

36. Soderback I, Bengtsson I, Ginsburg E, Ekholm J: Video feedback inoccupational therapy: its effects in patients with neglectsyndrome. Arch Phys Med Rehabil 1992, 73:1140-1146.

37. Lott A, Bisson E, Lajoie Y, McComas J, Sveistrup H: The effect oftwo types of virtual reality on voluntary center of pressuredisplacement. CyberTherapy & Behavior 2003, 5:477-485.

38. Rand D, Kizony R, Brown H, Feintuch U, Weiss PL: Effect of per-formance demands and constraints within virtual environ-ments. In The 9th Annual Cybertherapy Conference San Diego,California.

39. Witmer BG, Singer MJ: Measuring presence in virtual environ-ments: a presence questionnaire. Presence: Teleoperators and Vir-tual Environments 1998, 7:225-240.

40. Rand D, Kizony R, Weiss PL: VR rehabilitation for all: vivid GXversus sony playstation II eyetoy. Proceedings of the 5th Interna-tional Conference on Disability, Virtual Reality and Associated Technologies:Oxford, U.K 2004.

41. Weiss PL, Bialik P, Kizony K: Virtual reality provides leisure timeopportunities for young adults with physical and intellectualdisabilities. Cyberpsychol Behav 2003, 6:335-342.

42. Reid DT: The influence of a virtual reality leisure interventionprogram on the motivation of older adult stroke survivors: apilot study. Phys Occup Ther Geriatr 2003, 21(4):1-19.

43. Thornton M, Marshall S, McComas J, Finestone H, McCormick A,Sveistrup H: A virtual reality exercise program improves bal-ance and mobility in community-living adults with traumaticbrain injury. Proceedings of the 15th Congress of the International Soci-ety of Electrophysiology and Kinesiology, Boston, USA 2004.

44. Bisson Y, Constant B, Sveistrup H, Lajoie Y: Balance training forelderly: comparison between virtual reality and visual bio-feedback. In Proceedings of the 6th World Congress on Aging and Physi-cal Activity: London 2004.

45. Rand D, Katz N, Shahar M, Kizony R, Weiss PL: The virtual mall:development of a functional virtual environment for strokerehabilitation. Abstracts of the 55th Annual Conference of the IsraeliAssociation of Physical and Rehabilitation Medicine, Tel Aviv 2004.

46. Liepert J, Baunder H, Wolfgang HR, Miltner WH, Taub E, Weiller C:Treatment-induced cortical reorganization after stroke inhumans. Stroke 2000, 31:1210-1216.


http://a.parsons.edu/~jonah/thesis/jonah_thesis.pdf

http://a.parsons.edu/~jonah/thesis/jonah_thesis.pdf
















http://www.biomedcentral.com/info/publishing_adv.asp


journal of neuroengineering and rehabilitation biomed centralcognitive and higher level-cognitive...

Documents