united states court of appealsonline.wsj.com/public/resources/documents/nintendo.pdf ·...

NON-CONFIDENTIALNo. 2012-1252

United States Court Of AppealsFor The Federal Circuit

MOTIVA, LLC,Appellant,

v.

INTERNATIONAL TRADE COMMISSION,Appellee,

and

NINTENDO CO., LTD. AND NINTENDO OF AMERICA INC.,Intervenors.

_________________

ON APPEAL FROM THE UNITED STATES INTERNATIONAL TRADE COMMISSION

IN INVESTIGATION NO. 337-TA-743_________________

ANSWERING BRIEF OF INTERVENORS NINTENDO CO., LTD.AND NINTENDO OF AMERICA INC.

_________________

Joseph S. PrestaRobert W. FarisNIXON & VANDERHYE P.C.901 North Glebe Road11th FloorArlington, VA 22203(703) 816-4000

E. Joshua RosenkranzPeter A. BicksAlex V. ChachkesORRICK, HERRINGTON & SUTCLIFFE LLP51 West 52nd StreetNew York, NY 10019(212) 506-5000

Mark S. DaviesKatherine M. KoppORRICK, HERRINGTON & SUTCLIFFE LLP1152 15th Street, N.W.Washington, D.C. 20005(202) 339-8400

Case: 12-1252 Document: 46 Page: 1 Filed: 08/29/2012

i

CERTIFICATE OF INTEREST

Counsel for Intervenors Nintendo Co., Ltd. and Nintendo of

America Inc. certify the following:

1. The full name of the parties represented by us in this case

are:

Nintendo Co., Ltd. and Nintendo of America Inc.

2. The name of the real parties in interest (if the party named

in the caption is not the real party in interest) represented by us are:

Nintendo Co., Ltd. and Nintendo of America Inc.

3. The parent companies, subsidiaries (except wholly owned

subsidiaries), and affiliates that have issued shares to the public of the

parties represented by us are:

Nintendo Co., Ltd., whose stock is publicly traded in Japan, owns

100% of Nintendo of America Inc.

4. The names of all law firms and the partners or associates

that appeared for the parties now represented by us in the agency or

are expected to appear in this Court, are:


ii

Peter A. BicksE. Joshua RosenkranzMark S. DaviesAlex V. ChachkesElyse D. EchtmanSarah E. WalcavichRichard A. RinkemaJordan L. CoyleNicholas H. LamLauren B. MuldoonCyrus P.W. RieckKatherine M. KoppSteven E. Adkins (no longer withfirm)ORRICK, HERRINGTON & SUTCLIFFE

LLP1152 15th Street, NWWashington, D.C. 20005(202) 339-8400


5. No other appeal involving this civil action was previously

before this or any other appellate court. There are no pending cases

known to counsel that would directly affect or be directly affected by

this Court’s decision in the pending appeal.

Date: August 29, 2012 /s/ Mark S. Davies

Mark S. Davies

Attorney for Nintendo Co., Ltd. andNintendo of America Inc./Intervenors


TABLE OF CONTENTS

Page

iii

CERTIFICATE OF INTEREST ................................................................ i

TABLE OF AUTHORITIES.................................................................... vi

STATEMENT OF RELATED CASES .................................................... ix

INTRODUCTION..................................................................................... 1

JURISDICTIONAL STATEMENT.......................................................... 3

STATEMENT OF THE ISSUES.............................................................. 3

STATEMENT OF THE CASE ................................................................. 3

STATEMENT OF THE FACTS ............................................................... 5

A. Motiva’s Research And Marketing Efforts: The StartAnd Finish.............................................................................. 5

B. Motiva’s Patents..................................................................... 9

C. Motiva’s Patent Litigation ................................................... 16

1. Motiva Files An ITC Action Against Nintendo .......... 16

2. The ALJ Grants Nintendo’s Motion For SummaryDetermination ............................................................. 18

3. The ITC Reverses And Remands With Questions ..... 19

4. The ALJ Holds A Hearing, Answers The ITC’sQuestions, And Finds No Violation ............................ 20

5. The ITC Affirms .......................................................... 30

SUMMARY OF ARGUMENT ................................................................ 30

STANDARD OF REVIEW...................................................................... 33

ARGUMENT .......................................................................................... 34


TABLE OF CONTENTS(continued)

Page

iv

I. SUBSTANTIAL EVIDENCE SUPPORTS THE ALJ’SFINDING THAT MOTIVA WAS NOT IN THE PROCESSOF ESTABLISHING ANY DOMESTIC INDUSTRY .................. 34

A. Motiva’s Stale Licensing Efforts Do Not DemonstrateAny Tangible Steps Toward Establishing A DomesticIndustry................................................................................ 35

B. Motiva’s Lawsuit Against Nintendo Is Not A TangibleStep Toward Establishing A Domestic Industry................. 41

C. The Wii System’s Success Does Not Suggest MotivaHas Any Chance Of Establishing A Domestic IndustryIn The Future....................................................................... 46

II. SUBSTANTIAL EVIDENCE SUPPORTS THE ALJ’SFINDING THAT THE WII SYSTEM DOES NOTINFRINGE MOTIVA’S PATENTS ............................................... 51

A. Substantial Evidence Shows That The Wii SystemDoes Not “Track” Movement Of A User .............................. 51

B. Substantial Evidence Shows That The Wii SystemDoes Not Determine User “Movement” ............................... 54

1. Three Credible Witnesses Demonstrated ThatThe Wii Sensors Cannot And Do Not DeterminePosition Or Orientation .............................................. 55

2. Motiva’s Hodgepodge Of Movement And PositionInformation Arguments Are All Meritless ................. 59

CONCLUSION ....................................................................................... 75


TABLE OF CONTENTS(continued)

v

Material has been deleted from pages 1, 2, 6, 22, 26, 29, 43, 46-47, 51-

53, 55-64, 68-70, 72 and 73 of the Non-Confidential Answering Brief of

Intervenors Nintendo Co. Ltd. and Nintendo of America Inc. This

material is deemed confidential business information pursuant to 19

U.S.C. § 1337(n) and 19 C.F.R. § 210.5, and pursuant to the Protective

Order entered November 2, 2010. The material omitted from these

pages contains confidential deposition and hearing testimony,

confidential business information, confidential patent application

information, and confidential licensing information.


vi

TABLE OF AUTHORITIES

Page(s)FEDERAL CASES

Am. Silicon Techs. v. United States,261 F.3d 1371 (Fed. Cir. 2001) ...........................................................50

Bally/Midway Mfg. Co. v. ITC,714 F.2d 1117 (Fed. Cir. 1983) ..................................................... 38, 48

Beneficial Innovations, Inc. v. Blockdot, Inc.,Nos. 2:07-CV-263-TJW-CE, 2:07-CV-555-TJW-CE,2010 U.S. Dist. LEXIS 54151 (E.D. Tex. June 3, 2010).....................74

Erbe Elektromedizin GmbH v. ITC,566 F.3d 1028 (Fed. Cir. 2009) ...........................................................33

Finnigan Corp. v. ITC,180 F.3d 1354 (Fed. Cir. 1999) ...........................................................60

In re Katz Interactive Call Processing Patent Litig.,07-ml-01816-BRGK, 2008 WL 4952454(C.D. Cal. Feb. 21, 2008)....................................................................74

In re Nintendo Co., Ltd.,589 F.3d 1194 (Fed. Cir. 2009) ...........................................................16

InterDigital Commc’ns, LLC v. ITC,No. 2010-1093, 2012 U.S. App. LEXIS 15893(Fed. Cir. Aug. 1, 2012)................................................................. 39, 41

John Mezzalingua Assocs. v. ITC,660 F.3d 1322 (Fed. Cir. 2011) .......................................... 32, 33,34, 40

Ninestar Tech. Co. v. ITC,667 F.3d 1373 (Fed. Cir. 2012) ...........................................................33

Nippon Steel Corp. v. United States,458 F.3d 1345 (Fed. Cir. 2006) ...........................................................50


vii

Northrop Grumman Corp. v. Intel Corp.,325 F.3d 1346 (Fed. Cir. 2003) ...........................................................66

Retractable Technologies v. Becton, Dickinson & Co.,653 F.3d 1296 (Fed. Cir. 2011) ............................................... 65, 66, 67

SRAM Corp. v. AD-II Eng’g, Inc.,465 F.3d 1351 (Fed. Cir. 2006) ...........................................................74

St. Clair Intellectual Prop. Consultants, Inc. v. Canon Inc.,412 F. App’x 270 (Fed. Cir. 2011) .......................................................74

Tessera, Inc. v. ITC,646 F.3d 1357 (Fed. Cir. 2011) ................................................34, 71, 72

TianRui Group Co. v. ITC,661 F.3d 1322 (Fed. Cir. 2011) ........................................................... 33

Vita-Mix Corp. v. Basic Holding,581 F.3d 1317 (Fed. Cir. 2009) ..................................................... 70, 71

ADMINISTRATIVE CASES

Certain Coaxial Cable Connectors and Components Thereof andProds. Containing Same,Inv. No. 337-TA-650, USITC Pub. No. 4283 (Nov. 11, 2010).............41

Certain Digital Satellite Sys. Receivers and Components Thereof,Inv. No. 337-TA-392, USITC Pub. No. 3418 (Apr. 2011) ............. 45, 46

Certain Multimedia Display and Navigation Devices and Sys.,Components Thereof, and Prods. Containing the Same,Inv. No. 337-TA-694, USITC Pub. No. 4292 (Nov. 2011)............. 44, 45

Certain Stringed Musical Instruments & Components Thereof,Inv. No. 337-TA-586, USITC Pub. No. 4120 (Dec. 2009) ...................34

In re Certain Rotary Wheel Printing Sys.,Inv. No. 337-TA-185, USITC Pub. No. 1857 (May 1986)...................38


viii

FEDERALRULES & STATUTES

Fed. Cir. R. 30(j) ........................................................................................1

19 U.S.C. § 1337 .............................................................................. passim

28 U.S.C. § 1295(a)(6)................................................................................3

LEGISLATIVE HISTORIES

S. Rep. No. 100-71 (1987) .................................................................. 20, 34

H.R. Rep. No. 100-40 (1987).................................................. 20, 32, 34, 40

REGULATIONS

75 Fed. Reg. 68,379 (Nov. 1, 2010)............................................................4


ix

STATEMENT OF RELATED CASES

No other appeal from the proceeding below was previously before

this Court or any other appellate court.

The same parties and one of the patents involved in this appeal

are involved in a district court action, Motiva, LLC v. Nintendo Co.,

Ltd., et al., No. 2:10-cv-00349 (W.D. Wash. filed Nov. 10, 2008). That

action was filed in the Eastern District of Texas, but, on Nintendo’s

petition to the Federal Circuit, this Court ordered the case transferred

to the Western District of Washington. In re Nintendo Co., Ltd., et al.,

589 F.3d 1194, 1201 (Fed. Cir. 2009). In June 2010, the district court

stayed the action pending completion of the International Trade

Commission investigation underlying this appeal and completion of the

U.S. Patent and Trademark Office’s reexamination of a patent.


INTRODUCTION1

The individuals behind Motiva, LLC (“Motiva”) dreamed of riches

resulting from an advanced type of exercise equipment that wirelessly

and precisely tracked an individual’s movements. But the dreamers

gave up that fantasy years ago once reality intruded. Creating a

workable device required “vast amounts of money” (A7811), and

Motiva’s only funder walked away. The individuals moved on to other

interests, and their “prototype,” which was “not close to being

production ready,” lay fallow. A7820.

A7836. Today,

only the dream of “‘winnings’ from a lawsuit against Nintendo” of

America Inc. and Nintendo Co., Ltd. (“Nintendo”) (A7822) keeps Motiva

going. But, as the ALJ found both before (A7647) and after (A7837) an

evidentiary hearing, Motiva “failed to demonstrate that a domestic

industry ‘is in the process of being established,’” (id.), a necessary

1 In this brief, all “A” cites are to the parties’ Joint Appendix, and “Br.”cites refer to the appellant’s opening brief. “SVA” refers to theSupplemental Video Recording Media Appendix, which includesdemonstratives submitted to the ALJ (Respondent’s DemonstrativeExhibit Nos. RDX-154, -156, and -158) and filed with this Courtpursuant to Fed. Cir. R. 30(j). For the Court’s convenience, the relevantpatents are reproduced at the back of this brief. The page numbersfrom the Joint Appendix are retained.


kxs

Text Box

Confidential Material Omitted

2

requirement for bringing a complaint before the International Trade

Commission (“ITC” or “Commission”), 19 U.S.C. § 1337(a)(2).

The ALJ also went on to conclude that Nintendo’s Wii System

does not infringe Motiva’s patents. That conclusion is correct, and thus,

to say the least, supported by substantial evidence. The Wii System

does not “track” “movement” of a user, key limitations of Motiva’s

patents. Crediting Nintendo’s witnesses, the ALJ found that the

“evidence adduced at the hearing demonstrates that the three key

devices in the Wii Remote—the three-axis gyroscope, the three-axis

accelerometer, and the DPD—do not, in fact, track the movement of the

user or provide information regarding the position or orientation of the

user.” A7773. The ALJ found Nintendo’s witnesses credible:

A7780.

The ITC’s order adopting the careful factual determinations of the

ALJ should be affirmed.


kxs

Text Box


3

JURISDICTIONAL STATEMENT

Motiva invoked the ITC’s authority under Section 337 of the Tariff

Act of 1930, as amended. A8746. See 19 U.S.C. § 1337(b)(1). The ITC’s

final determination was issued on January 5, 2012. A7876. Motiva

filed a petition for review with this Court on March 5, 2012. A30,562.

Motiva notes this Court’s jurisdiction to review ITC final

determinations under 28 U.S.C. § 1295(a)(6). Br. 1.

STATEMENT OF THE ISSUES

1. Whether the ITC correctly found that Motiva failed to

establish that “an industry in the United States, relating to the articles

protected by” Motiva’s two patents (U.S. Patent No. 7,492,268 and U.S.

Patent No. 7,292,151) was “in the process of being established”

(19 U.S.C. § 1337)?

2. Whether substantial evidence supports the ITC’s factual

finding that Nintendo’s Wii System does not track user movement nor

determine the position or orientation of the user and thus does not

infringe Motiva’s two patents?


4

STATEMENT OF THE CASE

In September 2010, Motiva filed a proposed complaint at the ITC

naming as respondents Nintendo Co., Ltd. and Nintendo of America

Inc. A8746. The ITC issued a Notice of Investigation. See 75 Fed. Reg.

68,379 (Nov. 1, 2010). Administrative Law Judge Robert K. Rogers, Jr.

issued an Initial Determination granting Nintendo’s motion for

summary determination. A7631-49. The ITC vacated the summary

determination, posed a series of factual questions, and remanded for an

evidentiary hearing. A7858-69. The evidentiary hearing took place

from August 1 through August 5, 2011. A7662. The witnesses included

the two named inventors, six fact witnesses (including two Nintendo

employees), and six experts. A7663. Thereafter, the ALJ issued a

second Initial Determination, again finding no violation. A7855.

Motiva petitioned the ITC for review. A8537. The ITC rejected

the petition and ruled that “[t]he ALJ’s conclusion that Motiva has not

proven a violation of section 337 is correct” and “is the Commission’s

final determination.” A7876.2

2 Although the ITC determined to review portions of the InitialDetermination on its own initiative, see A7871, those issues are notraised in this appeal.


5

STATEMENT OF THE FACTS

A. Motiva’s Research And Marketing Efforts: The StartAnd Finish

Kevin Ferguson and Donald Gronachan had a dream. But like so

many get-rich dreams, theirs never even came close to fruition. They

had exactly one investor who abandoned them, got no traction from any

identified licensee, and never created a product—or even a passable

prototype of a product.

The dream was to develop an advanced technology that would

precisely measure human movement for the fitness and rehabilitation

market. A7810. The product they had in mind was a system consisting

of a screen, a base station, and a handheld controller. A9519. They

thought the product might be used for general exercise, athletic

performance training, and physical therapy and research. A30,638.

The Ferguson-Gronachan collaboration began in October 2003.

They formed Motiva as an “informal partnership.” A7810. Mr.

Gronachan had a day job at a company called Biodex Medical Systems.

A7811. Mr. Ferguson devoted time to developing some prototypes—

largely compressed in a 16-month period between October 2003 and

April 2005. A7823. During that period he tried to build a “proof-of-


6

concept” prototype and a “demonstration” prototype. A10,490; see also

A10,179-80, 20,717. The proof-of-concept prototype no longer exists.

A20,717. “The demonstration prototype cannot be described as

anything close to production-ready. . . . [T]he prototype lacks

industrial design, fine-tooling, functional engineering, and test phase

manufacturing.” A7819-20, 10,506-07. See also A7819 (describing the

prototype as having “exposed circuit boards, wiring, and sensors”). It

was the engineering equivalent of a stick figure, albeit an expensive

one: Motiva planned to sell the device for See, e.g., A20,708-

09.

Motiva’s development work was funded—albeit only briefly—by

the only investor the duo ever attracted. His name was David Smith.

Motiva claims that Mr. Smith contributed a sum total of to

engineering, research, and development. A7807-08, 20,523-24. Mr.

Ferguson accepted the majority of this funding— —as salary.

A7807, 7811. Motiva spent the rest on supplies for the prototypes. Id.

Within a year, Mr. Smith pulled the plug on the investment. He

told the duo in 2004 that he had lost interest in supporting the project.

A7811. He realized that Mr. Ferguson and Mr. Gronachan had no hope


kxs

Text Box


7

of bringing anything to market without “vast amounts of money”—far

“more money than he could afford to invest.” Id. By early 2005, Mr.

Smith abandoned the venture without even recovering his investment.

A7814. He did not bother retaining any ownership rights in the

inventors’ work. A7810, 20,532, 20,909, 30,403, 30,404-06.

With their only investor gone, Motiva ground to a halt. Mr.

Ferguson took a job at Liebert Corporation. Now that he had a day job,

Mr. Ferguson had virtually no time to devote to developing the product.

A9534, 10,179-80. While Mr. Ferguson continued for a couple of years

to dabble in the project when time permitted, by 2007 he had

completely abandoned the pretense of working on the invention. A9539.

In the entire four-year period of their collaboration—from 2003 to

2007—Mr. Ferguson and Mr. Gronachan made exactly three pitches in

the hopes of attracting a licensee.

The first pitch, in January 2005, was to James Reiss. Mr. Reiss

was Mr. Gronachan’s boss at Biodex. A7811. The meeting took place in

the basement of Mr. Ferguson’s home. A10,447. The pitch went

nowhere because the technology was not compatible with Biodex’s

products. A7812.


8

Fifteen months elapsed before Motiva had an opportunity for a

second pitch. In March 2006, Mr. Gronachan had several conversations

about the Motiva technology with Gregory Highsmith. A7813. Mr.

Highsmith is one of Mr. Gronachan’s “oldest, long standing friends.”

A20,668. Mr. Highsmith was employed by a commercial fitness

equipment manufacturer known as Life Fitness. “As a member of his

company’s innovation committee, Mr. Highsmith was in a perfect

position to introduce the Motiva technology to Life Fitness, but he did

not do so.” A7814. Life Fitness just was not interested. Id. Mr.

Highsmith stated that he never would have even bothered talking about

the Motiva device but for his “personal relationship” with Mr.

Gronachan. A20,668. See also A7814.

A good nine months later, in January 2007, came the third, and

last, pitch. Mr. Ferguson and Mr. Gronachan met with Michael

Lannon, the CEO of Koko Fitness. A7816. The meeting was held on a

Saturday because, as Mr. Lannon explained, it “wasn’t important

enough to take time out of our actual workweek.” Id. (quoting A10,458);

see also A10,456. It was not even a real opportunity. Koko Fitness was


9

“a small company” with “limited development resources.” A7816

(quoting A10,456).

After this third strike, having already abandoned their

development efforts, Mr. Ferguson and Mr. Gronachan abandoned all

further marketing efforts.

B. Motiva’s Patents

What Mr. Ferguson and Mr. Gronachan did manage to do was to

secure two patents. In July 2004, they filed a provisional patent

application. A7810. The application resulted in U.S. Patent No.

7,292,151, “Human Movement Measurement System,” A1-35, issued on

November 6, 2007, and U.S. Patent No. 7,492,268, A36-67, a

continuation of the ‘151 patent, issued on February 17, 2009.

Both patents relate “to a system and methods for setup and

measuring the position and orientation (pose) of transponders” for

the purposes of “exercise” and “rehabilitation.” A14 at 1:11-13

(emphasis added); A50 at 1:13-15 (emphasis added). Before setting out

the patent more fully, a few words are in order about the bolded terms.

The “position” of any object relative to another object at any

particular time can be described by “x, y and z” coordinates. The three


10

coordinates describe an object’s location in space relative to an origin

point. Here is a graphical image of this coordinate system:

So, for example, using this Court as the point of origin, one can describe

the position of the flag atop the White House in terms of its x, y, and z

relative to the front door of 717 Madison Place. If west-east is the

x-axis, north-south the y-axis, and up-down the z-axis, the position of

the flag relative to the Court would be (in feet) x=+413, y=+734, and

z=+95.

As paradoxical as it might at first sound, an object can move

without changing its position. That is because a change in location is

not the only sort of movement. A pigeon flying from the White House

across Lafayette Square to the Court is obviously moving in the sense of

position. But if the pigeon sits atop the flagpole and spins around, or


11

tilts forward or to the side, it is moving even though its position has not

changed.

An object’s rotation about an axis is referred to as “orientation.”

Orientation is measured as the amount of rotation along each of three

axes at any particular time. “Roll” is the degree of rotation around the

x-axis (e.g., a bird leaning left or right); “pitch” is the degree of rotation

around the y-axis (e.g., a bird looking up or down); and “yaw” is the

degree of rotation around the z-axis (e.g., a bird balancing on one leg

and spinning in circles). See, e.g., A10,306.

“Pose” refers to measurement of both “position” and “orientation.”

See A14 at 1:11-13 (“This invention relates to a system and methods for

setup and measuring the position and orientation (pose) of

transponders.”).

In sum, the pose of an object is described by using six

coordinates—three coordinates specific to position (x, y, z), and three

coordinates specific to orientation (roll, pitch, yaw).

Although not a disputed term, by way of background a

“transponder” is a wireless device that responds to a signal with a

signal of its own. The patent’s specification defines a transponder as a


12

“wireless communication and monitoring device that receives a specific

signal and automatically responds with a specific reply.” A14 at 2:18-

20. A transponder is often “hand-held.” A15 at 4:34-35.

The ‘151 patent discloses an invention that trains the user to

manipulate “the pose of the transponders through a movement

trajectory” for the purposes of exercise and rehabilitation. A14 at

1:13-17 (emphasis added). Immediately below is an image (a composite

of patent Figures 4A, 4B, and 6A, see A10, 12) illustrating how the

system determines the transponder’s position and orientation:


13

In the diagram above, the user (represented by the figure) holds a

transponder. A10. In response to a request from the computer, the

transponder sends a signal (indicated by the blue lines) to the receiver.

To determine the transponder’s position relative to the receiver, the

computer engages in a two-step process: (1) it measures how long it

takes the signal to reach the different points on the receiver; and (2) it

performs certain geometric and algebraic calculations to translate those

measurements into the distance on the x, y, and z axes. A17 at 8:9-11,

23 at 20:12-16, 27 at 27:55-60, 28 at 29:6-7. With slightly different

calculations, the computer can also determine the position of the

transponder relative to an “origin” point other than the receiver. A28 at

29:31-32.

To determine the orientation of the transponder, the computer

calculates the position of three separate points on the transponder. See

A28 at 29:39-45. The three positions define a unique plane, and that

plane reflects the transponder’s roll, pitch, and yaw. Id.


As explained abo

measurements at any

a “movement trajector

the user moves the tra

stores the position and

By recording the trans

user’s movement.


Transponder →

14

ve, position and orientation, or pose, are

particular moment in time. The patents refer to

y.” See, e.g., A14 at 1:14-15; A50 at 1:16-17. As

nsponder, the system repeatedly generates and

orientation measurements:

ponder’s poses over time, the system tracks the

15

The first claim in the patent reads as follows:

A system for tracking movement of a user, comprising:

a first communication device . . .;

a processing system . . . said processing system adapted todetermine movement information for said first communicationdevice and sending data signals to said first communication devicefor providing feedback or control data; and

wherein said first communication device receives and processessaid data signals from said processing system and wherein theoutput device provides sensory stimuli according to the receiveddata signals.

A31 at 35:39-54 (emphasis added). Claim 50 claims an “apparatus for

use in tracking movement of a user.” A32 at 38:47.

The ‘268 patent is a continuation of the ‘151 patent. Claim 1

claims a system for “tracking position of a user.” A66-67 at 34:59-35:8.

Claim 10 claims an “apparatus for use in conjunction with a remote

processing system for tracking position of a user.” A67 at 36:7-8.

Claim 15 is for a “system for tracking movement of a user.” A67 at

36:35.

In 2010, in two separate orders, the PTO ordered reexamination of

each patent. The examiner canceled some claims and confirmed


16

others.3 Nintendo has appealed to the Board of Patent Appeals as to

both patents.

C. Motiva’s Patent Litigation

1. Motiva Files An ITC Action Against Nintendo

In November 2008, Motiva commenced a patent infringement

action against Nintendo in the United States District Court for the

Eastern District of Texas. This Court ordered the litigation transferred

to the Western District of Washington, where Nintendo of America is

located. In re Nintendo Co., Ltd., et al., 589 F.3d 1194, 1198

(Fed. Cir. 2009). In June 2010, the district court in Washington stayed

the action pending the outcome of the reexamination proceedings at the

Board of Patent Appeals. A few months later, Motiva sued Nintendo in

the ITC, alleging that Nintendo violated Section 337 by importing and

selling the Wii game system and related items. A8746.

Motiva’s allegations center on the Wii Remote, the primary

controller for the Wii System. Here is a picture of the Wii Remote Plus:

3 As to the ’151 patent, the examiner canceled claims 1-11, 17-20, 26, 35,36, 38-43, 45-56, 58-64, 66, 67, 69-72, 77, 78, 82, 83, and 85-91. Theexaminer confirmed claims 12-16, 21-25, 27-34, 37, 44, 57, 65, 68, 73-76,79-81, and 84. As to the ‘268 patent, the examiner canceled claims 1, 3,and 13, confirmed claim 4, and found claims 2, 5-12, 14, and 15patentable.


17

As relevant here, the Wii Remote Plus has three key devices:

(1) a three-axis gyroscope

(2) a three-axis accelerometer

(3) a particular type of camera referred to as a Direct Pointing

Device (“DPD”).

The Wii System was released in the United States in November

2006, A8276, two full years after Mr. Smith stopped funding Motiva.

A7811. The Wii sells for approximately $149. A8278.

“Motiva argues that these three devices contained within the Wii

send movement and position information” to the Wii console. A7773.

The complaint asserted that the Wii System infringes three

independent claims and various dependent claims of Motiva’s patents.


18

All the asserted ‘151 claims require “tracking movement of a user.”4

A8746. All but one of the asserted ‘268 claims require “tracking

position of a user.”5 Id.

The ITC opened an investigation.

2. The ALJ Grants Nintendo’s Motion For SummaryDetermination

In Order No. 12, the ALJ granted Nintendo’s “motion for summary

determination that the economic prong of the domestic industry

requirement is not satisfied.” A7631. The ALJ noted that a patent

complainant invoking the ITC’s authority must establish that an

industry “relating to the articles protected by the patent . . . exists or is

in the process of being established” in the United States, A7634

(quoting 19 U.S.C. § 1337(a)(2)).

After summarizing the research and marketing effort set forth by

Motiva, the ALJ ruled that “Motiva’s investments do not support a

4 The complaint, as corrected, asserted eight claims dependent onclaim 1 (claims 16, 27-32, 44) and four claims dependent on claim 50(57, 68, 81, and 84).

5 The complaint asserted claim 1 of the ‘268 patent and 7 claims(claims 2-6 and 8-9) dependent on claim 1, as well as claim 10 and 4claims (11-14) dependent on claim 10. The complaint also assertedclaim 15, which refers to “tracking movement of a user.”


19

finding of either the existence of a domestic industry or that a domestic

industry was in the process of being established at the time of filing of

the complaint in this investigation.” A7637. The ALJ found that

“Motiva ceased investing in research and development, marketing, or

any step towards creating a domestic industry, in 2007.” A7648. The

ALJ reasoned that: (i) “Motiva does not directly dispute that it had

stopped investing in research and development and marketing for the

asserted patents in 2007”; (ii) “Motiva’s patent litigation activities are

not to be considered as part of the domestic industry analysis, and its

prosecution of ‘related applications’ is not relevant to the domestic

industry analysis”; and (iii) Motiva’s “assertion that once it wins its

patent infringement litigations” it will “reenter the market” is

“insufficient” to establish a domestic industry because to “find otherwise

would render the domestic industry requirement a nullity.” Id. The

ALJ granted summary determination to Nintendo. A7631.

3. The ITC Reverses And Remands With Questions

Motiva and the Office of Unfair Import Investigations petitioned

for review. The ITC reversed and remanded. The ITC explained that

the “legislative history indicates that an industry is ‘in the process of


20

being established’” if the patent owner “can demonstrate that he is

taking the necessary tangible steps to establish such an industry in the

United States,” A7863-64 (citing S. Rep. No. 100-71, at 130 (1987)), and

there is a “significant likelihood that the industry requirement will be

satisfied in the future,” A7864 (citing H.R. Rep. No. 100-40, at 157

(1987)). The ITC ruled that the “ALJ erred in declining to consider

Motiva’s activities that occurred before the issuance of the asserted

patents.” A7863 (citing A7672).

The ITC also found that Motiva had raised a “genuine issue of

material fact as to whether its district court litigation activities between

2007 and the present are related to licensing and/or product

development.” A7865. The ITC directed the ALJ to address “to the

extent necessary” several questions “relevant” either “to whether a

domestic industry exists” or “is in the process of being established.”

A7867.

4. The ALJ Holds A Hearing, Answers The ITC’sQuestions, And Finds No Violation

The ALJ holds the hearing. At the hearing, the ALJ heard

from nine witnesses called by Motiva: the two inventors; the one short-

term investor; two of the three targets of pitches (Mr. Highsmith and


21

Mr. Reiss); three experts; and a Nintendo engineer. A7663. Nintendo

called three experts, a Nintendo manager, and Mr. Barry French (the

CEO of Trazer Technologies and Mr. Ferguson’s former boss). Id.

At the hearing, Nintendo’s expert testified that “Motiva

abandoned any domestic industry that it might have had in process.”

A10,488. He based his conclusion “on the fact that Mr. Ferguson’s work

. . . waned significantly . . . in early 2005, and then stopped entirely in

January 2007” and that “Mr. Gronachan . . . did not put in any

significant effort at any time.” Id. In response to one of Motiva’s

experts, Mr. Bakewell prepared the following graph to illustrate the

point:


22

The ALJ finds no ITC authority over Motiva’s suit. Based on

a careful review of the trial record, the ALJ concluded that “Motiva has

failed to prove that a domestic industry ‘exists’ pursuant to Section

337(a)(2).” A7832. The ALJ summarized the record:

(1) well before the release of the Wii, Motiva was facing alack of funding due to the departure of David Smith;


kxs

Text Box


23

(2) prior to the Wii being released, Motiva met withpotential partners, but they showed little to no interest ininvesting in the Motiva technology;

(3) the one company that Motiva met with after the releaseof the Wii—Koko Fitness—was not interested in investing inthe technology for reasons other than the fact that theWii was already on the market;

(4) Motiva has offered no evidence that any potentialinvestor, partner, or licensee was dissuaded in investing inthe Motiva technology due to the Wii;

(5) the Wii was not in the same market as the Motivaproduct, and the two products would not have competed;

(6) Motiva’s actions in litigation are not consistent with acompany whose main purpose is to remove Nintendo fromthe market so that it can enter the market; and

(7) Motiva has not demonstrated that litigation was its onlyrecourse to salvage its business in the face of Nintendo’srelease of the Wii.

A7830-31 (emphasis added).

The ALJ also provided the following answers to the ITC’s

questions relevant to the existence of a domestic industry (with all

emphasis added):

Commission Question ALJ Findings

1[a]. What was the level of interestfrom potential manufacturers,investors, and licensees in Motiva’stechnology prior to the release ofthe Wii?

“I find that there was littleinterest from potentialmanufacturers, investors, andlicensees in Motiva’s technologyprior to release of the Wii.” A7814.


24

1[b]. Did Nintendo’s release of theWii cause this interest to decrease?

“I now find that Motiva has failedto demonstrate that there wasany decline in interest causedby the release of the Wii.” A7815.

1[c]. To what extent would theproduct(s) being developed byMotiva compete with Nintendo’sWii?

“I find that the product developedby Motiva would not competewith the Nintendo Wii.” A7817.

2. How close was Motiva’stechnology to beingcommercialized and/or productionready?

“I find that Motiva’s technologywas not close to beingincorporated in a commercial orproduction-ready product.” A7819.

3[a]. To what extent was Motiva’sshift in production-orientedactivities to litigation-orientedactivities a strategic businessdecision not caused by Nintendo’sactivities?

“I find no evidence to supportMotiva’s claim that the turn tolitigation was the result of beingrejected by potential investors dueto the presence of the Wii.” A7822.“Motiva’s litigation tacticsstrongly suggest that thepurpose behind the litigationwas to extract a monetaryaward either through damages ora financial settlement.” A7823.

3[b]. Could Motiva have continuedits commercialization effortswithout resorting to litigation?

“It was possible that Motiva couldhave continued itscommercialization efforts withoutresorting to litigation, but it wouldhave taken a new source ofmoney to do so.” A7823.

In addition to finding that no domestic industry existed, the ALJ

concluded that “Motiva has failed to demonstrate that a domestic


25

industry ‘is in the process of being established.’” A7832-37. The ALJ

found that “[f]rom 2003 to 2007, Motiva was taking tangible steps to

establish an industry in the United States.” A7833. But the “building”

and “market[ing]” “activities ended in 2007.” A7833-34. “Therefore,”

the ALJ found, “after 2007, Motiva abandoned its efforts to establish an

industry in the United States.” A7834 (citing A10,488-90). “Mr.

Bakewell’s credible testimony supports this conclusion.” Id. The ALJ

also rejected Motiva’s argument that the “litigation against Nintendo is

evidence of Motiva taking the necessary tangible steps to establish an

industry.” Id.

Not only was Motiva not taking tangible steps to establish an

industry, the ALJ found that “Motiva has not demonstrated that there

is a significant likelihood that the industry requirement will be satisfied

in the future.” Id. “Motiva could not demonstrate that there was any

significant interest in its technology prior to the Wii’s existence, and I

find that there is no reason to believe that manufacturers of fitness and

rehabilitation equipment will suddenly become interested in the

technology because Nintendo has been excluded from the market.”

A7835. Moreover, the ALJ found that “Motiva has not shown that it


26

has ever attracted interest from the video game industry.” Id. In

addition, the ALJ found that

A7836. The ALJ found

that “Motiva failed to demonstrate that a domestic industry ‘is in the

process of being established,’ pursuant to Section 337(a)(2).” A7837.

In so holding, the ALJ answered the ITC’s questions relevant to

any future industry:

3[c]. Was Motiva taking the“necessary tangible steps toestablish” a domestic industry?

“I find that after 2007, Motivaabandoned its efforts toestablish an industry in theUnited States.” A7834. “I do notfind that the litigation againstNintendo is evidence of Motivataking the necessary tangible stepsto establish an industry.” Id.

4[a]. Do the steps “taken [byMotiva] indicate a significantlikelihood that the industryrequirement will be satisfied in thefuture?”

4[b]. How likely is it that Motivawill have a domestic industry inthe future (1) if no relief is issuedagainst Nintendo or, alternatively,(2) if relief is granted againstNintendo?

“I conclude that Motiva has notdemonstrated that there is asignificant likelihood that theindustry requirement will besatisfied in the future, and thisdetermination is not dependent onthe Commission’s actions in thisinvestigation.” Id.


kxs

Text Box


27

The ALJ also finds no infringement. In addition to rejecting

Motiva’s complaint for failure to establish a condition for an ITC action,

the ALJ separately found that Nintendo’s Wii System does not infringe

the Motiva patents. A7854-55.

In so doing, the ALJ’s Initial Determination construed certain

claim terms. The phrase “tracking movement of a user” appears in the

preamble of independent claims 1 and 50 of the ‘151 patent. A31-32.

The ALJ ruled that the phrase means “tracking changes of position

and/or orientation of a user.” A7674.

The ALJ’s construction was different from Nintendo’s reading of

the patent. Nintendo argued that “tracking movement of a user”

required tracking the position and orientation of the user. Among

other things, Nintendo noted that the patent describes the invention as

“measuring the position and orientation (pose) of transponders.” A7673

(emphasis added). Nevertheless, the ALJ found that “tracking

movement of a user” can mean tracking either the position or

orientation of the transponder. A7672.

In addition to construing the claims to require “tracking changes

of position and/or orientation of a user,” the ALJ construed “position


28

information.” The ALJ explained that the “parties’ proposed

constructions do not vary greatly,” and “[t]he primary dispute centers

on Nintendo’s inclusion of ‘3D space.’” A7678. The ALJ found that

“[t]he specification explains that the invention may be used ‘for the

purposes of functional movement assessment for exercise, and physical

medicine and rehabilitation,’” Id. (citing A14 at 1:16-17), and “[s]uch

tracking requires knowledge of the user’s location in 3D space,” id.

(citing A10,330-31). Although Motiva invoked certain statements by

the examiner made during the reexamination of the ‘151 patent, the

ALJ found them “not persuasive” in light of the “broader standard for

claim construction during reexamination” and because the

“reexamination of the ‘151 patent is not complete.” A7679.

After construing the key terms, the ALJ found that “Motiva has

failed to show by a preponderance of the evidence that the accused

products infringe claim 1 of the ‘151 patent.” A7782. At the hearing,

the ALJ viewed live demonstrations in which Nintendo expert

Dr. J. Edward Colgate and Keizo Ohta, Nintendo’s Manager of the

Technology Group in the Entertainment Analysis and Development

Division (A7663), played Wii games and explained how the Wii creates


29

the appearance of tracking the user’s movement without knowing the

user’s position or orientation. A21,593-618, 30,447-56, 30,552-53,

30,576-79, 30,581-83; see generally, SVA at RDX-154, -156, and -158.

The ALJ found that the “evidence adduced at the hearing demonstrates

that the three key devices in the Wii Remote—the three-axis gyroscope,

the three-axis accelerometer, and the DPD—do not, in fact, track the

movement of the user or provide information regarding the position or

orientation of the user.” A7773. The ALJ concluded:

A7780. Instead,

the Wii

Id.

(emphasis added); see id. (“After careful review of the record evidence, I

find that Motiva has failed to meet its burden to demonstrate that the

Wii accused products track the movement of a user.”).

The ALJ applied the same analysis to find no infringement of the

‘268 patent.


kxs

Text Box


30

5. The ITC Affirms

Motiva appealed again to the ITC, but this time the ITC affirmed.

A7871. The Commission determined that “[t]he ALJ’s conclusion that

Motiva has not proven a violation of section 337 is correct and is the

Commission’s final determination. The investigation is terminated.”

A7876.

SUMMARY OF ARGUMENT

The Court should affirm the ITC’s decision to adopt the ALJ’s

careful determination. The ALJ gave two independent reasons for

rejecting Motiva’s complaint. Nothing in Motiva’s brief calls these

reasons into question.

I.A. Substantial evidence supports the ALJ’s finding that Motiva

was not “in the process” of “establishing” any domestic industry related

to these patents. In a detailed opinion issued after an extensive

hearing, the ALJ found that “after 2007, Motiva abandoned its efforts to

establish an industry in the United States.” A7834.

Motiva makes no effort to challenge the ALJ’s finding that its

licensing efforts have “ended.” Id. Motiva does not deny that its main

investor walked away, does not claim to have another funding source,


31

and does not claim to have done anything since 2007 (except sue

Nintendo). Motiva has stopped trying to establish any industry, and its

completed actions are not evidence that it will do any research or

marketing in the future.

B. Against these and similar dispositive factual findings, Motiva

makes a variety of limited and erroneous points. For example, much of

Motiva’s brief turns on its contention (at 44) that “[b]ut for the presence

of the Wii in the market, Motiva was—and still is—extremely close to

realizing its product-driven licensing goals.” The ALJ specifically

rejected this fiction. A7812-13 (citing A20,633-34). The ALJ also ruled

that “Motiva could not demonstrate that there was any significant

interest in its technology prior to the Wii’s existence,” and “that there is

no reason to believe that manufacturers of fitness and rehabilitation

equipment will suddenly become interested in the technology because

Nintendo has been excluded from the market.” A7835.

This Court has explained that “Congress recognized that the

Commission is fundamentally a trade forum, not an intellectual

property forum, and that only those intellectual property owners who

are ‘actively engaged in steps leading to the exploitation of the


32

intellectual property’ should have access to the Commission.” John

Mezzalingua Assocs. v. ITC, 660 F.3d 1322, 1327-28 (Fed. Cir. 2011)

(quoting H.R. Rep. No. 100-40, at 157 (1987)). Here, Motiva’s bald

assertion that it is a “licensing company” is not sufficient to allow it to

invoke the ITC’s jurisdiction. Motiva has nothing but its patents. If

Motiva can invoke the protections of the ITC, any and every patent

owner can do so. This Court does not permit such a lax definition of the

ITC’s role.

II. The Wii System does not infringe Motiva’s patents. Because

Motiva did not establish any basis for bringing its complaint to the ITC,

the ITC decision should be affirmed on that basis alone. Nevertheless,

the ALJ reached, and the ITC adopted, a second rationale for rejecting

Motiva’s complaint: The Wii System does not infringe Motiva’s patents.

A. Substantial evidence supports the Commission’s determination

that the Wii System does not “track” movement of a user. A7777-82.

The ALJ found that the Wii System does not take “complete

measurements” of user movement, does not measure user movement

with “reasonable accuracy,” and does not keep any “representation of

the movement history in memory.”


33

Motiva argues (at 25-26) that the ALJ’s construction of “tracking”

“directly contradicts the Commission’s apparent construction of

‘tracking movement of a user.’” Br. 24-25 (citing A7674 at ¶¶ 3-4). This

is inexplicable. The cited paragraphs do not construe the term

“tracking.” In any event, the Wii System does not “track” movement.

B. Moreover, and independently, substantial evidence supports

the ITC’s finding that the Wii System does not determine user

movement. As three witnesses that the ALJ found persuasive

explained, the Wii sensors cannot and do not determine the position or

orientation of the transponder. Motiva’s hodgepodge of movement and

position information arguments are all meritless.

STANDARD OF REVIEW

“[F]indings of fact [by the ITC] are reviewed to ascertain whether

they were supported by substantial evidence on the record as a whole.”

Erbe Elektromedizin GmbH v. ITC, 566 F.3d 1028, 1033

(Fed. Cir. 2009). The Court routinely affirms ITC decisions that are

supported by substantial evidence. See, e.g., Ninestar Tech. Co. v. ITC,

667 F.3d 1373, 1379 (Fed. Cir. 2012); TianRui Group Co. v. ITC, 661

F.3d 1322, 1337 (Fed. Cir. 2011); Mezzalingua, 660 F.3d at 1330.


34

Claim construction is an issue of law this court reviews de novo.

Tessera, Inc. v. ITC, 646 F.3d 1357, 1364 (Fed. Cir. 2011). Infringement

decisions are a question of fact reviewed for substantial evidence. Id.

ARGUMENT

I. SUBSTANTIAL EVIDENCE SUPPORTS THE ALJ’SFINDING THAT MOTIVA WAS NOT IN THE PROCESS OFESTABLISHING ANY DOMESTIC INDUSTRY

As the ALJ realized from the get-go, Motiva is not entitled to

invoke the ITC’s authority. Motiva could not prove that a domestic

industry is “in the process of being established,” 19 U.S.C. § 1337(a)(2),

without establishing: (1) “that [it] is taking the necessary tangible

steps to establish such an industry in the United States,” and (2) that

“there is a significant likelihood that the industry requirement will be

satisfied in the future.” Certain Stringed Musical Instruments &

Components Thereof, Inv. No. 337-TA-586, USITC Pub. No. 4120, at 13

(Dec. 2009) (Comm’n Op.) (quoting S. Rep. No. 100-71, at 130 (1987) &

H.R. Rep. No. 100-40, at 157 (1987)).

Motiva’s brief reads as if the ALJ made no findings. But in a

detailed opinion issued after an extensive hearing, the ALJ found that

“after 2007, Motiva abandoned its efforts to establish an industry in the


35

United States.” A7834. The decision is supported by substantial—

indeed, overwhelming—evidence and should be affirmed.

A. Motiva’s Stale Licensing Efforts Do Not DemonstrateAny Tangible Steps Toward Establishing A DomesticIndustry

The ALJ’s two opinions here call to mind the Monty Python “dead

parrot” sketch. There, John Cleese returns to the pet shop to register a

complaint about his dead Norwegian Blue parrot that had been nailed

to its perch. The shop owner insists that the bird is “resting,” “pining

for the fjords,” or simply “stunned.” Cleese responds by banging “Polly

Parrot” on the counter and rattles off several metaphors for death: the

bird “is no more,” “has ceased to be,” “bereft of life, it rests in peace.”

The sketch is one of the most famous in the history of British television

comedy.

The propped-up parrot here is Motiva’s technology, and the ALJ is

playing the role of Cleese by repeatedly and emphatically declaring the

technology dead. As the ALJ found both before (A7648) and after

(A7835-36) an evidentiary hearing, Motiva’s technology was “bereft of

life” by the time Motiva filed its complaint in 2010. Motiva’s only

investor quit in 2004. The ALJ noted that the “Motiva technology has


36

not been updated or improved since at least December 2007.” A7820

The ALJ found that Motiva’s licensing efforts “ended in 2007.” A7834-

35. “Thus, on October 1, 2010, the relevant date for determining a

domestic industry, it had been at least 3.5 years since the end of the

[sic] Motiva’s engineering, research and development, and

commercialization activities.” A7826. Motiva has no future products

and no potential customers. Instead, Motiva exists solely to seek a cut

in the hoped-for contingent lawsuit “‘winnings.’” A7822. The ALJ notes

that Motiva does not “directly” challenge this version of events. A7648.

Instead, Motiva emphasizes (at 40) the ALJ’s ruling that “Motiva

was taking tangible steps to establish an industry in the United States”

from 2003 to 2007.6 Motiva argues this finding is evidence that it was

seeking to establish a domestic industry when it filed this complaint in

2010.7 And these same steps form the basis of Motiva’s claim (at 48)

that the “public interest” supports ITC authority here.

6 On appeal, Motiva has abandoned its argument that ITC authoritycould be based on the “existence” of a relevant domestic industry.Compare A8023 with Br. at 10.

7 For purposes of this appeal, Nintendo does not dispute the ALJ’sfinding that Motiva’s activities before the end of 2007 were steps towardestablishing a licensing program. But the record is far from clear thatthe sporadic effort set forth by Motiva amounts to a “substantial”


37

But Motiva cannot turn old news into future prospects. The ALJ

found that “all” of Motiva’s licensing efforts “ended in 2007.” A7834.

“Motiva lost its primary source of funding when David Smith ended his

investment in late 2004, and Motiva has never found a suitable investor

to replace him,” id., nor did it try, A10,489. “Since the end of 2007,

Motiva has focused on nothing except suing Nintendo.” A7834. Motiva

did not have any executed licensing agreements, term sheets, letters of

intent, confidentiality agreements, licensing negotiations, trade show

demonstrations, or any offers made or received to license Motiva’s

technology. See A8751 (“Motiva has not licensed the Asserted

Patents.”). See also A20,686, 20,687, 20,689, 20,690, 20,706, 20,722,

20,723. The ALJ found that “after 2007, Motiva abandoned its efforts to

establish an industry in the United States.” A7834; see also id. (“Mr.

Bakewell’s credible testimony supports this conclusion” (citing A10,488-

90)).

Motiva makes no effort to challenge the ALJ’s finding that its

licensing efforts have “ended.” A7834. Motiva does not deny that Mr.

Smith walked away, does not claim to have another funding source, and

investment in licensing. Cf. Br. 51 (asserting there is “no dispute”regarding pre-2008 activities).


38

does not claim to have done anything since 2007 (except sue Nintendo,

discussed below in section I.C). Motiva has stopped trying to establish

any industry, and its completed actions are not evidence that it will do

any research or marketing in the future. Compare Bally/Midway Mfg.

Co. v. ITC, 714 F.2d 1117, 1122 (Fed. Cir. 1983) (“if there was an

existing . . . [domestic industry] when the complaint was filed, section

337(a) was satisfied.”) (emphasis added); see In re Certain Rotary Wheel

Printing Sys., Inv. No. 337-TA-185, USITC Pub. No. 1857, at 42 (May

1986) (holding that Bally/Midway “was attempting to take account of

the situation where an industry is destroyed in the course of a

Commission investigation”).

Although it is not even attempting to license its patents, Motiva

continues to refer to itself (at 39, 43) as a “product-driven licensing

company.” But Motiva has no products, no licenses, and no drive. It is

done. Motiva began marketing its technology as early as January 2005,

yet it has never come close to negotiating a license with a third party,

let alone having its technology incorporated into a commercial product.

A7820, 8751, 10,486-87, 20,686, 20,687, 20,689, 20,723. After Nintendo

observed that “Motiva has never offered to license its patents to


39

Nintendo,” Motiva wrote: “Nintendo has it right. Motiva’s litigation is

not about licensing Nintendo.” A30,364 (emphasis added).

Also meritless is Motiva’s claim (at 39) to a focus on the “video

game environment.” Its patents disavow any real interest in the video

game business. See A21 at 15:38-43 (“It is important to note that only

primitive forms of video game challenges would be considered, to take

into account the user’s cognitive awareness and physical limitations,

and the economics of software development for photo realistic virtual

environments and animation.”) (emphasis added); A57 at 15:29-34

(same). Motiva has made no effort to compete with Nintendo in the

video game market. A20,708-09.

But even if Motiva is a licensing company, it has not shown the

required “substantial licensing activities related to the asserted patent”

necessary to satisfy the future domestic industry requirement.

InterDigital Commc’ns, LLC v. ITC, No. 2010-1093, 2012 U.S. App.

LEXIS 15893, at *32 (Fed. Cir. Aug. 1, 2012). Since 2007, the ALJ

found that Motiva engaged in no licensing activities related to any

patents. A7834. Motiva complains that the ALJ “faults Motiva at

length for its lack of consummated licensing deals.” Br. 46 (citing


40

A7814) (emphasis added). That is rather like referring to a monk’s lack

of consummated marriages. The ALJ’s finding was not just that the

deals were not “consummated,” but that they never got past the first

date: The ALJ found that “nothing ever came of those meetings.”

A7814. They do nothing to show the required tangible steps toward a

future domestic industry.

As for public policy, “Congress recognized that the Commission is

fundamentally a trade forum, not an intellectual property forum, and

that only those intellectual property owners who are ‘actively engaged

in steps leading to the exploitation of the intellectual property’ should

have access to the Commission.” Mezzalingua, 660 F.3d at 1328

(quoting H.R. Rep. No. 100-40, at 157 (1987)). Motiva’s bald assertion

that it is a “licensing company” is not sufficient to allow it to complain

to the ITC. Motiva has nothing but its patents. If Motiva can invoke

the protections of the ITC, any and every patent owner can do so. That

is not the scheme Congress established. Id.


41

B. Motiva’s Lawsuit Against Nintendo Is Not A TangibleStep Toward Establishing A Domestic Industry

The ALJ rejected Motiva’s argument that its district court

“litigation against Nintendo is evidence of Motiva taking the necessary

tangible steps to establish an industry.” A7834.

In Coaxial Cable, the Commission held that for litigation activities

to constitute “exploitation” of a patent under Section 337(a)(3)(C), the

complainant must “clearly link” its litigation efforts to licensing.

Certain Coaxial Cable Connectors and Components Thereof and

Products Containing Same, Inv. No. 337-TA-650, USITC Pub. No. 4283,

at 51 (Apr. 14, 2010) (Comm’n Op.). As the Commission explained,

“[a]llowing patent infringement litigation activities alone to constitute a

domestic industry would place the bar for establishing a domestic

industry so low as to effectively render it meaningless.” Id. at 46.

Motiva does not dispute that this settled agency practice is the correct

understanding of Section 337(a)(3)(C). See InterDigital, 2012 U.S. App.

LEXIS 15893, at *32 (noting that “[i]f there were any ambiguity” about

the scope of Section 337(a)(3)(C), the Commission’s “consistent

interpretation of the statute . . . would be entitled to deference”).


42

Here, the ALJ concluded that “the litigation against Nintendo

does not relate to exploitation of the asserted patents.” A7832. Among

the ALJ’s reasons for so holding were:

“Emails between the inventors before the start of any litigation showthat they were interested in their potential ‘winnings’ from alawsuit against Nintendo.” A7822 (citing A10,548); A30,547 (“Themoney is suppose [sic] to come from Laniers [sic] cut of the winningsand not from our cut.”) (emphasis added).

“Motiva’s decision not to file a complaint at the ITC from the outsetor seek a preliminary injunction against Nintendo shows that Motivawas not concerned with taking swift actions to remove Nintendo fromthe market.” A7823.

“The inventors could have continued to seek investors, licensees, orpartners instead of, or in conjunction with, suing Nintendo.” A7824.

Moreover, the ALJ held that even if the litigation is related to the

patents, Motiva has not made a significant investment in the district

court litigation. “Motiva’s lawyers are working under a contingent fee

agreement.” A7831. “Motiva has not paid any attorney’s fees or

expenses related to Motiva’s litigation against Nintendo.” Id.

Despite all this, Motiva writes (at 52) that “the undisputed

evidence establishes that Motiva did not pursue litigation as a mere

patent owner.” According to Motiva, “it is important to remember that

Motiva’s litigation effort was in further service of its licensing goals.”


43

Br. 52. But it is Motiva that has amnesia. The ALJ found that “the

litigation against Nintendo does not relate to exploitation of the

asserted patents.” A7832. If this litigation is “not about licensing,”

A30,364, then it cannot be evidence of Motiva’s “substantial investment

in . . . licensing,” 19 U.S.C. § 1337(a)(3)(C).

Motiva’s litigation-inspired arguments also forget a host of other

adverse factual findings. Motiva says (at 54) that the evidence showed

“Motiva and several business partners alike all saw the Wii as a

substantial roadblock to Motiva’s success.” But, as detailed further

below (at 47-49), the ALJ rejected this view of events. Motiva “failed to

demonstrate that there was any decline in interest caused by the

release of the Wii.” A7815. Motiva invokes the wisdom of its founders

on the necessity of litigation. Br. 54. But the ALJ specifically rejected

this assertion: “I do not concur with the inventors’ assertion that

because of the Wii, they had no other recourse but to sue Nintendo.”

A7824. Motiva notes that it “invested in the litigation,” Br.

58, but the ALJ ruled this time “insubstantial.” A7831; see also A7817

(“I give no weight to this wholly unsupported testimony from clearly

interested witnesses.”).


kxs

Text Box


44

Motiva’s “public policy” points in support of its ‘litigation alone is

enough’ theory of ITC authority misconstrues what the ALJ held.

Motiva states that the ALJ “implicitly held that when a company seeks

financial compensation for patent infringement, as well as injunctive

relief, it necessarily belies a bona fide domestic industry.” Br. 55. But

the ALJ made no such global ruling, “implicitly” or otherwise. The ALJ

pointed to Motiva’s failure to invoke certain procedures, such as a

preliminary injunction or exclusion order. Those procedures are

designed to protect industry. Motiva’s decision not to invoke those

protective procedures is a decision that confirms the lack of an industry

to protect. Where, as here, the complainants are on record as excited

about their litigation “winnings” and have ended any efforts to license

or commercially exploit the patents, a patent owner cannot cite

litigation focused on monetary relief as the sole evidence of steps toward

establishing an industry.

Motiva cites Certain Multimedia Display and Navigation Devices

and Systems, Components Thereof, and Products Containing the Same,

Inv. No. 337-TA-694, USITC Pub. No. 4292 (Nov. 2011) (Final), for the

premise that its litigation efforts should have been given more weight in


45

the ALJ’s domestic industry analysis. In Certain Multimedia, the ITC

held that “section 337(a)(2) . . . requires complainant’s activities relate

to licensing.” Id. at 13. Here, however, the ALJ found as a matter of

fact that Motiva failed to establish any connection between its litigation

efforts and any licensing. No matter how frequently Motiva asserts

that it is in the licensing business, the ALJ found as a matter of fact

that its litigation against Nintendo had nothing to do with licensing.

A7830-31. Litigation unrelated to licensing, as Motiva concedes, cannot

establish a domestic industry. Br. 52.

Motiva’s reliance on Certain Digital Satellite System Receivers and

Components Thereof is similarly misplaced. Inv. No. 337-TA-392,

USITC Pub. No. 3418, at 11 (Apr. 2001) (Initial and Recommended

Determinations). The patentee in Certain Digital Satellite had reached

license agreements to the patent-in-suit with four licensees and every

single employee of the patentee was “responsible for maintaining [its]

system of identifying, approaching, and negotiating with prospective

licensees.” Id. at 10-11. As Motiva acknowledges, the Certain Digital

Satellite litigation efforts were “an extension of its licensing program.”


46

Id. at 11 (emphasis added). But here, there is no licensing program

other than the litigation effort.

C. The Wii System’s Success Does Not Suggest MotivaHas Any Chance Of Establishing A Domestic IndustryIn The Future

The ALJ ruled that “Motiva could not demonstrate that there was

any significant interest in its technology prior to the Wii’s existence,”

and “there is no reason to believe that manufacturers of fitness and

rehabilitation equipment will suddenly become interested in the

technology because Nintendo has been excluded from the market.”

A7835. In addition, the ALJ found that

A7836.

Against these dispositive findings, Motiva makes three limited

and erroneous points.

Much of Motiva’s brief turns on its contention that “[b]ut for the

presence of the Wii in the market, Motiva was—and still is—extremely

close to realizing its product-driven licensing goals.” Br. 44; see Br. 41-

42, 58-59. For example, Motiva cites the testimony of Reiss (he “would

still be willing to invest in a license to Motiva’s technology today if

Motiva has the power to exclude Nintendo”) and of Highsmith (“there is


kxs

Text Box


47

potential for a product that has [Motiva’s] characteristics”). Br. 44.

Motiva argues that “[s]ubstantial evidence supports a conclusion that

Motiva’s invention competes with the Wii.” Br. 50.

The ALJ explained why “the Wii Fit [an exercise game] would not

compete with the Motiva product.” A7818. “Even though the Wii Fit

may be viewed by some as a fitness product, it is not a serious fitness

product like the one described in the Motiva documentation.” Id.

“Based” on this review of the evidence, the ALJ found “that the Motiva

product was intended to be an expensive tool used by people in the

physical rehabilitation and fitness industries, while the Nintendo Wii is

a relatively inexpensive video game system for home consumers that is

intended to appeal to a mass market.” A7819. See, e.g., A20,708-09 (“Q:

Now, do you think a normal American consumer is going to pay

for a device and put it in their living room to play video games? A: No.”).

Motiva makes no effort to dispute these substantial facts the ALJ relied

on to find no competition between the Wii System and any Motiva

product.

Moreover, any argument that the November 2006 introduction of

the Wii System doomed Motiva’s domestic industry ignores the ALJ’s


kxs

Text Box


48

findings. “The proper date for determining whether [Motiva’s licensing

efforts] constituted an ‘industry’ entitled to protection under section 337

was the date on which the complaint was filed.” Bally/Midway, 714

F.2d at 1121. Motiva’s complaint was filed in September 2010. A8759.

But, as detailed above at 25, the ALJ properly found that Motiva’s

efforts, if any, to establish a domestic industry had ceased by 2007.

A7829. Any effect of the Wii System on Motiva’s prospects must have

likewise ended by 2007, three years before the complaint was filed here.

The ALJ also specifically rejected the fiction that Motiva’s

licensing efforts were hurt by the introduction of the Wii System.

“Neither Mr. Reiss nor Motiva has offered any evidence to support the

assertion that [Mr. Reiss’s company] would license the Motiva patents,

but for the presence of the Wii.” A7812. Instead, the ALJ found that

Mr. Reiss’s “real interest lies in excluding Nintendo from the market” to

protect his “physical therapy device” from competition from the Wii Fit

game. A7812-13 (citing A20,633-34). And the ALJ found that Mr.

Highsmith had “little interest” in Motiva; he never viewed the prototype

or patent application. A7814; see also A7815 (“[E]ven though Mr.

Highsmith was aware of Motiva’s technology prior to the Wii’s release,


49

he never showed enough interest to view the prototype, view Motiva’s

patents or patent applications, hold a meeting with the inventors in

[his] offices, or present the invention to [his company’s] innovation

committee.”).8

Motiva incorrectly states (at 59-60) that the ALJ’s findings are

“premised largely” on its noninfringement ruling. According to Motiva,

should this Court find that the Wii System infringes the Motiva patents

(it does not, see below at Section II), then the ALJ’s analysis of the

prospects of Motiva’s products also falls. But the ALJ expressly stated

that its factual findings were independent of the infringement ruling:

“Motiva has not demonstrated that there is a significant likelihood that

the industry requirement will be satisfied in the future, and this

determination is not dependent on the Commission’s actions in

this investigation.” A7834 (emphasis added). The ALJ noted

testimony by Motiva’s inventors to the contrary, but the ALJ did “not

find this testimony to be persuasive,” A7835, dismissing it as “self-

interested testimony of the inventors.” A7834. In short, regardless of

8 Among other things, the ALJ also rejected Motiva’s “claim that thetechnology is currently ready to be commercialized.” A7820 (“theMotiva prototype was not close to being production-ready”).


50

whether the Wii System does not or does infringe Motiva’s patents, the

ALJ found that Motiva still had no prospects of marketplace success.

No one was interested before the Wii System arrived, and no one would

be interested if the Wii System suddenly disappeared from the market.

For all these reasons, the ALJ was correct to find that the Wii

System’s success had nothing to do with the failure of Motiva’s

technology. But in any event, the Court need only find that substantial

evidence supports that conclusion. This Court affirms agency findings

where substantial evidence supports several competing conclusions.

Nippon Steel Corp. v. United States, 458 F.3d 1345, 1352

(Fed. Cir. 2006) (“‘[E]ven if it is possible to draw two inconsistent

conclusions from evidence in the record, such a possibility does not

prevent [the Commission’s] determination from being supported by

substantial evidence.’” (quoting Am. Silicon Techs. v. United States, 261

F.3d 1371, 1376 (Fed. Cir. 2001))).


51

II. SUBSTANTIAL EVIDENCE SUPPORTS THE ALJ’SFINDING THAT THE WII SYSTEM DOES NOT INFRINGEMOTIVA’S PATENTS

In light of how the Wii System actually operates, substantial

evidence supports the ITC’s conclusion that the Wii System does not

infringe Motiva’s patents and must be affirmed. First, the Wii System

does not “track” movement of the user—a fact which, by itself, is a basis

on which to affirm. Second, and independently, the Wii System does

not determine user “movement.” It does not determine the position or

orientation of the user (under the ALJ’s construction). Nor does it

determine the position and orientation of the user (under the proper

construction Motiva incorrectly says the ALJ actually applied).

Motiva’s scattershot effort to challenge these rationales is unpersuasive.

A. Substantial Evidence Shows That The Wii SystemDoes Not “Track” Movement Of A User

Substantial evidence supports the Commission’s determination

that the Wii System does not “track” movement of a user. A7777-82.

There was no dispute as to what one of skill in the art understood

“tracking” movement to mean. The only witness to testify on this topic,

Professor Colgate, explained that


kxs

Text Box


52

A7777

(citing A30,437-38); see also A10,320-21. This is also how the patents

and others of skill in the art use the term “tracking.” A10,321-30.

The ITC’s holding—that the Wii lacks each of the three elements

that define tracking—is supported by the documents and the testimony.

First,

A7777; see A30,430-33.

A7777-78.

A30,430.

Second, the ALJ found that the Wii does not measure user

movement with A7777.

A7777.


kxs

Text Box


53

Third, the ALJ found that the Wii System did not keep any

Id. In an effort

to contradict this finding, Motiva invokes its expert, Dr. Jaswinder Pal

Singh,

A9920; see also A7779. But the ALJ found that Dr. Singh was

not qualified as an expert on motion-sensing devices (A20,154); he was

an expert in computer systems and electrical engineering. A20,153.

The ALJ was justified in finding that Nintendo manager

A7779.

Id. (citing A30,566, 30,592, 30,593-94, 30,596).

A7779.

Id. (emphasis in original).


kxs

Text Box


54

Simply put, tracking a fictional position in a three-dimensional virtual

world is not the same as tracking the human playing the game.

Motiva argues that the ALJ’s construction of “tracking” “directly

contradicts the Commission’s apparent construction of ‘tracking

movement of a user.’” Br. 24-25 (citing A7674 at ¶¶ 3-4). Nonsense.

The cited paragraphs do not construe the term “tracking.” The

paragraphs construe “tracking movement of a user” to mean “tracking

changes of position and/or orientation of a user.” A7674 (emphasis

added). So as relevant here, all that one can glean from this claim

construction is that the Commission said “tracking” means “tracking.”

The bottom line is that the Wii System does not “track”

movement, and nothing in Motiva’s brief casts doubt on that fact. This

alone requires affirmance.

B. Substantial Evidence Shows That The Wii SystemDoes Not Determine User “Movement”

In light of the above, this Court need not proceed to the next step

and address whether the Wii System determines user “movement” as

required by Motiva’s patents. But should the Court reach the issue, the

answer is “No”: The ALJ correctly found that the Wii sensors do not

determine user “movement,” i.e., “changes of position and/or orientation


55

of a user” (or, consistent with Nintendo’s construction, changes of

position and orientation of a user). A7772. Motiva’s attacks on that

finding are meritless.

1. Three Credible Witnesses Demonstrated ThatThe Wii Sensors Cannot And Do Not DeterminePosition Or Orientation

To understand why the Wii System does not determine the

position or orientation of the user, it is useful to understand how the

Wii System actually works. As anyone who has played it or watched

the videos submitted with this brief might ask, isn’t the whole fun of the

Wii System that it responds to the player’s movement? See A7752

(“90% of consumer purchase decisions are based upon the Wii’s ability

to track motion”). In fact, the Wii System “trick[s]” the user into

thinking it knows the user’s movement. A20,439. The ALJ found that

the Wii “create[s] a convincing display of character movement on the

screen through animations based on real time computations that merely

give the appearance of tracking player movements.” A7780 (emphasis

added).

The ALJ found Nintendo’s witnesses credible:


kxs

Text Box


56

A7780. Mr. Ohta participated in the development of the Wii prototype

and designed many of the Wii’s most popular games, including Wii

Sports. A21,395, 21,401. He explained, in testimony endorsed by the

ALJ, how the illusion of tracking user movement was implemented in

the popular Wii Tennis game.


kxs

Text Box


57

Thus, the Wii System determines which tennis animation to

display without determining either the position or orientation.

A7762 (citing A21,479-80, 21,482, 30,575). See also SVA

at RDX-156.

The record includes various other examples that confirm the Wii

System works without tracking the user’s position or orientation.


kxs

Text Box


58

Dr. Colgate explains:

A30,452.

So too with boxing. As demonstrated in the Wii Boxing video,

SVA at RDX-154, the


kxs

Text Box


59

In sum, the Wii System does not determine either the position or

orientation of the user.

2. Motiva’s Hodgepodge Of Movement And PositionInformation Arguments Are All Meritless

Regarding the alleged infringement of the movement claim term,

Motiva raises a host of scattered arguments that are all meritless.

1. In an attempt to circumvent the ALJ’s findings, Motiva now

emphasizes that the gyroscope measures the “changes” in orientation by

measuring angular velocity. Motiva asserts that the ALJ erred by

evaluating “infringement based on whether the Wii determines the

‘position or orientation’ of the user instead of changes in the position

and/or orientation of a user.” Br. 22 (Motiva’s emphasis). Motiva

argues that the Wii’s gyroscope detects “changes in orientation over

time.” Id. at 23 (Motiva’s emphasis).


kxs

Text Box


60

But it is Motiva’s arguments that have changed. Before trial and

at the hearing, Motiva argued that the Wii System “determine[s]”

(A30,129) or “describes” (A9797) the orientation of the transponder.

Only later, in the petition to the ITC, did Motiva advance its newfound

theory that what matters is that the Wii System determines orientation

“changes.” See, e.g., A8602. The law is clear that arguments not raised

before the ALJ are waived. Finnigan Corp. v. ITC, 180 F.3d 1354, 1363

(Fed. Cir. 1999) (“The argument at the trial and appellate level should

be consistent, thereby ensuring a clear presentation of the issue to be

resolved.”).

In any event, the Wii System does not measure changes in the

orientation or position of the user.

It is undisputed that gyroscopes detect only “angular velocity.”

Br. 13. Angular velocity is the speed of rotation around an axis. A7755;

see A20,160; 20,174.

A20,160, 20,174; see also A30,423.


kxs

Text Box


61

A7775 (citing A20,412-15). Motiva agrees that the

gyroscope “cannot detect where an object is facing at a particular

moment in time.” Br. 13. The ALJ also embraced testimony from

A7776-77; see also A30,423-24, 30,430-31.

A7776-77.

Motiva argues that the accelerometers determine “changes in

position.” Br. 14 (emphasis added). It is undisputed that

accelerometers do nothing but measure acceleration. As with the

orientation argument, Motiva’s “changes” position argument is new and


kxs

Text Box


62

waived. See, e.g., A20,194

Moreover, the ALJ cited

substantial evidence in support of the finding that the accelerometers

do not determine position or any changes in position.

A7775.

A20,285

The ALJ also again credited Dr. Colgate’s testimony.

A7774.

A7775.

A7774.

Gravity provides a constant acceleration on every object; all objects


kxs

Text Box


63

register acceleration even when not moving. Id. (citing A30,417-18).

Therefore, one cannot simply calculate out the gravitational

acceleration because, for example,

A30,553.

A7775 (citing A20,234).

To the extent Motiva argues that the accelerometers determine

orientation,

A7776. Motiva’s patents concede that accelerometers can

only be used to determine pitch and roll when the device is “static.” A30

at 33:60-61; see also A30,418.

Furthermore, the ALJ found that Mr. Rabin

A7775. The ALJ

noted that Mr. Rabin had explained his presentation to Wii developers


kxs

Text Box


64

A7775-76. But

the ALJ noted that Mr. Rabin had

A7776. In other words,

Id.

And the ALJ found that

Id.

Id. Of course, measuring only

pitch and roll of a stationary transponder cannot determine user

movement.

In short, Motiva’s last-minute change in argument would be

unpersuasive even if it had not been waived.

Moreover, Motiva’s new infringement argument still requires

“tracking”—something the Wii System does not do. See above at II.A.

2. Next, Motiva argues that “movement” and “position” do not

necessarily involve movement or position in all three spatial


kxs

Text Box


65

dimensions. Br. 15-18, 26. It is enough, Motiva argues, if a device can

figure out “that an object is ‘above’ or ‘to the left of’ another object.” Id.

at 26.

The ALJ’s opinion demonstrates why Motiva is wrong to insist

that two-dimensional detection is enough under the claims. The ALJ

noted that the patent “specification explains that the invention may be

used ‘for the purposes of functional movement assessment for exercise,

and physical medicine and rehabilitation.’” A7678. “Such tracking,”

the ALJ held, “requires knowledge of the user’s location in 3D space.”

Id. A “user” is a human being, not a cartoon, which means that the user

resides, and moves, in three-dimensional space. Tracking in fewer than

three dimensions is not tracking “the user” as the claims require.

“Motiva cites no evidence from the ‘151 patent that supports a finding

that the claimed invention may track position in only one or two

dimensions.” A7679. Thus, the ALJ found “it appropriate to include

express reference to 3D space in the proposed construction.” A7678.

Motiva contends that the plain and ordinary meaning of position

and movement information does not require three-dimensional

tracking. In support of this view, Motiva invokes (at 26) Retractable


66

Technologies v. Becton, Dickinson & Co., 653 F.3d 1296, 1306

(Fed. Cir. 2011), and (at 19) Northrop Grumman Corp. v. Intel Corp.,

325 F.3d 1346, 1355-56 (Fed. Cir. 2003). Those cases stand for the rule

that “statements from the description of the preferred embodiment . . .

are just that—descriptions of a preferred embodiment” and that such

statements do not disavow claim scope absent a clear disclaimer.

Northrop Grumman, 325 F.3d at 1356; see also Retractable, 653 F.3d at

1306 (specification disclosed an embodiment that operates by “cutting”

and mere statements about difficulties with prior art “cutting” type

devices did not disclaim claim scope).

This rule, however, has no bearing here because the ALJ did not

find that the asserted patents disclosed and then disclaimed

embodiments that track in one or two dimensions. Rather, the ALJ

found that every embodiment of “the invention” tracks the user in three

dimensions, and Motiva cited “no evidence . . . that the claimed

invention may track position in only one or two dimensions.” A7679.

This finding is confirmed by the specification’s statement that “[t]he

present invention described can be used . . . to measure frequency and

amplitude of body sway in three dimension (3D) space.” A21 at 16:8-13.


67

And it is further confirmed by the statement that “[t]he present

invention” can be used to determine whether an “individual’s movement

trajectory varied from the intended two dimensional (2D) reference

movement trajectory by deviation from the planar path into the

uninvolved spatial dimension,” which would require tracking in three

dimensions. A22 at 18:3-16. Thus, where the specification recites that

“the invention” measures in three dimensions and only discloses

embodiments that track the user in three dimensions, tracking

movement and position must be in three dimensions. That is the only

construction that properly “tether[s] the claims to what the

specifications indicate the inventor actually invented.” Retractable, 653

F.3d at 1305.

None of Motiva’s arguments come close to overcoming the

conclusion compelled by the specification. Motiva cites (at 15-16, 20) to

a “dual axis inertial sensor” in a preferred embodiment. But the

specification explains that this sensor is optional; the sensor is used to

provide supplemental pitch and roll information. A28 at 30:43-31:50;

see also A30,538. Moreover, the asserted patents acknowledge that

such dual axis inertial sensors can only measure pitch and roll when


68

“static,” making them useless for tracking movement. A30 at 33:60-61.

Motiva mischaracterizes a “Limits of Stability” test by asserting

that it measures body sway in just two dimensions. Br. 16. The

specification states twice that “body sway [is measured] in three

dimension [sic] (3D) space.” A21 at 16:12-13, 22 at 7:23-24.

See A30,539-40

Motiva also mischaracterizes the specification by stating that it

“describes tracking using ‘mechanical, inertial, acoustical or

electromagnetic radiation sensors.’” Br. 19. The passage that Motiva

cites, however, concerns the prior art—not the claimed invention:

“Known are commercial tracking and display systems that employ . . .

mechanical, inertial, acoustical or electromagnetic radiation sensors to

determine a mobile object’s position and orientation, referred to

collectively as pose.” A14 at 1:18-21.


kxs

Text Box


69

Motiva also cites to three prior art references that allegedly

disclose tracking humans in fewer than three dimensions. Br. 16-18.

As the ALJ recognized, however, Motiva’s citations to the prior art

merely shift the focus away from the central inquiry: what one skilled

in the art would understand “tracking” a “user” to mean in the asserted

patents. A7678-79.

As explained above, supra at 60-64 neither the gyroscopes nor the

accelerometers determine, much less track, position or orientation.

Motiva also emphasizes the Wii System’s camera, the DPD. Br. 37-39.

But, again, the ALJ’s fact findings are dispositive.

The ALJ had substantial evidence to support the finding that the

DPD does not determine position or orientation in any dimension. Dr.

Colgate

A7773.

A7774.

Thus, as demonstrated at the hearing,


kxs

Text Box


70

A7773. Indeed,

Id.

3. Motiva’s brief includes an attempt to circumvent its burden

under the substantial evidence standard by claiming that the ALJ first

construed “movement information” correctly but then “applied a

construction for ‘movement information’ that was inconsistent with its

initial construction.” Br. 32.

Motiva is wrong on the law, wrong on what the ALJ did here, and

wrong on the correct claim construction.

Motiva asserts that when an ALJ construes terms in one fashion

and then applies them in an arguably different fashion, the de novo

standard of review applies. Id. at 32. In support of its argument,

Motiva incorrectly describes the holding of Vita-Mix Corp. v. Basic

Holding, 581 F.3d 1317 (Fed. Cir. 2009). Motiva contends that Vita-Mix

stands for the proposition that “a lower court’s application of a

construction that is ‘inconsistent with its earlier claim construction’ is


kxs

Text Box


71

[reviewed] de novo to determine correct construction.” Br. 32. Vita-Mix,

however, says no such thing. Vita-Mix was an appeal from a summary

judgment of non-infringement, and thus the Court merely applied the

settled rule that grants of summary judgment are reviewed de novo.

Vita-Mix, 581 F.3d at 1323.

More instructive here is Tessera, Inc. v. ITC, 646 F.3d 1357 (Fed.

Cir. 2011), which Motiva asserts, without explanation, does not apply.

Br. 32-33. Like Motiva, Tessera won its claim construction, but the ITC

nevertheless found no infringement. Tessera, 646 F.3d at 1363-64.

Like Motiva, Tessera appealed to this Court and argued that the ITC

“initially adopted a correct claim construction, but ‘halfway through its

infringement analysis, the ITC inexplicably switched to an incorrect

claim construction’” and, thus, the ITC’s finding of noninfringement

should be reviewed “de novo, as an error in claim construction.” Id. at

1364. The Court, however, found that Tessera could not argue that the

ITC applied the wrong construction because it adopted Tessera’s

proposed construction and thus its “contention at best [was] a

disagreement over the Commission’s application of Tessera’s

construction to the accused . . . devices.” Id. Thus, the Court held that


72

Tessera’s appeal was a challenge of the ITC’s “infringement

determination,” and should be “review[ed] for substantial evidence.” Id.

For the same reasons, the substantial evidence burden applies here.

In any event, the ALJ in fact applied the construction of “tracking

movement of a user” as “tracking changes of position and/or orientation

of a user.” The ALJ adopted Motiva’s construction of “movement

information,” construing it to mean “information about changes in

position and/or orientation.” A7674, 7676. The ALJ referred to this

construction at the outset of the infringement discussion: “The question

becomes whether or not the accused products ‘track changes of position

and/or orientation of a user.’ I find that they do not.” A7772. The ALJ

then described Motiva’s arguments regarding the gyroscope,

accelerometer, and DPD, and held they “do not, in fact, track the

movement of the user or provide information regarding the position or

orientation of the user.” A7773 (emphasis added).

A7777. And, indeed,

that is Nintendo’s view. But the ALJ ruled differently. A7674.


kxs

Text Box


73

Nothing in this accurate recitation of Dr. Colgate’s testimony suggests

that the ALJ had a change of heart.

A7777.

The ALJ was citing Dr. Colgate for the point that the Wii System

determines neither position nor orientation.

Nintendo continues to disagree with the ALJ’s construction of

movement. But the Court can and should resolve this appeal without

deciding whether the claims require determining both position and

orientation. As the ALJ held, the Wii System does not track either a

user’s position or a user’s orientation. A7773. There is no need to

decide whether a device that tracks only position would infringe the

claims.

4. Motiva repeatedly invokes the reexamination process of its

patents. Br. 14, 16, 18, 19-20, 27-31. But the reexamination is ongoing

and in any event not meaningfully relevant here.

Motiva emphasizes its “own actions” during reexamination. Br.

27-30. According to Motiva, its decision to press on with rejected claims

rather than canceling or amending the claims suggests that it did not

disavow the broader claim scope. Br. 28. But nothing in the ALJ’s


kxs

Text Box


74

claim construction turns on whether or not Motiva disavowed any claim

scope.

Motiva complains (at 28) that the ALJ should not have

disregarded the (still incomplete) reexamination record. Motiva cites

cases that stand for the proposition that a patent owner’s statements

during an incomplete prosecution can limit the claims.9 The cases also

suggest that any points raised by an examiner in a prosecution can be

evaluated for substantive persuasiveness.10

Under this authority, the ALJ’s decision to give little weight to the

incomplete examination was proper. Motiva seeks to use incomplete

proceedings to expand rather than narrow its claims. And, as the ALJ

9 See, e.g., In re Katz Interactive Call Processing Patent Litig., 07-ml-01816-BRGK (FFMx), 2008 WL 4952454, at *5 (C.D. Cal. Feb. 21, 2008)(“[T]he prosecution history can often inform the meaning of the claimlanguage . . . making the claim scope narrower than it would otherwisebe.”) (emphasis added), aff’d 639 F.3d 1303 (Fed. Cir. 2011); BeneficialInnovations, Inc. v. Blockdot, Inc., Nos. 2:07-CV-263-TJW-CE, 2:07-CV-555-TJW-CE, 2010 U.S. Dist. LEXIS 54151, at *8 (E.D. Tex. June 3,2010) (“[S]tatements made by a patentee during reexamination todistinguish a claim from the prior art may serve to limit the scope of theclaim.”) (emphasis added).

10 See, e.g., St. Clair Intellectual Prop. Consultants, Inc. v. Canon Inc.,412 F. App’x 270, 276-77 (Fed. Cir. 2011); SRAM Corp. v. AD-II Eng’g,Inc., 465 F.3d 1351, 1357, 1359 (Fed. Cir. 2006) (rejecting constructionadopted by examiner after three reexaminations).


75

found, “the statements made by the Examiner regarding claim

construction are not persuasive.” A7679; see also supra II.A. Motiva

repeatedly notes (at 14, 16, 18, 19-20) that the examiner adopted

Nintendo’s definition of “movement.” But the statements that Motiva

cites were made under the express qualification that these are not

Nintendo’s arguments for the purposes of litigation because federal

courts use a narrower standard than the PTO in reexamination. The

ALJ’s treatment of the reexamination proceedings provides no basis for

reversal.

CONCLUSION

For the foregoing reasons, the ITC’s decision should be affirmed.


76

Date: August 29, 2012 Respectfully submitted,

/s/ Mark S. Davies

Mark S. DaviesKatherine M. KoppORRICK, HERRINGTON & SUTCLIFFE LLP1152 15th Street, N.W.Washington, D.C. 20005(202) [email protected]

E. Joshua RosenkranzPeter A. BicksAlex V. ChachkesORRICK, HERRINGTON & SUTCLIFFE LLP51 West 52nd StreetNew York, NY 10019(212) 506-5000


Attorneys for Nintendo Co., Ltd. andNintendo of America Inc./Intervenors


77

CERTIFICATE OF COMPLIANCEUNDER FEDERAL RULES OF APPELLATE PROCEDURE

32(A)(7) AND FEDERAL CIRCUIT RULE 32

Counsel for Intervenors Nintendo Co., Ltd. and Nintendo of

America Inc. certify that the brief contained herein has a proportionally

spaced 14-point typeface, and contains 13,791 words, based on the

“Word Count” feature of Word 2007, including footnotes and endnotes.

Pursuant to Federal Rule of Appellate Procedure 32(a)(7)(B)(iii) and

Federal Circuit Rule 32(b), this word count does not include the words

contained in the Certificate of Interest, Table of Contents, Table of

Authorities, Abbreviations, and Statement of Related Cases.

Dated: August 29, 2012 Respectfully submitted,

/s/ Mark S. DaviesMark S. Davies


CERTIFICATE OF SERVICE

I certify that on the 29th day of August, 2012, the foregoing Non-

Confidential Answering Brief of Intervenors Nintendo Co., Ltd. and

Nintendo of America Inc. was electronically filed using the Court’s

CM/ECF System, which will automatically serve all counsel of record.

I further certify that two copies of the foregoing Non-Confidential

Answering Brief of Intervenors Nintendo Co., Ltd. and Nintendo of

America Inc. will be served at the time the paper copies of the brief are

submitted to the Court:

Clark S. CheneyWayne W. HerringtonDominic L. BianchiU.S. International Trade Commission500 E Street, S.W., Suite 707Washington, D.C. [email protected]

Counsel for Appellee International Trade Commission

Christopher D. BanysLanier Law Firm, P.C.2200 Geng Rd.Suite 200Palo Alto, CA [email protected]

Counsel for Appellant Motiva, LLC

/s/ Mark S. DaviesMark S. Davies


Addendum


Motiva, LLC v. International Trade CommissionFed. Cir. Appeal No. 2012-1252

Index to Addendum

DOCUMENT PAGE NO.

Patent No. 7,292,151, Dated November 6, 2007....................... A1-35

Patent No. 7,492,268, Dated February 17, 2009 .................... A36-67


,-,

UNITED STATES DEPARTMENT OF COMMERCE

United States Patent and Trademark Office

June 08, 20lO

THIS IS TO CERTIFY THAT ANNEXED HERETO IS A TRUE COpy FROM

THE RECORDS OF THIS OFFICE OF:

U.S. PATENT: 7,292,151

ISSUE DATE: November 06,2007

By Autbority of tbe

Under Secretary of Commerce for Intellectual Property and Director of tbe United States Patent and Trademark Office

,-,



June 08, 20lO



U.S. PATENT: 7,292,151

ISSUE DATE: November 06,2007

By Autbority of tbe

Under Secretary of Commerce for Intellectual Property and Director of tbe United States Patent and Trademark Office

JX-001

A000001


(12) United States Patent Ferguson et al.

(54) HUMAN MOVEMENT MEASUREMENT SYSTEM

(76) Inventors: Kevin Ferguson, 8156 Camp den Lakes Blvd., Dublin, OH (US) 43016; Donald Gronachan, 4 Spiral Rd., Holtsville, NY (US) 11742

( *) Notice: Subject to any disclaimer, the tenn of this patent is extended or adjusted under 35 U.S.c. 154(b) by 47 days.

(21) Appl. No.: 111187,373

(22) Filed: Jul. 22, 2005

(65) Prior Publication Data

US 2006/0022833 Al Feb. 2, 2006

Related U.S. Application Data

(60) Provisional application No. 60/592,092, filed on luI. 29,2004.

(51) Int. CI. G08B 23/00 (2006.01)

(52) U.S. CI. ............................... 340/573.1; 340/407.1; 434/114

(58) Field of Classification Search ............. 340/573.1,

(56)

340/573.4,539.12,539.13,539.22,407.1, 340/825.36; 3811315; 4341112,114

See application file for complete search history.

References Cited

U.S. PATENT DOCUMENTS

4,337,049 A 611982 Connelly .................... 434/247 4,375,674 A 311983 Thornton .................... 364/559 4,627,620 A 1211986 Yang ....................... 273/1 GC 4,631,676 A 1211986 Pugh .......................... 364/413 4,645,458 A 211987 Williams .................... 434/251

4,695,953 A 911987 Blair et al. ................. 364/410 4,702,475 A 1011987 Elstein et aI ............. 273/1 GC 4,751,642 A 611988 Silva et aI . ................. 364/413 4,817,950 A 411989 Goo ....................... 273/148 B

111111 1111111111111111111111111111111111111111111111111111111111111 US007292151B2

(10) Patent No.: US 7,292,151 B2 Nov. 6,2007 (45) Date of Patent:

4,912,638 A 3/1990 Pratt .......................... 600/595

4,925,189 A 5/1990 Braeunig ................ 273/148 B 5,148,154 A 9/1992 Mackay et al. ............. 3401712 5,184,295 A 2/1993 Mann ......................... 364/410

5,214,615 A 5/1993 Baur .......................... 367/128 5,227,985 A 7/1993 DeMenthon ................ 364/559

5,229,756 A 7/1993 Kosugi et al. .............. 340/706 5,239,463 A 8/1993 Blair ............................. 463/3

5,255,211 A 10/1993 Redmond ................... 365/401

5,288,078 A 2/1994 Capper et al. .......... 273/148 B 5,320,538 A 6/1994 Baum ......................... 434/307

5,347,306 A 9/1994 Nitta .......................... 348/578 5,372,365 A 12/1994 McTeigue et al. ....... 273/182.2

5,375,610 A * 12/1994 LaCourse et al. ........... 600/595

(Continued)

FOREIGN PATENT DOCUMENTS

WO PCTlUS96117580 5/1997

OTHER PUBLICATIONS

Reality built for two: a virtual reality tool, Symposium on Interactive 3D Graphics, ACM Press webpages from http://portal.acm.org/ citation.cfm?id~91385.91409&dl+ACM&type~series&i (Jun. 10, 2004) 1-4.

(Continued)

Primary Examiner-Toan N. Pham (74) Attorney, Agent, or Firm-Standley Law Group LLP

(57) ABSTRACT

A system for measuring the position of transponders for testing and training a user to manipulate the position of the transponders while being guided by interactive and sensory feedback through a bidirectional communication link to a processing system for the purpose of functional movement assessment for exercise and physical rehabilitation.

91 Claims, 10 Drawing Sheets



(76) Inventors: Kevin Ferguson, 8156 Camp den Lakes Blvd., Dublin, OH (US) 43016; Donald Gronachan, 4 Spiral Rd., Holtsville, NY (US) 11742

( *) Notice: Subject to any disclaimer, the tenn of this patent is extended or adjusted under 35 U.S.c. 154(b) by 47 days.

(21) Appl. No.: 111187,373

(22) Filed: Jul. 22, 2005


US 2006/0022833 Al Feb. 2, 2006



(51) Int. CI. G08B 23/00 (2006.01)

(52) U.S. CI. ............................... 340/573.1; 340/407.1; 434/114

(58) Field of Classification Search ............. 340/573.1,

(56)

340/573.4,539.12,539.13,539.22,407.1, 340/825.36; 3811315; 4341112,114


References Cited


4,337,049 A 611982 Connelly .................... 434/247 4,375,674 A 311983 Thornton .................... 364/559 4,627,620 A 1211986 Yang ....................... 273/1 GC 4,631,676 A 1211986 Pugh .......................... 364/413

4,645,458 A 211987 Williams .................... 434/251

4,695,953 A 911987 Blair et al. ................. 364/410

4,702,475 A 1011987 Elstein et aI ............. 273/1 GC 4,751,642 A 611988 Silva et aI . ................. 364/413

4,817,950 A 411989 Goo ....................... 273/148 B

111111111111111111111111111111111111111111111111111111111111111111111111111 US007292151B2

(10) Patent No.: US 7,292,151 B2 Nov. 6,2007 (45) Date of Patent:

4,912,638 A 3/1990 Pratt .......................... 600/595

4,925,189 A 5/1990 Braeunig ................ 273/148 B 5,148,154 A 9/1992 Mackay et al. ............. 3401712 5,184,295 A 2/1993 Mann ......................... 364/410

5,214,615 A 5/1993 Baur .......................... 367/128

5,227,985 A 7/1993 DeMenthon ................ 364/559

5,229,756 A 7/1993 Kosugi et al. .............. 340/706 5,239,463 A 8/1993 Blair ............................. 463/3

5,255,211 A 10/1993 Redmond ................... 365/401

5,288,078 A 2/1994 Capper et al. .......... 273/148 B 5,320,538 A 6/1994 Baum ......................... 434/307

5,347,306 A 9/1994 Nitta .......................... 348/578

5,372,365 A 12/1994 McTeigue et al. ....... 273/182.2

5,375,610 A * 12/1994 LaCourse et al. ........... 600/595

(Continued)


WO PCTlUS96117580 5/1997

OTHER PUBLICATIONS

Reality built for two: a virtual reality tool, Symposium on Interactive 3D Graphics, ACM Press webpages from http://portal.acm.org/ citation.cfm?id~91385.91409&dl+ACM&type~series&i (Jun. 10, 2004) 1-4.

(Continued)

Primary Examiner-Toan N. Pham (74) Attorney, Agent, or Firm-Standley Law Group LLP

(57) ABSTRACT



JX-001.0002

A000002


US 7,292,151 B2 Page 2

u.s. PATENT DOCUMENTS 6,400,452 Bl 6,430,997 Bl 6,487,906 Bl * 6,515,593 Bl * 6,720,876 Bl 6,749,432 B2 6,765,726 B2 6,774,885 Bl 6,834,436 B2 * 6,876,496 B2

6/2002 Maynard ................. 356/141.1

5,385,519 A 5,405,152 A 5,423,554 A 5,429,140 A 5,466,200 A 5,469,740 A 5,474,083 A * 5,485,402 A * 5,495,576 A 5,516,105 A 5,524,637 A 5,577,981 A 5,580,249 A 5,584,700 A 5,587,937 A 5,591,104 A 5,597,309 A 5,616,078 A 5,638,300 A 5,641,288 A 5,645,077 A 5,656,904 A 5,659,691 A 5,702,323 A 5,703,623 A 5,704,837 A 5,711,304 A * 5,715,834 A 5,720,619 A 5,759,044 A 5,785,630 A 5,785,631 A 5,790,076 A 5,790,124 A 5,792,031 A 5,812,257 A 5,838,816 A 5,846,086 A 5,850,201 A 5,872,438 A 5,888,172 A 5,890,995 A 5,913,727 A 5,929,782 A * 5,963,891 A 5,989,157 A 6,004,243 A 6,028,593 A 6,043,873 A 6,050,822 A 6,050,963 A 6,054,951 A 6,066,075 A 6,073,489 A 6,077,201 A 6,088,091 A 6,098,458 A 6,100,896 A 6,119,516 A * 6,132,337 A 6,152,856 A 6,162,191 A 6,164,973 A 6,181,343 Bl 6,183,259 Bl 6,198,528 Bl 6,244,987 Bl 6,308,565 Bl 6,346,045 B2 6,361,507 Bl 6,366,272 Bl

111995 Hsu et al. ..................... 482/57 411995 Katanics et al ............. 273/438 611995 Davis ......................... 273/437 711995 Burdea ....................... 600/587

1111995 Uerich .......................... 482/4 1111995 French et al ............. 73/379.04 1211995 Church et al. .............. 600/546

111996 Smith et al. ................ 702/160 211996 Ritchey ...................... 395/125 511996 Eisenbrey et al. ...... 273/148 B 611996 Erickson ..................... 1281779

1111996 Jarvik ........................... 482/4 1211996 Jacobsen et al. .............. 434/11 1211996 Feldnian et al ............. 434/247 1211996 Massie ....................... 364/578

111997 Andrus .......................... 48217 111997 Riess ......................... 434/258 411997 Oh ................................ 463/8 611997 Johnson ................. 364/551.01 611997 Zaenglein, Jr. ... ... ... ..... 434/21 711997 Foxlin ........................ 1281774 811997 Lander .................. 318/568.12 811997 Durward et al. ............ 395/329

1211997 Poulton ......................... 482/8 1211997 Hall ........................... 345/158

111998 Iwasaki et al ................ 463/38 111998 Dower ....................... 600/523 211998 Bergamasco et al. ....... 1281782 211998 Fisslinger ................... 434/336 611998 Redmond ................... 434/307 711998 Bobick .......................... 482/4 711998 Heidecke ....................... 482/5 811998 Sypniewski ................. 342/365 811998 Fischer et al. .............. 345/435 811998 Alton ... ... ..... ... ... ... ...... 482178 911998 Teitel et al .............. 356/141.4

1111998 Holmberg ................... 382/157 1211998 Bizzi et al. ................. 434/247 1211998 Lasko-Harvill ................ 345/8 211999 Roston .................. 318/568.11 311999 Andras et al. .. ... ... ... ... ... 48217 411999 Bobick .......................... 482/4 611999 Ahdoot. ... ..... ... ... ... ...... 463/38 711999 Stark et al. ............ 340/870.01

1011999 Walker et al. .............. 702/150 1111999 Walton .......................... 482/4 1211999 Ewart ............................ 482/8 212000 Rosenberg ................... 434/11 3/2000 Ramer.. ... ..... ... ... ... 356/139.03 4/2000 Faughnn ...................... 434/11 4/2000 Johnson et al. ............. 600/595 4/2000 Sypniewski ................. 342/465 5/2000 Poulton ......................... 482/8 6/2000 French et al ............. 73/379.04 612000 Cheng ......................... 482/57 7/2000 Ramer ..................... 356/141.5 8/2000 French et al ............. 73/379.04 812000 Strohecker et al. ......... 345/427 912000 Hock ...................... 73/379.01

1012000 Krupka .......................... 482/8 1112000 Studor ........................... 482/8 1212000 Foxlin ........................ 600/595 1212000 Macri ......................... 434/247

112001 Lyons ........................ 345/358 212001 Macri ......................... 434/247 3/2001 Maynard ................. 356/141.1 6/2001 Ohsuga ......................... 482/4

10/2001 French et al ............. 73/379.04 212002 Rider ... ... ..... ... ... ... ...... 463/31 3/2002 Foxlin ........................ 600/595 4/2002 Rosenberg .................. 345/156

8/2002 French et al. ............ 73/379.04 12/2002 Hock ...................... 73/379.01 212003 Stark et al. ............ 340/870.07 4/2004 Burgess ................... 340/568.1 6/2004 French et al. ............... 434/247 7/2004 French et al. ............... 359/630 8/2004 Even-Zohar ................ 345/156

1212004 Townsend et al. ............ 33/512 412005 French et al. ............... 359/630

200210183961 Al 12/2002 French et al. ............... 702/150

OTHER PUBLICATIONS

Europe is Bursting with Virtual Reality Ideas, But Developers Are Critically Strapped for Cash, webpages from https:llwww/lexis. comlresearch/retrieve? _m~66dI7057c 1 b77fl97 aledb9f5 fadb87 d &_browseType~Text, (Jan. 1993) 1-2. Allard, P., et al, Three-Dimensional Analysis of Human Movement, Human Kinetics (1995) 3, 8-14. Brownstein, B., et al, Functional Movement in Orthopedic and Sports Physical Therapy, Churchill Livingstone (1997), 15. Brugger, W., et ai, Computer-aided tracking of body motions using a c.c.d.-image sensor, Med. BioI. Eng. & Comput, (Mar. 1978), 207-210. Codella, c., et ai, Interactive Simulation in a Multi-Person Virtual World ACM (May 3-7, 1992), 329-334. DeLoura, M., et ai, Game Programming Gems, Charles River Media, (2000) 200-204. Greenleaf, W.J., DataGlove, DataSuit, and virtual reality Advanced technology for people with disabilities, Proceedings of the Seventh Annual Conference 'Technology and Persons with Disabilities,' (Mar. 1992) 211-214. Kasvand, T., et al, Computers and the Kinesiology of Gait, Comput. BioI. Med. Pergamon Press (1976) vol. 6 111-120. Kenmochi, A., et al, A network virtual reality skiing system-system overview and skiing movement estimation, Symbiosis of Human and Artifact, (Jul. 1995) 423-428. Kraus, A., Matrices for Engineers, Hemisphere Publishing Corp. (1987) 118-120, 124-126. Lengyel, E., Mathematics for 3D Game Programming & Computer Graphics, Charles River Media (2004) 76-78, 467-468. Medved, v., Towards a virtual reality-assisted movement diagnostics-an outline, Robotica (Jan.-Feb. 1994) vol. 12, 55-57. Mulder, A., Human movement tracking technology, School of Kinesiology, Simon Fraser University (Jul. 1994) 1-14. Ruby, D., Biomechanics-how computers extend athletic performance to the body's far limits, Popular Science (Jan. 1982) 58-60. Sandweiss, J., et al, Biofeedback and Sports Science, Plenum Press New York (1985) 1-201. Scarborough, E.L., Enhancement of Audio Localization Cue Synthesis by Adding Environmental and Visual Cues, Air Force Inst. Of Tech., Wright-Patterson AFB, OH School of Engineering (Dec. 1992) 1-4. Smith, J., et ai, Virtual Batting Cage and Human Model, Virtual Human http://www.cs.berkeley.edu/rcdavis/classes/cs294/, (Jun. 17, 2004)1-5. Zetu, D., et al, Extended range tracking for remote virtual realityaided facility management, Department of Mechanical Engineering The University of Illinois at Chicago, http://alpha.me.uic.eduldan/ NsfPaper/nsfl.htrnl, (Apr. 19, 2004)1-9. Codamotion: The science of real-time motion capture and analysis, webpages from http://www.charndyn.comlindex.htrnl. (Apr. 17, 2004) 1. Irex, Virtual Reality Technologies, webpages from http://www. irexonline.comlhow_it_works.htrn, (Apr. 19, 2004) 1-2. Polhemus, PATRIOT: The Fast and Affordable Digital Tracker, www.polhemus.com. (Feb. 2004) 1-2. Polhemus, LIBERTY: The Forerunner in Electromagnetic Tracking Technology, www.polhemus.com. (May 2003) 1-2. Success Story Profile: Innovative Sports Training, Motion Monitor, (2002) 1-2.

* cited by examiner

JX-001.0003

A000003


u.s. Patent Nov. 6,2007 Sheet 1 of 10 US 7,292,151 B2

10

---------) y FIG-1A

y FIG-1B

JX-001.0004

A000004



(-l \----- ) 1----..... 1 1 1 1 1 1 1 1 1 1 1 ~ ) ..... _-",

FIG-2A ~ I

FIG-2B

1 kg I , ! I \

FIG-2C

FIG-2D

JX-001.0005

A000005



T •

Evaluate msplay PIN Entry I Security & Navigation

Requirements Controls

+[reqUiredl " ",

Request ---) CI) Authentication rn 0 .c a...

[not required] User Session b [identification] 'c :::J

Authenticatef _____ J 0 CI)

(/) User

+ User Session

[loaded]

.. Configure Display D Session UI Graphical

• Incons of Name & ID

1 Descriptor

-,,' ,-"

Display Graphical ~ Request ,,-"

Incons of Workload Program ---" Intensity & Limits, Selection ..

Measurement User Session Criteria, Stimulus [setup] rh Properties, ect.

I Process i I I

Program Display D CI) / ...---_./ rn I Request Graphical 0

Request I .c

a... Incons of Session ---"" " Parameters 0.. " Setup ~ " :::J

Request " & Instruction ...- " CI) (/) Program Text

User Session ---" Setup ..

[setup] rh User Session

Process I [setup] rh Session +-----"

J

Parameters Process l

l Program ...---_./

Parameters

J + +

[A or B]

( Deployment Phase ) FIG-3A

JX-001.0006

A000006



CI) CI) o

.J:: 0... ..... c ..... CI) 0 E S ~

"'i5.. CI)

Cl

( Deployment Phase) i

User Session

[save] rh

Process 1M Deployment

Requirements

--------, Program

[load]

-------,

Evaluate Secondary 1M Requirement Display Graphical

Incon for Detail of Modular Length,

Weighted, or Augmentative Force

Resistance Attachments

[not required]

.. / _.c._-. l 1

/ i / i

/ i r-~~----~I i

Fasten Primary 1M

i i

+ Program

[Deployment] I i i i i

, i \ i \.. J --r

\ Display

Graphical Incon of

User Body

[ d] Location & rea y Strap Holster

~echan;sm

[required]

Fasten Secondary 1M

[ready]

--- ...... i ..

Program

[Deployment]

i i i i i i i

\. i \ .. I --T-'

'\.

Display Graphical Incon of

User Body Location &

Strap Holster Mechanism

Calibration Phase FIG-3B

JX-001.0007

A000007



Q) ff) c

..c c..

Request

( Calibration Phase)

Process Requirements of

1M Calibration

Display GraphicalD. Incon of

Primary 1M Calibration Status

Determine Secondary 1M Requirement

~[ not requ;red]

[required] Display GraphicalD.

.---. Primary 1M Pose :B Modification

---" I I ? se~noc~;a;f 1M

",----+< Calibration Status

c E o c:

'--' Process

Primary 1M Pose

Modulate Aural/Visual/Tactile

Stimuli rh

Evaluate Primary 1M

1 Program

[Calibration]

1 1 1 J

----'

.---. ..c u ..... c E o c:

'--'

.-;"

r----=-......... --:--"'" ;/;'" Request

Secondary 1M Pose Modification

Process Secondary 1M

Pose


Stimuli rh

---"" 1 I

t Program

[Calibration]

I I 1

---)

Pose tatch .... (_ ... _"'_'"_,, ........ __ ,..

'-----<0 Match Pose to D. Intial Reference

Movement Trajectory

Evaluate Secondary 1M

pose~MatCh P--,..."_"_"_"..;" ____ _

'----__ --< Match Pose to D. [match] Intial Field

'--__ ~~ ..... ___ .....,~---[m-t.:.t-c-h] Position

( Execution Phase) FIG-3C

JX-001.0008

A000008



,,---[Iearn]------..

Query Leam

Modality

[normal]

---, , Program [Record]

User Session ---'" [Setup] I

I Q) I I In I • c Process .s::. I c.. Primary 1M Program I:: Pose [Execution] 0

:;:::; :::l I 0 Process I Q)

I x Learn w I

Terminus I I I I

[not done] I J ---

[done]

Process Program Terminus

Program [not done]

[Save] [done] • )

FIG-3D Data Analysis Phase

JX-001.0009

A000009


u.s. Patent Nov. 6,2007 Sheet 7 of 10

---------L-~ _~---

--=---J-~) ---=--~==---] ,-----------~

/ /

/ .,-..---,..'"

I /

/ I

I

I

I I

I I

FIG-4B

US 7,292,151 B2

JX-001.0010

A000010


u.s. Patent Nov. 6,2007

Remote Position Processor

Analog Signal Processor Interface

Receiver Constellation

Sensor &: «access»

Preamplifier

I J

r---~

r.t, Amplifer &:

BW Filter (I I

«a..:c:~~_....l

Comparator

Absolute Value

«access»

~ tI.I tI.I CD o o c !II. r---'"'I

Peak Detect &:

Sample-Hold ....

I

I I I I I I I

«access» I I~-

«access»

~-------------------

L..-U_S_B_2_.0_J ..... ----- --

FIG-5

Communication Interface

L...-___ ---I

Sheet 8 of 10

Digital Signal Processor Interface

Processor

AID

US 7,292,151 B2

TImebase

Conversion

r.t, ~--I 2 I .s I

<II I L..----r--..,---:-r

r.t, Phase-Locked

Loop

TIming Capture &: PWM

I ..... _- .....

J I

\ I I I

------./ I

~ <II <II CD o o c ~

I I I

r.t, Software

Calculation Engine

I I I

r----

(

I I I

------'"

r.t,

Digital Filter

r------------I «1M status» I I

_./

20

Radio Link

30

Parallel/Serial

I/o

I --_./

I J

---

JX-001.0011

A000011



14

FIG-6A

FIG-6B

FIG-6e

JX-001.0012

A000012



Transponder

Stimuli Interface Processor I r===n.

I "k ~ ~ r.L, 28MHz

r-l White LED ~ & Driver e .... D Timebase

r.L, ..... Vibrator 8 .c::::::;L «clock»,t Motor ~ ~ r.L, & Driver l) )t. PWM

r r-l

r.L, «access>~ Audio «configure»

Annuciator f----'

r===n.l ~ "" & Driver r===il T

r.L, II> r.L, CD u AID u Software

,.-- c Calculation Conversion ~"

Sensor Interface - Engine

=jJ ~""f;9""'J, r..

r===il <)g. r.L, E

Ultrasonic :::;; ------- ~ Transmitter r.L,

J Parallel/Serial

~ess»T «access» I/o "

r-l T r.L, ~ ':-r.L, Accelerometer ~( :::l

/ Heart Rate 01 t;::

Receiver ,.. <:

"', 0 u ,.... , ¥

T I Dual Axis I

low g I I I I

I I

Communication Interface I Polar

compatible ~ uS~»T ~ r----' ? «uLr I/O» ----,

r.L, r.L, Graphic LCD

~ r- Switch I/O & Touch Screen n ( )

r-l «1M msg»

20-...." r.L,

~'\

FIG-7 Radio Link

30 (1)-----------_ _ 900MHz ISM

JX-001.0013

A000013


US 7,292,151 B2 1

HUMAN MOVEMENT MEASUREMENT SYSTEM

This application claims the benefit of u.s. Provisional Application No. 60/592,092, filed luI. 29, 2004, which is hereby incorporated by reference in its entirety.

BACKGROUND AND SUMMER OF THE INVENTION

This invention relates to a system and methods for setup and measuring the position and orientation (pose) of transponders. More specifically, for training the user to manipulate the pose of the transponders through a movement trajectory, while guided by interactive and sensory feedback means, for the purposes of functional movement assessment for exercise, and physical medicine and rehabilitation.

Known are commercial tracking and display systems that employ either singularly, or a hybrid fusion thereof, mechanical, inertial, acoustical or electromagnetic radiation sensors to determine a mobile object's position and orientation, referred to collectively as pose.

The various commercial tracking systems are broadly classified by their relative or absolute position tracking capability, in which system the pose of a mobile object is measured relative to a fixed coordinate system associated with either combination of receiver(s) or passive or active transmitter( s) housing mounted on the user. The tracking system's components may be tethered with obvious inherent movement restrictions, or use wireless communication means to remotely transmit and process the information and allow for greater mobility and range of movement.

Typically these tracking systems are utilized for biomechanics and gait analysis, motion capture, or performance animation and require the sensors to be precisely mounted on the joints. Various means of presenting the tracking information in a visual display are employed, such as Heads-Up Display (HUD) , that provide occluded or seethrough visibility of the physical world, or Fixed-Surface Display (FSD), such as computer desktop monitors, depending upon the simulation and immersive quality required for the application. The application may require various degrees of aural, visual, and tactile simulation fidelity and construct direct or composite camera views of the augmented or three dimensional (3D) virtual reality environment to elicit interactive user locomotion and/or object manipulation to enhance the user's performance and perception therein. The tracked object may be represented in the virtual environment in various forms, i.e., as a fully articulated anthropoid or depicted as a less complex graphical primitive. The rendering strategy employed depends upon the degree of photo realism required with consideration to its computational cost and the application's proprioception requirements.

Tracking technologies possess certain inherent strengths and limitations dependent upon technology, human factors, and environment that need consideration when discussing their performance metrics. Regardless of differentiating resolution and accuracy performance benchmarks, many implementations suffer from varying degrees of static and dynamic errors, including spatial distortion, jitter, stability, latency, or overshoot from prediction algorithms. Some human factors include perceptual stability and task performance transparency, which are more subjective in nature. And environmental issues such as line-of-sight, sensor attachment, range, and multiple-object recognition, need to be considered when selecting the optimal technology for the most robust application development. Irrespective of the

2 intrinsic strengths and weaknesses of the tracking technology employed, ultimately the user's satisfaction with the system's utilization and efficacy, including the production of reliable, easily understood, measurable outcomes, will dictate the overall success of the device.

This invention's system and methods facilitates biomechanical tracking and analysis of functional movement. In the preferred embodiment, this invention is low cost, robust,

10 easy to deploy, noninvasive, unobtrusive, and conveys intuitive and succinct information to the user to execute movement properly and provides performance indicators of said movement for feedback purposes. One feature of the present invention provides for an interactive tracking system

15 because the sensor functionality, or referred to herein as active transponders or transponders, is integrated with local user input control, and real-time sensory interfaces on the same device. The transponder is a wireless communication and monitoring device that receives a specific signal and

20 automatically responds with a specific reply. In one embodiment, the invention provides functional movement assessment based upon the relative measures of limb pose with respect to two positions defined by the transponders. The transponders can operate independently or work in unison to

25 process and share computational tasks and information between the local databases. This decentralized, distributed processing scheme allows the configuration and coordination of the training session, and processing and analysis of the measurements to occur without requiring expensive

30 auxiliary computer and display systems to manage the same, and without relying on costly software development of complex synthetic environments for visualization purposes. Also, the user can manage the applications and performance databases off-line on a remote computer system with Inter-

35 net connectivity to customize and configure the system parameters in advance of their session.

The present invention is designed to provide such system and methods for high-fidelity tracking or registration of the poses of active transponders and engage the user to pur-

40 posely manipulate the transponders' pose along a prescribed or choreographed movement trajectory in order to train and assess functional movement capability. In the preferred embodiment, the system is comprised of two subsystems: (1) a subsystem comprised of one or more active transpon-

45 ders, which, in its most sophisticated implementation, responds to periodic requests from another component of the system to radiate or transmit a signal for purposes of absolute position tracking; processes an embedded inertial sensor for relative orientation tracking and absolute tracking

50 refinement; and provides an essentially real-time aural, visual, and tactile sensory interfaces to the user, and (2) a subsystem comprised of a centralized position processor system or unit and receiver constellation unit, collectively referred to as the processor unit, which is essentially a signal

55 processor that synchronizes the transponders' periodicity of radiating signal and other operational states; collectively receives and processes the radiated signal; iteratively calculates the transponders instantaneous pose and convolution, thereof; and continually exchanges this information,

60 and its analysis thereof, with the transponders and/or auxiliary host computer system in essentially real-time via a combined wireless and tethered communication means. This real-time bidirectional exchange of information allows for proper transponder identification, coordination, and the

65 accurate measurement of pose, thereof, and timely actuation of the sensory interfaces for optimal user regulated closedloop control.

JX-001.0014

A000014


US 7,292,151 B2 3

The transponder is broadly classified by its level of hardware and software configuration that define its scope of intelligence, sensory support, and configuration. The degree

4 of movement control or smoothness. In summary, one embodiment of the present invention is comprised of:

of intelligence is detennined by its capability to locally access, process, and modifY the database. Further, either transponder classification can be sub-classified by its manipulative requirements. In one embodiment, where multiple transponders are used, a principle transponder is consciously and deliberately moved along the reference movement trajectory, while a subordinate transponder serves as an 10

anchor or secondary reference point elsewhere on the locomotion system whose kinematics are not necessarily controlled by the user's volition.

1) a means to create a single movement vector whose endpoints are defined by the locations of at least two transponders, wherein, the expansion and contraction of the vector's length is calculated, analyzed, and reported in essentially real-time;

2) a means to create a single movement vector whose endpoints are defined by the locations of two transponders, wherein, a representative point along the vector length is referenced and its higher-order derivatives are computed by mathematical numerical processes, wherein the result is calculated, analyzed, and reported in essentially real-time; and,

An interactive transponder, preferably, has significant intelligence; supports relative and absolute tracking capabilities; provides complete sensory stimuli support; provides for functional enhancement through attachment of modular, extension pieces; and provides a user display and input system to control the training session. In the preferred embodiment, the interactive transponder is primarily held in the hand to facilitate more complex user input and greater sensory intimacy. Conversely, in another embodiment, the fixed transponder has limited intelligence; supports only the absolute pose tracking capability; provides no sensory stimuli support; and is usually mounted to a fixed site on the limb or trunk.

A combination of transponder deployment strategies may be required depending on the training session's objectives, such as two interactive transponders grasped by each hand; or alternatively, an interactive transponder, and a fixed transponder attached to the limb or trunk; or lastly, two fixed transponders attached to the limb(s) and/or trunk.

In one embodiment, this invention proposes to elicit movement strategies based on the deployment of at least two transponders that define the endpoints of a movement vector whose relative translation and rotation is measured and evaluated for the assessment of functional movement capability, including but not limited to, limb range of motion and its control thereof, limb strength conditioning, and overall proprioception and hand-eye coordination skills, and overall body movement. This registration system measures a single movement vector whose endpoints are comprised of an anchor point, i.e. one that is located in a less dynamic frame

15 3) a means to correlate said vector's length and at least one other measure consisting of a higher-order derivative, to the reference movement trajectory, wherein the result is calculated, analyzed, and reported in essentially real-time. A registration system for practical functional movement

20 applications should clearly convey information to the user regarding his movement quality while he perfonns the task, without compromising or distracting from said execution by uunecessary head movements or change in eye gaze and normal focus. Poor visualization strategies that distract the

25 user are ineffectual for promoting heads-up, immersive interaction, and the alphanumerical information it imparts often can not be consciously processed fast enough to elicit corrective action. This system provides for both a local, standalone sensory interface as a primary feedback aid, or

30 alternatively, an interface to a remote fixed-surface display for greater visualization and simulation capabilities. The visual stimulus could be modulated to warn of range violations, or provide signals for purposes of movement cadence and directional cueing. A principle interactive transponder is

35 typically hand-held, which is naturally in close proximity to the user's aural and visual sensory field during most upper extremity movements, or, conversely, the visual stimulus may be viewed through a mirrored or reflective means if not in optimal line-of-sight. A remote fixed-surface display

40 might augment the immersive quality of the user's experience by providing control of a view camera of a simulated computer environment, and display of the transponders and/or interactive objects' static or dynamic poses within the computer display's skewed through-the-window perspective

45 projection. In summary, one embodiment of the present invention is comprised of:

of reference, e.g., such as the trunk or abdomen, and another more distal location fixed on or held by a limb or extremity, e.g., the hand, ann, or leg. As this movement vector is translated and rotated through space by the act of the user modifYing the pose of the principle transponder in concert with the reference movement trajectory, the vector's length 50

will expand and contract relative to the proximity of principle transponder with respect to the subordinate transponder. The vector's length conveys unique and explicit information regarding the user's movement efficiency and biomechanical leverage. For example, by attaching a fixed subordinate transponder at the hips and a fixed principle transponder on the upper ann, the biomechanics of the act of lifting a box or similar object can be elegantly qualified. If the user assumes a poor lifting technique, i.e. legs locked with the trunk severely flexed with head down and the arms 60

stretched out beyond the basis of support, the vector's length would consistently be measured longer than compared to a good lifting technique, i.e., legs bent at knees with the back straight, head gaze up, and arms close to body. Also, the measurement(s) of higher-order derivatives derived from 65

numerical mathematical processes of a reference point described by the vector would provide additional indication

1) a means for modulating an embedded luminescent display organized and oriented into a directional-aiding pattern, by varying its degree of intensity and color, or other physical characteristics, to provide a visual display stimulus. This sensory interface is excited at a rate, repetition, or pattern proportional to the pose error of the transponders' movement trajectory compared to the reference movement trajectory;

55 2) a means to view said visual display stimulus with the aid of a mirror(s) or other reflective means;

3) a means for the real-time projection of sound or speech commands through an audio device to provide warning, alann, instructional, and motivational aid, and/or additional cueing upon encroachment of static and dynamic limitlboundary conditions defined by the reference move-ment trajectory;

4) a means for real-time tactile feedback including, but not limited to, modulation of the rotational properties of a vibrator motor proportional to the pose error of the transponders' movement vector compared to the refer-ence movement trajectory;

JX-001.0015

A000015


US 7,292,151 B2 5

5) a means for combining the excitation of said stimuli proportional to the pose error of the transponders' movement vector compared to the reference movement trajectory; and,

6) a means to coordinate the real-time, periodic parametric update and modulation of the stimuli imparted by the sensory interfaces within the transponders from a processing unit by means of a wireless communication linle This invention addresses the need for an intuitive, inter-

active method to instruct, create, and deliver a movement 10

trajectory command without necessarily relying on preprogrammed, regimented movement trajectories. The registration system can be configured via remote setup at the principle transponder to pre-record and choreograph a freeform movement trajectory of the principle transponder with 15

the intent of the user mimicking the same said path. This impromptu learning modality can expedite the session down time between different users and movement scenarios, and accommodate users' high anthropometric variability in range of movement. In summary, one embodiment of the 20

present invention is comprised of: 1) a means is to provide a movement trajectory learning

modality that allows the user to calibrate and create the desired endpoints, midpoints, and/or total reference movement trajectory through user progranliller entry of an 25

input device resident on the transponder; 2) a means to process and save a movement trajectory using

a computationally efficient Catmull-Rom spline algorithm or other similar path optimizing algorithms to create control points along key points of the movement trajec- 30

tory that define the optimally smoothest path intersecting the control points;

3) a means to provide database management by a processing unit via a wireless communication link or, alternatively, through user data entry of an input device resident on the 35

interactive transponder; and, 4) a means to access, edit, and store the program and/or

databases to nonvolatile memory operably coupled to the principle transponders for the purpose of automating the creation, delivery, storage, and processing of movement 40

trajectories. Customized user programs and databases would be downloaded from a central repository or relevant website in advance of the training session to the transponder from the user's home location via the Internet or other convenient locales having networked Internet 45

access, and transported to the systems remote physical location, and uploaded into the system's memory, and executed as the application program. This a priori process of remote selection, download, and transfer of programmatic content and database would minimize the user's 50

decision making and input during product utilization by offering only relevant and customized progranlilling material of their choosing targeted for their specific exercise, fitness, or rehabilitation goals. Performance data could be saved indefinitely in the database's nonvolatile 55

memory, until an upload process was performed through the said network so the database could be transferred to another location for purposes of, but not limited to, registration, processing, archival, and normative perfor-mance evaluation, etc. 60

An exemplary list of specific data structures contributing to or affecting the means for automating the creation, delivery, storage, and processing of movement trajectories described below may be stored within the non-volatile memory of the 65

transponder or position processor which may use highdensity serial FLASH, although other types of memory may

6 be used such as SmartMedia, Compact Flash, etc. Additionally, the memory device interface should not be limited to internal, but may include external media devices, such as USB FLASH Key or other portable media means, that may have inter-operability with other computerized devices. The data structures may include: Modulation & Feedback Thresholds/Triggers Properties

the aural, visual, tactile interfaces require threshold settings which determine their excitation or stimulation characteristics. These settings can be derived from previous performance data or defaults determined from normative data, or modified in real-time, by algorithmic methods including moving averages, standard deviations, interpolation based upon goal-oriented objectives, etc.

Normative Performance-performance data collected over a large population of users through controlled studies, that is distilled down into specific user categories based upon certain demographics that the user may compare and rank hislher results. This data may be initially embedded within the transponders or position processor non-volatile memory and may be augmented or modified auto mati -cally or by user volition when connected to the Internet.

Competitive Ranking-applications which have a predominate point goal-oriented purpose would allow access to a global ranking file archive accessed through the Internet or automatically via updated executive files. This ranking file would be created through an analysis of user participation and publishing of his/her results through Internet Web-based services.

Downloadable Executive Programs & Configurations-new software programs, including new features, enl13ncements, bug fixes, adjustments, etc., could be downloaded to the transponder through an Internet connection. Graphics images would be stored in compressed or uncompressed binary forms, i.e., bitmap, gif, jpeg, etc. This new programs could be transferred to any suitable computerized position processor unit located at a remote facility via the transponder's wireless link. Therefore, the user's transponder is the node that establishes the portable network capabilities of the system, not necessarily the computerized position processor.

Custom Menu Interfaces-specialized activities may require more advanced (or simplified) interfaces dependent upon the users' cognitive abilities and interactive specificity. This menu may include interactive queries or solicit information regarding the user's daily goals, subjective opinions or overall impression of the activity and ones performance which could be incorporated in the Motivation Index described below.

Report Generation Tools and Templates-XML, HTML or other authoring language used to create documents on the Web that would provide an interactive browser-based user interface to access additional performance data analysis and report generation tools and templates that may not be available or offered with the standard product.

Custom Performance Algorithms---certain application-specific performance analysis may require dynamically linked algorithms that process and calculate non-standard or specialized information, values, units, physical measurements, statistical results, predictive behaviors, filtering, numerical analysis including differentiation and integration, convolution and correlation, linear algebraic matrices operations to compute data pose and scaling transformation, and proprietary types. One example of a proprietary type is Motivation Index, a composite numerical value derived from a weighted average of statistical performance indicators and subjective user input includ-

JX-001.0016

A000016


US 7,292,151 B2 7

ing relative scoring improvements, conformity to ROM pattern, lengthy activity access duration, high access rate, relative skill level improvement, daily goal achievement, etc., that could represent the overall level of enthusiasm and satisfaction, the user has for a particular activity.

Range of Motion (ROM) Pattern Generator-the ROM pattern requires some key control points to be captured along the desired trajectory and stored in order that the algorithm can calculate an optimally smooth path, in real-time, during the comparative analysis phase.

8 optimal tracking location based upon collectively maximizing the ultrasonic source's energy received at the transducer interface. This invention addresses the practicality and robustness of

the registration system when used in either indoor or outdoor environments, and especially when the tracking volnme likely contains potentially occluding objects, i.e., an uninvolved limbs or clothing, that become potential sources of competing, reflected paths. The preferred embodiment of the

10 registration system utilizes the time of flight (TOF) measurement of ultrasonic acoustic waves due to its immunity from interference from the visible and near-visible electromagnetic spectrum and its superior ability to overcome most

ROM Pattern Capture & Replay-the ROM pattern can be can saved to memory in real-time by discrete position samples versus time depending upon the resolution desired and memory limitations and later played back on 15

the transponder or remote display for analysis.

multi-path reflections problems by simple gated timing of the initial wave front. Upon command from the processor unit, the transponders produce a few cycles burst of ultra-

Activity Specific Attributes-includes Reps/Sets, Duration, Pause, Heart Rate Limits, intra-activity delay, level, point scalars, energy expenditure, task-oriented triggers, etc., and other parametric data that controls intensity, execution rate and scoring criteria for the activity.

Instructional Information-textual, graphical, or animationbased instruction, advice, coaching, activity description, diagramed transponder deployment and intra-device connectivity, etc. that facilitates the intuitiveness, understanding, and usage of the system. The form of instruction may include music files saved in various formats, including Wave, MP3 or other current or future audio data compression formats, and video files saved in MPEG or other current or future video data compression formats.

Real-time Data Management-proprietary data management protocols that reside above the communication driver layer that manage the real-time, synchronous and asynchronous exchange of data between transponder(s) and position processor. This would provide an essential real-time sharing of activity data, analysis, and feedback stimulus thresholds, or coordination of multiple transponder configurations, or for a collaboration of same or different user requirements to complete a similar activity objective. This invention addresses the need for adaptability of the

registration system to different movement measurement scenarios. In one embodiment, it utilizes a versatile, modular configuration and mounting of the transponders onto the user. The efficient deployment of the transducers between different users' and from task to task requires a universal mounting scheme to provide consistent localization and pose of the transponders at the desired measurement sites on user's body. Also, to compensate for the receivers' finite tracking volume when stationary, the receiver constellation unit may be mechanically modified to optimize its tracking properties by conveniently repositioning it in closer proximity to the expected transponders movement trajectories and line-of-sight, thereof. In summary, one embodiment of the present invention is comprised of:

1) a means to quickly and effIciently alter the location of the transponders using a fastening system designed to quickly attach and dispose various forms of transponder assemblies;

2) a means to augment the physical properties, i.e., weight and length, of the principle transponder with adjunct electromechanical components that provide variations in biomechanical leverage for isotonic and isometric utilization; and,

3) a means to allow the user to manually alter the geometry and pose of the receiver constellation unit to facilitate an

sonic energy and the transducers of the receiver constellation unit are stimulated and mechanically resonate accordingly, upon the wave front arrival. The processor unit's

20 analog signal processing circuits transform the mechanical energy into electrical signals that resemble tapered sinusoidal waveforms, which another electronic circuit triggers upon using an adaptive threshold technique which, in turn, the processor unit detects and calculates TOF timestamps

25 indicating the wave front arrival. In the preferred embodiment, the system overcomes the ultrasonic technology's intrinsic challenge of precisely triggering on same the waveform location and provides consistent unambiguous trigger detection by complementing the adaptive threshold tech-

30 nique with a software timestamp correction algorithm, which includes in part, a digital over-sampling and averaging timestamp algorithm, a relative timestamp correction scheme utilizing a predictive algorithm of higher-order Taylor series based derivatives, and an absolute timestamp

35 correction scheme that minimizes the range error based upon discrete biasing of timestamps.

Further, in the preferred embodiment, the processor unit utilizes the absolute and relative trigger timestamps in a multi-modal trilateration algorithm for the measurement of

40 three-dimensional (3D) translations and rotations of the transponders. The primary trilateration calculation is derived by an application of Pythagoream theorem involving a point position solution based-upon range measurements from at least three (3) points, versus the well-known triangulation

45 method which uses bearing angles of two cameras of known pose. Additionally, the system's main accuracy limitation is mostly affected by the temperature variability of outdoor environments and its influence on the speed of sound in air value. This algorithm mitigates this problem by mathemati-

50 cally computing the speed of sound every analysis period provided at least five (5) receivers and a transponder synchronizing means are utilized. If the integrity of the synchronizing signal is temporarily compromised, the system automatically employs a variation of the trilateration algo-

55 rithm that uses the last known speed of sound value. In the preferred embodiment, the maximum update rate,

and hence the major contributor to the latency of the position calculation, is determined by the typical acoustical reverberation, typically between 20 to 100 ms, encountered in an

60 indoor environment. Since the transponders are held or fixed on the user's body and, therefore, are mobile, the TOF measurements will experience an additional latency effect. A Kalman filter is used as a prediction/estimation strategy to minimize and compensate for the latency effect. The pre-

65 diction algorithm uses a higher-order Taylor series based derivatives and augmentative inertial sensor data. Its predictive refinement is dependent upon predefined models of

JX-001.0017

A000017


US 7,292,151 B2 9

expected movement conditions. Because functional movement is episodic, having periods of stillness interspersed with bursts of motion activity, a multi-modal filtering strategy is preferably employed to handle the unpredictable jerkiness at the start of motion and relatively predictable, smooth motion afterwards. In summary, the preferred embodiment of the present invention is comprised of: 1) a means to detect the same carrier wave cycle of ultra

sonic energy using a software correction algorithm requiring multiple, consecutive TOF acquisitions as input for 10

the digital over-sampling and averaging algorithm, the calculation of a higher-order numerical differentiation of the past and current TOF information as input for the predictive algorithm of higher-order Taylor series based derivatives used for the relative TOF correction, and a 15

measurement of the intra-pulse time intervals of consecutive TOF acquisitions as input for the absolute TOF correction scheme that minimizes the range error based upon selective biasing of the TOFs;

2) a means to utilize a dual matrix formulation of the 20

trilateration algorithm, and a calculation strategy thereof, which decision is dependent upon the integrity of the system's communication link, synchronization condition, and the desired measurement accuracy; and,

3) a means to coordinate the information transfer between 25

transponders and the processor unit so that their contribution to the resultant movement vector calculation can be measured without intra-signal interference. These goals will be attained by such system and methods

that are comprised of the user's interaction described by the 30

following steps as set forth as the preferred embodiment: 1) Authenticate user access and open user session from a

local or remote database;

10 FIGS. 2A-2D illustrates example extension pieces for the

present invention; FIGS. 3A-3D illustrate one example of process flows for

the present invention; FIGS. 4A and 4B illustrate a sample application of the

present invention; FIG. 5 illustrates a block diagram of the remote process

ing system of the present invention; FIGS. 6A-6C illustrate example receiver configurations of

the present invention; and FIG. 7 illustrates a block diagram of the components of

one embodiment of the transponder of the present invention.

DETAILED DESCRIPTION OF INVENTION

The present invention provides a practical, versatile measurement tool for the assessment of the user's manipulation strategy of the transponder 10 or transponders along a reference movement trajectory. Moreover, the system and methods measure and analyze the kinematics of the relative translations and rotations of the limbs or extremities with respect to each other or to a more inertial reference location on or off body as the transponders are manipulated. This information provides useful insight on biomechanical demands and anthropometric factors that influence human movement efficiency and control. Although measurement performance metrics are important design criteria, it's equally important to provide intuitive and motivating program instruction and administration, and to provide comprehensive analysis and integration of the motion data in a form that is objective and easily interpreted. This system improves upon the practicality and user interactive aspects of setup, deployment, calibration, execution, feedback, and data interpretation of a tracking system designed for func-2) Setup user training session, i.e., workload limitations,

measurement criteria, and audio/visual/tactile stimuli; 35 tion human movement. 3) Select training program and configure its options; 4) Deploy the transponders as instructed to predefined

locations of users locomotion system to create at least one transponder movement vector;

5) Calibrate the transponder movement vector to establish its 40

reference pose; 6) Create a movement trajectory using learn mode, if

required; 7) Initiate the start of session; 8) Determine the instantaneous pose of transponder move- 45

ment vector relative to its reference pose from a periodic temporal iteration of this step;

9) Perform qualitative and quantitative statistical analysis of accumulated measured poses of the transponder movement vector relative to the pattern of instantaneous poses 50

defined by the reference movement trajectory; 10) Update the major transponders sensory interfaces to

modulate said system parameters in a periodic temporal iteration of this step;

11) End the session once program objectives have been 55

obtained; 12) Analyze the results by interacting with local and/or

remote databases;

Human movement is a response to external environmental forces which requires the accurate coordination of the distal segment( s) to compensate for these forces. Skillful coordination of human movement is dependent upon the cohesive interaction of multiple sensory systems, including visual, vestibular, with the musculoskeletal system. More specifi-cally, the challenges and goals of cognitive spatial mapping, (2) minimization of energy expenditure, (3) maintaining stability, (4) steering and accommodation strategies for various environments, (5) dynamic equilibrinm, (6) active propulsion and weight support, and (7) core locomotion pattern should be relationally considered to properly assess hnman movement. Therefore, it is preferable to engage the interaction of these sensory systems during a training session to promote the desired functional movement outcome. Because many movements persist for 400-500 ms, enough time is allowed for the initiation of the movement and for user correction based upon visual and kinesthetic information acquired during the time of the movement. However, the implemented means of visual feedback should be not be distracting or interfering with the task at hand. In the preferred embodiment, this system engages the sensory systems with non-distracting, intuitive, embedded aural, visual, and tactile stimuli which provide real-time indication 13) Provide numerical, graphical, and/or animated informa

tion indicating desired performance measurements. 60 of the principle transponder pose error with respect to the reference movement trajectory.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. lA illustrates one example of a deployment apparatus of the present invention;

FIG. IB illustrates one example of hand-held form for the transponder of the present invention;

In order to conduct a time efficient training session, this registration system attempts to minimize the encumbering experimental setup and calibration procedures characteristic

65 of more complex and higher cost motion analysis technology. These complementary systems serve important academic or clinical oriented research needs or for motion

JX-001.0018

A000018


US 7,292,151 B2 11

capture for computer animation purposes and strive for highly accurate measurement of joint motion data in terms of angular displacement. Therefore, the integrity and reliability of their motion data is dependent upon proper sensor setup and calibration.

For instance, single axis goniometer-based systems usually require specially designed harnesses to hold the monitor and are firmly strapped or taped over the joint to avoid relative motion artifacts. Usually these devices are tethered and their fit, weight, and constraining mechanical linkages can impose limitations on the joint motion and cause discomfort for the user. Most optical or video-based systems require the placement of numerous active or passive markers over landmarks, such as the joints' center of rotation. These systems should guarantee sufficient environmental illumination and contrast between markers and background to function optimally. Also, these systems are severely affected by occluded markers that may disappear for long periods of time due to rotations and line-of-sight limitations. Other video-based systems do not use markers but require the assignment of the body's joints manually or through computerized automation during data analysis, making real-time analysis arduous and real-time feedback virtually impossible.

In the preferred embodiment, the system doesn't require complicated, time consuming sensor setup and calibration by virtue of it minimalist sensor requirements and uncomplicated sensor mounting. Instead, it requires only the deployment of a sensor on the body (in one embodiment a dual sensor group on a combination oflimb(s) and or trunk) and doesn't enforce stringent movement protocol, but encourages free-form, unrestrictive movement of the transponders.

The transponder's preferred deployment means, include either insertion into a universal strap and holster apparatus (FIG. lA) that secures on the user's limb, extremity, or trunk, including, but not limited to, the hip, ankle, knee, wrist, upper arm, neck, waist or an augmentative mechanical attachment to one or a combination of modular extension pieces shaped into a hand-held form (FIG. IB). A strap or torx-like clip and holster design provides a firm, yet light weight and comfortable mounting location away from areas that clothing and or uninvolved limbs may occlude.

The modular extension piece is either an instrumented sensory type designed to support alternative tactile stimulus device or alternative configurations of aural, visual, and tactile feedback types, or non-instrumented, weighted extension pieces as shown in FIGS. 2A-2D. All modular extension pieces may be of various physical dimensions and intrinsic weight, with a captive handle design that preferably requires zero grip strength to grasp. Alternatively, the modular extension piece may be coupled to the transponder through a fixed or flexible, segmented, articulated coupling to accommodate attachment of additional transponders and/ or other modular extension pieces. These components would quickly assemble to each other using a spherical snap joint or twist snap latch, or similar mechanism, to provide quick alteration of form and function when used for different movement trajectory scenarios.

In one embodiment, the weighted extension attachments (FIG. 2A) are offered in fixed gradations of one (1) kilogram increments or other convenient unit of measure and either be indicated as such with a numerical label, quantitative mark, or color-code feature, or combination thereof. For upper extremity evaluation, the weighted extension piece integrated into a zero-grip handle would enhance the improve-

12 ment of musculature strength of the limb, while not compromising the user's endurance with a potentially fatiguing hand grasp requirement.

In one embodiment (FIG. 2B), the tactile type provides force feedback functionality by controlling the rotational speed of an embedded vibrator motor in the shaft. Alternatively, the visual type (FIG. 2C) may be comprised of a series of light emitting diodes that could be uniformly

10 embedded along the length of the handle or transponder and their intensities variably controlled therein. It should be appreciated that a simple, economical mirrored or reflective surface placed in front of the user's visual field could provide sufficient real-time indication of the user's subjec-

15 tive conformity to the said movement trajectory while allowing non-distracting viewing of this visual stimulus. For example, a program that requires the user to reposition the principle interactive transponder through an arc-like movement trajectory in the midsagittal plane through out a range

20 of motion begiuning from the waist upwards until parallel to shoulder height. As the user performs the movement, the visual sensory interface could be proportionally excited if the user moves too quickly, or hesitates too long, or produces shaky or erratic episodic motions, or is beyond the pre-

25 scribed bounds of the movement arc. The light stimulus is easily viewed in the mirror and would indicate corrective action in his or her movement strategy, while appropriate aural commands may be issued simultaneously to encourage the same correction. Regardless of the sensory interface

30 type, its control and excitation properties will be determined by some statistical aspect of the user's conformity to and progression through the movement trajectory.

The hand-held transponder may include a modular extension piece with an embedded graphic display device and

35 associated input means to allow the user to setup, operate, provide visual feedback, and view performance results of the device usage without additional remote display means. More specifically, a software-controlled user interface could provide certain visual prompts in a menu oriented presentation,

40 to instruct the user on (1) device setup, i.e., aural, visual, and tactile feedback parameters, types of program start and termination cues, program intensity based on ratio of amount of repetitions, sets, and rest periods or categorical gradation of challenge, learn mode behavior, etc., (2) scrol-

45 lable program selection with brief descriptions including objective, desired measurement, i.e., range of motion, energy, accuracy, speed, etc., and instructive information, and (3) alphanumeric and/or graphical display of measured performance data and other biophysical data and its analysis

50 thereof, displayed in standard plotted forms including line, bar, and pie charts, etc. It is important to note that the user input process is intuitive and streamlined so as not to detract from the practicality and user friendliness of the system. Only relevant applications and its control thereof will be

55 sequestered from the database and presented to the user. In one embodiment, two or more transponders and exten

sion pieces, or combinations thereof, may be assembled at their endpoints with a universal spring coupling. The assembled device could be grasped in both hands and bent

60 in various rotational angles about the spring coupling's axis. Isotonic strength conditioning programs can be developed due to the force resistance feedback supplied by the spring. A multi-transponder assembly in the form of a flexible rod or staff could provide an indication of balance of upper

65 extremity strength and proprioceptive function dependent upon the angular closure rate and rotational imbalance and orientation deviation from initial starting position.

JX-001.0019

A000019


US 7,292,151 B2 13

Additionally, in the preferred embodiment, the modular extension pieces have provisions for other attachable apparatus (FIG. 2D) that can augment the program's intensity or difficulty. For example, an eyelet is embedded in the end of the extension piece and is designed to attach an elastomeric band, such as the type manufactured by Theraband®. By securing the other open stirrup end of the band to the user's foot, isotonic strength conditioning programs can be developed due to the force resistance feedback supplied by the elastomeric band. Moving the transponder through a movement trajectory is now made more restrictive and challenging.

APPLICATION EXAMPLES

An example training session deploying a dual transponder group is now described that may be designed to improve the range of motion, strength, and coordination of shoulder abduction in a user. The training session would primarily serve as an exercise aid that provides essential feedback to the user so that he/she learns to progressively improve the manipulation of the transponder through the reference movement trajectory, while benefiting from increased shoulder range of motion and strength improvement.

In advance of the training session, a software application is operated from a host computer that provides a utility for baseline configuration and management of the system's and transponder's local databases, and/or access to other remote databases, and for the real-time interface to the data flow between the system's components. The application's navigation and selections are presented to the user through a typical graphical user interface like Microsoft Windows® XP operating system. A generalized step-wise procedure requires the administrator or user to (1) select the desired program and features from a menu screen list, and (2) to initiate a communication process that causes the program parameters to be transferred to the processor unit through a standard computer communication protocol, i.e. serial, USB, ethernet, etc., whereupon, (3) the information is subsequently processed and transferred into the transponders local memory via a wireless communication link, and, finally, (4) the transponder's software program accesses this database to manage the device utilization and configuration of the local display means. Alternatively, a Compact FLASH-based memory card, embedded serial FLASH, or a similar nonvolatile memory device provides the user an additional specialized database supporting remote data collection capabilities. This database would be preprogrammed in advance and the resultant performance data retained, even if the device's power is lost, or for extended unsupervised exercise sessions conducted remotely from the host computer system or when the host computer system is unattached or unavailable. After the session is completed, the user would be queried if the results are to be saved for later analysis or would automatically be saved, dependent upon device setup. This data could be retrieved at a later time when the system is once again attached to a host computer system, and the software utility could be commanded to upload the database.

Henceforth, the following procedural description refers to the activity dependencies diagrammed in FIGS. 3A-3D that the user would encounter while operating the system.

During the Security Phase (FIG. 3A), the user may be requested to provide a security authentication code for validation, which opens access to his/hers custom programs

14 limited to, workload intensity, measurement criteria, sensory interface properties, and reporting features. A program menu list would indicate name, ID, and a brief description, or alternatively, be represented by a detailed graphical icon. When the program is selected, other program-specific options can be setup.

During the Deployment Phase (FIG. 3B), and dependent upon the program's objectives, a suitable combination of transponder types will be mounted on the user's body as

10 instructed by the program. This example requires the assembly of a hand-held interactive transponder with graphical display, and a weighted extension piece coupled therein to be grasped by the hand on the same side as the affected shoulder. Another subordinate transponder 12 is placed into

15 a holster assembly strapped around the lower quadriceps on the same side. This setup is shown in FIG. 4.

During the Calibration Phase (FIG. 3C), a simple calibration procedure may be requested to evaluate transponder function and specific user range of motion constraints.

20 Typically, this information is determined beforehand and saved in the system's database. Also, practicality of this system is claimed for lack of extensive calibration requirements.

Dependent upon the program's options, a user-defined 25 movement trajectory may be created prior to program start

in lieu of executing the predefined version. The learn mode could be utilized to quickly choreograph free-form movement trajectories and save them into the transponder's non-volatile memory for later execution. The learn mode

30 would be accessed through the user interface and instruct the management of the control point assignment by pressing the push button switch at the appropriate junctures of movement discontinuity or, preferably, allowing automated assignment by the software. In the preferred embodiment, a computa-

35 tionally efficient Catmull-Rom spline is used to define a three dimensional (3D) curve that passes through all the control points along the movement trajectory path. Ifmanually interceding, the user is instructed to press the push button once at each major juncture in the movement trajec-

40 tory, but, preferably, for no more than a few locations, until the desired end of range of motion is reached as shown in FIG. 4B. Similarly, the return path may be similarly defined or he/she may elect to use the same forward path in reverse. These control points are registered by the processor unit and

45 transferred and saved to the transponders' memory to serve as the control points for the real-time calculation of a Catmull-Rom spline. The Catmull-Rom spline is calculated in real-time from the desired initial starting point to provide a continuous set of position points representing the

50 "learned" reference movement trajectory. After the program is selected or the learn mode complete,

the user may be instructed to alter the pose of the transponders to satisfY the initial starting conditions of the program. Either one or a combination of sensory interfaces could be

55 excited by the principle transponder to cause the user to direct or steer it towards the desired start point. For instance, the visual sensory interface could sequentially extinguish or dim its peripheral light sources to converge to a central light source as the principle transponder is positioned closer to the

60 desired start point. Alternatively, the aural sensory interface could change its tonality and loudness as the start point is approached. Or alternatively, the tactile sensory interface could be modulated to provide less force feedback as the start point is approached.

in the training session. Next, during the Setup Phase (FIG. 65

3A), the user can configure global options or select the desired program. The global options may include, but are not

During the Execution Phase (FIG. 3D), the transponders are continually manipulated along the reference movement trajectory to the best of the user's skill and fidelity, within

JX-001.0020

A000020


US 7,292,151 B2 15

the bounds of the user's physical limitation, until an aural, visual, or tactile response is given that indicates the activity volume has been successfully completed or a sufficient number of confonnity violations or failures have been registered. The processor unit calculates the instantaneous pose coordinates of the transponders every analysis interval and periodically communicates this information to the transponders via the wireless communication linle As the principle transponder is moved in mimicry to the reference movement trajectory the confonnity error between the actual 10

and reference movement trajectory is calculated periodically

16 increased risk for falls, balance disorders can shorten attention span, disrupt normal sleep patterns, cause excessive fatigue, increase dependence on others and reduce quality of life. It is not uncommon for individuals with a history of balance problems to regain their balance control through accurate diagnosis followed by specific medical treatment and/or rehabilitation exercises.

The present invention described can be used as a testing and training device for balance improvement under both static and dynamic conditions.

One testing and training scenario for postural stability would be to measure frequency and amplitude of body sway in three dimension (3D) space while feet remain in a fixed position. This task can be performed in both a double or

in real-time to determine the characteristics of feedback quality to be elicited by the sensory interfaces for the user's closed-loop control to correct his/her manipulation strategy. For example, the conformity error may be calculated from statistical processes based upon the standard deviation of the least mean squared (LSM) principle transponder's position error compared to the reference movement trajectory, or based upon, or combination thereof, a threshold magnitude

15 single leg stance to test for bilateral symmetry relating to balance. Another modification of the test would be to

of some multi-order numerical differentiation of said move- 20

ment to indicate a "smoothness" quality of translation and rotation along the movement trajectory path.

perfonn each test with eyes both open and closed to help determine the contribution of the visual component to overall balance ability. Tracking body sway while creating the illusion of motion through proper visual cueing on a display means would be another test to help detennine the reliance on specific sensory components of balance. Delivering repetition of protocols with increasing difficult oscillation thresholds with biofeedback of successes and failures of

25 such predetermined goals is one way to train to improve balance.

Alternatively, a host computer system could provide an auxiliary processing and display means to allow another software program to access the transponder's calculated positional data through an application programmer's interface and use this data to alter the pose of a graphical primitive in proportion to the motions of the transponders within the context of computer generated virtual environment. The dynamic control of objects in the computer 30

generated virtual environment could be used to augment the local sensory interfaces of the transponders through an interactive, goal-oriented video game modality. The video game challenges could be increased over time based upon some scoring criteria of successful manipulation of the 35

principally controlled on-screen graphical object with respect to cueing derived from other secondary static or dynamically moving objects. It is important to note that only primitive fonns of video game challenges would be considered, to take into account the user's cognitive awareness and 40

physical limitations, and the economics of software development for photo realistic virtual environments and animation. Also, this auxiliary computer display means would offer an alternative visualization method of interactive and immersive video feedback aid to enhance the application 45

presentation. Additional examples of how the present invention may be

applied are described as follows:

Balance The body has the ability to maintain balance under both

static and dynamic conditions. In static conditions, such as

The transponder can deliver aural, visual, and tactile stimuli to queue the individual to the degree of frequency and amplitude of body oscillations. The aural and tactile components provide the only means of feedback when the testing and training are perfonned with eyes closed or the visual field is compromised. Examples include, but are not limited to, (1) an audio signal increasing and decreasing in volnme proportional to the amplitude of body sway, (2) a vibration action proportional to frequency of body oscillations, and (3) a light source illuminated when both frequency and amplitude goals are achieved. Multiple transponders can be used to evaluate and reinforce proper balance posture by communicating position information of certain body segments in relationship to others. An example would be the comparison of position of the head with respect to the hips while generating a vibration action if an excessive forward lean of the head as compared to the hips is recognized.

Another test for balance would be to test ones Limits of Stability (LOS). This test refers to ones ability to effectively operate within their sway envelope. The sway envelope or LOS is defined as the maximal angle a person's body can achieve from vertical without losing balance. An individual with healthy balance is capable ofleaning (swaying) within

50 a known sway envelope and recover back to a centered position without the need for a secondary maneuver such as a step, excessive bend at the torso or ann swinging. LOS for bilateral stance in nonnal adults is 8 degrees anterior, 4 in standing, the body strives to efficiently maintain posture

(often referred to as postural stability) with minimal movement. In dynamic conditions such as in walking or sports 55

play, the body strives to maintain balance while perfonning

degrees posterior and 8 degrees laterally in both directions. The present invention described can be used as a testing

and training device for balance control during movement or perturbations within a desired sway envelope. Through proper visual queuing represented on the display means that defines a normal sway envelope, the amount of body dis-

in an ever changing environment. The ability to maintain balance is a complex process that depends on three major sensory components. The sensory systems include visual, vestibular and proprioception. For example, we rely on our visual system (eyes) to tell us if the environment around us is moving or still; we rely on our vestibular system (inner ears) to tell us if we are upright or leaning, standing still or moving; and we rely on our proprioceptive system (feet and joints) to tell us if the surface we are standing on is uneven or moving. If balance problems develop, they can cause profound disruptions in your daily life. In addition to

60 placement can be measured from vertical stance. The transponder can deliver aural, visual, and tactile

stimuli to queue the individual as to when he or she has achieved the desired range of their sway envelope, then assess the individual's ability to return back to a vertical

65 stance. Examples include, but are not limited to, (1) a vibration action when the user varies (meanders) from the desired movement path, (2) an array of lights change inten-

JX-001.0021

A000021


US 7,292,151 B2 17

sity and pattern as the individual successfully approaches the intended target, (3) an audio signal is generated when the individual has maintained a stable position with respect to proper visual queuing represented on the display means for a selected period of time. Multiple transponders can be used to evaluate and reinforce proper balance posture by communicating position information of certain body segments in relationship to others. An example would be the comparison of position of the head with respect to the hips while generating a vibration action if an excessive forward lean of the head as compared to the hips is recognized.

Dynamic balance can be evaluated while having the individual perform coordinated movements which specifically challenge the various components of balance in a dynamic nature. Such movements include, but are not limited to jumping, hopping, and walking. These movements can be performed with eyes both open and closed, during interaction with static or dynamic visual queuing on the display means. The ability to perform these dynamic balance tasks with comparisons to others of similar sex, age or disability can be assessed. Example measurements may include, but are not limited to, (1) amount of body sway in three dimension (3D) space, (2) time to complete specific task, and (3) effects of fatigue on balance ability.

Balance training in both static and dynamic conditions can be easily achieved by providing specific visual queuing

18 the display means to instruct and motivate the individual through the proper testing procedure.

The present invention described can be used as a testing and training device for individuals involved in physical rehabilitation or general fitness to improve ROM. Proper visual queuing can be represented on a display means to motivate individuals to extend their range of motion beyond their current capabilities.

The transponder can deliver aural, visual, and tactile 10 feedback that alerts the individual to successes or failures in

proper execution of each repetition. An example of tactile feedback would be the transponders are vibrated if the individual's movement trajectory varied from the intended two dimensional (2D) reference movement trajectory by

15 deviation from the planar path into the uninvolved spatial dimension. An array of light sources could increase illumination in intensity and repetition as the ROM goal was approached and an audio tone could signal the individual they have achieved the desired pause time at the proper

20 ROM. Multiple transponders can be deployed to determine the

contribution of each joint or anatomical structure where more then one joint is involved in the ROM movement (example; shoulder and scapular in overhead reaching). The

25 vector sum of each transponder movement in a specific axis can be added together to determine the total ROM. The ROM of one joint in a two joint motion can be subtracted from the total ROM to determine the contribution of a single on the display means, which challenge the individual to

perform repetitive and progressively more difficult balance drills. Performance reports can be generated to establish a 30

baseline, isolate specific strengths and weaknesses within the specific sensory and motor control aspects of balance, and document progression and improvements.

joint in a two joint movement.

Human Performance Testing and Training There are many devices that test the strength and speed of

isolated joint movements, for example, the leg extension and bicep curl. This information has value in testing both healthy The transponder can deliver aural, visual, and tactile

stimuli to queue the individual as to when he or she has achieved the desired balance task. By example, a vibration action is produced proportional to the frequency of a body segment oscillation after the user lands from a hop test and attempts to stabilize and maintain proper postural balance. When the individual finally stabilizes and achieves correct postural balance, an audio signal indicates the task has successfully completed. Multiple transponders can be used to evaluate and reinforce proper balance posture by communicating position information of certain body segments in relationship to others. An example would be the comparison of position of the head with respect to the hips while generating a vibration action if an excessive forward lean of the head as compared to the hips is recognized.

Range of Motion (ROM)

The present invention described can be used as a testing and training device to determine the range of motion within

35 individuals, athletes and individuals whose strength and speed capabilities may be compromised by injury, disease, poor conditioning or simply age. Recently in the field of human performance, it has been recognized that the analysis of the mobility of the isolated joint, although providing some

40 value, does not offer enough information to determine how the body will perform during functional movements. Functional movements are defined as movements equal to those encountered on the athletic field, in the work environment or while performing activities of daily living. Functional move-

45 ments involve the movement and coordination of multiple joints and muscle groups acting together to perform a more complex task then a single, isolated joint movement.

The present invention described can be used as a testing and training device for functional movement improvement.

50 By tracking various registration points on the body with respect to each other or to an off-body registration point, performance measurements of functional movements can be assessed, such as jumping, cutting, turning, bounding, hop-a joint. Range of Motion is the normal distance and direction

through which a joint can move. Limited ROM is a relative term indicating that a specific joint or body part cannot move 55

through its normal and full ROM. Motion may be limited by

ping, shuttling, etc. The present invention described can be used as a testing

and training device for individuals involved in physical rehabilitation, general fitness or sports performance enhancement to improve their functional movement abilities. Proper visual queuing can be represented on the display

a mechanical problem within the joint that prevents it from moving beyond a certain point, by swelling of tissue around the joint, by spasticity of the muscles, or by pain. Diseases that prevent a joint from fully extending, over time may produce contracture deformities, causing permanent inability to extend the joint beyond a certain fixed position.

The present invention described can be used to test the starting point and end point which an individual is capable of moving a body part, typically a limb and associated joint(s). Comparisons to age and sex based normative data can be made. Proper visual queuing can be represented on

60 means to instruct and motivate individuals to perform specific functional movements.

The transponder can deliver aural, visual, and tactile feedback of proper movement execution. Examples include, but are not limited to, (1) an audio signal alerting the user

65 that the desired performance stance is incorrect, (2) the light sources illuminate when the desired speed is achieved in a first step quickness drill, (3) a vibration action to indicate the

JX-001.0022

A000022


US 7,292,151 B2 19

limits of tracking range, (4) a vibration action proportional to the magnitude of a biophysical measurement during the interaction with visual queues represented on the display means, (5) a vibration action when the body or limb position does not correlate well to the desired body or limb position of the visual queuing represented on the display means, (6)

20

an audio signal indicating start, stop and pause periods or other controlling events, (7) an audio signal indicating proper body alignment or posture has been compromised, and (8) an audio signal indicating the relationship of desired 10

target heart rate to a desired threshold.

oriented perpendicular to the horizontal plane. Further, as indicated in the preceding figures, the transducers vertical axes are oriented 90° with respect to the typical vertical axis orientation of the transponder's transmitter to improve acoustic coupling in the vertical plane, a consideration for overhead, upper extremity tracking. Although this causes some reduction in the lateral registration bounds, the compromise provides a more symmetric field about the middle or primary location of tracking interest.

In the preferred embodiment, the overall size of the receiver constellation unit is predicated on a phenomenon referred to as Geometric Dilution of Precision (GDOP). The solution of a unique three-dimensional location based upon trilateration requires the precise resolution of the common

Hardware Description In the preferred embodiment, the processor unit is com

prised principally of a constellation of five (5) ultrasonic transducers and signal processing circuitry, thereof, and a signal processor that interfaces to this receiver group, performs the pose calculations, and interfaces to the transponders and host computer databases. The following interface descriptions for the processor unit are based upon the dependency flow represented by FIG. 5.

The sensors 14 preferably used for the receiver constellation unit are cylindrically-shaped ultrasonic transducers, for example, the model US40KR-0l 40 kHz PVDF ultrasonic receivers manufactured by Measurement Specialties Inc., which provide adequate acoustic pressure sensitivity and exhibit 360 degree onmidirectional broad beam response along the horizontal plane. The onmidirectional characteristic, albeit in one plane only, is very desirable to minimize line-of-sight occlusion. Because of its low resonance Q value, the rising and decay times are much faster than conventional ceramic transmitters. This reduces its power requirements since less burst drive duration is needed to achieve sufficient triggering thresholds at the receiver. This transducer type is also utilized similarly in the transponders to provide the potential for the most optimal acoustic coupling.

The receiver constellation unit is preferably mounted on a fixed support base, and has a pivoting and/or swiveling mechanical linkage which provides an adjustable mechanism for configuration of the receiver constellation unit's inertial frame of reference relative to the tracking field. In the preferred embodiment, it is strategically positioned and oriented in proximity to the tracking field in order (1) to minimize line-of-sight degradation with respect to the expected transponder orientation, (2) to optimize registration resolution with respect to field volume size, and (3) to satisfY the mathematical restrictions of performing trilateration calculations based upon the solution of simultaneous linear equations. It should be noted that the trilateration matrices may be solved if the matrices have a rank of five, and are non-singular, i.e., the matrix detenninant is nonzero. In the preferred embodiment, the geometric parameters and their coordinate location of the receiver constellation must insure linear independence of the columns of the matrices and to avoid the matrices from becoming singular.

One example geometrical pennutation of the receiver constellation unit that satisfies these rules is shown in FIG. 6A. It occupies a volnme of approximately 8 cu. ft. and essentially fixes the transducers in a way that defines two primary orthogonal, bisecting planes defined by three noncollinear points each. Another preferred implementation that occupies nearly the same volume is shown in FIG. 6B and is characterized by its S-shaped curve and tilted with respect to the horizontal plane. Another preferred implementation that occupies nearly the same volume is shown in FIG. 6C and is characterized by its helical or logarithmic spiral shape

15 intersection of multiple spheres circumscribed by the distance between each transmitter and receiver transducer. Each sphere has an inexact radius due to system noise and measurement resolution. Therefore, the intersection becomes a volnme instead of a point and the size of the

20 volnme is dependent upon the radii of the intersecting spheres as well as the distance between the spheres' centers. As the radii get larger with respect to the distance between the centers, i.e., the transmitter is farther down range, the spheres begin to appear more and more tangential to one

25 another and the intersection volnme increases, although not necessarily symmetrical in all dimensions. Therefore, to minimize position uncertainty, the receiver transducers should be separated from each other as much as practical proportions allow with respect to the confines of the tracking

30 field volume as the above said geometric examples provide. This receiver constellation unit can be repositioned with

respect to the tracking field by a simple mechanical adjustment as shown in the preceding figures. The mechanical adjustment raises and lowers and changes the length and

35 pivot axis of the cantilever ann which is fixed to a ground base support.

Because the receiver constellation unit operates a distance from the processor unit, each receiver preferably has an associated pre-amplifier circuit to convert the high-input

40 impedance piezoelectric signal into a low-level voltage proportional to the acoustic signal energy impinging the transducers sufficient in order to accurately transmit the signals to the processor unit. In one embodiment, a highinput impedance AC amplifier design with 30 dB gain can be

45 utilized. The preferred operational amplifier is the OPA373 manufactured by Texas Instruments. It was chosen for its low 1 pA input bias current, high 6 MHz GBW, and low-voltage single supply operation. The amplifier is configured as a non-inverting type with the high-pass cutoff

50 frequency set at 1 kHz. The overall circuitry is preferably enclosed in a metal shield to minimize electromagnetic noise coupling into the highly sensitive amplifier inputs. In addition, a local, regulated power supply is included to allow for a wide range of input voltage supply and provide sufficient

55 power supply rejection to compensate for the noise susceptibility of remote power distribution. All the pre-amplifier circuits' power and signal counections preferably originate from the processor unit.

The processor unit subsystem preferably consists of an 60 analog signal processing interface that provides (1) addi

tional voltage amplification and filtering of base band signal from the preamplifiers, (2) absolute value function, (3) peak detection function, and (4) analog-to-digital comparator function to provide support for an adaptive threshold means.

65 The adaptive threshold technique provides robust triggering of the most proximal ultrasonic source at a precise temporal point along the traversing sinusoidal waveform of the elec-

JX-001.0023

A000023


US 7,292,151 B2 21

trical signal. Essentially, a new threshold signal is recalculated each analysis period based upon a small percentage reduction of the last peak wavefonn detected. Therefore, the tracking range is not necessarily restricted due to an arbitrarily high threshold setting and the noise immunity is improved as the threshold tracks the wavefonn envelope and not transient disturbances. An alternative automatic gain control strategy for the amplification function is unnecessary since the trigger threshold will adjust to the signal level instead. In the preferred embodiment, the threshold faithfully tracks the peak to minimize integer period phase errors, so the amplifier's gain is set to prevent signal saturation from occurring when the receiver constellation unit and transponders are in closest proximity during normal use.

In one embodiment, an amplifier and BW (band width) filter circuit receives the output from the sensor and preamplifier circuit and provides additional amplification and low-pass filtering to condition it for reliable threshold triggering and input to other analog signal processing circuitry. A dual amplifier configuration may be used to provide an additional gain of 40 dB, AC coupling to remove DC offsets of the preamplifier outputs and long cable losses, and low-pass filter to reject noise beyond the interest signal's bandwidth. The first stage amplifier may be configured as a non-inverting type with a gain of20 dB. The low-impedance DC input signal is effectively blocked by the coupling capacitor in series at its non-inverting input with a high-pass frequency cutoff set at 20 kHz. This gain stage feeds a second amplifier configured as low-pass, 2nd order Butterworth MFB filter. This filter type provides smooth pass band response and reduced sensitivity to component tolerances. The second stage low-pass frequency cutoff is set at 80 kHz with a pass band gain of 20 dB.

An absolute value circuit receives the output of the amplifier and BW filter circuit and converts the bipolar signal into a unipolar form for magnitude detection. A dual amplifier configuration may be used to provide highly accurate full wave rectification of the millivolt-level signal. The first stage amplifier feedback switches to control the distribution of input current between the two signal paths dependent upon the input signal polarity. For a positive input voltage the input current will be positive which forward biases Dl and reverse biases D2. This configures the 1st

stage as an inverter driving the inverting input resistor of the 2nd stage, which is also configured as an inverter because its non-inverting input is held at virtual ground due to the non-conducting path of D2. This effectively creates a combined circuit of two cascaded inverters for an overall gain of +1. For a negative input signal its input current is negative which forward biases D2 and reverse biases Dl. This configures the 1st stage as an inverter driving the noninverting input of the 2nd stage which changes the sign of the circuit gain. In this mode, the input current is shared between two paths to the input of the 2nd stage, where _2/3 of the input current flows around the 1 st feedback stage and -l!3 flows in the opposite path around the 2nd stage feedback path for a net gain of -1.

In the preferred embodiment, a peak detect and samplehold circuit receives the output of the absolute value circuit and registers a peak value that is required to set a magnitude threshold precisely at some percentage of full-scale of the peak. A dual amplifier configuration may be used to provide the highest ratio of high output slew rate to low droop. The first stage is typically in negative saturation until the input voltage rises and exceeds the peak previously stored on the sample capacitor at the inverting input. Now the amplifier acts as a unity gain buffer and the input voltage charges the

22 sample capacitor which faithfully tracks the rising voltage. Once the input voltage diminishes in magnitude, the first blocking diode reverse biases and the sample capacitor holds an accurate replica of the highest voltage attained with minimal droop because of the low input bias current of the amplifier and elimination ofleakage altogether in the second blocking diode by bootstrapping its cathode at the same potential provided by the low-impedance buffer of the second output stage. An electronic switch and bleed resistor

10 allow the voltage across the sample capacitor to be reset by the processor during power up and after the triggering event is recorded so the adaptive threshold value can be refreshed each cycle. A 1 st order Butterworth filter may be used at the input to smooth false in-band transients that could disrupt

15 the peak accuracy detection. In the preferred embodiment, a comparator circuit

receives the output from the peak detect and sample-hold circuit to convert the analog signal to digital form for high-speed triggering operation of the processor. The pre-

20 ferred device is the MAX941 which is manufactured by Maxim. A percentage of the peak threshold is used to set the inverting input. When the non-inverting voltage exceeds the inverting voltage, the comparator's output will trip and produce a high-true logic pulse that triggers the processor. A

25 latch control input allows the processor to disable the comparator action to prevent urmecessary triggering during the reverberation phase and to prevent potentially disruptive noisy output chattering near threshold crossover beyond its hysteretic immunity. The percentage of threshold level is

30 predetermined through the scaling resistors to be set low enough to trigger on the rising edge of the signal's first crest at the furthest range of transponder operation, but high enough above the intrinsic system noise level and external noise caused by reverberation and other ultrasonic sources.

35 Once the first crest is registered, subsequent crests may be triggered at their zero-crossing representing the most precise timing registration by momentarily disabling the samplehold circuit. Because of the longer duration trigger receptivity window, early multiple reflections are mitigated by

40 transducer placement at least 3.5 em away from adjacent planar surfaces, so the reflected acoustic energy doesn't produce a canceling effect of the direct acoustic energy of the later crests. Once a sufficient number of crests have been registered, then the triggering window is blanked for the

45 remainder of the analysis period by latching the comparator's value.

In the preferred embodiment, a digital signal processing interface is connected to the analog signal processing interface to transfonn the analog trigger processing into digital

50 position information. The digital filter circuit receives output from the com

parator circuit and preferably consists of a digital low-pass filter implemented in a complex programmable logic device (CPLD) that serves to precondition the comparator circuit's

55 digital outputs. The preferred device is an AT1504ASVL CPLD which is manufactured by Atmel. Base band system noise or other glitches potentially occurring in the analog signal processor interface, but prior to the actually arrival of the ultrasonic signal, could cause a threshold disruption that

60 registers a "runt" pulse as a false trigger condition. The "runt" pulse would be misinterpreted as the actual TOF trigger and cause serious error in the position calculation. An ANDINOR one-hot state machine design may be used to ignore level transitions that are not stable for at least Ih

65 system clock frequencyx8 states, so only transitions of 4 flS or greater are passed through. The system clock delays introduced by the digital filter's synchronous state machine

JX-001.0024

A000024


US 7,292,151 B2 23

affect all channels the same and are, therefore, effectively eliminated by the inherent dependency on relative measurement.

In the preferred embodiment, the processor and digital filter circuits receive the output from the analog processor and provide controlling signals therein. The preferred processor circuit is the MC9S08GB60 which is manufactured by Motorola Inc. It is a low-cost, high-performance 8-bit microcontroller device that provides all the aforementioned hardware circuits integrated into one convenient device. The calculation circuit is abstracted from embedded 60 KB FLASH for program memory with in-circuit programmable capability and 4 KB RAM for data memory. The time base circuit is preferably comprised of an external, high-noise immnnity, 4.0 MHz system clock, which multiplies this by the internal frequency-locked loop for a bus clock of 40.0 MHz and single instruction execution time of 25 llS. This clock also provides all the capture and control timing fnnctionality for the other specified circuits. Multiple parallel I/O ports and dedicated asynchronous serial communication signals provide for the digital control of the analog signal processing and commnnication interfaces, respectively.

The timing capture-control circuit receives the output from the digital filter circuit representing the arrival of the TOF triggers to detennine the relative TOF propagation of the ultrasonic acoustic wave as it passes through the receiver constellation nnit. More specifically, it is comprised of a five channel 16-bit timer input capture module with programmable interrupt control that provides edge detection and 50 llS timing precision to automatically register the TOF triggers timestamps asynchronously without using inefficient and less accurate software polling means.

24 ber and Product Description strings. A host computer may enumerate and access this device utilizing the manufacturer's virtual COM port device drivers using a USB channel.

In the preferred embodiment, the radio link circuit is comprised of a wireless bidirectional communication interface to preferably (1) broadcast a synchronization signal to control the transponders interoperability, (2) to receive other transponder sensor data, including, but not limited to, accelerometer, heart rate, battery, user I/O status, (3) to provide

10 control messages for the transponders' sensory interfaces, and (4) to provide means to configure transponders' local databases. The preferred wireless communication link is based upon the AT86RF211, a highly integrated, low-power FSK transceiver optimized for license-free ISM band opera-

15 tions from 400 MHz to 950 MHz. and manufactured by Atmel. It supports data rates up to 64 kbps with data clock recovery and no Manchester Encoding required. The device has a three wire microprocessor interface that allows access of read/write registers to setup the frequency selection,

20 transmission mode, power output, etc. or get infonnation about parameters such as battery, PLL lock state, etc. In normal mode, any data entering its input channel is immediately radiated or any desired signal collected by the aerial is demodulated and transferred to the microprocessor as

25 reshaped register bit information. In wake-up mode, the device periodically scans for an expected message sequence and broadcasts an interrupt if a correct message is detected.

In the preferred embodiment, at least three (3) consecutive TOF timestamps are registered for each receiver during the

30 acquisition phase. Preferably, the transponder's transducer emits a multi-cycle ultrasonic acoustic burst of at least ten cycles in duration so that sufficient energization of the receiver transducer is realized and at least three crests of the

The phase-locked loop circuit receives the output from the timing capture-control circuit and is preferably comprised of 35

a three channel, 16-bit timer compare module is implemented as an all-digital phase locked loop (ADPLL), which synchronizes the capture window and blanking functions with respect to the reference input channel. It is comprised primarily of a free-running 16-bit timer configured to peri- 40

odically interrupt the processor dependent upon a precise convergence of its period and phase to the reference trigger source, by means of an over/under count matching and correction technique.

waveform can be properly registered. At low signal levels when ultrasonic acoustic coupling is poor, this requirement may fail and an invalid tracking status is asserted. Prefer-ably, the reference receiver transducer of the receiver constellation unit is positioned in closest proximity to the acoustic signal source so that it is the first transducer to be affected by the initial wave front. This reference receiver provides the overall system timing and state machine control for the phase-locked loop circuit, so that the processing, calculation, and commnnication tasks are executed in a deterministic and efficient fashion.

The AID conversion circuit receives the output from the 45

amplifier and BW filter circuit and consists of an eight channel lO-bit analog-to-digital converter used to monitor channel offsets and magnitudes for range and polarity errors and correction. This information is utilized by the calculation circuit as input to the TOF software correction algo- 50

rithm to detennine the slope of the wavefonn crest.

It should be appreciated that a high-resolution ultrasonic acoustic tracking system that depends upon threshold detection means has an inherent nncertain trigger dilemma. This uncertainty arises because of the multi-cycle nature of the transmitted signal's waveform and the associated difficulty detecting the exact temporal location for consecutive analysis periods when the signal's magnitude may vary greatly

In the preferred embodiment, the serial communication circuit is comprised of two asynchronous serial communication interfaces that are connected between the calculation circuit and host link and radio link circuits of the communication interface. The host link provides a 115K bit per second (baud) bi-directional communication link to an auxiliary host computer system through a Serial-to-Universal Serial Bus bridge. The preferred device is the CP2101 which is manufactured by Silicon Laboratories. It supports the conversion of a fully asynchronous serial data bus protocol, with buffering and handshaking support, to an integrated Universal Serial Bus (USB) Function Controller and Transceiver and internal clock providing USB 2.0 full-speed compliancy. An integrated 512 bit EEPROM stores the required USB device descriptors, including the Vendor ID, Product ID, Serial Number, Power Descriptor, Release num-

depending upon the efficiency of the acoustic coupling, the distance between transmitter and receiver, and signal-tonoise ratio of the signal processing techniques. If a threshold

55 is set near one of the minor crests of the wavefonn during the last analysis period, then it is conceivable a slight reduction of magnitude of the waveform during the next analysis period may fall slightly below the set threshold and actually not be triggered nntil the next larger excursion of

60 the wavefonn occurs. This would create a TOF error proportional to the period of the acoustic waveform or its intra-pulse interval and have a detrimental affect on the measurement accuracy. This analog processing described above establishes trigger thresholds that allow no more than

65 a single intra-pulse interval of nncertainty, but that is still inadequate for high-resolution measurements. Although a technique is known that controls the largest peak profile of

JX-001.0025

A000025


US 7,292,151 B2 25

the transmitter acoustic signal and claims to provide an absolute trigger condition, this procedure is difficult to reliably tune and control among different transducer types.

26

In the preferred embodiment of the invention, no modulation of the acoustic signal is required. Rather, the adaptive threshold method is augmented with a TOF software correction algorithm that unambiguously detennines the correct TOF based upon a means to detect the same carrier wave cycle of ultrasonic energy every period. The software correction algorithm requires multiple, consecutive TOF acquisitions as input for the digital over-sampling and averaging algorithm, the calculation of a higher-order numerical differentiation of the past and current TOF information as input for the predictive algorithm of higher-order Taylor series based derivatives used for the relative TOF correction, and a measurement of the intra-pulse time intervals of consecutive TOF acquisitions as input for the absolute TOF correction scheme that minimizes the range error based upon selective biasing of the TOFs.

correction algorithm. The initial condition that precedes the start of the relative compensation algorithm may be due to the resumption of a stable, locked tracking state after recovery from a fault condition and, therefore, requires computation of a set of reference TOFs producing minimum range error as a starting basis. The algorithm utilizes a wireless synchronization means to determine a reference TOF calculation between the transponder and reference sensor of the receiver constellation. By computing the reference range

10 distance by the product of the reference TOF and speed of sound in air, this reference range may be compared to the range calculated from the matrices solutions described below. By iteratively and sequential increasing and decreas-

15 ing the TOFs by a single intra-pulse time interval and applying the input to matrices formulations described below, all possible combinations of compensation are permutated and tested, which produces a unique set of TOFs that minimize the error between the calculated range distance

The calculation circuit preferably processes multiple, consecutive TOF acquisitions to effectively improve the timing resolution that proportionally affects position accuracy and precision. The digital filter discussed above introduces quantization errors because of its discrete operation. And minor fluctuations in the acoustical coupling produces timing jitter or uncertainty in the triggered zero-crossings of the acoustic sinusoidal. A Gaussian average or mean value

20 with respect to the reference range distance. This unique set of initial TOFs serves as the starting basis for the relative compensation algorithm. In the preferred, embodiment, this absolute compensation algorithm works most effectively when (1) the wireless synchronization means is tightly

of multiple TOF is a simple and effective filter strategy. Due

25 coupled to the excitation of the acoustic source, (2) the synchronizing signal's arrival is timed by the same mechanism that times the arrival of the reference transducer's acoustic signal, and (3) the coordinate locations of the

to the possibility of poor acoustic coupling or misalignment, and distant transponder location from the processor unit, the 30

number of detectable triggered zero-crossings may vary for a fixed duration of multi -cycle ultrasonic acoustic burst. The averaging algorithm automatically adjusts to this condition by only including TOFs whose delta changes fall within the expected range of the nominal intra-pulse interval defined by 35

the transmission properties of the acoustic source. The nominal intra-pulse interval is detennined and utilized by the following compensation schemes.

The calculation circuit preferably processes a relative TOF correction algorithm based upon a predictive tuned 40

algorithm that requires higher-order numerical differentiation calculation of the past and current TOFs. This compensates the TOFs that may have registered one intra-pulse interval earlier or later than the nominally expected time due to the trigger dilemma described above. By fonnulating 45

these derivatives into a truncated 2nd order Taylor series expansion and weighting the terms contribution, an estimate of expected TOF is calculated and compared to the actual TOF through an iterative error minimization calculation. A minimized error that results in a delta time change indicative 50

of a discrete intra-pulse interval increase or decrease due to an early or late TOF, respectively, produces a characteristic value that directs the algorithm to compensate the actual TOF by the intra-pulse interval and restore it to its correct value. In the preferred embodiment, this relative compen- 55

sation algorithm works most effectively when, (1) the maximally expected inter-period TOF change is less than the discrete intra-pulse interval, (2) the TOF inter-period processing is contiguous, (3) the TOF increase or decrease is no more than a single intra-pulse interval, and (3) the Taylor 60

series tenns are suitably weighted in the prediction algorithm.

The calculation circuit preferably processes an absolute TOF correction algorithm at least once initially, when the phase-locked loop is stable, but may be perfonned every 65

analysis period depending on computational resources, that determines the initial set of TOF values for the relative

sensors of the receiver constellation are established to a high degree of accuracy.

The calculation circuit preferably employs two software methods of trilateration calculation to estimate transponder position, wherein the particular method used depends upon the availability of a synchronizing signal and the accuracy desired. The first method is based on a relative TOF calculation and the speed of sound is treated as a constant estimated at ambient indoor room temperature. The second method requires calculation of an additional TOF timestamp between the transponder and reference receiver, but calculates the speed of sound as an unknown every analysis period, and thus improves measurement accuracy. The first method eliminates the global system timing variances and delays due to the multiplicity of signal conditioning circuitry and eliminates the need for a controlling signal means synchronized at the generation of the transmission of the ultrasonic acoustic wave. The second method also employs relative TOF calculation but requires an additional synchronization signal from the processor unit to determine the absolute TOF between transponder and reference receiver. Since the absolute TOF is based upon a single chaunel only, its timing latencies can be readily accounted for and easily corrected. This method computes the speed of sound every analysis period, provided the synchronization signal is detected, without need for additional hardware temperature processing or requiring more then five (5) receivers, and automatically accounts for the system's main accuracy limitation of speed of sound in air as defined by Eq.1.1, if uncorrected, yields a 1.6 mmlm ranging error for every 10 C. temperature shift. If the synchronization signal is not detected and, therefore, the second method is not resolvable, the last calculated speed of sound can be utilized within the first method's calculation to minimize error.

c=34.6 rn/s+0.5813 rn/s(Tc--25c C.) (1.1)

The TOF timestamps and speed of sound values are input into linear independent algebraic equations in a matrix

JX-001.0026

A000026


US 7,292,151 B2 27

formulation to solve for the unknown transponder(s) position, in a form as shown in Eq.2.1,

A·X =B (2.1) 5

all al2 al3 a14

A= a2l a22 a23 a24

a3l a32 a33 a34

a4l a42 a43 a44 10

Xl

X= X2

X3

X4 15

bl

b2 B=

28 Since each receiver is fixed at a distance Di from the

transponder as determined by the receiver constellation geometry and because the acoustic waves propagate spherically, by using Pythagorean's theorem the following set of range equations are defined in Eqs.5.1-5,

(x l-U )2+(y l-V)2+(Z 1-W)2 =D12 (5.1)

(x2-u )2+(Y2-V)2+(Z2-W)2 =D22 (5.2)

(xru)2+(yrv)2+(zrw)2=D32 (5.3)

(X4-U)2+(y4-V)2+(Z4-W)2=Dl (5.4)

(XS-U)2+(yS-V)2+(ZS-W)2=Ds2 (5.5)

Equivocally, the four (4) non-reference receivers are preferably located at an incremental distance relative to the reference receiver, so by substitution of the incremental distance defined by Eq.6.1, the following set ofrelativistic

b3

b4 20 range equations are defined by Eqs.6.2-5,

To solve for the unknowns X, Eq.2.1 is rearranged as shown in Eq.3.1, whereas the inverse of A requires computation of the cofactor matrix AC for the adjoint and determi- 25

nant calculations for Eq.3.2 and Eq.3.3, respectively,

X=A-l.B=(A'l.B IAI

(3.1)

30

(6.1)

(6.2)

(6.3)

(6.4)

(XS-U)2+(yS-V)2+(ZS-W)2~(Dl+CLl.TlS)2 (6.5)

By expanding and rearranging the terms of Eqs.6.2-5, a set All A2l

(A'l = Al2 A22

Al3 A23

A3l

A32

A33

A4l

A42

A43

(3.2) of four linear algebraic equations and four unknowns for the first method algorithm, depicted in the matrix form of Eq.2.1, is defined by Eq.7.1,

A14 A24 A34 A44 35

(3.3)

To setup the coefficient matrix A, the utilization of five (5) 40

receivers produces the following set of relative TOF equations defined by EqsA.1-4,

(4.1)

(4.2) 45

(4.3)

(4.4)

The receiver locations are fixed within the system's inertial reference frame, while the transponder(s) are mobile with respect to the same reference frame and are defined as follows,

S(xo,yo,zo)==S(u,v,w)~unknown transponder location

50

55

Xl -X2 Yl - Y2 Zl - Z2

2 Xl -X3 Yl - Y3 Zl -Z3

Xl -X4 Yl - Y4 Zl - Z4

Xl -Xs Yl - Ys Zl -Zs

-cLl.Tl2

-cLl.Tl3

-cLl.T14

-cLl.TlS

U (cLl.Tl2)2 + Ri - R~

(cLl.Tl3)2 + Ri - R~

W (cLl.T14)2 + Ri - R~

where Rf = xf + yf + zf for 5 ;:: i ;:: 1

(7.1)

(7.2)

Alternatively, if the second method algorithm is used, the unknown range of the reference receiver Dl can be substituted by Eq.8.1,

Dl ~cToj, Ll.Tol~time of flight (TOF) from S(u,v,w) to S(xjo Yl' Zl) (8.1)

And, by rearranging terms, it is depicted in the matrix form defined by Eq.9.1,

Xl -X2 Yl - Y2 Zl -Z2 -(Ll.TOlLl.Tl2 + o.SLl.TI2) u Ri - R~ (9.1)

Xl -X3 Yl - Y3 Zl -Z3 -(Ll.TOlLl.Tl3 + o.SLl.TI3) Ri - R~ 2

Xl -X4 Yl - Y4 Zl -Z4 -(Ll.TOlLl.T14 + o.SLl.Ti4) W Ri - R~

Xl -Xs Yl - Ys Zl -Zs -(Ll.TOlLl.TlS + o.SLl.Tls) c2 Ri - R~

JX-001.0027

A000027


US 7,292,151 B2 29

Although similar results may be obtained by application of more computational efficient processes such as pivotal condensation or Crout's decomposition, the application of Cramer's rule was used to evaluate the first-order determinant in Eq.3.3 using second-order determinants from Laplace expansion. The final transponder(s) position equations are defined by Eqs.1O.1-S.

30 By examination of the matrices element equivalency of

Eqs.l1.2-3 and manipulation of terms so that the angles may be found using the inverse tangent function, the following rotation equations Eqs.12.1-3 are derived,

(12.1)

(10.1) 10 (12.2)

(10.2) 4' X3 sin8,sin8y ) Y3 -

8 cos8, - 1 z = at X3

(12.3)

(10.3) 15 cosBz_1 == cos()z from previous iteration

(10.4) 8, = atan(~) for 1st iteration

(12.4)

(12.5)

(10.5) 20 These calculations are performed through iterative step

processes which inherit angular approximations of the preceding steps until the final desired angular accuracy is achieved by assuming the conditions of Eqs.12A-S. There-

(10.6)

(10.7)

(10.8) 25 fore the rotation 8z ' roll, is first approximated by Eq.12.S; then the rotation 8x ' pitch, is approximated by Eq.12.1; and then the final rotation 8y , yaw or tum, is approximated by Eq.12.2. The next approximation of 8z utilizes the previous value of 8z in Eq.12.3 and the similar steps are preferably

If the first method is used, D, the range of the transponder to the reference receiver from Eq.lOA may be calculated as a redundant confirmation of the Eqs.1O.1-3 calculations, provided the frame of reference origin and location of the reference receiver are identical or their offsets accounted for. If the second method is used, C, the speed of sound in air, from Eq.lOA must be computed every analysis period if its value is anticipated to be used in the first method in the absence of a synchronization signal.

The orientation of the transponders can be derived from a similar utilization of the above algorithms for a transponder configured with a triad of ultrasonic transmitters. The transducers are preferably arranged in a triangular plane at the transponder of sufficient area for the desired angular resolution. The sequential excitation of each transducer and subsequent calculation of position by the aforementioned methods provides suffIcient information to determine orientation by the inverse kinematic calculations of Eqs.l1.1-4, where the analysis is simplified by assuming the origin of rotations occurs about Tl and Tl23 represents the initial relative position matrix from this origin and Tl23 is the transformed or forward kinematic position matrix.

[

0 X2COS()y + Z2smBy

Rx(8)Ry(8)T123" 0 s1ll8x(X2S1118y -Z2cos8y)

o -cOS8x(X2S1118y - Z2cos8y)

X3cosB

y [ X3 smBxsmBy

-x3cosBxsmBy

30 repeated until the desired accuracy is achieved. The transcendental functions may be evaluated through a conventional look-up table or by a power series expansion.

Preferably, the overall analysis period duration is effectively trebled until the three (3) transducers' positions are

35 calculated, which reduces the system's frequency response and imposes an increased latency effect. Typically, robust absolute orientation processing requires more stringent lineof-sight operation and is reserved for more sensitive, less dynamic, and reduced ROM movement trajectories, e.g.,

40 balance and sway. Therefore, the latency effect is less noticeable upon the real-time performance of the sensory interfaces.

In the preferred embodiment, the interactive hand-held transponders support a dual axis inertial sensor, which is

45 operably configured to provide tilt (pitch and roll) orientation in its horizontal mounting plane. The inertial sensor is mounted in the intended operational horizontal plane with respect to the systems inertial frame of reference. Once the sensors signals has been converted to an acceleration value

(11.1)

(11.2)

r

Xl COS()z + Yl sinBz X2 COS()z + Y2 sinBz X3 COS()z + Y3 sinBz r Rz({)r1 TI23 == -Xl sinBz + Yl cosBz -x2sinBz + Y2COS()z -x3sinBz + Y3cos Bz

~ ~ ~

(11.3)

.: Xl = YI = Zl = 0 A Y2 = 0 A Y3 = Z3 = 0 for initial orientation (11.4)

JX-001.0028

A000028


US 7,292,151 B2 31

that varies between +/-1 g the tilt in degrees is calculated as shown in Eqs.13.1-2, for pitch and roll, respectively.

<I>~a sin(A)1 g) (13.1)

<I>~a sin(Ajl g) (13.2)

This outside-in ultrasonic tracking implementation, where the transponders are mounted on the mobile object, produces inherent temporal delays due to the finite TOF registration and calculation delays after the transponder has already moved into a different position before the measurement is complete. This overall latency period is compensated and minimized through use of a Kalman filter data processing algorithm to estimate the pose of the transponder by optimally and recursively combining past history, new measurements, and a priori models and information. Generally speaking, the Kalman filter is a digital filter with timevarying gains that are optimally determined through a stochastic dynamical model of the motion. The overall goal is to minimize filter lag while providing suffIcient smoothing of the motion data.

An adaptive, multi dynamic model is developed based upon the kinematic quality of the expected movement trajectory. The predictive kinematic model for the Kalman filter is depicted in matrix form utilizing a truncated 2nd order Taylor series expansion as below in Eqs.14.1-2,

[:L =[~ ~ttl +[:l (14.1)

[1 0 1: : o~:" 11:[.[:[ (14.2)

The Kalman filter is now described for a single dimension, although it is utilized for prediction and smoothing for all position dimensions. The predictor stages consist of the calculation of the state and the error covariance projection equations. The state projector equation, EqlS.l, utilizes a discrete time-sampled difference equation of r calculated from Eq.1S.2. In other words, the numerically derived velocity and acceleration components of motion are linearly combined with the previously a priori position to estimate the new position. The corrector stages consist of sequential computation of the gain, updated state estimate, and updated error covariance equations. The a posteriori state estimate, Eq.1S.4, is based on a linear combination of the weighted measurement residual and the last state estimate.

(15.1)

(15.2)

(15.3)

(15.4)

(15.5)

32 defined as a small constant and based upon the actual static timing variance empirically measured. The smaller this value the more confidence there exists in the systems' measurement capability.

In the preferred embodiment, the product of the numerically-derived 1st and 2nd order derivatives of the measured position scaled by a frequency dependent gain provides a computationally practical adaptive dynamic process noise estimate model. The derivative product term increases Qk

10 proportionally for higher velocity and acceleration components of motion, e.g., quick, abrupt directional changes, which effectively increases the gain and, therefore, means more confidence exists in the measurement rather than the estimate. This provides faithful, low-latency response to

15 high-frequency motions. Conversely, the frequency scaling term decreases the predictive "overshoot" characteristic of lower power, repetitive motion, e.g. slower, cyclic, ROM trajectories, which effectively decreases the gain and, therefore, means more confidence exists in the estimate rather

20 than the measurement. It should be appreciated this filter implementation provides superior tracking fidelity and comparable smoothing characteristics as compared to practical lengths of finite impulse response running-average filters and various low-orders infinite impulse response filters. It

25 achieves enough predictive response to compensate for the inherent TOF and computation latencies, while providing and comparable smoothing properties of other filter types.

30

QK==IKq[ (Zk_1-Zk_3)(Zk_1- 2Zk_2 +Z 1_3)sin(zk_1-zk_3)]1 (16.1)

(16.2)

In the preferred embodiment, a three dimensional (3D) piecewise cubic curve interpolates a movement trajectory for smoothing and reduced sample storage for greater

35 memory effIciency. Preferably, four (4) sequential discrete control points of the n-length set of control points, the sample resolution dependent upon the desired movement granularity, and corresponding timestamp are needed to calculate in real-time the interpolated position between any

40 pair of control points. A Catmull-Rom spline algorithm is the preferred method in that the path intersects the control points and would best approximate a movement that may have acute directional changes. The Catmull-Rom spline algorithm is defined by Eqs.17.1-3, where the geometry matrix

45 Gk represents the matrix of three dimensional (3D) control points.

50

55

60

0 -1 2 -1

1 2 0 -5 3 Ck(J.l) = G"i

0 4 -3

0 0 -1

-O.SJ.l + J.l2 - O.SJ.l3

1 - 2.SJ.l2 + l.SJ.l3 Ck(J.l)=Gk

O.SJ.l + 2J.l2 - l.SJ.l3

-O.SJ.l2 + O.SJ.l3

(17.1)

J.l

J.l2

J.l3

(17.2)

(17.3)

The new error covariance projector, Eq.1S.2, is it's previously computed value combined with the current process noise covariance, Qk' which is tuned by an example model 65

derived from the measured motion dynamics shown in Eq.16.1. The gain's measurement noise covariance, Rio is

The fl value is normalized and represents the % value between the 2nd 3rd control points. To calculate the interpolated value between the 1 st and 2nd or the n-l th and nth control points, the value of first control point of the pair and the value of the last control point pair are doubly entered into the

JX-001.0029

A000029


US 7,292,151 B2 33 34

associated drivers. The preferred LED device is a CMD87 manufactured by Chicago Miniature Lamp. These LEDs' intensity is controlled by a white LED driver. The preferred white LED driver device is a MAX1570 manufactured by Maxim. The white LED driver provides a maximum 120 rnA constant current source to each LED for optimal uniform luminescence. The drive current can be proportionally regulated through external pulse width modulation (PWM) means from the processor circuit to modulate its brightness

geometry matrix, respectively. The appropriate dflldt is determined by the desired rate of playback of movement trajectory. To playback at the same rate as the recorded session, and assuming fairly constant velocity, a timestamp should also be saved at each control point registration so that the fl calculation is correctly scaled by the delta time interval. The n-Iength set of control points would be manually registered by the user pressing a switch or automatically post processed by a sorting method where a control point is registered at the tangents of the trajectory having sufficient magnitude and/or experience sign changes which indicates discontinuous or non-monotonic movement.

The major functional interfaces of the transponder unit preferably include the sensory interface, transducer interface, processor, and communication interface. The following descriptions of the transponder unit are based upon the dependence flow represented by FIG. 6.

10 level. Additionally, an electronic switch is connected in series to each LED drive to individually control its active state. By simultaneously controlling the PWM duty cycle and active state of each LED, the light strobe can appear to smoothly migrate along the linear array in spite of its

15 discontinuous operation.

The sensor interface refers to the collective support for the ultrasonic transmitter, heart rate receiver, and accelerometer circuits. The ultrasonic transmitter circuit is preferably gated by a pulse-width modulated (PWM) digital signal at nominally 0.8% duty cycle of the 40 kHz resonant frequency, e.g., a single 250 fls pulse every analysis period, by the processor circuit. The radiated ultrasonic signal strength is controlled by gating a MOSFET transistor switch at a duty cycle which optimally energizes the transducer's series resonant tank circuit for sufficient duration. The resonant circuit's reactive components include an impedance matching inductor, the transducer's intrinsic capacitance, and a small damping resistive load. At resonance, a electrical damped sinusoidal 30

with a potential up to -400 V pk_pk is developed across the transducer to sufficiently drive it at acoustical power levels practical for the system's intended range of operation. Enabling a lower duty cycle control through means of a software algorithm monitoring the transponders range 35

would effectively lower the transponders power consumption and radiate less ultrasonic acoustic energy for close range operation when signal saturation and clipping is undesirable. Conversely, a higher duty cycle control would radiate greater ultrasonic energy to compensate for less 40

efficient, non-optimal acoustical coupling orientations of the transponder with respect to the receiver constellation. Optionally, two additional transducers may be driven in unison or sequentially from a different transponder assembly

Preferably, the stimuli interface circuit provides the primary aural stimulus by means of a 4 kHz piezo buzzer. The preferred device is SMT-3303-G manufactured by Projects Unlimited. This electromechanical buzzer requires an exter-

20 nal transistor drive circuit and digital control signal gated at a rate near its resonant frequency. The buzzer inputs are connected to and controlled by PWM means from the processor circuit to provide a gross volume adjustment which is dependent upon the amplitude of the drive signal.

25

to support measurement of absolute rotation about a single 45

or multiple axes, or provide calculated positional redundancy for certain diffIcult line-of-sight applications.

The heart rate receiver circuit wirelessly receives a 5 kHz heart rate signal from a Polar® transmitter belt. The transmitter, worn around the chest, electrically detects the heart 50

beat and starts transmitting a pulse corresponding to each heart beat. The receiver captures the signal and generates a corresponding digital pulse which is received by the timing capture-control circuit of the processor interface. A software algorithm processes the signal with known time-based aver- 55

aging and an adaptive window filter techniques to remove any extraneous artifact or corruption caused by interfering sources.

Additionally, the stimuli interface circuit provides the primary tactile stimulus by means of a vibrator motor. The driver for the vibrator motor enables a 120 rnA DC current source to excite the motor armature. The preferred driver device is the MAX1748 manufactured by Maxim. The rotational speed of the motor's armature is controlled by PWM means from the processor circuit.

The processor circuit preferably receives input from the stimuli interface, sensor interface, and the communication interface and provides controlling signals therein. The preferred processor circuit is the MC9S08GB60 which is manufactured by Motorola Inc. It is a low-cost, highperformance 8-bit microcontroller device that integrates the specialized hardware circuits into one convenient device. The software calculation engine circuit operates from an embedded 60 KB FLASH for program memory with in-circuit progranJillable capability and 4 KB RAM for data memory. The time base circuit is preferably comprised of an external, high-noise immunity, 4.0 MHz system clock, which multiplies this value by the internal frequency-locked loop for a bus clock of 40.0 MHz and single instruction execution time of 25 llS. This clock also provides all the capture and control timing requirements for the other specified circuits. Multiple parallel I/O ports and dedicated asynchronous serial communication signals provide digital control for the circuits of the parallel/serial I/O circuit.

In the preferred embodiment, the graphic LCD and touch screen circuit is the primary user input device for database management for an interactive transponder configuration. For example, it may be a 128x64 graphical liquid crystal display system (LCD) and associated 4-pin touch screen input device. A preferred LCD device is the 51553 manufactured by Optrex and the preferred touch screen device is the TSG-51 manufactured by Apollo Displays. LCD display The accelerometer circuit consists of a low cost +/-1.5 g

dual axis accelerometer that can measure both dynamic, e.g. vibration, and static, e.g. gravity or tilt, acceleration. If the accelerometer is oriented so both its axes are parallel to the earth's surface it can be used as a two axis tilt sensor with a roll and pitch axis.

60 information, configuration commands, and bitmaps images can be loaded through the software calculation engine via a parallel memory interface to emulate a graphical user interface. A touch screen input device is connected to a controller circuit to decode soft key presses at areas over the graphical

The stimuli interface circuit provides the primary visual sensory interface preferably comprised of a linear array of five (5) bright, white light emitting diodes (LED) and

65 object. Preferably, the key presses are registered, filtered, decoded, and processed by the controller and then transferred to the software calculation engine via an interrupt

JX-001.0030

A000030


US 7,292,151 B2 35

driven asynchronous serial communication channel of the I/O interface. A preferred LCD controller is the UR7HCTS manufactured by Semtech.

The timing capture-control circuit provides controlling means for the stimuli interface and portions of the sensor interface. The stimuli interface is preferably comprised of a five channel 16-bit timer PWM module with programmable interrupt control which provides 250 llS timing resolution to automatically modulate the circuits' drivers through variable duty cycle control. 10

36 device, which further comprises a second visual display for providing visual stimuli to a user in combination with the first visual display.

6. The system of claim 1, wherein: the output device comprises an array of light emitting

devices. 7. The system of claim 1, wherein: the output device provides audible stimuli to the user. 8. The system of claim 1, wherein:

In the preferred embodiment, the AID conversion circuit receives the output from the accelerometer circuit and consists of a two channel lO-bit analog-to-digital converter used determine the rotational angle of roll and pitch in the accelerometer deviates from its horizontal plane orientation. 15

This information is communicated to the signal processor via the radio link.

the output device provides tactile stimuli to the user. 9. A system according to claim 1, wherein the first

communication device sends ultrasonic signals received by the processing system for determining movement information for the first communication device.

10. A system according to claim 1, wherein said system comprises a registration system adapted to be configured remotely by said first communication device.

In the preferred embodiment, the radio link circuit is comprised of a wireless bi -directional communication interface (with a receiver and transmitter shown generally at 20 and 30) to (1) receive a synchronization signal for control of the transponders interoperability, (2) to transfer acquired local sensor data, including, but not limited to, accelerometer, heart rate, battery, user I/O status, to processor unit and

11. A system according to claim 10, wherein said regis-20 tration system allows the user to record a reference move

ment trajectory remotely using said first communication device.

12. A system according to claim 1, wherein said first communication device is further comprised of:

(3) to provide means to configure its local database from 25

command of processor unit. The preferred wireless communication link is based upon the AT86RF211, a highly integrated, low-power FSK transceiver optimized for licensefree ISM band operations from 400 MHz to 950 MHz. and manufactured by Atmel. Its key features are described 30

above.

a user input device and display and wherein said first communication device is configured with multiple training applications and wherein the user may choose one training application to activate, and wherein said user may download additional training applications to said communication device.

13. A system according to claim 12, wherein said user input device may be used to configure options customized for the user.

In the preferred embodiment, the switch I/O circuit uses a SPST push button switch for user input to control the system's operational states, start and stop program execution, and function as feedback input to the program. A preferred device is the KSS231 SPST pushbutton switch manufactured by ITT Industries.

What is claimed is: 1. A system for tracking movement of a user, comprising: a first communication device comprising a transmitter for

transmitting signals, a receiver for receiving signals and an output device, said first communication device adapted to be hand-held;

a processing system, remote from the first communication device, for wirelessly receiving said transmitted signals from said first communication device, said processing system adapted to determine movement information for said first communication device and sending data signals to said first communication device for providing feedback or control data; and

wherein said first communication device receives and processes said data signals from said processing system and wherein the output device provides sensory stimuli according to the received data signals.

14. A system according to claim 12, wherein said user 35 input device may be used to authenticate user access and

open a user session. 15. A system according to claim 12, wherein said user

input device may be used to calibrate said first communication device to establish a reference pose or reference

40 traj ectory. 16. A system according to claim 1, wherein said first

communication device is adapted to accept various mechanical extensions pieces depending on the application desired.

17. A system according to claim 1, wherein said first 45 communication device transmits accelerometer signals to

said processing system.

50

18. A system according to claim 1, wherein said first communication device transmits heart rate signals to said processing system.

19. A system according to claim 1, wherein said first communication device is further comprised of:

2. A system according to claim 1, wherein said first 55

communication device is a transponder.

an inertial sensor and wherein said first communication device transmits signals containing orientation information to said processing system.

20. A system according to claim 1, wherein said first communication device is further comprised of: 3. The system of claim 1, wherein the first communication

device further comprises: a first visual display for providing an interactive interface

for the user. 4. The system of claim 3, further comprising: a display device in communication with the processing

system for providing sensory stimuli for the user according to the transmitted signals from the first communication device.

5. The system of claim 4, wherein the display device indicates the movement direction of the first communication

a nonvolatile memory. 21. A system according to claim 20, wherein the first

60 communication device is adapted to download customized user programs from the Internet to be uploaded to a remote system as the application program.

65

22. A system according to claim 20, wherein performance algorithms are stored in said memory.

23. A system according to claim 22, where said performance algorithms calculate custom information personal to a user in real-time.

JX-001.0031

A000031


US 7,292,151 B2 37

24. A system according to claim 22, wherein said performance algorithm produces a Motivation Index that represents the overall level of enthusiasm or enjoyment for a particular activity.

38 37. A system according to claim 36, wherein said receiver

array is in the form of an S-shaped curve. 38. A system according to claim 36, wherein position

information is calculated based on time of flight measurement of said ultrasonic signals.

39. A system according to claim 38, wherein position information can be calculated without interference from occluding objects.

25. A system according to claim 22, wherein said performance algorithm produces a composite numerical value derived from a weighted average of statistical performance indicators and subjective user input including at least one of the following items: relative scoring improvements, conformity to a range of motion pattern, duration of participation, high activity access rate, relative skill level improvement, daily goal achievement.

40. The system of claim 1, wherein the processing system 10 is adapted to determine position information.

41. The system of claim 40, wherein said processing system is adapted to determine the error between the actual movement information of said first communication device and a movement information defined by a reference move-

26. A system according to claim 20, wherein a user's session data can be saved to said memory for later retrieval.

27. A system according to claim 1, further comprising: a second communication device, adapted to be hand held,

in electrical communication with the first communication device, with the processing system adapted to determine movement information of the second com-

15 ment trajectory.

munication device relative to the first communication 20

device.

42. A system according to claim 41, wherein said processing system is adapted to send feedback signals to said first communication device based on said error.

43. The system according to claim 42, wherein the output device provides feedback stimuli to the user in response to the received feedback signals.

28. A system according to claim 27, wherein said processing system is adapted to determine movement information for both said first and second communication devices and to calculate a displacement vector from said movement information.

44. A system according to claim 43, wherein said feedback stimuli are aural instructions to the user for guiding the user's movements to conform to said reference movement

25 traj ectory. 45. A system according to claim 43, wherein said feed

back stimuli are aural cues informing the user of encroachments of threshold conditions.

29. A system according to claim 28, wherein said processing system is adapted to compare said calculated displacement vector to a reference vector position and to calculate a numerical result.

30. A system according to claim 29, wherein said processing system sends feedback signals to said first communication device based on said numerical result.

46. A system according to claim 43, wherein said output 30 device is an array of light emitting devices adapted to be

strobed at an intensity, rate or pattern proportional to said

31. A system according to claim 30, wherein a user's movement efficiency can be determined. 35

32. A system according to claim 27, wherein said processing system is adapted to determine movement information for both said first and second communication devices and wherein a vector is calculated and compared to a desired reference vector to calculate a numerical result and wherein 40

said processing system sends feedback signals to said first communication device based on said numerical result, said first communication device further comprised of an output device for providing feedback stimuli to the user in response to said received feedback signals. 45

33. A system according to claim 27, wherein said processing system is adapted to determine movement information for both said first and second communication devices

:~e~h:~~i:~e~::' :a~:t;::e:~d e!~~:?c~:n~~~~~~~ 50

devices are adapted to communicate with each other for synchronization purposes.

34. The system of claim 27, wherein the second communication device comprises:

an output device for providing sensory stimuli to said user according to said received data signals.

35. A system according to claim 1, wherein said signals transmitted from said first communication device are radio frequency signals.

36. A system according to claim 1, further comprising:

55

60

a receiver array in data communication with said processing system for receiving ultrasonic signals from said first communication device and wherein said receiver array sends data to said processing system for use in 65

calculating movement information for said first communication device.

error. 47. The system of claim 1, wherein the processing system

is adapted to determine acceleration information of the first communication device.

48. The system of claim 1, wherein: the first communication device further comprises a sensor

for determining tilt information of the first communication device; and the first communication device is adapted for transmitting the tilt information to the processing system.

49. The system of claim 1, wherein: the first communication device comprises an interactive

interface such that movement of the first communication device controls the movement of an object in a computer generated virtual environment.

50. An apparatus for use in tracking movement of a user, comprising:

a transmitter for transmitting signals; a receiver for receiving signals wirelessly from a remote

processing system; wherein the apparatus is hand-held; wherein the receiver is adapted to receive feedback or

control data signals from the processing system, the feedback or control data signals derived from processed information including movement information of the apparatus; and

wherein the receiver receives the data signals from the processing system and wherein the apparatus processes the received data signals to provide feedback or control information to the user.

51. An apparatus according to claim 50, wherein said apparatus is further comprised of:

a display for providing an interactive interface for the user.


JX-001.0032

A000032


US 7,292,151 B2 39

an output device for providing sensory stimuli to said user according to said received data signals.

53. An apparatus according to claim 52, wherein said output device is an array of light emitting devices.

54. An apparatus according to claim 52, wherein the output device provides audible stimuli to the user.

55. An apparatus according to claim 50, wherein said processing system is adapted to determine the error between the actual movement infonnation of said apparatus and a reference movement trajectory.

40 71. An apparatus according to claim 50, further comprised

of: an inertial sensor and wherein apparatus transmits signals

containing orientation infonnation to the processing system.

72. An apparatus according to claim 50, further comprised of:

a nonvolatile memory. 73. An apparatus according to claim 72, wherein said

10 apparatus is adapted to download customized user programs from the Internet to be uploaded to the processing system as the application program.

56. An apparatus according to claim 55, wherein the processing system is adapted to send feedback signals to said apparatus based on said error and wherein said apparatus is further comprised of an output device for providing feedback stimuli to the user in response to said received 15

feedback signals.

74. An apparatus according to claim 72, wherein performance algorithms are stored in said memory.

75. An apparatus according to claim 74, wherein said perfonnance algorithm produces a Motivation Index that represents the overall level of enthusiasm or enjoyment for a particular activity.

57. An apparatus according to claim 56, wherein said feedback stimuli are aural instructions to the user for guiding the user's movements to conform to said reference movement trajectory.

58. An apparatus according to claim 56, wherein said feedback stimuli are aural cues infonning the user of encroachments of threshold conditions.

59. An apparatus according to claim 56, wherein said output device is an array oflight emitting devides adapted to be strobed at an intensity, rate or pattern proportional to said error between the movement of said apparatus compared to said reference movement trajectory.

60. An apparatus according to claim 50, wherein said transmitted signals by said transmitter are ultrasonic signals received by the processing system for determining movement infonnation for said apparatus.

61. An apparatus according to claim 50, wherein the processing system is adapted with a registration system adapted to be configured remotely by said apparatus.

62. An apparatus according to claim 61, wherein the registration system allows the user to record a reference movement trajectory remotely using apparatus.


a user input device and display and wherein said apparatus is configured with multiple training applications, each of which is selectively activated with the user input device.

64. An apparatus according to claim 63, wherein said user input device may be used to authenticate user access and open a user session.

65. An apparatus according to claim 63, wherein said user input device may be used to calibrate said apparatus to establish a reference pose or reference trajectory.

66. An apparatus according to claim 50, further comprising:

a remote visual display in communication with the processing system for providing visual stimuli for the user.

76. An apparatus according to claim 74, wherein said 20 perfonnance algorithm produces a composite numerical

value derived from a weighted average of statistical performance indicators and subjective user input including at least one of the following items: relative scoring improvements, conformity to a range of motion pattern, duration of partici-

25 pation, high activity access rate, relative skill level improvement, daily goal achievement.

77. An apparatus according to claim 72, wherein a user's session data can be saved to said memory for later retrieval.

78. An apparatus according to claim 50 adapted to operate 30 in conjunction with a receiver array in data communication

with the processing system for receiving ultrasonic signals from apparatus and wherein said receiver array sends data to the processing system for use in calculating movement information for said first apparatus.

35 79. An apparatus according to claim 78, wherein said

receiver array is in the form of an S-shaped curve. 80. An apparatus according to claim 79, wherein move

ment information is calculated based on time of flight 40 measurement of said ultrasonic signals.

81. An apparatus according to claim 50 further comprised of an output device and wherein said apparatus processes said feedback data from the processing system and provides stimulus from said output device to cue the user to move in

45 a predetennined direction to assess the user's ability to balance.

82. An apparatus according to claim 50 further comprised of an output device and wherein said apparatus processes said feedback data from the processing system and provides

50 stimulus from said output device to cue the user based on the movement information of said apparatus in reference to at least one of: a desired range of motion and a desired location.

67. An apparatus according to claim 50 wherein said 55

transmitted signals by said transmitter are radio frequency signals for transmitting information to the remote processing system.

83. An apparatus according to claim 82, wherein the desired range of motion be established by placing targets in the real or virtual world at predetennined locations.

84. An apparatus according to claim 50, wherein the apparatus is a first transponder adapted for communicating with a second transponder, also hand held by the user. 68. An apparatus according to claim 50, wherein said

apparatus is adapted to accept various mechanical exten- 60

sions pieces depending on the application desired. 69. An apparatus according to claim 50, wherein said

apparatus transmits accelerometer signals to the processing system.

70. An apparatus according to claim 50, wherein said 65

apparatus transmits heart rate signals to the processing system.

85. An apparatus for use in tracking movement of a user, said apparatus being hand-held, comprising:

a transmitter for transmitting signals wirelessly to a remote processing system;

a receiver for receiving signals from the processing system, wherein the received signals are feedback signals derived from comparing movement infonnation of the apparatus with a reference movement infonnation;

JX-001.0033

A000033


US 7,292,151 B2 41

an output device for providing stimuli to the user, wherein the feedback signals are used to initiate aural stimuli to the user; and

a user-actuated button for providing input to the apparatus.

86. An apparatus according to claim 85, wherein said transmitted signals by said transmitter are ultrasonic signals for use in obtaining movement information for said apparatus.

87. An apparatus according to claim 85 wherein said aural stimuli are cues to guide the user's movements to conform to a desired reference movement information.

88. An apparatus according to claim 87, wherein said aural signals are cues to the user for proper movement

42 execution to increase range of motion of a predetermined body part of the user.

89. An apparatus according to claim 85, wherein said aural stimuli are cues to train the user for a predetermined physical activity.

90. An apparatus according to claim 85, wherein said aural stimuli provides the user with on-going performance information and where movement information of said apparatus is collected over a period to time to determine the

10 user's ability to perform a particular movement or activity. 91. An apparatus according to claim 85, wherein said

button is actuated by the user to signal an end of a user movement.

* * * * *

JX-001.0034

A000034


UNITED STATES PATENT AND TRADEMARK OFFICE

CERTIFICATE OF CORRECTION

PATENT NO. : 7,292,151 B2 APPLICATION NO. : 111187373 DATED : November 6, 2007 INVENTOR(S) : Kevin Ferguson and Donald Gronachan

Page 1 of 1

It is certified that error appears in the above-identified patent and that said Letters Patent is hereby corrected as shown below:

In column 1, line 8, please delete "SUMMER" and insert -- SUMMARY --.

In column 7, lines 11-12, please delete "can be can saved" and insert -- can be saved --.

In column 39, line 25, please delete "devides" and insert -- devices --.

Signed and Sealed this

Twenty Second Day of April, 2008

JON W, DUDAS Director of the United States Patent and Trademark Office

JX-001.0035

A000035




June 08, 2010



U.S. PATENT: 7,492,268

ISSUE DATE: February 17,2009

By Authority of the

Under Secretary of Commerce for Intellectual Property and Director of the United States Patent and Trademark Office

N. WILLIAMS

Certifying Officer



June 08, 2010



U.S. PATENT: 7,492,268

ISSUE DATE: February 17,2009

By Authority of the

Under Secretary of Commerce for Intellectual Property and Director of the United States Patent and Trademark Office

N. WILLIAMS

Certifying Officer

JX-003

A000036




(75) Inventors: Kevin Ferguson, Dublin, OH (US); Donald Gronachan, Holtsville, NY (US)

(73) Assignee: Motiva LLC, Dublin, OH (US)

( *) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.c. 154(b) by 0 days.

This patent is subject to a terminal disclaimer.

(21) Appl. No.: 11/935,578

(22) Filed: Nov. 6, 2007


US 2008/0061949 Al Mar. 13,2008


(63) Continuation of application No. 111187,373, filed on luI. 22, 2005, now Pat. No. 7,292,151.


(51) Int. Cl. G08B 23/00 (2006.01)

(52) U.S. Cl. .................. 340/573.1; 340/407.1; 4341114 (58) Field of Classification Search .............. 340/573.1,

(56)

340/573.4,539.12,539.13,539.22,407.1, 340/825.36; 3811315; 4341112, 114


References Cited


4,337,049 A 4,375,674 A 4,627,620 A 4,631,676 A

611982 Connelly 311983 Thorton

1211986 Yang 1211986 Pugh

111111 1111111111111111111111111111111111111111111111111111111111111 USOO7492268B2

(10) Patent No.: US 7,492,268 B2 (45) Date of Patent: *Feb.17,2009

4,645,458 A 2/1987 Williams 4,695,953 A 9/1987 Blair et al. 4,702,475 A 10/1987 Elstein et al. 4,751,642 A 6/1988 Silva et al. 4,817,950 A 4/1989 Goo 4,912,638 A 3/1990 Pratt 4,925,189 A 5/1990 Braeunig 5,148,154 A 9/1992 MacKay et al. 5,184,295 A 2/1993 Mann 5,214,615 A 5/1993 Baur 5,227,985 A 7/1993 DeMenthon 5,229,756 A 7/1993 Kosugi et al. 5,239,463 A 8/1993 Blair 5,255,211 A 10/1993 Redmond 5,288,078 A 2/1994 Capper et al.

(Continued)


WO PCT /uS9617 580 5/1997

OTHER PUBLICATIONS

Allard, P., et ai, Three-Dimensional Analysis of Human Movement, Human Kinetics (1995) 3,8-14.

(Continued)

Primary Examiner-Toan N Pham (74) Attorney, Agent, or Firm-Standley Law Group LLP

(57) ABSTRACT





(75) Inventors: Kevin Ferguson, Dublin, OH (US); Donald Gronachan, Holtsville, NY (US)

(73) Assignee: Motiva LLC, Dublin, OH (US)

( *) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.c. 154(b) by 0 days.

This patent is subject to a terminal disclaimer.

(21) Appl. No.: 11/935,578

(22) Filed: Nov. 6, 2007


US 2008/0061949 Al Mar. 13,2008


(63) Continuation of application No. 111187,373, filed on luI. 22, 2005, now Pat. No. 7,292,151.


(51) Int. Cl. G08B 23/00 (2006.01)

(52) U.S. Cl. .................. 340/573.1; 340/407.1; 4341114 (58) Field of Classification Search .............. 340/573.1,

(56)

340/573.4,539.12,539.13,539.22,407.1, 340/825.36; 3811315; 4341112, 114


References Cited


4,337,049 A 4,375,674 A 4,627,620 A 4,631,676 A

611982 Connelly 311983 Thorton

1211986 Yang 1211986 Pugh

111111111111111111111111111111111111111111111111111111111111111111111111111 US007492268B2

(10) Patent No.: US 7,492,268 B2 (45) Date of Patent: *Feb.17,2009

4,645,458 A 2/1987 Williams 4,695,953 A 9/1987 Blair et al. 4,702,475 A 10/1987 Elstein et al. 4,751,642 A 6/1988 Silva et al. 4,817,950 A 4/1989 Goo 4,912,638 A 3/1990 Pratt 4,925,189 A 5/1990 Braeunig 5,148,154 A 9/1992 MacKay et al. 5,184,295 A 2/1993 Mann 5,214,615 A 5/1993 Baur 5,227,985 A 7/1993 DeMenthon 5,229,756 A 7/1993 Kosugi et al. 5,239,463 A 8/1993 Blair 5,255,211 A 10/1993 Redmond 5,288,078 A 2/1994 Capper et al.

(Continued)


WO PCTlUS9617580 5/1997

OTHER PUBLICATIONS

Allard, P., et ai, Three-Dimensional Analysis of Human Movement, Human Kinetics (1995) 3,8-14.

(Continued)

Primary Examiner-Toan N Pham (74) Attorney, Agent, or Firm-Standley Law Group LLP

(57) ABSTRACT



JX-003.0002

A000037


US 7,492,268 B2 Page 2

u.s. PATENT DOCUMENTS 6,308,565 Bl 6,315,673 Bl 6,346,045 B2 6,361,507 Bl 6,366,272 Bl 6,400,452 Bl 6,430,997 Bl 6,487,906 Bl 6,515,593 Bl 6,545,661 Bl 6,720,876 Bl 6,749,432 B2 6,765,726 B2 6,774,885 Bl 6,834,436 B2 6,876,496 B2 7,158,118 B2 7,236,156 B2 7,239,301 B2 7,262,760 B2 7,359,121 B2 * 7,414,611 B2

1012001 French et al. 1112001 Kopera et al. 212002 Rider 3/2002 Foxlin

5,320,538 A 5,347,306 A 5,372,365 A 5,375,610 A 5,385,519 A 5,405,152 A 5,423,554 A 5,429,140 A 5,466,200 A 5,469,740 A 5,474,083 A 5,485,402 A * 5,495,576 A 5,516,105 A 5,524,637 A 5,577,981 A 5,580,249 A 5,584,700 A 5,587,937 A 5,591,104 A 5,597,309 A 5,616,078 A 5,638,300 A 5,641,288 A 5,645,077 A 5,656,904 A 5,659,691 A 5,702,323 A 5,703,623 A 5,704,837 A 5,711,304 A 5,715,834 A 5,720,619 A 5,759,044 A 5,785,630 A 5,785,631 A 5,790,076 A 5,790,124 A 5,792,031 A 5,812,257 A 5,838,816 A 5,846,086 A 5,850,201 A 5,872,438 A 5,888,172 A 5,890,995 A 5,913,727 A 5,929,782 A * 5,963,891 A 5,989,157 A 6,004,243 A 6,028,593 A 6,043,873 A 6,050,822 A 6,050,963 A 6,054,951 A 6,066,075 A * 6,073,489 A 6,077,201 A 6,088,091 A 6,098,458 A 6,100,896 A 6,119,516 A 6,132,337 A 6,152,856 A 6,162,191 A 6,164,973 A 6,181,343 Bl 6,183,259 Bl 6,198,528 Bl 6,244,987 Bl

611994 911994

1211994 1211994

111995 411995 611995 711995

1111995 1111995 1211995

111996 211996 511996 611996

1111996 1211996 1211996 1211996

111997 111997 411997 611997 611997 711997 811997 811997

1211997 1211997

111998 111998 211998 211998 611998 711998 711998 811998 811998 811998 911998

1111998 1211998 1211998 211999 311999 411999 611999 711999

1011999 1111999 1211999 212000 3/2000 4/2000 4/2000 4/2000 5/2000 6/2000 6/2000 7/2000 8/2000 8/2000 912000

1012000 1112000 1212000 1212000

112001 2/2001 3/2001 6/2001

Baum Nitta McTeigue et al. LaCourse et al. Hsu et al. Katanics et al. Davis Burdea Uerich French et al. Church et al. Smith et al. ................. 702/160 Ritchey Eisenbrey et al. Erickson Jarvik Jacobsen et al. Feldman et al. Massie Andrus et al. Riess Oh Johnson Zaenglein, Jr. Foxlin Lander Durward et al. Poulton Hall Iwasaki et al. Dower Bergamasco et al. Fisslinger Redmond Bobick Heidecke Sypniewski Fischer et al. Alton Teitel et al. Holmberg Bizzi et al. Lasko-Harvill Roston Andras et al. Bobick Andoot Stark et al. Walker et al. Walton Ewart Ro senberg et al. Ramer et al. Faughnn Johnson et al. Sypniewski

340/870.01

Poulton ......................... 482/8 French et al. Cheng Ramer French et al. Strohecker et al. Hock Krupka et al. Studor Foxlin Macri Lyons Macri Maynard Ohsuga

200210183961 Al 2006/0028446 Al

4/2002 Rosenberg et al. 6/2002 Maynard 8/2002 French et al.

12/2002 Hock 212003 Stark et al. 4/2003 Goschyet al. 4/2004 Burgess 6/2004 French et al. 7/2004 French et al. 8/2004 Even-Zohar

12/2004 Townsend et al. 412005 French et al. 112007 Liberty 6/2007 Liberty et al. 7/2007 Liberty et al. 8/2007 Liberty 4/2008 French et al. ............... 359/630 8/2008 Liberty

12/2002 French et al. 212006 Liberty et al.

OTHER PUBLICATIONS

Brownstein, B., et al, Functional Movement in Orthopedic and Sports Physical Therapy, Churchill Livingstone (1997), 15. Brugger, W., et al, Computer-aided tracking of body motions using a c.c.d.-image sensor, Med. BioI. Eng. & Comput, (Mar. 1978),207-210. Codella, c., et ai, Interactive Simulation in a Multi-Person Virtual WorldACM (May 3-7,1992),329-334. DeLoura, M., et ai, Game Programming Gems, Charles River Media, (2000) 200-204. Greenleaf, W.J., DataGlove, DataSuit, and virtual reality Advanced technology for people with disabilities, Proceedings of the Seventh Annual Conference 'Technology and Persons with Disabilities,' (Mar. 1992).211-214 Kasvand, T., et ai, Computers and the Kinesiology of Gait, Comput. BioI. Med. Pergamon Press (1976) vol. 6 111-120. Kenmochi, A., et ai, A netwotk virtual reality skiing system-system overview and skiing movement estimation, Symbiosis of Human and Artifact, (Jul. 1995) 423-428. Kraus, A., Matrices for Engineers, Hemisphere Publishing Corp. (1987) 118-120, 124-126. Lengyel, E., Mathematics for 3D Game Programming & Computer Graphics, Charles River Media (2004) 76-78, 467-468. Medved, v., Towards a virtual reality-assisted movement diagnostics-an outline, Robotica (Jan.-Feb. 1994) vol. 12, 55-57. Mulder, A., Human movement tracking technology, School of Kinesiology, Simon Fraser University (Jul. 1994) 1-14. Ruby, D., Biomechanics-how computers extend athletic performance to the body's far limits, Popular Science (Jan. 1982) 58-60. Sandweiss, J., et al, Biofeedback and Sports Science, Plenum Press New York (1985) 1-201. Scarborough, E.L., Enhancement of Audio Localization Cue Synthesis by Adding Environmental and Visual Cues, Air Force Inst. Of Tech., Wright-Patterson AFB, OH. School of Engineering (Dec. 1992) 1-4. Smith, J., et ai, Virtual Batting Cage and Human Model, Virtual Human http://www.cs. berkeley.edu/'rcdavisl classes/cs294/, (Jun. 17,2004)1-5. Zetu, D., et al, Extended range tracking for remote virtual realityaided facility management, Department of Mechanical Engineering The University of Illinois at Chicago, http://alpha.me.uic.edu/dan/ NsfPaper/nsfl.htrnl, (Apr. 19,2004)1-9. Codamotion: The science of real-time motion capture and analysis, webpages from http://www.chamdyn.comlindex.htrnl. (Apr. 17, 2004) 1. IREX, Virtual Reality Technologies, webpages from http://www. irexonline.comlhow_it_works.htrn, (Apr. 19,2004) 1-2.

JX-003.0003

A000038


US 7,492,268 B2 Page 3

Polhemus, Patriot: The Fast and Affordable Digital Tracker, www. polhemus. com, (Feb. 2004) 1-2. Polhemus, Liberty: The Forerunner in Electromagnetic Tracking Technology, www.polhemus.com. (May 2003) 1-2. Success Story Profile: Innovative Sports Training, Motion Monitor, (2002) 1-2. Reality built for two: a virtual reality tool, Symposium on Interactive 3D Graphics, ACM Press webpages from http://portal.acm.org/cita-

tion.cfm?id~91385.91409&dl+ACM&type~series&i (Jun. 10, 2004) 1-4. Europe is Bursting with Virtual Reality Ideas, But Developers Are Critically Strapped for Cash, webpages fromhttps:llwww/lexis.com! research/retrieve? _m~66dI7057cl b77fl97aledb9f5fadb 87d&_browseType~Text, (Jan. 1993) 1-2.

* cited by examiner

JX-003.0004

A000039


u.s. Patent Feb. 17,2009 Sheet 1 of 10 US 7,492,268 B2

10

----------"') Y FIG-1A

-~-ooIO~O 1 ~~: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

'"--I ~ __

~-------------- --------) y FIG-1B

JX-003.0005

A000040



FIG-2A ~ I

FIG-2B

1 kg I I \

FIG-2C

FIG-2D

JX-003.0006

A000041



, t

Evaluate Display PIN Entry I Security & Navigation

Requirements Controls

~I ./ . Request Q) [requlred]--. AuthentiCation}"" - - -"") . II) c

-'= c.. [not required] User Session ~ [identification] °C :::l

Authenticatet ____ J u Q)

U) User

• User Session

[loaded]

t Configure Display ~ Session UI Graphical

.. Incons of Name & ID

1 Descriptor ,,/"

Display Graphical D Request ,,"

'" Incons of Workload Program ---" Intensity & Limits, Selection +

Measurement User Session Criteria, Stimulus [setup] rh Properties, ect.

I Process I I

Program Display ~ Q) I 1+---_ .... I

II) I C I Request Graphical

-'= Request c.. Incons of Session ---"'" ,,~

Parameters a.

~ ,,-

:::l Setup -" Request

~" & Instruction +' ,,~

Q) U) Program Text

User Session ---" Setup ~

[setup] rh User Session

I Process [setup] rh Session +-----"

J

Parameters Process J

l Program 1+---_ ....

Parameters

i ~ [A or B]

( Deployment Phase FIG-3A

JX-003.0007

A000042


u.s. Patent Feb. 17,2009 Sheet 4 of 10

( Deployment Phase) i

User Session [save] rh

Process 1M Deployment

Requirements

--------,

Program

[load] - ______ 1

Evaluate Secondary 1M Requirement

US 7,492,268 B2

Display Graphical Incon for Detail

of Modular Length, Weighted, or

Augmentative Force Resistance

Attachments

[not required]

/ _c __ ,> 1

" i / i / i

,-~+.--~/ i i i

Fasten Primary 1M

• Program

[Deployment] I i i i i i

, i \.. J --t-,

Display Graphical Incon of

User Body

[ d] Location &: rea y Strap Holster

~eChan;sm

[required]

Fasten Secondary 1M

[ready]

--- ...... i ..

Program

[Deployment]

i i i i i i i

, i \ .. J --t--'

\ Display

Graphical Incon of

User Body Location &:

Strap Holster Mechanism

Calibration Phase FIG-3B

JX-003.0008

A000043



Request

( Calibration Phase

Process Requirements of

1M Calibration

Display GraphicalD. Ineon of

Primary 1M Calibration Status

Determine Secondary 1M Requirement

~!nOI requiredl~ [required] Display Graphical~

,......., Primary 1M Pose -'=

---, 1 se~n;~~a~f 1M

~---+<y calib~:~~n Status .B Modification o E o I::

I I

~ Request ' ---""

CI) L......J Process

Primary 1M Program ,......., -'= o +' o

Secondary 1M Pose Modification 1

I If) o

-'= c.. Pose

Modulate Au ra I/Visua I/Tacti Ie

Stimuli rh

Evaluate Primary 1M Pose Match (~,

[Calibration]

I I I J

----'

-----<! Mot~' Pose to" Intial Reference

Movement Trajectory

E o I::

L......J

Process Secondary 1M

Pose


Stimuli rh

Evaluate Secondary 1M

+ Program

[Calibration]

: I I

__ ~J

'------< Match Pose to ~

pose~MatCh P-,.,_"_,,_,_,,"""--__ .,..

[match] Intial Field

____ ~~ ..... ___ ~ ____ [_t .... a~tc_h] Position

( Execution Phase FIG-3C

JX-003.0009

A000044


u.s. Patent Feb. 17,2009

~--[Iearn]---......

Request Primary 1M Pose ---,

Modification • Program

[Record]

User Session

[Setup]

CI> I Cf) I 0 .s::. I

c.. I:: 0

:,J :::l 0

Process CI> x Learn I.LI

Terminus

[not done]

[done]

Program

[Save]

• )

FIG-3D

Sheet 6 of 10

Query Learn

Modality

[normal]

Process Primary 1M

Pose

Process Program Terminus

[done]

US 7,492,268 B2

---, I I I

+ Program

[Execution] I I I I I I I I I J

--~

[not done]

Data Analysis Phase

JX-003.0010

A000045



---------L-~ --r------=---I-~) -:~-:--~=:---]

"~------~

/ .-.-.-

--- -""" ......

10

I I

/

I I

I

I

I I

I I

FIG-4B

JX-003.0011

A000046


u.s. Patent Feb. 17,2009

Remote Position Processor

Analog Signal Processor Interface

Receiver Constellation

Sensor & Preamplifier

«access»

r----I J

I-I I I I J r--------

r, Amplifer &

BW Filter

J

,I I

«a.:c:~~_...J

r, Comparator

I I I I I I I

«access»

Absolute Value

«access»

Peak Detect &

Sample-Hold

«access»

~-------------------

L..-U_S_B_2_'O_J ..... ----- --

FIG-5

Communication Interface

Link

L...-___ ---'

Sheet 8 of 10

Digital Signal Processor Interface

Processor

US 7,492,268 B2

r, ~-~, Timebase .s I AID

Conversion .s I en I M I

L..-____ .r ...... --.... 1

\ I I I

r, Phase-Locked

Loop

r-

I "" ) I I I I

L...-_...,._---' ~ I 15 I I "t) --l-

~-) (

Timing Capture & PWM

J

------./ I

~ en en CI) o o c ~

I I I

r, Software

Calculation Engine

I I I

r---- r,

Para lIel/Seria I

I/O (

I I I

_____ J

r, Digital Filter

r------------I «1M status» I I

-.I

20

Radio Link

30

I ---.I

~~..--=-=-.;--n ---

JX-003.0012

A000047



FIG-6A

FIG-6E

FIG-6e

JX-003.0013

A000048



Transponder

Stimuli Interface Processor I =jl

I 1-, Fit n r.L, 28MHz

White LED ~ -----, & Driver e ... ~ Tlmebase r.L, .....

Vibrator 8 .c:::::::n «clock»l Motor ~ ¥ r.L, & Driver >- IJ }I. PWM

f-, -----, r.L, «access>~

Audio «configure» Annuciator

-----'

& Driver r===n. ~ r===n. T (J)

r.L, (J) r.L, CI) 0

AID 0 Software ,- 0

Calculation Conversion ¥1"0._ Sensor Interface ~ Engine

Fit lioonngu .. J ~ =n ()~

r.L, E

Ultrasonic ~

Transmitter r.L, ¥

J Parallel/Serial

~ess»T «access» I/O " -----, T r.L,

~ -::-r.L, Accelerometer ~()

/ :::l

Heart Rate 0> t;::

Receiver " .:: 0

-""'''" 0

" ¥ i I Dual Axis I

low g I I I I

I I

Communication Interface I Polar

( I compatible ~ user I/O» ..... ( ,.----'

-----, T -----, «user I/O» r.L, r.L,

Graphic LCD I/o &

r---- ,- Switch Touch Screen n ( ) -----, «1M msg»

20, r.L, ~'\

FIG-7 Radio Link

30 ( )------------ - 900MHz ISM

JX-003.0014

A000049


US 7,492,268 B2 1

HUMAN MOVEMENT MEASUREMENT SYSTEM

This application is a continuation of U.S. Ser. No. 111187, 373 filed 22 luI. 2005, now U.S. Pat. No. 7,292,151, issued 6 Nov. 2007, which claims the benefit of U.S. 60/592,092, filed 29 luI. 2004. Each of these applications is incorporated by reference as if fully recited.

BACKGROUND OF THE ART AND SUMMARY OF THE INVENTION

This invention relates to a system and methods for setup and measuring the position and orientation (pose) of transponders. More specifically, for training the user to manipulate the pose of the transponders through a movement trajectory, while guided by interactive and sensory feedback means, for the purposes of functional movement assessment for exercise, and physical medicine and rehabilitation.

Known are commercial tracking and display systems that employ either singularly, or a hybrid fusion thereof, mechanical, inertial, acoustical or electromagnetic radiation sensors to determine a mobile object's position and orientation, referred to collectively as pose.

The various commercial tracking systems are broadly classified by their relative or absolute position tracking capability, in which system the pose of a mobile object is measured relative to a fixed coordinate system associated with either combination of receiver(s) or passive or active transmitter(s) housing mounted on the user. The tracking system's components may be tethered with obvious inherent movement restrictions, or use wireless communication means to remotely transmit and process the information and allow for greater mobility and range of movement.

Typically these tracking systems are utilized for biomechanics and gait analysis, motion capture, or performance animation and require the sensors to be precisely mounted on the joints. Various means of presenting the tracking information in a visual display are employed, such as Heads-Up Display (HUD), that provide occluded or see-through visibility of the physical world, or Fixed-Surface Display (FSD), such as computer desktop monitors, depending upon the simulation and immersive quality required for the application. The application may require various degrees of aural, visual, and tactile simulation fidelity and construct direct or composite camera views of the augmented or three dimensional (3D) virtual reality environment to elicit interactive user locomotion and/or object manipulation to enhance the user's performance and perception therein. The tracked object may be represented in the virtual environment in various forms, i.e., as a fully articulated anthropoid or depicted as a less complex graphical primitive. The rendering strategy employed depends upon the degree of photo realism required with consideration to its computational cost and the application's proprioception requirements.

Tracking technologies possess certain inherent strengths and limitations dependent upon technology, human factors, and environment that need consideration when discussing their performance metrics. Regardless of differentiating resolution and accuracy performance benchmarks, many implementations suffer from varying degrees of static and dynamic errors, including spatial distortion, jitter, stability, latency, or overshoot from prediction algorithms. Some human factors include perceptual stability and task performance transparency, which are more subjective in nature. And environmental issues such as line-of-sight, sensor attachment, range, and multiple-object recognition, need to be considered when

2 selecting the optimal technology for the most robust application development. Irrespective of the intrinsic strengths and weaknesses of the tracking technology employed, ultimately the user's satisfaction with the system's utilization and efficacy, including the production of reliable, easily understood, measurable outcomes, will dictate the overall success of the device.

This invention's system and methods facilitates biomechanical tracking and analysis of functional movement. In the

10 preferred embodiment, this invention is low cost, robust, easy to deploy, noninvasive, unobtrusive, and conveys intuitive and succinct information to the user to execute movement properly and provides performance indicators of said movement for feedback purposes. One feature of the present inven-

15 tion provides for an interactive tracking system because the sensor functionality, or referred to herein as active transponders or transponders, is integrated with local user input control, and real-time sensory interfaces on the same device. The transponder is a wireless communication and monitoring

20 device that receives a specific signal and automatically responds with a specific reply. In one embodiment, the invention provides functional movement assessment based upon the relative measures of limb pose with respect to two positions defined by the transponders. The transponders can oper-

25 ate independently or work in unison to process and share computational tasks and information between the local databases. This decentralized, distributed processing scheme allows the configuration and coordination of the training session, and processing and analysis of the measurements to

30 occur without requiring expensive auxiliary computer and display systems to manage the same, and without relying on costly software development of complex synthetic environments for visualization purposes. Also, the user can manage the applications and performance databases off-line on a

35 remote computer system with Internet connectivity to customize and configure the system parameters in advance of their session.

The present invention is designed to provide such system and methods for high-fidelity tracking or registration of the

40 poses of active transponders and engage the user to purposely manipulate the transponders' pose along a prescribed or choreographed movement trajectory in order to train and assess functional movement capability. In the preferred embodiment, the system is comprised of two subsystems: (1) a sub-

45 system comprised of one or more active transponders, which, in its most sophisticated implementation, responds to periodic requests from another component of the system to radiate or transmit a signal for purposes of absolute position tracking; processes an embedded inertial sensor for relative

50 orientation tracking and absolute tracking refinement; and provides an essentially real-time aural, visual, and tactile sensory interfaces to the user, and (2) a subsystem comprised of a centralized position processor system or unit and receiver constellation unit, collectively referred to as the processor

55 unit, which is essentially a signal processor that synchronizes the transponders' periodicity of radiating signal and other operational states; collectively receives and processes the radiated signal; iteratively calculates the transponders instantaneous pose and convolution, thereof; and continually

60 exchanges this information, and its analysis thereof, with the transponders and/or auxiliary host computer system in essentially real-time via a combined wireless and tethered communication means. This real-time bi-directional exchange of information allows for proper transponder identification,

65 coordination, and the accurate measurement of pose, thereof, and timely actuation of the sensory interfaces for optimal user regulated closed-loop control.

JX-003.0015

A000050


US 7,492,268 B2 3

The transponder is broadly classified by its level of hardware and software configuration that define its scope of intelligence, sensory support, and configuration. The degree of intelligence is detennined by its capability to locally access, process, and modify the database. Further, either transponder classification can be sub-classified by its manipulative requirements. In one embodiment, where multiple transponders are used, a principle transponder is consciously and deliberately moved along the reference movement trajectory, while a subordinate transponder serves as an anchor or sec- 10

ondary reference point elsewhere on the locomotion system whose kinematics are not necessarily controlled by the user's volition.

4 a means to create a single movement vector whose end

points are defined by the locations of at least two transponders, wherein, the expansion and contraction of the vector's length is calculated, analyzed, and reported in essentially real-time;

a means to create a single movement vector whose endpoints are defined by the locations of two transponders, wherein, a representative point along the vector length is referenced and its higher-order derivatives are computed by mathematical numerical processes, wherein the result is calculated, analyzed, and reported in essentially real-time; and,

a means to correlate said vector's length and at least one other measure consisting of a higher-order derivative, to the reference movement trajectory, wherein the result is calculated, analyzed, and reported in essentially real-time.

A registration system for practical functional movement applications should clearly convey information to the user regarding his movement quality while he performs the task, without compromising or distracting from said execution by uunecessary head movements or change in eye gaze and normal focus. Poor visualization strategies that distract the user are ineffectual for promoting heads-up, immersive inter-

An interactive transponder, preferably, has significant intelligence; supports relative and absolute tracking capabili- 15

ties; provides complete sensory stimuli support; provides for functional enhancement through attachment of modular, extension pieces; and provides a user display and input system to control the training session. In the preferred embodiment, the interactive transponder is primarily held in the hand 20

to facilitate more complex user input and greater sensory intimacy. Conversely, in another embodiment, the fixed transponder has limited intelligence; supports only the absolute pose tracking capability; provides no sensory stimuli support; and is usually mounted to a fixed site on the limb or trunk. 25 action, and the alphanumerical information it imparts often

can not be consciously processed fast enough to elicit corrective action. This system provides for both a local, standalone sensory interface as a primary feedback aid, or alternatively,

A combination of transponder deployment strategies may be required depending on the training session's objectives, such as two interactive transponders grasped by each hand; or alternatively, an interactive transponder, and a fixed transponder attached to the limb or trunk; or lastly, two fixed transpon- 30

ders attached to the limb(s) and/or trunk. In one embodiment, this invention proposes to elicit move

ment strategies based on the deployment of at least two transponders that define the endpoints of a movement vector whose relative translation and rotation is measured and evalu- 35

ated for the assessment of functional movement capability, including but not limited to, limb range of motion and its control thereof, limb strength conditioning, and overall proprioception and hand-eye coordination skills, and overall body movement. This registration system measures a single move- 40

ment vector whose endpoints are comprised of an anchor point, i.e. one that is located in a less dynamic frame of reference, e.g., such as the trunk or abdomen, and another more distal location fixed on or held by a limb or extremity, e.g., the hand, arm, or leg. As this movement vector is trans- 45

lated and rotated through space by the act of the user modifying the pose of the principle transponder in concert with the reference movement trajectory, the vector's length will expand and contract relative to the proximity of principle transponder with respect to the subordinate transponder. The 50

vector's length conveys unique and explicit infonnation regarding the user's movement efficiency and biomechanical leverage. For example, by attaching a fixed subordinate transponder at the hips and a fixed principle transponder on the upper arm, the biomechanics of the act of lifting a box or 55

similar object can be elegantly qualified. If the user assumes a poor lifting technique, i.e. legs locked with the trunk severely flexed with head down and the anns stretched out beyond the basis of support, the vector's length would consistently be measured longer than compared to a good lifting 60

technique, i.e., legs bent at knees with the back straight, head gaze up, and arms close to body. Also, the measurement( s) of higher-order derivatives derived from numerical mathemati-cal processes of a reference point described by the vector would provide additional indication of movement control or 65

smoothness. In summary, one embodiment of the present invention is comprised of:

an interface to a remote fixed-surface display for greater visualization and simulation capabilities. The visual stimulus could be modulated to warn of range violations, or provide signals for purposes of movement cadence and directional cueing. A principle interactive transponder is typically hand-held, which is naturally in close proximity to the user's aural and visual sensory field during most upper extremity movements, or, conversely, the visual stimulus may be viewed through a mirrored or reflective means if not in optimal lineof-sight. A remote fixed-surface display might augment the immersive quality of the user's experience by providing control of a view camera of a simulated computer environment, and display of the transponders and/or interactive objects' static or dynamic poses within the computer display's skewed through-the-window perspective projection.

In summary, one embodiment of the present invention is comprised of:

a means for modulating an embedded luminescent display organized and oriented into a directional-aiding pattern, by varying its degree of intensity and color, or other physical characteristics, to provide a visual display stimulus. This sensory interface is excited at a rate, repetition, or pattern proportional to the pose error of the transponders' movement traj ectory compared to the reference movement trajectory;

a means to view said visual display stimulus with the aid of a mirror(s) or other reflective means;

a means for the real-time projection of sound or speech commands through an audio device to provide warning, alarm, instructional, and motivational aid, and/or additional cueing upon encroachment of static and dynamic limit/boundary conditions defined by the reference movement trajectory;

a means for real-time tactile feedback including, but not limited to, modulation of the rotational properties of a vibrator motor proportional to the pose error of the transponders' movement vector compared to the reference movement trajectory;

JX-003.0016

A000051


US 7,492,268 B2 5

a means for combining the excitation of said stimuli proportional to the pose error of the transponders' movement vector compared to the reference movement trajectory; and,

a means to coordinate the real-time, periodic parametric update and modulation of the stimuli imparted by the sensory interfaces within the transponders from a processing unit by means of a wireless communication linle

This invention addresses the need for an intuitive, interactive method to instruct, create, and deliver a movement tra- 10

jectory command without necessarily relying on pre-programmed, regimented movement trajectories. The registration system can be configured via remote setup at the principle transponder to pre-record and choreograph a freeform movement trajectory of the principle transponder with 15

the intent of the user mimicking the same said path. This impromptu learning modality can expedite the session down time between different users and movement scenarios, and accommodate users' high anthropometric variability in range of movement. In summary, one embodiment of the present 20

invention is comprised of: a means is to provide a movement trajectory learning

modality that allows the user to calibrate and create the desired endpoints, midpoints, and/or total reference movement trajectory through user programmer entry of 25

an input device resident on the transponder; a means to process and save a movement trajectory using a

computationally efficient Catmull-Rom spline algorithm or other similar path optimizing algorithms to create control points along key points of the movement 30

trajectory that define the optimally smoothest path intersecting the control points;

a means to provide database management by a processing unit via a wireless communication link or, alternatively, through user data entry of an input device resident on the 35

interactive transponder; and, a means to access, edit, and store the program and/or data

bases to nonvolatile memory operably coupled to the principle transponders for the purpose of automating the creation, delivery, storage, and processing of movement 40

trajectories. Customized user programs and databases would be downloaded from a central repository or relevant website in advance of the training session to the transponder from the user's home location via the Internet or other convenient locales having networked Inter- 45

net access, and transported to the systems remote physi-cal location, and uploaded into the system's memory, and executed as the application program.

This a priori process of remote selection, download, and transfer of programmatic content and database would mini- 50

mize the user's decision making and input during product utilization by offering only relevant and customized programming material of their choosing targeted for their specific exercise, fitness, or rehabilitation goals. Performance data could be saved indefinitely in the database's nonvolatile 55

memory, until an upload process was performed through the said network so the database could be transferred to another location for purposes of, but not limited to, registration, processing, archival, and normative performance evaluation, etc.

6 but may include external media devices, such as USB FLASH Key or other portable media means, that may have interoperability with other computerized devices. The data structures may include:

Modulation & Feedback Thresholds/Triggers Propertiesthe aural, visual, tactile interfaces require threshold settings which determine their excitation or stimulation characteristics. These settings can be derived from previous performance data or defaults determined from normative data, or modified in real-time, by algorithmic methods including moving averages, standard deviations, interpolation based upon goal-oriented objectives, etc.

Normative Performance-performance data collected over a large population of users through controlled studies, that is distilled down into specific user categories based upon certain demographics that the user may compare and rank hislher results. This data may be initially embedded within the transponders or position processor non-volatile memory and may be augmented or modified automatically or by user volition when counected to the Internet.

Competitive Ranking-applications which have a predominate point goal-oriented purpose would allow access to a global ranking file archive accessed through the Internet or automatically via updated executive files. This ranking file would be created through an analysis of user participation and publishing of hi s/her results through Internet Web-based servIces.

Downloadable Executive Programs & Configurationsnew software programs, including new features, enhancements, bug fixes, adjustments, etc., could be downloaded to the transponder through an Internet connection. Graphics images would be stored in compressed or uncompressed binary forms, i.e., bitmap, gif, jpeg, etc. This new programs could be transferred to any suitable computerized position processor unit located at a remote facility via the transponder's wireless link. Therefore, the user's transponder is the node that establishes the portable network capabilities of the system, not necessarily the computerized position processor.

Custom Menu Interfaces-specialized activities may require more advanced (or simplified) interfaces dependent upon the users' cognitive abilities and interactive specificity. This menu may include interactive queries or solicit information regarding the user's daily goals, subjective opinions or overall impression of the activity and ones performance which could be incorporated in the Motivation Index described below.

Report Generation Tools and Templates-XML, HTML or other authoring language used to create documents on the Web that would provide an interactive browser-based user interface to access additional performance data analysis and report generation tools and templates that may not be available or offered with the standard product.

Custom Performance Algorithms---certain applicationspecific performance analysis may require dynamically linked algorithms that process and calculate non-standard or specialized information, values, units, physical measurements, statistical results, predictive behaviors, filtering, numerical analysis including differentiation and integration,

An exemplary list of specific data structures contributing to or affecting the means for automating the creation, delivery, storage, and processing of movement trajectories described below may be stored within the non-volatile memory of the transponder or position processor which may use high-density serial FLASH, although other types of memory may be used such as SmartMedia, Compact Flash, etc. Additionally, the memory device interface should not be limited to internal,

60 convolution and correlation, linear algebraic matrices operations to compute data pose and scaling transformation, and proprietary types. One example of a proprietary type is Motivation Index, a composite numerical value derived from a weighted average of statistical performance indicators and

65 subjective user input including relative scoring improvements, conformity to ROM pattern, lengthy activity access duration, high access rate, relative skill level improvement,

JX-003.0017

A000052


US 7,492,268 B2 7

daily goal achievement, etc., that could represent the overall level of enthusiasm and satisfaction, the user has for a particular activity.

Range of Motion (ROM) Pattem Generator-the ROM pattem requires some key control points to be captured along the desired trajectory and stored in order that the algorithm can calculate an optimally smooth path, in real-time, during the comparative analysis phase.

ROM Pattem Capture & Replay-the ROM pattem can be can saved to memory in real-time by discrete position 10

samples versus time depending upon the resolution desired and memory limitations and later played back on the transponder or remote display for analysis.

Activity Specific Attributes-includes Reps/Sets, Duration, Pause, Heart Rate Limits, intra-activity delay, level, 15

point scalars, energy expenditure, task-oriented triggers, etc., and other parametric data that controls intensity, execution rate and scoring criteria for the activity.

Instructional Information-textual, graphical, or animation-based instruction, advice, coaching, activity description, 20

diagramed transponder deployment and intra-device connectivity, etc. that facilitates the intuitiveness, understanding, and usage of the system. The form of instruction may include music files saved in various formats, including Wave, MP3 or other current or future audio data compression formats, and 25

video files saved in MPEG or other current or future video data compression formats.

8 environments, and especially when the tracking volume likely contains potentially occluding objects, i.e., an uninvolved limbs or clothing, that become potential sources of competing, reflected paths. The preferred embodiment of the registration system utilizes the time of flight (TOF) measurement of ultrasonic acoustic waves due to its immunity from interference from the visible and near-visible electromagnetic spectrum and its superior ability to overcome most multi-path reflections problems by simple gated timing of the initial wave front. Upon command from the processor unit, the transponders produce a few cycles burst of ultrasonic energy and the transducers of the receiver constellation unit are stimulated and mechanically resonate accordingly, upon the wave front arrival. The processor unit's analog signal processing circuits transform the mechanical energy into electrical signals that resemble tapered sinusoidal waveforms, which another electronic circuit triggers upon using an adaptive threshold technique which, in turn, the processor unit detects and calculates TOF timestamps indicating the wave front arrival. In the preferred embodiment, the system overcomes the ultrasonic technology's intrinsic challenge of precisely triggering on same the waveform location and provides consistent unambiguous trigger detection by comple-menting the adaptive threshold technique with a software timestamp correction algorithm, which includes in part, a digital over-sampling and averaging timestamp algorithm, a relative timestamp correction scheme utilizing a predictive algorithm of higher-order Taylor series based derivatives, and

Real-time Data Management-proprietary data management protocols that reside above the communication driver layer that manage the real-time, synchronous and asynchronous exchange of data between transponder(s) and position processor. This would provide an essential real-time sharing

30 an absolute timestamp correction scheme that minimizes the range error based upon discrete biasing of timestamps.

of activity data, analysis, and feedback stimulus thresholds, or coordination of multiple transponder configurations, or for a collaboration of same or different user requirements to 35

complete a similar activity objective. This invention addresses the need for adaptability of the

registration system to different movement measurement scenarios. In one embodiment, it utilizes a versatile, modular configuration and mounting of the transponders onto the user. 40

The efficient deployment of the transducers between different users' and from task to task requires a universal mounting scheme to provide consistent localization and pose of the transponders at the desired measurement sites on user's body. Also, to compensate for the receivers' finite tracking volume 45

when stationary, the receiver constellation unit may be mechanically modified to optimize its tracking properties by conveniently repositioning it in closer proximity to the expected transponders movement trajectories and line-ofsight, thereof. In summary, one embodiment of the present 50

invention is comprised of:

Further, in the preferred embodiment, the processor unit utilizes the absolute and relative trigger timestamps in a multi-modal trilateration algorithm for the measurement of three-dimensional (3 D) translations and rotations of the transponders. The primary trilateration calculation is derived by an application of Pythagoream theorem involving a point position solution based-upon range measurements from at least three (3) points, versus the well-known triangulation method which uses bearing angles of two cameras of known pose. Additionally, the system's main accuracy limitation is mostly affected by the temperature variability of outdoor environments and its influence on the speed of sound in air value. This algorithm mitigates this problem by mathematically computing the speed of sound every analysis period provided at least five (5) receivers and a transponder synchro-nizing means are utilized. If the integrity of the synchronizing signal is temporarily compromised, the system automatically employs a variation of the trilateration algorithm that uses the last known speed of sound value.

In the preferred embodiment, the maximum update rate, and hence the major contributor to the latency of the position calculation, is determined by the typical acoustical reverberation, typically between 20 to 100 ms, encountered in an

a means to quickly and efficiently alter the location of the transponders using a fastening system designed to quickly attach and dispose various forms of transponder assemblies;

a means to augment the physical properties, i.e., weight and length, of the principle transponder with adjunct electro-mechanical components that provide variations in biomechanical leverage for isotonic and isometric utilization; and,

55 indoor environment. Since the transponders are held or fixed on the user's body and, therefore, are mobile, the TOF measurements will experience an additional latency effect. A Kalman filter is used as a prediction/estimation strategy to minimize and compensate for the latency effect. The predic-

a means to allow the user to manually alter the geometry and pose of the receiver constellation unit to facilitate an optimal tracking location based upon collectively maximizing the ultrasonic source's energy received at the transducer interface.

This invention addresses the practicality and robustness of the registration system when used in either indoor or outdoor

60 tion algorithm uses a higher-order Taylor series based derivatives and augmentative inertial sensor data. Its predictive refinement is dependent upon predefined models of expected movement conditions. Because functional movement is episodic, having periods of stillness interspersed with bursts of

65 motion activity, a multi-modal filtering strategy is preferably employed to handle the unpredictable jerkiness at the start of motion and relatively predictable, smooth motion afterwards.

JX-003.0018

A000053


US 7,492,268 B2 9

In sunnnary, the preferred embodiment of the present invention is comprised of:

10 FIGS. 2A-2D illustrate example extension pieces for the

present invention; a means to detect the same carrier wave cycle of ultrasonic

energy using a software correction algorithm requiring multiple, consecutive TOF acquisitions as input for the digital over-sampling and averaging algorithm, the calculation of a higher-order numerical differentiation of the past and current TOF infonnation as input for the predictive algorithm of higher-order Taylor series based derivatives used for the relative TOF correction, and a 10

measurement of the intra-pulse time intervals of consecutive TOF acquisitions as input for the absolute TOF correction scheme that minimizes the range error based upon selective biasing of the TOFs;

FIGS. 3A-3D illustrate one example of process flows for the present invention;

FIGS. 4A and 4B illustrate a sample application of the present invention;

FIG. 5 illustrates a block diagram of the remote processing system of the present invention;

FIGS. 6A-6C illustrate example receiver configurations of the present invention; and

FIG. 7 illustrates a block diagram of the components of one embodiment of the transponder of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS a means to utilize a dual matrix fonnulation of the trilat- 15

eration algorithm, and a calculation strategy thereof, which decision is dependent upon the integrity of the system's connnunication link, synchronization condition, and the desired measurement accuracy; and,

a means to coordinate the information transfer between 20

transponders and the processor unit so that their contribution to the resultant movement vector calculation can

The present invention provides a practical, versatile measurement tool for the assessment of the user's manipulation strategy of the transponder 10 or transponders along a reference movement trajectory. Moreover, the system and methods measure and analyze the kinematics of the relative translations and rotations of the limbs or extremities with respect

be measured without intra-signal interference. These goals will be attained by such system and methods

that are comprised of the user's interaction described by the following steps as set forth as the preferred embodiment:

a. Authenticate user access and open user session from a local or remote database;

b. Setup user training session, i.e., workload limitations, measurement criteria, and audio/visual/tactile stimuli;

c. Select training program and configure its options; d. Deploy the transponders as instructed to predefined loca

tions of users locomotion system to create at least one transponder movement vector;

e. Calibrate the transponder movement vector to establish its reference pose;

f. Create a movement trajectory using learn mode, if required;

g. Initiate the start of session; h. Detennine the instantaneous pose of transponder move

ment vector relative to its reference pose from a periodic temporal iteration of this step;

i. Perfonn qualitative and quantitative statistical analysis of accumulated measured poses of the transponder movement vector relative to the pattern of instantaneous poses defined by the reference movement trajectory;

j. Update the major transponders sensory interfaces to modulate said system parameters in a periodic temporal iteration of this step;

k. End the session once program objectives have been obtained;

1. Analyze the results by interacting with local and/or remote databases;

m. Provide numerical, graphical, and/or animated infonnation indicating desired perfonnance measurements.

BRIEF DESCRIPTION OF THE DRAWINGS

to each other or to a more inertial reference location on or off body as the transponders are manipulated. This infonnation

25 provides useful insight on biomechanical demands and anthropometric factors that influence human movement efficiency and control. Although measurement perfonnance metrics are important design criteria, it's equally important to provide intuitive and motivating program instruction and

30 administration, and to provide comprehensive analysis and integration of the motion data in a fonn that is objective and easily interpreted. This system improves upon the practicality and user interactive aspects of setup, deployment, calibration, execution, feedback, and data interpretation of a tracking

35 system designed for function human movement. Human movement is a response to external enviroumental

forces which requires the accurate coordination of the distal segment( s) to compensate for these forces. Skillful coordination of human movement is dependent upon the cohesive

40 interaction of multiple sensory systems, including visual, vestibular, with the musculoskeletal system. More specifically, the challenges and goals of cognitive spatial mapping, (2) minimization of energy expenditure, (3) maintaining stability, (4) steering and acconnnodation strategies for various

45 environments, (5) dynamic equilibrium, (6) active propulsion and weight support, and (7) core locomotion pattern should be relationally considered to properly assess human movement. Therefore, it is preferable to engage the interaction of these sensory systems during a training session to promote

50 the desired functional movement outcome. Because many movements persist for 400-500 ms, enough time is allowed for the initiation of the movement and for user correction based upon visual and kinesthetic infonnation acquired during the time of the movement. However, the implemented

55 means of visual feedback should be not be distracting or interfering with the task at hand. In the preferred embodiment, this system engages the sensory systems with nondistracting, intuitive, embedded aural, visual, and tactile

The disclosed embodiments will be better understood 60 stimuli which provide real-time indication of the principle transponder pose error with respect to the reference movement trajectory. when reference is made to the accompanying drawings,

wherein identical parts are identified with identical reference numbers and wherein:

FIG. lA illustrates one example of a deployment apparatus of the present invention;

FIG. IB illustrates one example of hand-held fonn for the transponder of the present invention;

In order to conduct a time efficient training session, this registration system attempts to minimize the encumbering experimental setup and calibration procedures characteristic

65 of more complex and higher cost motion analysis technology. These complementary systems serve important academic or clinical oriented research needs or for motion capture for

JX-003.0019

A000054


US 7,492,268 B2 11

computer animation purposes and strive for highly accurate measurement of joint motion data in terms of angular displacement. Therefore, the integrity and reliability of their motion data is dependent upon proper sensor setup and calibration.

For instance, single axis goniometer-based systems usually require specially designed harnesses to hold the monitor and are firmly strapped or taped over the joint to avoid relative motion artifacts. Usually these devices are tethered and their fit, weight, and constraining mechanical linkages can impose limitations on the joint motion and cause discomfort for the user. Most optical or video-based systems require the placement of numerous active or passive markers over landmarks, such as the joints' center of rotation. These systems should guarantee sufficient environmental illumination and contrast between markers and background to function optimally. Also, these systems are severely affected by occluded markers that may disappear for long periods of time due to rotations and line-of-sight limitations. Other video-based systems do not use markers but require the assignment of the body's joints manually or through computerized automation during data analysis, making real-time analysis arduous and real-time feedback virtually impossible.

In the preferred embodiment, the system doesn't require complicated, time consuming sensor setup and calibration by virtue of it minimalist sensor requirements and uncomplicated sensor mounting. Instead, it requires only the deployment of a sensor on the body (in one embodiment a dual sensor group on a combination oflimb(s) and or trunk) and doesn't enforce stringent movement protocol, but encourages free-form, unrestrictive movement of the transponders.

The transponder's preferred deployment means, include either insertion into a universal strap and holster apparatus (FIG.IA) that secures on the user's limb, extremity, or trunk, including, but not limited to, the hip, ankle, knee, wrist, upper arm, neck, waist or an augmentative mechanical attachment to one or a combination of modular extension pieces shaped into a hand-held form (FIG. IB). A strap ortorx-like clip and holster design provides a firm, yet light weight and comfortable mounting location away from areas that clothing and or uninvolved limbs may occlude.

The modular extension piece is either an instrumented sensory type designed to support alternative tactile stimulus device or alternative configurations of aural, visual, and tactile feedback types, or non-instrumented, weighted extension pieces as shown in FIGS. 2A-2D. All modular extension pieces may be of various physical dimensions and intrinsic weight, with a captive handle design that preferably requires zero grip strength to grasp. Alternatively, the modular extension piece may be coupled to the transponder through a fixed or flexible, segmented, articulated coupling to accommodate attachment of additional transponders and/or other modular extension pieces. These components would quickly assemble to each other using a spherical snap joint or twist snap latch, or similar mechanism, to provide quick alteration of form and function when used for different movement trajectory scenarIos.

In one embodiment, the weighted extension attachments (FIG. 2A) are offered in fixed gradations of one (1) kilogram increments or other convenient unit of measure and either be indicated as such with a numerical label, quantitative mark, or color-code feature, or combination thereof. For upper extremity evaluation, the weighted extension piece integrated into a zero-grip handle would enhance the improvement of musculature strength of the limb, while not compromising the user's endurance with a potentially fatiguing hand grasp requirement.

12 In one embodiment (FIG. 2B), the tactile type provides

force feedback functionality by controlling the rotational speed of an embedded vibrator motor in the shaft. Alternatively, the visual type (FIG. 2C) may be comprised of a series of light emitting diodes that could be uniformly embedded along the length of the handle or transponder and their intensities variably controlled therein. It should be appreciated that a simple, economical mirrored or reflective surface placed in front of the user's visual field could provide sufficient real-

10 time indication of the user's subjective conformity to the said movement trajectory while allowing non-distracting viewing of this visual stimulus. For example, a program that requires the user to reposition the principle interactive transponder through an arc-like movement trajectory in the midsagittal

15 plane through out a range of motion beginning from the waist upwards until parallel to shoulder height. As the user performs the movement, the visual sensory interface could be proportionally excited if the user moves too quickly, or hesitates too long, or produces shaky or erratic episodic motions,

20 or is beyond the prescribed bounds of the movement arc. The light stimulus is easily viewed in the mirror and would indicate corrective action in his or her movement strategy, while appropriate aural commands may be issued simultaneously to encourage the same correction. Regardless of the sensory

25 interface type, its control and excitation properties will be determined by some statistical aspect of the user's conformity to and progression through the movement trajectory.

The hand-held transponder may include a modular extension piece with an embedded graphic display device and

30 associated input means to allow the user to setup, operate, provide visual feedback, and view performance results of the device usage without additional remote display means. More specifically, a software-controlled user interface could provide certain visual prompts in a menu oriented presentation,

35 to instruct the user on (1) device setup, i.e., aural, visual, and tactile feedback parameters, types of program start and termination cues, program intensity based on ratio of amount of repetitions, sets, and rest periods or categorical gradation of challenge, learn mode behavior, etc., (2) scrollable program

40 selection with brief descriptions including objective, desired measurement, i.e., range of motion, energy, accuracy, speed, etc., and instructive information, and (3) alphannmeric and/or graphical display of measured performance data and other biophysical data and its analysis thereof, displayed in stan-

45 dard plotted forms including line, bar, and pie charts, etc. It is important to note that the user input process is intuitive and streamlined so as not to detract from the practicality and user friendliness of the system. Only relevant applications and its control thereof will be sequestered from the database and

50 presented to the user. In one embodiment, two or more transponders and exten

sion pieces, or combinations thereof, may be assembled at their endpoints with a universal spring coupling. The assembled device could be grasped in both hands and bent in

55 various rotational angles about the spring coupling's axis. Isotonic strength conditioning programs can be developed due to the force resistance feedback supplied by the spring. A multi-transponder assembly in the form of a flexible rod or staff could provide an indication of balance of upper extrem-

60 ity strength and proprioceptive function dependent upon the angular closure rate and rotational imbalance and orientation deviation from initial starting position.

Additionally, in the preferred embodiment, the modular extension pieces have provisions for other attachable appara-

65 tus (FIG. 2D) that can augment the program's intensity or difficulty. For example, an eyelet is embedded in the end of the extension piece and is designed to attach an elastomeric

JX-003.0020

A000055


US 7,492,268 B2 13

band, such as the type manufactured by Theraband®. By securing the other open stirrup end of the band to the user's foot, isotonic strength conditioning programs can be developed due to the force resistance feedback supplied by the elastomeric band. Moving the transponder through a movement trajectory is now made more restrictive and challenging.

APPLICATION EXAMPLES

An example training session deploying a dual transponder group is now described that may be designed to improve the range of motion, strength, and coordination of shoulder abduction in a user. The training session would primarily serve as an exercise aid that provides essential feedback to the user so that he/she learns to progressively improve the manipulation of the transponder through the reference movement trajectory, while benefiting from increased shoulder range of motion and strength improvement.

In advance of the training session, a software application is operated from a host computer that provides a utility for baseline configuration and management of the system's and transponder's local databases, and/or access to other remote databases, and for the real-time interface to the data flow between the system's components. The application's navigation and selections are presented to the user through a typical graphical user interface like Microsoft Windows® XP operating system. A generalized step-wise procedure requires the administrator or user to (1) select the desired program and features from a menu screen list, and (2) to initiate a communication process that causes the program parameters to be transferred to the processor unit through a standard computer communication protocol, i.e. serial, USB, ethernet, etc., whereupon, (3) the information is subsequently processed and transferred into the transponders local memory via a wireless communication link, and, finally, (4) the transponder's software program accesses this database to manage the device utilization and configuration of the local display means. Alternatively, a Compact FLASH-based memory card, embedded serial FLASH, or a similar nonvolatile memory device provides the user an additional specialized database supporting remote data collection capabilities. This database would be preprogrammed in advance and the resultant performance data retained, even if the device's power is lost, or for extended unsupervised exercise sessions conducted remotely from the host computer system or when the host computer system is unattached or unavailable. After the session is completed, the user would be queried if the results are to be saved for later analysis or would automatically be saved, dependent upon device setup. This data could be retrieved at a later time when the system is once again attached to a host computer system, and the software utility could be commanded to upload the database.

Henceforth, the following procedural description refers to the activity dependencies diagrammed in FIGS. 3A-3D that the user would encounter while operating the system.

During the Security Phase (FIG. 3A), the user may be requested to provide a security authentication code for validation, which opens access to hislhers custom programs in the training session. Next, during the Setup Phase (FIG. 3A), the user can configure global options or select the desired program. The global options may include, but are not limited to, workload intensity, measurement criteria, sensory interface properties, and reporting features. A program menu list would indicate name, ID, and a brief description, or alternatively, be represented by a detailed graphical icon. When the program is selected, other program-specific options can be setup.

14 During the Deployment Phase (FIG. 3B), and dependent

upon the program's objectives, a suitable combination of transponder types will be mounted on the user's body as instructed by the program. This example requires the assembly of a hand-held interactive transponder with graphical display, and a weighted extension piece coupled therein to be grasped by the hand on the same side as the affected shoulder. Another subordinate transponder 12 is placed into a holster assembly strapped around the lower quadriceps on the same

10 side. This setup is shown in FIG. 4. During the Calibration Phase (FIG. 3C), a simple calibra

tion procedure may be requested to evaluate transponder function and specific user range of motion constraints. Typically, this information is determined beforehand and saved in

15 the system's database. Also, practicality of this system is claimed for lack of extensive calibration requirements.

Dependent upon the program's options, a user-defined movement trajectory may be created prior to program start in lieu of executing the predefined version. The learn mode

20 could be utilized to quickly choreograph free-form movement trajectories and save them into the transponder's nonvolatile memory for later execution. The learn mode would be accessed through the user interface and instruct the management of the control point assignment by pressing the push

25 button switch at the appropriate junctures of movement discontinuity or, preferably, allowing automated assignment by the software. In the preferred embodiment, a computationally efficient Catmull-Rom spline is used to define a three dimensional (3D) curve that passes through all the control points

30 along the movement trajectory path. If manually interceding, the user is instructed to press the push button once at each major juncture in the movement trajectory, but, preferably, for no more than a few locations, until the desired end of range of motion is reached as shown in FIG. 4B. Similarly, the return

35 path may be similarly defined or he/she may elect to use the same forward path in reverse. These control points are registered by the processor unit and transferred and saved to the transponders' memory to serve as the control points for the real-time calculation of a Catmull-Rom spline. The Catmull-

40 Rom spline is calculated in real-time from the desired initial starting point to provide a continuous set of position points representing the "learned" reference movement trajectory.

After the program is selected or the learn mode complete, the user may be instructed to alter the pose of the transponders

45 to satisfY the initial starting conditions of the program. Either one or a combination of sensory interfaces could be excited by the principle transponder to cause the user to direct or steer it towards the desired start point. For instance, the visual sensory interface could sequentially extinguish or dim its

50 peripheral light sources to converge to a central light source as the principle transponder is positioned closer to the desired start point. Alternatively, the aural sensory interface could change its tonality and loudness as the start point is approached. Or alternatively, the tactile sensory interface

55 could be modulated to provide less force feedback as the start point is approached.

During the Execution Phase (FIG. 3D), the transponders are continually manipulated along the reference movement trajectory to the best of the user's skill and fidelity, within the

60 bounds of the user's physical limitation, until an aural, visual, or tactile response is given that indicates the activity volume has been successfully completed or a sufficient number of conformity violations or failures have been registered. The processor unit calculates the instantaneous pose coordinates

65 of the transponders every analysis interval and periodically communicates this information to the transponders via the wireless communication link. As the principle transponder is

JX-003.0021

A000056


US 7,492,268 B2 15

moved in mimicry to the reference movement trajectory the conformity error between the actual and reference movement trajectory is calculated periodically in real-time to determine the characteristics of feedback quality to be elicited by the sensory interfaces for the user's closed -loop control to correct hislher manipulation strategy. For example, the conformity error may be calculated from statistical processes based upon the standard deviation of the least mean squared (LSM) principle transponder's position error compared to the reference movement trajectory, or based upon, or combination thereof, 10

a threshold magnitude of some multi-order numerical differentiation of said movement to indicate a "smoothness" qual-ity of translation and rotation along the movement trajectory path.

16 One testing and training scenario for postural stability

would be to measure frequency and amplitude of body sway in three dimension (3D) space while feet remain in a fixed position. This task can be performed in both a double or single leg stance to test for bilateral symmetry relating to balance. Another modification of the test would be to perform each test with eyes both open and closed to help determine the contribution of the visual component to overall balance ability. Tracking body sway while creating the illusion of motion through proper visual cueing on a display means would be another test to help determine the reliance on specific sensory components of balance. Delivering repetition of protocols with increasing difficult oscillation thresholds with biofeedback of successes and failures of such predetermined goals is

Alternatively, a host computer system could provide an auxiliary processing and display means to allow another software program to access the transponder's calculated positional data through an application programmer's interface and use this data to alter the pose of a graphical primitive in proportion to the motions of the transponders within the context of computer generated virtual enviroument. The dynamic control of objects in the computer generated virtual environment could be used to augment the local sensory interfaces of the transponders through an interactive, goal-oriented video game modality. The video game challenges could be increased over time based upon some scoring criteria of successful manipulation of the principally controlled on-screen graphical object with respect to cueing derived from other secondary static or dynamically moving objects. It is important to note that only primitive forms of video game challenges would be considered, to take into account the user's cognitive awareness and physical limitations, and the economics of software development for photo realistic virtual environments and animation. Also, this auxiliary computer display means would offer an alternative visualization method of interactive and immersive video feedback aid to enhance the application presentation.

15 one way to train to improve balance.

Additional examples of how the present invention may be applied are described as follows:

Balance The body has the ability to maintain balance under both

static and dynamic conditions. In static conditions, such as in standing, the body strives to efficiently maintain posture (often referred to as postural stability) with minimal movement.

The transponder can deliver aural, visual, and tactile stimuli to queue the individual to the degree of frequency and amplitude of body oscillations. The aural and tactile components provide the only means of feedback when the testing

20 and training are performed with eyes closed or the visual field is compromised. Examples include, but are not limited to, (1) an audio signal increasing and decreasing in volume proportional to the amplitude of body sway, (2) a vibration action proportional to frequency of body oscillations, and (3) a light

25 source illuminated when both frequency and amplitude goals are achieved. Multiple transponders can be used to evaluate and reinforce proper balance posture by communicating position information of certain body segments in relationship to others. An example would be the comparison of position of

30 the head with respect to the hips while generating a vibration action if an excessive forward lean of the head as compared to the hips is recognized.

Another test for balance would be to test ones Limits of Stability (LOS). This test refers to ones ability to effectively

35 operate within their sway envelope. The sway envelope or LOS is defined as the maximal angle a person's body can achieve from vertical without losing balance. An individual with healthy balance is capable ofleaning (swaying) within a known sway envelope and recover back to a centered position

40 without the need for a secondary maneuver such as a step, excessive bend at the torso or arm swinging. LOS for bilateral stance in normal adults is 8 degrees anterior, 4 degrees posterior and 8 degrees laterally in both directions.

The present invention described can be used as a testing 45 and training device for balance control during movement or

perturbations within a desired sway envelope. Through proper visual queuing represented on the display means that defines a normal sway envelope, the amount of body displace-

In dynamic conditions such as in walking or sports play, the body strives to maintain balance while performing in an ever changing environment. The ability to maintain balance is a complex process that depends on three major sensory components. The sensory systems include visual, vestibular and 50

proprioception. For example, we rely on our visual system (eyes) to tell us if the environment around us is moving or still;

ment can be measured from vertical stance. The transponder can deliver aural, visual, and tactile

stimuli to queue the individual as to when he or she has achieved the desired range of their sway envelope, then assess the individual's ability to return back to a vertical stance. we rely on our vestibular system (iuner ears) to tell us if we are

upright or leaning, standing still or moving; and we rely on our proprioceptive system (feet and joints) to tell us if the 55

surface we are standing on is uneven or moving. If balance problems develop, they can cause profound disruptions in your daily life. In addition to increased risk for falls, balance disorders can shorten attention span, disrupt normal sleep patterns, cause excessive fatigue, increase dependence on 60

others and reduce quality of life. It is not uncommon for individuals with a history of balance problems to regain their balance control through accurate diagnosis followed by specific medical treatment and/or rehabilitation exercises.

The present invention described can be used as a testing 65

and training device for balance improvement under both static and dynamic conditions.

Examples include, but are not limited to, (1) a vibration action when the user varies (meanders) from the desired movement path, (2) an array oflights change intensity and pattern as the individual successfully approaches the intended target, (3) an audio signal is generated when the individual has maintained a stable position with respect to proper visual queuing represented on the display means for a selected period of time. Multiple transponders can be used to evaluate and reinforce proper balance posture by communicating position information of certain body segments in relationship to others. An example would be the comparison of position of the head with respect to the hips while generating a vibration action if an excessive forward lean of the head as compared to the hips is recognized.

JX-003.0022

A000057


US 7,492,268 B2 17

Dynamic balance can be evaluated while having the individual perform coordinated movements which specifically challenge the various components of balance in a dynamic nature. Such movements include, but are not limited to jumping, hopping, and walking. These movements can be performed with eyes both open and closed, during interaction with static or dynamic visual queuing on the display means. The ability to perform these dynamic balance tasks with comparisons to others of similar sex, age or disability can be assessed. Example measurements may include, but are not 10

limited to, (1) amount of body sway in three dimension (3D) space, (2) time to complete specific task, and (3) effects of fatigue on balance ability.

Balance training in both static and dynamic conditions can be easily achieved by providing specific visual queuing on the 15

display means, which challenge the individual to perform repetitive and progressively more difficult balance drills. Performance reports can be generated to establish a baseline, isolate specific strengths and weaknesses within the specific sensory and motor control aspects of balance, and document 20

progression and improvements. The transponder can deliver aural, visual, and tactile

stimuli to queue the individual as to when he or she has achieved the desired balance task. By example, a vibration action is produced proportional to the frequency of a body 25

segment oscillation after the user lands from a hop test and attempts to stabilize and maintain proper postural balance. When the individual finally stabilizes and achieves correct postural balance, an audio signal indicates the task has successfully completed. Multiple transponders can be used to 30

evaluate and reinforce proper balance posture by communicating position information of certain body segments in relationship to others. An example would be the comparison of position of the head with respect to the hips while generating a vibration action if an excessive forward lean of the head as 35

compared to the hips is recognized.

Range of Motion (ROM)

18 vidual's movement trajectory varied from the intended two dimensional (2D) reference movement trajectory by deviation from the planar path into the uninvolved spatial dimension. An array oflight sources could increase illumination in intensity and repetition as the ROM goal was approached and an audio tone could signal the individual they have achieved the desired pause time at the proper ROM.

Multiple transponders can be deployed to determine the contribution of each joint or anatomical structure where more then one joint is involved in the ROM movement (example; shoulder and scapular in overhead reaching). The vector sum of each transponder movement in a specific axis can be added together to determine the total ROM. The ROM of one j oint in a two joint motion can be subtracted from the total ROM to determine the contribution of a single joint in a two joint movement.

Human Performance Testing and Training

There are many devices that test the strength and speed of isolated joint movements, for example, the leg extension and bicep curl. This information has value in testing both healthy individuals, athletes and individuals whose strength and speed capabilities may be compromised by injury, disease, poor conditioning or simply age. Recently in the field of human performance, it has been recognized that the analysis of the mobility of the isolated joint, although providing some value, does not offer enough information to determine how the body will perform during functional movements. Functional movements are defined as movements equal to those encountered on the athletic field, in the work environment or while performing activities of daily living. Functional movements involve the movement and coordination of multiple joints and muscle groups acting together to perform a more complex task then a single, isolated joint movement.

The present invention described can be used as a testing and training device for functional movement improvement. By tracking various registration points on the body with The present invention described can be used as a testing

and training device to determine the range of motion within a joint. Range of Motion is the normal distance and direction through which a joint can move. Limited ROM is a relative term indicating that a specific joint or body part cannot move through its normal and full ROM. Motion may be limited by

40 respect to each other or to an off-body registration point, performance measurements of functional movements can be assessed, such as jumping, cutting, turning, bounding, hopping, shuttling, etc.

a mechanical problem within the joint that prevents it from 45

moving beyond a certain point, by swelling of tissue around the joint, by spasticity of the muscles, or by pain. Diseases that prevent a joint from fully extending, over time may produce contracture deformities, causing permanent inability

The present invention described can be used as a testing and training device for individuals involved in physical rehabilitation, general fitness or sports performance enhancement to improve their functional movement abilities. Proper visual queuing can be represented on the display means to instruct and motivate individuals to perform specific functional move-

to extend the joint beyond a certain fixed position. 50 ments. The present invention described can be used to test the

starting point and end point which an individual is capable of moving a body part, typically a limb and associatedjoint(s). Comparisons to age and sex based normative data can be made. Proper visual queuing can be represented on the dis- 55

play means to instruct and motivate the individual through the proper testing procedure.

The present invention described can be used as a testing and training device for individuals involved in physical rehabilitation or general fitness to improve ROM. Proper visual 60

queuing can be represented on a display means to motivate individuals to extend their range of motion beyond their current capabilities.

The transponder can deliver aural, visual, and tactile feedback that alerts the individual to successes or failures in 65

proper execution of each repetition. An example of tactile feedback would be the transponders are vibrated if the indi-

The transponder can deliver aural, visual, and tactile feedback of proper movement execution. Examples include, but are not limited to, (1) an audio signal alerting the user that the desired performance stance is incorrect, (2) the light sources illuminate when the desired speed is achieved in a first step quickness drill, (3) a vibration action to indicate the limits of tracking range, (4) a vibration action proportional to the magnitude of a biophysical measurement during the interaction with visual queues represented on the display means, (5) a vibration action when the body or limb position does not correlate well to the desired body or limb position of the visual queuing represented on the display means, (6) an audio signal indicating start, stop and pause periods or other controlling events, (7) an audio signal indicating proper body alignment or posture has been compromised, and (8) an audio signal indicating the relationship of desired target heart rate to a desired threshold.

JX-003.0023

A000058


US 7,492,268 B2 19

Hardware Description In the preferred embodiment, the processor unit is com

prised principally of a constellation of five (5) ultrasonic transducers and signal processing circuitry, thereof, and a signal processor that interfaces to this receiver group, performs the pose calculations, and interfaces to the transponders and host computer databases. The following interface descriptions for the processor unit are based upon the dependency flow represented by FIG. 5.

The sensors 14 preferably used for the receiver constella- 10

tion unit are cylindrically-shaped ultrasonic transducers, for example, the model US40KR-0l 40 kHz PVDF ultrasonic receivers manufactured by Measurement Specialties Inc., which provide adequate acoustic pressure sensitivity and exhibit 360 degree onmidirectional broad beam response 15

along the horizontal plane. The onmidirectional characteristic, albeit in one plane only, is very desirable to minimize line-of-sight occlusion. Because of its low resonance Q value, the rising and decay times are much faster than conventional ceramic transmitters. This reduces its power requirements 20

since less burst drive duration is needed to achieve sufficient triggering thresholds at the receiver. This transducer type is also utilized similarly in the transponders to provide the potential for the most optimal acoustic coupling.

The receiver constellation unit is preferably mounted on a 25

fixed support base, and has a pivoting and/or swiveling mechanical linkage which provides an adjustable mechanism for configuration of the receiver constellation unit's inertial frame of reference relative to the tracking field. In the preferred embodiment, it is strategically positioned and oriented 30

in proximity to the tracking field in order (1) to minimize line-of-sight degradation with respect to the expected transponder orientation, (2) to optimize registration resolution with respect to field volume size, and (3) to satisfy the mathematical restrictions of performing trilateration calculations 35

based upon the solution of simultaneous linear equations. It should be noted that the trilateration matrices may be solved if the matrices have a rank of five, and are non-singular, i.e., the matrix determinant is non-zero. In the preferred embodiment, the geometric parameters and their coordinate location 40

of the receiver constellation must insure linear independence of the colunms of the matrices and to avoid the matrices from becoming singular.

One example geometrical permutation of the receiver constellation unit that satisfies these rules is shown in FIG. 6A. It 45

20 trilateration requires the precise resolution of the common intersection of multiple spheres circumscribed by the distance between each transmitter and receiver transducer. Each sphere has an inexact radius due to system noise and measurement resolution. Therefore, the intersection becomes a volume instead of a point and the size of the volume is dependent upon the radii of the intersecting spheres as well as the distance between the spheres' centers. As the radii get larger with respect to the distance between the centers, i.e., the transmitter is farther down range, the spheres begin to appear more and more tangential to one another and the intersection volume increases, although not necessarily symmetrical in all dimensions. Therefore, to minimize position uncertainty, the receiver transducers should be separated from each other as much as practical proportions allow with respect to the confines of the tracking field volume as the above said geometric examples provide.

This receiver constellation unit can be repositioned with respect to the tracking field by a simple mechanical adjustment as shown in the preceding figures. The mechanical adjustment raises and lowers and changes the length and pivot axis of the cantilever arm which is fixed to a ground base support.

Because the receiver constellation unit operates a distance from the processor unit, each receiver preferably has an associated pre-amplifier circuit to convert the high-input impedance piezoelectric signal into a low-level voltage proportional to the acoustic signal energy impinging the transducers sufficient in order to accurately transmit the signals to the pro-cessor unit. In one embodiment, a high-input impedance AC amplifier design with 30 dB gain can be utilized. The pre-ferred operational amplifier is the OPA373 manufactured by Texas Instruments. It was chosen for its low 1 pA input bias current, high 6 MHz GBW, and low-voltage single supply operation. The amplifier is configured as a non-inverting type with the high-pass cutoff frequency set at 1 kHz. The overall circuitry is preferably enclosed in a metal shield to minimize electromagnetic noise coupling into the highly sensitive amplifier inputs. In addition, a local, regulated power supply is included to allow for a wide range of input voltage supply and provide sufficient power supply rejection to compensate for the noise susceptibility of remote power distribution. All the pre-amplifier circuits' power and signal connections preferably originate from the processor unit.

The processor unit subsystem preferably consists of an analog signal processing interface that provides (1) additional voltage amplification and filtering of base band signal from the preamplifiers, (2) absolute value function, (3) peak detection function, and (4) analog-to-digital comparator function

occupies a volume of approximately 8 cu. ft. and essentially fixes the transducers in a way that defines two primary orthogonal, bisecting planes defined by three non-collinear points each. Another preferred implementation that occupies nearly the same volume is shown in FIG. 6B and is characterized by its S-shaped curve and tilted with respect to the horizontal plane. Another preferred implementation that occupies nearly the same volume is shown in FIG. 6C and is characterized by its helical or logarithmic spiral shape oriented perpendicular to the horizontal plane. Further, as indicated in the preceding figures, the transducers vertical axes are oriented 90° with respect to the typical vertical axis orientation of the transponder's transmitter to improve acoustic coupling in the vertical plane, a consideration for overhead, upper extremity tracking. Although this causes some reduction in the lateral registration bounds, the compromise provides a more symmetric field about the middle or primary location of tracking interest.

50 to provide support for an adaptive threshold means. The adaptive threshold technique provides robust triggering of the most proximal ultrasonic source at a precise temporal point along the traversing sinusoidal waveform of the electrical signal. Essentially, a new threshold signal is recalculated each

In the preferred embodiment, the overall size of the receiver constellation unit is predicated on a phenomenon referred to as Geometric Dilution of Precision (GDOP). The solution of a unique three-dimensional location based upon

55 analysis period based upon a small percentage reduction of the last peak waveform detected. Therefore, the tracking range is not necessarily restricted due to an arbitrarily high threshold setting and the noise immunity is improved as the threshold tracks the waveform envelope and not transient

60 disturbances. An alternative automatic gain control strategy for the amplification function is unnecessary since the trigger threshold will adjust to the signal level instead. In the preferred embodiment, the threshold faithfully tracks the peak to minimize integer period phase errors, so the amplifier's gain

65 is set to prevent signal saturation from occurring when the receiver constellation unit and transponders are in closest proximity during normal use.

JX-003.0024

A000059


US 7,492,268 B2 21

In one embodiment, an amplifier and BW (band width) filter circuit receives the output from the sensor and preamplifier circuit and provides additional amplification and lowpass filtering to condition it for reliable threshold triggering and input to other analog signal processing circuitry. A dual amplifier configuration may be used to provide an additional gain of 40 dB, AC coupling to remove DC offsets of the preamplifier outputs and long cable losses, and low-pass filter to reject noise beyond the interest signal's bandwidth. The first stage amplifier may be configured as a non-inverting type with a gain of 20 dB. The low-impedance DC input signal is effectively blocked by the coupling capacitor in series at its non-inverting input with a high-pass frequency cutoff set at 20 kHz. This gain stage feeds a second amplifier configured as low-pass, 2nd order Butterworth MFB filter. This filter type provides smooth pass band response and reduced sensitivity to component tolerances. The second stage low-pass frequency cutoff is set at 80 kHz with a pass band gain of20 dB.

An absolute value circuit receives the output of the amplifier and BW filter circuit and converts the bipolar signal into a unipolar form for magnitude detection. A dual amplifier configuration may be used to provide highly accurate full wave rectification of the millivolt-level signal. The first stage amplifier feedback switches to control the distribution of input current between the two signal paths dependent upon the input signal polarity. For a positive input voltage the input current will be positive which forward biases Dl and reverse biases D2. This configures the 1st stage as an inverter driving the inverting input resistor of the 2nd stage, which is also configured as an inverter because its non-inverting input is held at virtual ground due to the non-conducting path ofD2. This effectively creates a combined circuit of two cascaded inverters for an overall gain of + 1. For a negative input signal

22 In the preferred embodiment, a comparator circuit receives

the output from the peak detect and sample-hold circuit to convert the analog signal to digital form for high-speed triggering operation of the processor. The preferred device is the MAX941 which is manufactured by Maxim. A percentage of the peak threshold is used to set the inverting input. When the non-inverting voltage exceeds the inverting voltage, the comparator's output will trip and produce a high-true logic pulse that triggers the processor. A latch control input allows the

10 processor to disable the comparator action to prevent unnecessary triggering during the reverberation phase and to prevent potentially disruptive noisy output chattering near threshold crossover beyond its hysteretic immunity. The percentage of threshold level is predetermined through the scal-

15 ing resistors to be set low enough to trigger on the rising edge of the signal's first crest at the furthest range of transponder operation, but high enough above the intrinsic system noise level and external noise caused by reverberation and other ultrasonic sources. Once the first crest is registered, subse-

20 quent crests may be triggered at their zero-crossing representing the most precise timing registration by momentarily disabling the sample-hold circuit. Because of the longer duration trigger receptivity window, early multiple reflections are mitigated by transducer placement at least 3.5 cm away from

25 adjacent planar surfaces, so the reflected acoustic energy doesn't produce a canceling effect of the direct acoustic energy of the later crests. Once a sufficient number of crests have been registered, then the triggering window is blanked for the remainder of the analysis period by latching the com-

30 parator's value. In the preferred embodiment, a digital signal processing

interface is connected to the analog signal processing interface to transform the analog trigger processing into digital position information.

The digital filter circuit receives output from the compara-tor circuit and preferably consists of a digital low-pass filter implemented in a complex programmable logic device (CPLD) that serves to precondition the comparator circuit's digital outputs. The preferred device is an AT1504ASVL

its input current is negative which forward biases D2 and 35

reverse biases Dl. This configures the 1st stage as an inverter driving the non-inverting input of the 2nd stage which changes the sign of the circuit gain. In this mode, the input current is shared between two paths to the input of the 2nd stage, where _2/3 of the input current flows around the 1 st feedback stage and -1f3 flows in the opposite path around the 2nd stage feedback path for a net gain of -1.

40 CPLD which is manufactured by Atmel. Base band system noise or other glitches potentially occurring in the analog signal processor interface, but prior to the actually arrival of the ultrasonic signal, could cause a threshold disruption that registers a "runt" pulse as a false trigger condition. The "runt"

In the preferred embodiment, a peak detect and samplehold circuit receives the output of the absolute value circuit and registers a peak value that is required to set a magnitude threshold precisely at some percentage of full-scale of the peak. A dual amplifier configuration may be used to provide the highest ratio of high output slew rate to low droop. The first stage is typically in negative saturation until the input voltage rises and exceeds the peak previously stored on the sample capacitor at the inverting input. Now the amplifier acts

45 pulse would be misinterpreted as the actual TOF trigger and cause serious error in the position calculation. AnANDINOR one-hot state machine design may be used to ignore level transitions that are not stable for at least Ih system clock frequencyx8 states, so only transitions of 4 flS or greater are

50 passed through. The system clock delays introduced by the digital filter's synchronous state machine affect all channels the same and are, therefore, effectively eliminated by the inherent dependency on relative measurement.

as a unity gain buffer and the input voltage charges the sample capacitor which faithfully tracks the rising voltage. Once the input voltage diminishes in magnitude, the first blocking diode reverse biases and the sample capacitor holds an accurate replica of the highest voltage attained with minimal droop because of the low input bias current of the amplifier and elimination ofleakage altogether in the second blocking diode by bootstrapping its cathode at the same potential provided by the low-impedance buffer of the second output stage. An electronic switch and bleed resistor allow the voltage across the sample capacitor to be reset by the processor during power up and after the triggering event is recorded so the adaptive threshold value can be refreshed each cycle. A 1 st

order Butterworth filter may be used at the input to smooth 65

false in-band transients that could disrupt the peak accuracy detection.

In the preferred embodiment, the processor and digital 55 filter circuits receive the output from the analog processor and

provide controlling signals therein. The preferred processor circuit is the MC9S08GB60 which is manufactured by Motorola Inc. It is a low-cost, high-performance 8-bit microcontroller device that provides all the aforementioned hard-

60 ware circuits integrated into one convenient device. The calculation circuit is abstracted from embedded 60 KB FLASH for program memory with in -circuit programmable capability and 4 KB RAM for data memory. The time base circuit is preferably comprised of an external, high-noise immunity, 4.0 MHz system clock, which multiplies this by the internal frequency-locked loop for a bus clock of 40.0 MHz and single instruction execution time of251lS. This clock also provides

JX-003.0025

A000060


US 7,492,268 B2 23

all the capture and control timing functionality for the other specified circuits. Multiple parallel I/O ports and dedicated asynchronous serial communication signals provide for the digital control of the analog signal processing and communication interfaces, respectively.

24 power output, etc. or get infonnation about parameters such as battery, PLL lock state, etc. In nonnal mode, any data entering its input channel is immediately radiated or any desired signal collected by the aerial is demodulated and transferred to the microprocessor as reshaped register bit information. In wake-up mode, the device periodically scans for an expected message sequence and broadcasts an interrupt if a correct message is detected.

In the preferred embodiment, at least three (3) consecutive TOF timestamps are registered for each receiver during the acquisition phase. Preferably, the transponder's transducer emits a multi-cycle ultrasonic acoustic burst of at least ten cycles in duration so that sufficient energization of the receiver transducer is realized and at least three crests of the

The timing capture-control circuit receives the output from the digital filter circuit representing the arrival of the TOF triggers to detennine the relative TOF propagation of the ultrasonic acoustic wave as it passes through the receiver constellation unit. More specifically, it is comprised of a five 10

channel l6-bit timer input capture module with programmable interrupt control that provides edge detection and 50 11 s timing precision to automatically register the TOF triggers timestamps asynchronously without using inefficient and less accurate software polling means. 15 waveform can be properly registered. At low signal levels

when ultrasonic acoustic coupling is poor, this requirement may fail and an invalid tracking status is asserted. Preferably, the reference receiver transducer of the receiver constellation

The phase-locked loop circuit receives the output from the timing capture-control circuit and is preferably comprised of a three channel, l6-bit timer compare module is implemented as an all-digital phase locked loop (ADPLL), which synchronizes the capture window and blanking functions with respect 20

to the reference input channel. It is comprised primarily of a free-running l6-bit timer configured to periodically interrupt the processor dependent upon a precise convergence of its period and phase to the reference trigger source, by means of

unit is positioned in closest proximity to the acoustic signal source so that it is the first transducer to be affected by the initial wave front. This reference receiver provides the overall system timing and state machine control for the phase-locked loop circuit, so that the processing, calculation, and communication tasks are executed in a deterministic and efficient

an over/under count matching and correction technique. 25 fashion. The AID conversion circuit receives the output from the

amplifier and BW filter circuit and consists of an eight channell O-bit analog-to-digital converter used to monitor channel offsets and magnitudes for range and polarity errors and correction. This infonnation is utilized by the calculation 30

circuit as input to the TOF software correction algorithm to determine the slope of the wavefonn crest.

It should be appreciated that a high-resolution ultrasonic acoustic tracking system that depends upon threshold detection means has an inherent uncertain trigger dilemma. This uncertainty arises because of the multi-cycle nature of the transmitted signal's wavefonn and the associated difficulty detecting the exact temporal location for consecutive analysis periods when the signal's magnitude may vary greatly depending upon the efficiency of the acoustic coupling, the distance between transmitter and receiver, and signal-tonoise ratio of the signal processing techniques. If a threshold is set near one of the minor crests of the wavefonn during the

In the preferred embodiment, the serial communication circuit is comprised of two asynchronous serial communication interfaces that are connected between the calculation 35

circuit and host link and radio link circuits of the communication interface. The host link provides a l15K bit per second (baud) bi -directional communication link to an auxiliary host computer system through a Serial-to-Universal Serial Bus bridge. The preferred device is the CP210l which is manufactured by Silicon Laboratories. It supports the conversion of a fully asynchronous serial data bus protocol, with buffering and handshaking support, to an integrated Universal Serial Bus (USB) Function Controller and Transceiver and internal clock providing USB 2.0 full-speed compliancy. An integrated 512 bit EEPROM stores the required USB device descriptors, including the Vendor ID, Product ID, Serial Number, Power Descriptor, Release number and Product Description strings. A host computer may enumerate and access this device utilizing the manufacturer's virtual COM port device drivers using a USB channel.

In the preferred embodiment, the radio link circuit is comprised of a wireless bi-directional communication interface to preferably (1) broadcast a synchronization signal to control the transponders interoperability, (2) to receive other transponder sensor data, including, but not limited to, accelerometer, heart rate, battery, user I/O status, (3) to provide control messages for the transponders' sensory interfaces, and (4) to provide means to configure transponders' local databases. The preferred wireless communication link is based upon the AT86RF2ll, a highly integrated, low-power FSK transceiver optimized for license-free ISM band operations from 400 MHz to 950 MHz. and manufactured by Atmel. It supports data rates up to 64 kbps with data clock recovery and no Manchester Encoding required. The device has a three wire microprocessor interface that allows access of read/write registers to setup the frequency selection, transmission mode,

last analysis period, then it is conceivable a slight reduction of magnitude of the wavefonn during the next analysis period may fall slightly below the set threshold and actually not be

40 triggered until the next larger excursion of the wavefonn occurs. This would create a TOF error proportional to the period of the acoustic wavefonn or its intra-pulse interval and have a detrimental affect on the measurement accuracy. This analog processing described above establishes trigger thresh-

45 olds that allow no more than a single intra-pulse interval of uncertainty, but that is still inadequate for high-resolution measurements. Although a technique is known that controls the largest peak profile of the transmitter acoustic signal and claims to provide an absolute trigger condition, this proce-

50 dure is difficult to reliably tune and control among different transducer types.

In the preferred embodiment of the invention, no modulation of the acoustic signal is required. Rather, the adaptive threshold method is augmented with a TOF software correc-

55 tion algorithm that unambiguously determines the correct TOF based upon a means to detect the same carrier wave cycle of ultrasonic energy every period. The software correction algorithm requires multiple, consecutive TOF acquisitions as input for the digital over-sampling and averaging

60 algorithm, the calculation of a higher-order numerical differentiation of the past and current TOF information as input for the predictive algorithm of higher-order Taylor series based derivatives used for the relative TOF correction, and a measurement of the intra-pulse time intervals of consecutive TOF

65 acquisitions as input for the absolute TOF correction scheme that minimizes the range error based upon selective biasing of the TOFs.

JX-003.0026

A000061


US 7,492,268 B2 25

The calculation circuit preferably processes multiple, consecutive TOF acquisitions to effectively improve the timing resolution that proportionally affects position accuracy and precision. The digital filter discussed above introduces quantization errors because of its discrete operation. And minor fluctuations in the acoustical coupling produces timing jitter or uncertainty in the triggered zero-crossings of the acoustic sinusoidal. A Gaussian average or mean value of multiple TOF is a simple and effective filter strategy. Due to the possibility of poor acoustic coupling or misalignment, and dis- 10

tant transponder location from the processor unit, the nnmber of detectable triggered zero-crossings may vary for a fixed duration of multi-cycle ultrasonic acoustic burst. The averaging algorithm automatically adjusts to this condition by only including TOFs whose delta changes fall within the expected 15

range of the nominal intra-pulse interval defined by the transmission properties of the acoustic source. The nominal intrapulse interval is determined and utilized by the following compensation schemes.

The calculation circuit preferably processes a relative TOF 20

correction algorithm based upon a predictive tuned algorithm that requires higher-order numerical differentiation calculation of the past and current TOFs. This compensates the TOFs that may have registered one intra-pulse interval earlier or later than the nominally expected time due to the trigger 25

dilemma described above. By formulating these derivatives into a truncated 2nd order Taylor series expansion and weighting the terms contribution, an estimate of expected TOF is calculated and compared to the actual TOF through an iterative error minimization calculation. A minimized error that 30

results in a delta time change indicative of a discrete intrapulse interval increase or decrease due to an early or late TO F, respectively, produces a characteristic value that directs the algorithm to compensate the actual TOF by the intra-pulse interval and restore it to its correct value. In the preferred 35

embodiment, this relative compensation algorithm works most effectively when, (1) the maximally expected interperiod TOF change is less than the discrete intra-pulse interval, (2) the TOF inter-period processing is contiguous, (3) the TOF increase or decrease is no more than a single intra-pulse 40

interval, and (3) the Taylor series terms are suitably weighted in the prediction algorithm.

26 ferred, embodiment, this absolute compensation algorithm works most effectively when (1) the wireless synchronization means is tightly coupled to the excitation of the acoustic source, (2) the synchronizing signal's arrival is timed by the same mechanism that times the arrival of the reference transducer's acoustic signal, and (3) the coordinate locations of the sensors of the receiver constellation are established to a high degree of accuracy.

The calculation circuit preferably employs two software methods of trilateration calculation to estimate transponder position, wherein the particular method used depends upon the availability of a synchronizing signal and the accuracy desired. The first method is based on a relative TOF calculation and the speed of sound is treated as a constant estimated at ambient indoor room temperature. The second method requires calculation of an additional TOF timestamp between the transponder and reference receiver, but calculates the speed of sound as an unknown every analysis period, and thus improves measurement accuracy. The first method eliminates the global system timing variances and delays due to the multiplicity of signal conditioning circuitry and eliminates the need for a controlling signal means synchronized at the generation of the transmission of the ultrasonic acoustic wave. The second method also employs relative TOF calculation but requires an additional synchronization signal from the processor unit to determine the absolute TOF between transponder and reference receiver. Since the absolute TOF is based upon a single channel only, its timing latencies can be readily accounted for and easily corrected. This method computes the speed of sound every analysis period, provided the synchronization signal is detected, without need for additional hardware temperature processing or requiring more then five (5) receivers, and automatically accounts for the system's main accuracy limitation of speed of sound in air as defined by Eq. 1.1, if uncorrected, yields a 1.6 mmlmranging error for every 10 C. temperature shift. If the synchronization signal is not detected and, therefore, the second method is not resolvable, the last calculated speed of sound can be utilized within the first method's calculation to minimize error.

c=34.6 mis+0.5813 m/s(Tc-25C C.) (1.1)

The calculation circuit preferably processes an absolute TOF correction algorithm at least once initially, when the phase-locked loop is stable, but may be performed every 45

analysis period depending on computational resources, that determines the initial set ofTOF values for the relative correction algorithm. The initial condition that precedes the start

The TOF timestamps and speed of sound values are input into linear independent algebraic equations in a matrix formulation to solve for the unknown transponder(s) position, in

50 a form as shown in Eq. 2.1, of the relative compensation algorithm may be due to the resnmption of a stable, locked tracking state after recovery from a fault condition and, therefore, requires computation of

55

a set of reference TOFs producing minimum range error as a starting basis. The algorithm utilizes a wireless synchronization means to determine a reference TOF calculation between the transponder and reference sensor of the receiver constellation. By computing the reference range distance by the product of the reference TOF and speed of sound in air, this reference range may be compared to the range calculated from the matrices solutions described below. By iteratively and sequential increasing and decreasing the TO F s by a single 60

intra-pulse time interval and applying the input to matrices formulations described below, all possible combinations of compensation are permutated and tested, which produces a unique set ofTOFs that minimize the error between the calculated range distance with respect to the reference range 65

distance. This unique set of initial TOFs serves as the starting basis for the relative compensation algorithm. In the pre-

all al2 a13 al4 Xl bl (2.1)

A·X =B A= a2l a22 a23 a24

X= X2

B= b2

a31 a32 a33 a34 X3 b3

G41 a42 G43 G44 X4 b4

To solve for the unknowns X, Eq. 2.1 is rearranged as shown in Eq. 3.1, whereas the inverse of A requires computation of the cofactor matrix AC for the adjoint and determinant calculations for Eq. 3.2 and Eq. 3.3, respectively,

(3.1)

JX-003.0027

A000062


US 7,492,268 B2 27

-continued All A2l A3l A4l (3.2)

(A'l = Al2 A22 A32 A42

Al3 A23 A33 A43

A14 A24 A34 A44

(3.3) 10

To setup the coefficient matrix A, the utilization of five (5) receivers produces the following set of relative TOF equations defined by Eqs. 4.1-4,

Ll.T12=Tr Tl (4.1)

Ll.T13=TrTl (4.2)

Ll.T14=T4-Tl (4.3)

Ll.TlS=Ts-Tl (404)

15

20

The receiver locations are fixed within the system's inertial reference frame, while the transponder(s) are mobile with 25

respect to the same reference frame and are defined as follows,

S(Xi'Yi,zJfor 5~i~1 =>fixed receiver locations

30 S(xo,Yo,zo)=S(u,v,w)~>unknown transponder location

Since each receiver is fixed at a distance Di from the transponder as determined by the receiver constellation geometry and because the acoustic waves propagate spherically, by 35

using Pythagorean's theorem the following set of range equations are defined in Eqs. 5.1-5,

(x l-U)2 +(Yl-V)2 +(z 1-w)2~D12 (5.1)

(xru)2+(yrv)2+(zrw)2~D22 (5.2) 40

(xru)2+(yrv)2+(Z3-W)2~D32 (5.3)

(X4-U)2+(y4-V)2+(Z4-W)2~Dl (504) 45

(xs-u )2+(yS-V)2+(ZS-w)2~Ds 2 (5.5)

Equivocally, the four (4) non-reference receivers are preferably located at an incremental distance relative to the reference receiver, so by substitution of the incremental distance 50

defined by Eq. 6.1, the following set of relativistic range equations are defined by Eqs. 6.2-5,

Di=Dl+C~Tll for 5~i~2 (6.1)

55

(X2-u)2+(y2-V)2+(zrw)2~(Dl+cLl.Td2 (6.2)

(xru)2+(yrv)2+(zrw)2~(Dl+cLl.T13)2 (6.3)

(X4-U)2 +(y 4-V)2 +(Z4-w)2~(Dl +CLl.T14)2 (604) 60

(xs-u )2+(yS-V)2+(ZS-w)2~(Dl +cLl.TlS )2 (6.5)

28

Xl -X2 Yl - Y2 Zl - Z2 -c!:J.T12

2 Xl -X3 Yl - Y3 Zl -Z3 -c!:J.Tl3

Xl -X4 Yl - Y4 Zl - Z4 -CLl.T14

Xl -XS Yl - Ys Zl -ZS -CLl.TlS

where R?=x/+y?+z? for 5~i~1

U

W

Dl

(cLl.T12)2 + Ri - R~

(cLl.Tl3)2 + Ri - R~

(cLl.T14)2 + Ri - R~

(cLl.Tld + Ri - R~

(7.1)

(7.2)

Alternatively, if the second method algorithm is used, the unknown range of the reference receiver D 1 can be substituted by Eq. 8.1,

Dl =cTOl> Ll.Tol~>time of flight (TOF) from S(u,v,w,) to S(XVYi,Zl) (8.1)

And, by rearranging terms, it is depicted in the matrix form defined by Eq. 9.1,

a.

Xl -X2 Yl - Y2 Zl -Z2 -( Ll.TOl Ll.T 12 + 0.5Ll.Tf2) u R2 1 (9.1)

Xl -X3 Yl - Y3 Zl -Z3 -(Ll.TOlLl.Tl3 + 0.5Ll.Tf3) R2 1 2

Xl -X4 Yl - Y4 Zl -Z4 -(Ll.TOlLl.T14 + 0.5Ll.Tf4) W R2 1

-(Ll.TOlLl.TlS + 0.5Ll.Tfs) c2 R2 1 Xl -Xs Yl - Ys Zl -Zs

Although similar results may be obtained by application of more computational efficient processes such as pivotal condensation or Crout's decomposition, the application of Cramer's rule was used to evaluate the first-order determinant in Eq. 3.3 using second-order determinants from Laplace expansion. The final transponder(s) position equations are defined by Eqs. 10.1-8.

IA21 u=w IA21

V=-IAI

(10.1)

(l0.2)

(l0.3)

(lOA)

(10.5)

(10.6)

(10.7)

(10.8)

By expanding and rearranging the terms ofEqs. 6.2-5, a set of four linear algebraic equations and four unknowns for the first method algorithm, depicted in the matrix form ofEq. 2.1, is defined by Eq. 7.1,

If the first method is used, D, the range of the transponder to the reference receiver from Eq. 10.4 may be calculated as a

65 redundant confirmation of the Eqs. 10.1-3 calculations, provided the frame of reference origin and location of the reference receiver are identical or their offsets accounted for. If the

JX-003.0028

A000063


US 7,492,268 B2 29

second method is used, C, the speed of sound in air, from Eq. 10.4 must be computed every analysis period if its value is anticipated to be used in the first method in the absence of a synchronization signal.

The orientation of the transponders can be derived from a similar utilization of the above algorithms for a transponder configured with a triad of ultrasonic transmitters. The transducers are preferably arranged in a triangular plane at the transponder of sufficient area for the desired angular resolution. The sequential excitation of each transducer and subsequent calculation of position by the aforementioned methods provides sufficient information to detennine orientation by the inverse kinematic calculations of Eqs. 11.1-4, where the analysis is simplified by assuming the origin of rotations occurs about Tl and Tl23 represents the initial relative position matrix from this origin and T\23 is the transfonned or forward kinematic position matrix.

(11.1)

X2COS()y + Z2sinBy

sin8Ax2sin8y - z2cos8y)

-cos8x(x2sin8y - z2cos8y)

X3cos8y [(11.2)

x3sinBxsinBy

-x3cosOxsinBy

[

Xl cos(h + Yl sinB2

- Xl sinB2 + Yl cos(h

z;

X2 cos(h + Y2 sinB2

- x2 sinB2 + Y2 eosB2

Z2

X3 eose2 + Y3 sinB2 [

- x3 sinB2 + Y3 eose2

Z,

(11.3)

(1104)

By examination of the matrices element equivalency of Eqs. 11.2-3 and manipulation of terms so that the angles may be found using the inverse tangent function, the following rotation equations Eqs. 12.1-3 are derived,

30 functions may be evaluated through a conventional look-up table or by a power series expansion.

Preferably, the overall analysis period duration is effectively trebled until the three (3) transducers' positions are calculated, which reduces the system's frequency response and imposes an increased latency effect. Typically, robust absolute orientation processing requires more stringent lineof-sight operation and is reserved for more sensitive, less dynamic, and reduced ROM movement trajectories, e.g., bal-

10 ance and sway. Therefore, the latency effect is less noticeable upon the real-time perfonnance of the sensory interfaces.

In the preferred embodiment, the interactive hand-held transponders support a dual axis inertial sensor, which is operably configured to provide tilt (pitch and roll) orientation

15 in its horizontal mounting plane. The inertial sensor is mounted in the intended operational horizontal plane with respect to the systems inertial frame of reference. Once the sensors signals has been converted to an acceleration value that varies between +/-1 g the tilt in degrees is calculated as

20 shown in Eqs. 13.1-2, for pitch and roll, respectively.

25

<I>~a sin(A)l g) (13.1)

<I>~a sin(A)l g) (13.2)

This outside-in ultrasonic tracking implementation, where the transponders are mounted on the mobile object, produces inherent temporal delays due to the finite TOF registration and calculation delays after the transponder has already moved into a different position before the measurement is

30 complete. This overall latency period is compensated and minimized through use of a Kalman filter data processing algorithm to estimate the pose of the transponder by optimally and recursively combining past history, new measurements, and a priori models and information. Generally speak-

35 ing, the Kalman filter is a digital filter with time-varying gains that are optimally determined through a stochastic dynamical model of the motion. The overall goal is to minimize filter lag while providing sufficient smoothing of the motion data.

(12.1) 40

An adaptive, multi dynamic model is developed based upon the kinematic quality of the expected movement trajectory. The predictive kinematic model for the Kalman filter is depicted in matrix fonn utilizing a truncated 2nd order Taylor series expansion as below in Eqs. 14.1-2, ( -z, ) By = atan , , .

cos8x(x3cos82 + Y3 sm8,)

(12.2)

(

, x3sin81 sin8y ) Y3 - -c-O--;S8;-23----'-

(}z = atan " x3

(12.3)

COS 82 _ 1 =cos 82 from previous iteration (1204)

82 = atan(~) for 1st iteration (12.5)

These calculations are perfonned through iterative step processes which inherit angular approximations of the preceding steps until the final desired angular accuracy is achieved by assuming the conditions of Eqs. 12.4-5. Therefore the rotation 8z ' roll, is first approximated by Eq. 12.5; then the rotation 8x ' pitch, is approximated by Eq. 12.1; and then the final rotation 8y , yaw or tum, is approximated by Eq. 12.2. The next approximation of8z utilizes the previous value of8z in Eq. 12.3 and the similar steps are preferably repeated until the desired accuracy is achieved. The transcendental

45

[:L =[~ ~ttl +[:l (14.1)

[1°[: :' O~[[l[:[ (14.2)

50

The Kalman filter is now described for a single dimension, 55 although it is utilized for prediction and smoothing for all

position dimensions. The predictor stages consist of the calculation of the state and the error covariance projection equations. The state projector equation, Eq 15.1, utilizes a discrete time-sampled difference equation of r calculated from Eq.

60 15.2. In other words, the numerically derived velocity and acceleration components of motion are linearly combined with the previously a priori position to estimate the new position. The corrector stages consist of sequential computation of the gain, updated state estimate, and updated error

65 covariance equations. The a posteriori state estimate, Eq. 15.4, is based on a linear combination of the weighted measurement residual and the last state estimate.

JX-003.0029

A000064


US 7,492,268 B2 31

(15.1)

(15.2)

Fk Kk =---(Pk + Rk )

(15.3)

10

(15.4)

(15.5)

The new error covariance projector, Eq. 15.2, is it's previ- 15

ously computed value combined with the current process noise covariance, Qk' which is tuned by an example model derived from the measured motion dynamics shown in Eq. 16.1. The gain's measurement noise covariance, Rk , is defined as a small constant and based upon the actual static 20

timing variance empirically measured. The smaller this value the more confidence there exists in the systems' measurement capability.

32

1 2 0 -5 3 J.l Ck(J.l) = Gk 2 0 1 4 -3 J.l2

o -1 2 -1 11

Ck(J.l)=Gk

o 0 -1 1 J.l3

-O.SJ.l + J.l2 - O.SJ.l3

1 - 2.SJ.l2 + l.SJ.l3

O.SJ.l + 2J.l2 - I.SJ.l3

-O.SJ.l2 + O.SJ.l3

(17.1)

(17.2)

(17.3)

The fl value is normalized and represents the % value between the 2nd

3rd control points. To calculate the interpolated value between the 1 st and 2nd orthen-l th and nth control points, the value of first control point of the pair and the value of the last control point pair are doubly entered into the geometry matrix, respectively. The appropriate dflldt is determined by the desired rate of playback of movement trajectory.

25 To playback at the same rate as the recorded session, and assuming fairly constant velocity, a timestamp should also be saved at each control point registration so that the fl calculation is correctly scaled by the delta time interval. The n-length set of control points would be manually registered by the user

In the preferred embodiment, the product of the numerically-derived 1st and 2nd order derivatives of the measured position scaled by a frequency dependent gain provides a computationally practical adaptive dynamic process noise estimate model. The derivative product term increases Qk proportionally for higher velocity and acceleration components of motion, e.g., quick, abrupt directional changes, which effectively increases the gain and, therefore, means more confidence exists in the measurement rather than the estimate. This provides faithful, low-latency response to high-frequency motions. Conversely, the frequency scaling 35

term decreases the predictive "overshoot" characteristic of lower power, repetitive motion, e.g. slower, cyclic, ROM trajectories, which effectively decreases the gain and, therefore, means more confidence exists in the estimate rather than 40

the measurement. It should be appreciated this filter implementation provides superior tracking fidelity and comparable smoothing characteristics as compared to practical lengths of finite impulse response running-average filters and various low-orders infinite impulse response filters. It achieves 45

enough predictive response to compensate for the inherent TOF and computation latencies, while providing and comparable smoothing properties of other filter types.

30 pressing a switch or automatically post processed by a sorting method where a control point is registered at the tangents of the trajectory having sufficient magnitude and/or experience sign changes which indicates discontinuous or non-mono-tonic movement.

The major functional interfaces of the transponder unit preferably include the sensory interface, transducer interface, processor, and communication interface. The following descriptions of the transponder unit are based upon the dependence flow represented by FIG. 6.

The sensor interface refers to the collective support for the ultrasonic transmitter, heart rate receiver, and accelerometer circuits. The ultrasonic transmitter circuit is preferably gated by a pulse-width modulated (PWM) digital signal at nominally 0.8% duty cycle of the 40 kHz resonant frequency, e.g., a single 250 flS pulse every analysis period, by the processor

(16.1) 50

circuit. The radiated ultrasonic signal strength is controlled by gating a MOSFET transistor switch at a duty cycle which optimally energizes the transducer's series resonant tank circuit for sufficient duration. The resonant circuit's reactive components include an impedance matching inductor, the

(16.2)

In the preferred embodiment, a three dimensional (3D) piecewise cubic curve interpolates a movement trajectory for smoothing and reduced sample storage for greater memory efficiency. Preferably, four (4) sequential discrete control points of the n-length set of control points, the sample resolution dependent upon the desired movement granularity, and corresponding timestamp are needed to calculate in real-time the interpolated position between any pair of control points. A Catmull-Rom spline algorithm is the preferred method in that the path intersects the control points and would best approximate a movement that may have acute directional changes. The Catmull-Rom spline algorithm is defined by Eqs. 17.1-3, where the geometry matrix G k represents the matrix of three dimensional (3D) control points.

transducer's intrinsic capacitance, and a small damping resistive load. At resonance, a electrical damped sinusoidal with a potential up to -400 V pk_pk is developed across the transducer

55 to sufficiently drive it at acoustical power levels practical for the system's intended range of operation. Enabling a lower duty cycle control through means of a software algorithm monitoring the transponders range would effectively lower the transponders power consumption and radiate less ultra-

60 sonic acoustic energy for close range operation when signal saturation and clipping is undesirable. Conversely, a higher duty cycle control would radiate greater ultrasonic energy to compensate for less efficient, non-optimal acoustical coupling orientations of the transponder with respect to the

65 receiver constellation. Optionally, two additional transducers may be driven in unison or sequentially from a different transponder assembly to support measurement of absolute

JX-003.0030

A000065


US 7,492,268 B2 33

rotation about a single or multiple axes, or provide calculated positional redundancy for certain difficult line-of-sight applications.

34 clock also provides all the capture and control timing requirements for the other specified circuits. Multiple parallel I/O ports and dedicated asynchronous serial communication signals provide digital control for the circuits of the parallel/ serial I/O circuit.

In the preferred embodiment, the graphic LCD and touch screen circuit is the primary user input device for database management for an interactive transponder configuration. For example, it may be a 128x64 graphical liquid crystal display

The heart rate receiver circuit wirelessly receives a 5 kHz heart rate signal from a POLAR® transmitter belt. The transmitter, wom around the chest, electrically detects the heart beat and starts transmitting a pulse corresponding to each heart beat. The receiver captures the signal and generates a corresponding digital pulse which is received by the timing capture-control circuit of the processor interface. A software algorithm processes the signal with known time-based averaging and an adaptive window filter techniques to remove any extraneous artifact or corruption caused by interfering sources.

10 system (LCD) and associated 4-pin touch screen input device. A preferred LCD device is the 51553 manufactured by Optrex and the preferred touch screen device is the TSG-51 manufactured by Apollo Displays. LCD display information, configuration commands, and bitmaps images can be loaded

The accelerometer circuit consists of a low cost +/-1.5 g dual axis accelerometer that can measure both dynamic, e.g. vibration, and static, e.g. gravity or tilt, acceleration. If the accelerometer is oriented so both its axes are parallel to the earth's surface it can be used as a two axis tilt sensor with a roll and pitch axis.

15 through the software calculation engine via a parallel memory interface to emulate a graphical user interface. A touch screen input device is connected to a controller circuit to decode soft key presses at areas over the graphical object. Preferably, the key presses are registered, filtered, decoded,

20 and processed by the controller and then transferred to the software calculation engine via an interrupt driven asynchronous serial communication channel of the I/O interface. A preferred LCD controller is the UR7HCTS manufactured by Semtech.

The stimuli interface circuit provides the primary visual sensory interface preferably comprised of a linear array of five (5) bright, white light emitting diodes (LED) and associated drivers. The preferred LED device is a CMD87 manufactured by Chicago Miniature Lamp. These LEDs' intensity 25

is controlled by a white LED driver. The preferred white LED driver device is a MAX1570 manufactured by Maxim. The white LED driver provides a maximum 120 rnA constant current source to each LED for optimal uniform luminescence. The drive current can be proportionally regulated 30

through external pulse width modulation (PWM) means from the processor circuit to modulate its brightness level. Additionally, an electronic switch is connected in series to each LED drive to individually control its active state. By simultaneously controlling the PWM duty cycle and active state of 35

each LED, the light strobe can appear to smoothly migrate along the linear array in spite of its discontinuous operation.

Preferably, the stimuli interface circuit provides the primary aural stimulus by means of a 4 kHz piezo buzzer. The preferred device is SMT-3303-G manufactured by Projects 40

Unlimited. This electro-mechanical buzzer requires an external transistor drive circuit and digital control signal gated at a rate near its resonant frequency. The buzzer inputs are connected to and controlled by PWM means from the processor circuit to provide a gross volume adjustment which is depen- 45

dent upon the amplitude of the drive signal.

The timing capture-control circuit provides controlling means for the stimuli interface and portions of the sensor interface. The stimuli interface is preferably comprised of a five channel 16-bit timer PWM module with programmable interrupt control which provides 250 llS timing resolution to automatically modulate the circuits' drivers through variable duty cycle control.

In the preferred embodiment, the AID conversion circuit receives the output from the accelerometer circuit and consists of a two channell O-bit analog-to-digital converter used determine the rotational angle of roll and pitch in the accelerometer deviates from its horizontal plane orientation. This information is communicated to the signal processor via the radio link.

In the preferred embodiment, the radio link circuit is comprised of a wireless bi-directional communication interface (with a receiver and transmitter shown generally at 20 and 30) to (1) receive a synchronization signal for control of the transponders interoperability, (2) to transfer acquired local sensor data, including, but not limited to, accelerometer, heart rate, battery, user I/O status, to processor unit and (3) to provide means to configure its local database from command of processor unit. The preferred wireless communication link is based upon the AT86RF211, a highly integrated, lowpower FSK transceiver optimized for license-free ISM band

Additionally, the stimuli interface circuit provides the primary tactile stimulus by means of a vibrator motor. The driver for the vibrator motor enables a 120 rnA DC current source to excite the motor armature. The preferred driver device is the MAX1748 manufactured by Maxim. The rotational speed of the motor's armature is controlled by PWM means from the processor circuit.

50 operations from 400 MHz to 950 MHz. and manufactured by Atmel. Its key features are described above.

In the preferred embodiment, the switch I/O circuit uses a SPST push button switch for user input to control the system's operational states, start and stop program execution,

55 and function as feedback input to the program. A preferred device is the KSS231 SPST pushbutton switch manufactured by ITT Industries.

The processor circuit preferably receives input from the stimuli interface, sensor interface, and the communication interface and provides controlling signals therein. The preferred processor circuit is the MC9S08GB60 which is manufactured by Motorola Inc. It is a low-cost, high-performance 8-bit microcontroller device that integrates the specialized hardware circuits into one convenient device. The software 60

calculation engine circuit operates from an embedded 60 KB FLASH for program memory with in-circuit programmable capability and 4 KB RAM for data memory. The time base circuit is preferably comprised of an external, high-noise immunity, 4.0 MHz system clock, which multiplies this value 65

by the internal frequency-locked loop for a bus clock of 40.0 MHz and single instruction execution time of 25 llS. This

What is claimed is: 1. A system for tracking position of a user, comprising: a first communication device, adapted for being attached

to, or held by, the user, comprising: a transmitter for transmitting signals; a receiver for receiving signals; and an output device; and

a processing system, remote from the first communication device, for wirelessly receiving the signals transmitted by the transmitter, determining position information for

JX-003.0031

A000066


US 7,492,268 B2 35

the first communication device and sending data signals to the receiver to provide feedback or control data to the user;

wherein the first communication device receives and processes the received data signals and generates sensory stimuli for the user, based on the received data signals and delivered through the output device.

2. The system of claim 1, further comprising:

a second communication device, in electrical communica- 10

tion with the first communication device and in wireless communication with the processing system, and adapted for being attached to, or held by, the user;

wherein the processing system is remote from the second communication device and is adapted for detennining 15

the position of the second communication device relative to at least one of the first communication device and the processing system.

3. The system of claim 1, wherein:

the first communication device comprises a user input device adapted for communication with the processing system through the transmitter.


20

the user input device is adapted for calibrating the first 25

communication device to establish a reference position.

5. The system of claim 1, wherein: the processing system is adapted to determine acceleration

information of the first communication device.


the sensory stimuli are at least one of: aural, visual or tactile.


30

35 the processing system determines posItion infonnation

without interference from occluding objects.

8. A method for tracking position of a user, comprising the steps of:

providing a system according to claim 1;

establishing a wireless communication between the first communication device and the processing system;

40

exchanging data signals between the first communication device and the processing system, the first communication device sending position information data signals to 45

the processing system and the processing system sending feedback or control data signals to the first communication device; and

providing sensory stimuli to the user through the output 50

device, the sensory stimuli based upon the received feedback or control data signals.

36 9. The method of claim 8, wherein: the system further comprises a user input device adapted

for communication with the processing system through the transmitter; and

the step of exchanging data signals further comprises sending user input data signals to the processing system.

10. An apparatus for use in conjunction with a remote processing system for tracking position of a user, comprising:

a transmitter for transmitting position information signals to the remote processing system;

a receiver for receiving feedback or control data signals wirelessly from the remote processing system, the data signals derived from processed signals from the transmitter; and

an output unit for delivering sensory stimuli to the user, based upon the data signals.

11. The apparatus of claim 10, wherein: the data signals are derived by comparing the position

information transmitted to reference position infonnation.

12. The apparatus of claim 10, further comprising: a first and a second communication device, each adapted

for being attached to, or held by, the user; and each of the communication devices is in communicative

contact with the processing system directly or through the other communication device.

13. The apparatus of claim 12, further comprising: a user input device deployed on at least one of the commu

nication devices and adapted for communication with the processing system through the transmitter.

14. The apparatus of claim 10, further comprising: an interactive interface such that movement of the appara

tus controls the movement of an object in a computer generated virtual environment.

15. A system for tracking movement of a user, comprising: a first communication device, adapted for being attached

to, or held by, the user, comprising: a transmitter for transmitting signals; a receiver for receiving signals; and an output device; and

a processing system, remote from the first communication device, for wirelessly receiving the signals transmitted by the transmitter, detennining position information for the first communication device and sending data signals to the receiver to provide feedback or control data to the user;

wherein the first communication device receives and processes the received data signals and generates sensory stimuli for the user, based on the received data signals and delivered through the output device.

* * * * *

JX-003.0032

A000067


united states court of appealsonline.wsj.com/public/resources/documents/nintendo.pdf ·...

Documents