method, computer program and apparatus for … · the present invention relates to motion tracking....
TRANSCRIPT
METHOD, COMPUTER PROGRAM AND APPARATUS FOR DETERMINING
MOTION INFORMATION
FIELD OF THE INVENTION
The present invention relates to motion
tracking. More particularly, the present invention
relates to a method, a computer program and an
apparatus for determining motion information of an
object from a video signal.
BACKGROUND OF THE INVENTION
Motion capture is a term that describes the
process of recording motion and translating the motion
onto to a digital model. Presently there are available
different kinds of commercial products which are based
on various technologies, e.g. optical systems and non-
optical systems. Some of the optical systems are based
on a usage of special optical markers the movements of
which are recorded on a video. To reproduce movement
the video is analyzed using various techniques.
Examples of the non-optical systems include e.g.
various kinds of inertial sensors. When using
intertial sensors the measurement information has to
be transmitted to a receiving entity either in a wired
or wireless manner.
The existing motion capture systems have
several drawbacks. Often the systems, e.g. optical
systems are complex, require much processing power and
need special equipment to work. Furthermore, if non-
optical systems are used, there is a need for a
special receiver receiving the transmission from the
wired or wireless transmitter.
Based on the above there is an obvious need
for a solution that would mitigate and/or alleviate
the above drawbacks.
1
5
10
15
20
25
30
35
SUMMARY OF THE INVENTION
According to an aspect there is provided a
method for determining motion information of an object
from a video signal. The method comprises: receiving
the video signal, the video signal comprising a
plurality of frames representing motion; analysing the
video signal to determine at least one identifiable
object in the plurality of frames of the video signal;
and determining motion information of the object
orthogonal to the two dimensional planar motion of the
video signal by tracking changes in optical properties
of the at least one identifiable object in the
plurality of frames of the video signal.
According to another aspect of the invention
there is provided a computer program comprising
program code, which when executed on a processor
implement the method of any of claims 1 – 10.
According to another aspect of the invention
there is provided an apparatus for determining motion
information of an object from a video signal. The
apparatus comprises: a processor; means for receiving
the video signal; and a memory comprising a computer
program, which when executed by the processor is
configured to: receive the video signal, the video
signal comprising a plurality of frames representing
motion; analyse the video signal to determine at least
one identifiable object in the plurality of frames of
the video signal; and determine motion information of
the object orthogonal to the two dimensional planar
motion of the video signal by tracking changes in
optical properties of the at least one identifiable
object in the plurality of frames of the video signal.
According to yet another aspect of the
invention there is provided a system for determining
motion information of an object from a video signal.
The system comprises: a video camera; at least one
identifiable object attachable to a target; an
2
5
10
15
20
25
30
35
apparatus comprising a processor; means for receiving
the video signal; and a memory comprising a computer
program, which when executed by the processor is
configured to: receive the video signal, the video
signal comprising a plurality of frames representing
motion; analyse the video signal to determine at least
one identifiable object in the plurality of frames of
the video signal; and determine motion information of
the object orthogonal to the two dimensional planar
motion of the video signal by tracking changes in
optical properties of the at least one identifiable
object in the plurality of frames of the video signal.
The advantages of the invention relate to
simplicity. With the invention it is possible to track
three-dimensional motion by using only one video
camera. Furthermore, since the motion information is
coded into a video signal, the decoding can be done
regardless of the location.
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are included
to provide a further understanding of the invention
and constitute a part of this specification,
illustrate embodiments of the invention and together
with the description help to explain the principles of
the invention. In the drawings:
Figure 1 discloses a flow diagram of a method
according to one embodiment of the invention,
Figure 2 discloses an initial arrangement
according to one embodiment of the invention,
Figures 3A and 3B disclose an embodiment for
decoding three-dimentional movement from a video
signal,
Figures 4A and 4B disclose another embodiment
for decoding three-dimentional movement from a video
signal,
3
5
10
15
20
25
30
35
Figures 5A and 5B disclose another embodiment
for decoding three-dimentional movement from a video
signal,
Figure 6 discloses an apparatus according to
one embodiment of the invention, and
Figure 7 disclosed a system according to one
embodiment of the invention.
DETAILED DESCRIPTION OF THE INVENTION
Reference will now be made in detail to the
embodiments of the present invention, examples of
which are illustrated in the accompanying drawings.
Figure 1 discloses a block diagram of
illustrating one embodiment of the invention. At step
100, a video signal is received. The video signal
comprises a plurality of frames. The signal itself may
consist any suitable frame rate, e.g. 24fps or 30fps
or higher or even less. Furthermore, the pixel size of
a single frame depends on a camera used to record the
video signal. At step 102 the video signal is analysed
to determine at least one identifiable object in the
plurality of frames of the video signal. The
identifiable object refers e.g. to a particular shape,
colour or optical element which is identifiable from a
frame of the video signal. At step 104 motion
information of the object orthogonal to the two
dimensional planar motion of the video signal is
determined by tracking changes in optical properties
of the at least one identifiable object in the
plurality of frames of the video signal. In general
this means that motion information of the object
orthogonal to the two dimensional planar motion of the
video signal can be determined from a two-dimensional
video signal. This makes possible e.g. to capture and
determine movement information of the object in each
three dimension based on a video signal from a single
camera.
4
5
10
15
20
25
30
35
Figure 2 illustrates an initial arrangement
of the invention at hand. A video camera 24 is
recording movements 20 of at least one object. The
movements happen three-dimensionally, but the video
camera 24 captures only a two-dimensional motion plane
22 of the movements 24. Thus, the objective of the
invention is how to capture three-dimensional
movements from a video signal presenting only a two-
dimensional motion plane.
Figures 3A and 3B disclose an embodiment in
which motion information of an identifiable object 30
orthogonal to the two-dimentiosional planar motion
component of the object 30 is determined by tracking
changes in opticl properties of the objec 30t.
In the embodiment of Figures 3A and 3B, the
two-dimensional motion component of the object 30 is
determined from the video signal by using e.g. one or
more markers (not shown in Figures 3A and 3B). The
marker, e.g. a passive marker is coated e.g. with a
retroreflective material to reflect light back to a
camera. The two-dimensional motion can then be tracked
by tracking the position of the marker in the two-
dimentional planar video signal.
In Figures 3A and 3B represent consecutive
frames of the video signal, and motion information of
the object 30 orthogonal to the two-dimensional planar
motion component is calculated from changes in the
relative size of the at least one identifiable object
30 in the frames. In Figure 3A the object 30 is
represented with a black circle having position
coordinates x1, y1, z1. It is evident to a skilled
person that there might not be any fixed coordinate
system in use. A more important piece of information
is the difference in coordinates (positions) between
two consecutive frames. In Figure 3B the object 30 has
moved (x2, y2, z2) compared to the position in the
5
5
10
15
20
25
30
35
previous frame (Figure 2A). In this embodiment only
the object 30 moves and the camera stays in place.
In Figure 3A the relative size of the object
30 in the video signal has a fist value. In Figure 3B
the relative size of the object 30 is bigger. Since
the camera stays in place, the change in the relative
size of the identifiable object 30 represents movement
of the object to a direction orthogonal to the two-
dimensional planar motion component determined e.g.
from the earlier mentioned marker. In short, the
bigger the object 30 (the black circle in this case)
is in a data frame, the closer it is to the camera.
Furthermore, the accuracy of determining of
motion in the direction orthogonal to the two-
dimensional planar motion component may also depend on
the total pixel size and the frame rate of the camera.
For example, an inexpensive web camera may have a
frame rate of 30 frames per second (fps) and a total
pixel size of 640x480. It is evident that if the
object moves fast, the movement is captured more
accurately with a higher fps value. Furthermore, the
size of a single pixel in a 640x480 web camera is much
bigger than e.g. in a better three megapixel camera
(e.g. 2048x1536 pixels). Thus, the accuracy of the
motion information of the object 30 orthogonal to the
two-dimensional planar motion component is more
accurate when the total pixel size and the frame rate
of the camera increases.
Furthermore, it was mentioned above that a
marker is separate from the light source. In another
embodiment, the marker and the light source comprise a
single element, i.e. the light source act also as a
marker.
Figures 4A and 4B disclose another embodiment
of the invention at hand. In the embodiment of Figures
4A and 4B, motion of an object 40 in a direction
orthogonal to the two-dimensional planar motion
6
5
10
15
20
25
30
35
component based on the optical properties of the
object 40.
In the embodiment of Figures 4A and 4B, the
two-dimensional motion component of the object 40 is
determined from the video signal by using e.g. one or
more markers 42. The marker 42, e.g. a passive marker
is coated e.g. with a retroreflective material to
reflect light back to a camera. The two-dimensional
motion can then be tracked by tracking the position of
the marker 42 in the two-dimentional planar video
signal.
In the embodiment of Figures 4A and 4B, the
object 40 is a light source. The ligth source is e.g.
a light emitting diode (LED). Depth information, i.e.
the third motion component of the object 40 is
determined from the signals of the light source in the
video signal. The object 40 (light source) is attached
to a target. The target is e.g. a hand, first, foot or
any other target the movement of which is to be
tracked. As said above, the movement of the target is
coded in a predetermined manner into signals of the
light source 40 so that it is possible to determine
the last motion component needed from the video
signal. The coding is based on e.g. a usage of at
least one accelerometer. The accelerometer itself may
be based on any possible and applicable technology.
The measurements of the accelerometer are coded into
light signals so that it is possible to decode the
previously encoded motion information afterwards. The
coding is executed so that both the quantity and
direction can be decoded from the signals of the light
source 40. Furthermore, the intensity of light may
also be used in the coding procedure, e.g. to indicate
whether the motion is away or toward the camera.
In short, a video camera is used to record
movements of a target. The target is e.g. a body part
of a human being, e.g. a fist. The body part is
7
5
10
15
20
25
30
35
equipped with at least one accelerometer which
provides information about the moving body part. An
accelerometer tracks movements of the body part, e.g.
a fist. Measurements of the accelerometer are coded
into a light signal of at least one light source 40 in
a predetermined manner. It is evident that the target
may be equipped with multiple acceletometer in
different locations and thus also multiple light
sources 40 providing coded movement information of the
multiple accelerometers.
Since the video camera is used to record
movements of the target, it also tracks the light
source 40. In other words, the coded light signal of
the light source 40 appears on the video signal. The
video signal is then transmitted to a recipient via a
communication network, e.g. the Internet.
Alternatively, the video may be locally connected to a
receiving processing device. At the receiving end, a
data processing device, e.g. a computer is provided
with a software application which is arranged to
analyse the video signal to track two-dimensional
planar motion component of the object. The two-
dimensional planar motion component is determined e.g.
based on signals provided by the at least one passive
markers or by any other suitable technique. Several
techniques for tracking two-dimensional planar motion
of a target are known in the art, and therefore they
are not described here in more detail. In addition to
the two-dimensional planar motion component, the
software application determines motion in a direction
orthogonal to the two-dimensional planar motion
component from the signals of the light source, the
signals of the light source having been modulated
based on motion of the object, e.g. the fist already
mentioned above. Based on the two-dimensional planar
motion component and the modulated light signals, the
software application is able to determine movements
8
5
10
15
20
25
30
35
three-dimensionally. The decoding algorithm used in
the software application to decode light signals from
the video signal is apparent to a skilled person. In
short, the software application combines the two-
dimensional information and motion information from
the light source into three-dimensional motion
information.
Furthermore, it was mentioned above that a
marker is separate from the light source. In another
embodiment, the marker and the light source comprise a
single element, i.e. the light source act also as a
marker.
A clear advantage of the solution disclosed
in Figures 4A and 4B is that three-dimensional motion
tracking can be achieved with only one camera and that
there are no special requirements for the camera
itself. Of course, it is evident that more accurate
cameras may provide more accurate video signal. A
further advantege is that decoding can be done
practically anywhere without special equipment, only
the software application is needed to decode the video
signal.
Figures 5A and 5B disclose another embodiment
of the invention at hand. In the embodiment of Figures
5A and 5B, motion of an object in each three dimension
is determined based on the optical properties of the
object.
In the embodiment of Figures 5A and 5B, the
object is a set of three light sources: a red light
source 50, a green light source 52 and a blue light
source 54. Each of the light sources comprise e.g. a
light emitting diode (LED). The individual light
sources may be set so close to each other that they
appear as a single point of light to an ourside
observer. Instead of three led light sources it is
possible to use e.g. layered displays (light sources)
where the three RGB components are superimposed.
9
5
10
15
20
25
30
35
The light sources 50, 52, 54 are attached to
a target. The target is e.g. a hand, foot or any other
target the movement of which is to be tracked. The
movement of the target is coded in a predetermined
manner into signals of the light sources 50, 52, 54 so
that it is possible to determine the last motion
component needed from the video signal.
The coding is based on e.g. a usage of at
least one accelerometer. The object may comprise a
built-in three-axis accelerometer for tracking motion.
In another embodiment, the accelerometer is an
external component from the object. The accelerometer
itself may be based on any possible and applicable
technology. The measurements of the accelerometer are
coded into light signals so that it is possible to
decode the previously encoded motion information
afterwards. In this embodiment, each axis (X, Y, Z) is
coded into a corresponding light source. For example,
the X-axis is coded into the red light source 50, the
Y-axis is coded into the green light source 52 and the
Z-axis is coded into the blue light source 54. The
coding is executed so that both the quantity and
direction can be decoded from the signals of the light
sources 50, 52, 54. Furthermore, the intensity of
light may also be used in the coding procedure, e.g.
to indicate whether the motion is away or toward the
camera. In another embodiment, intensities of the
three light sources may be used in the coding
procedure. The coding procedure may then e.g. use the
proportions of the intensities of the light sources in
the coding procedure.
In short, a video camera is used to record
movements of a target. The target is e.g. a body part
of a human being, e.g. a fist. The body part is
equipped with a three-axis accelerometer which
provides information about the moving body part. An
accelerometer tracks movements of the body part, e.g.
10
5
10
15
20
25
30
35
a fist. Measurements of the accelerometer are coded
into light signals of three light sources 50, 52, 54
in a predetermined manner.
Since the video camera is used to record
movements of the target, it also tracks the light
sources 50, 52, 54. In other words, the coded light
signal of the light sources 50, 52, 54 appears on the
video signal. The video signal is then transmitted to
a recipient via a communication network, e.g. the
Internet. Alternatively, the video may be locally
connected to a receiving processing device. At the
receiving end, a data processing device, e.g. a
computer is provided with a software application which
is arranged to analyse the video signal to determine
motion in a first direction from signals of the red
light source, the signals of the red light source
having been modulated based on motion in the first
direction. Furthermore, the software application
determines motion in a second direction from signals
of the green light source, the signals of the green
light source having been modulated based on motion in
the second direction and motion in a third direction
from signals of the blue light source, the signals of
the blue light source having been modulated based on
motion in the third direction.
Based on the decoded three-dimensional motion
components, the software application is able to
determine movements three-dimensionally. The decoding
algorithm used in the software application to decode
light signals from the video signal itsels is apparent
to a skilled person. In short, the software
application decodes all the needed three-dimensional
motion information from the video signal, where the
video signal comprises RGB coded motion information.
A clear advantage of the solution disclosed
in Figures 5A and 5B is that three-dimensional motion
tracking can be achieved with only one camera and that
11
5
10
15
20
25
30
35
there are no special requirements for the camera
itself. Of course, it is evident that more accurate
cameras may provide more accurate video signal. A
further advantege is that decoding can be done
practically anywhere without special equipment, only
the software application is needed to decode the video
signal.
In one embodiment of Figures 5A and 5B,
decoding motion information from the three light
sources is performed a bit differently. Motion is
first determined from signals of a first light source.
The signals of the first light source have previously
been modulated as a function of absolute sum of
acceleration. The second light source acts as static
reference signal. Direction of the motion is then
determined from signals of a third light source. Thus,
the three-dimentional motion is determined based on
the decoded signals from the first, second and third
light sources.
Figure 6 discloses an apparatus 60 according
to one embodiment of the invention. The apparatus 60
comprises a processor 64 to which is connected a
memory 62 and a video signal receiver 64. The video
signal receiver receives the video signal e.g. via a
local connection or via a data interface connection,
e.g. via a local area network interface of the
apparatus. Although Figure 6 discloses only one memory
62 conneted to the processor 64, the apparatus may
also include other memories. The memory 62 comprises a
softare application which is configured to implement
the motion decoding step disclosed in the description
of Figures 3-5.
The apparatus 60 may also include a receiver
configured to receive a radio transmission. The
processor 64 is then configured to determine motion in
a direction orthogonal to the two-dimensional planar
motion component by simultaneously recording motion
12
5
10
15
20
25
30
35
data received by means of the radio transmission from
the object. The information received from the radio
transmission can be used to combine the planar motion
information from the video signal with the motion data
received by the radio transmission and to correct and
reference the motion information sent by the radio
transmission. The receiver may receive the same
information also via a wired interface. In the case
of radio transmission, the earlier disclosed
identifiable object may itself comprise a radio
transmitter tranmitting information provided by the
accelerometer. In another embodiment, the radio
transmitter may be a separate entity from the object.
Figure 7 discloses a system according to one
embodiment of the invention. The system comprises one
or more identifiable objects 70. The objects have been
described in more detail previously in the
description. The objects are a tracked with a video
camera 72. The video camera 70 has a connection to a
processing apparatus 74 decoding motion information
from the video signal. The connection may be wireless
or wired. Furthermore, The decoding process itself is
implemented e.g. with a software application running
in the processing apparatus. The processing apparatus
itself may be any possible device, e.g. a personal
computer, a laptop computer, a hand-held computer, a
mobile terminal, a mobile phone, a gaming console, a
personal digital assistant etc.
The solution disclosed in the invention can
be used e.g. in gaming applications and/or
apparatuses, various sports, computer vision
applications, telerehabilitation etc.
The exemplary embodiments can include, for
example, any suitable servers, workstations, PCs, lap-
top computers, personal digital assistants (PDAs), In-
ternet appliances, handheld devices, cellular tele-
phones, smart phones, wireless devices, other devices,
13
5
10
15
20
25
30
35
and the like, capable of performing the processes of
the exemplary embodiments. The devices and subsystems
of the exemplary embodiments can communicate with each
other using any suitable protocol and can be imple-
mented using one or more programmed computer systems
or devices.
One or more interface mechanisms can be used
with the exemplary embodiments, including, for exam-
ple, Internet access, telecommunications in any suit-
able form (e.g., voice, modem, and the like), wireless
communications media, and the like. For example, em-
ployed communications networks or links can include
one or more wireless communications networks, cellular
communications networks, 3G communications networks,
Public Switched Telephone Network (PSTNs), Packet Data
Networks (PDNs), the Internet, intranets, a combina-
tion thereof, and the like.
It is to be understood that the exemplary em-
bodiments are for exemplary purposes, as many varia-
tions of the specific hardware used to implement the
exemplary embodiments are possible, as will be appre-
ciated by those skilled in the hardware and/or soft-
ware art(s). For example, the functionality of one or
more of the components of the exemplary embodiments
can be implemented via one or more hardware and/or
software devices.
The exemplary embodiments can store informa-
tion relating to various processes described herein.
This information can be stored in one or more memo-
ries, such as a hard disk, optical disk, magneto-
optical disk, RAM, and the like. One or more databases
can store the information used to implement the exem-
plary embodiments of the present inventions. The data-
bases can be organized using data structures (e.g.,
records, tables, arrays, fields, graphs, trees, lists,
and the like) included in one or more memories or sto-
rage devices listed herein. The processes described
14
5
10
15
20
25
30
35
with respect to the exemplary embodiments can include
appropriate data structures for storing data collected
and/or generated by the processes of the devices and
subsystems of the exemplary embodiments in one or more
databases.
All or a portion of the exemplary embodiments
can be conveniently implemented using one or more gen-
eral purpose processors, microprocessors, digital sig-
nal processors, micro-controllers, and the like, pro-
grammed according to the teachings of the exemplary
embodiments of the present inventions, as will be ap-
preciated by those skilled in the computer and/or
software art(s). Appropriate software can be readily
prepared by programmers of ordinary skill based on the
teachings of the exemplary embodiments, as will be ap-
preciated by those skilled in the software art. In ad-
dition, the exemplary embodiments can be implemented
by the preparation of application-specific integrated
circuits or by interconnecting an appropriate network
of conventional component circuits, as will be appre-
ciated by those skilled in the electrical art(s).
Thus, the exemplary embodiments are not limited to any
specific combination of hardware and/or software.
Stored on any one or on a combination of com-
puter readable media, the exemplary embodiments of the
present inventions can include software for control-
ling the components of the exemplary embodiments, for
driving the components of the exemplary embodiments,
for enabling the components of the exemplary embodi-
ments to interact with a human user, and the like.
Such software can include, but is not limited to, de-
vice drivers, firmware, operating systems, development
tools, applications software, and the like. Such com-
puter readable media further can include the computer
program product of an embodiment of the present inven-
tions for performing all or a portion (if processing
is distributed) of the processing performed in imple-
15
5
10
15
20
25
30
35
menting the inventions. Computer code devices of the
exemplary embodiments of the present inventions can
include any suitable interpretable or executable code
mechanism, including but not limited to scripts, in-
terpretable programs, dynamic link libraries (DLLs),
Java classes and applets, complete executable pro-
grams, Common Object Request Broker Architecture
(CORBA) objects, and the like. Moreover, parts of the
processing of the exemplary embodiments of the present
inventions can be distributed for better performance,
reliability, cost, and the like.
As stated above, the components of the exem-
plary embodiments can include computer readable medium
or memories for holding instructions programmed ac-
cording to the teachings of the present inventions and
for holding data structures, tables, records, and/or
other data described herein. Computer readable medium
can include any suitable medium that participates in
providing instructions to a processor for execution.
Such a medium can take many forms, including but not
limited to, non-volatile media, volatile media, trans-
mission media, and the like. Non-volatile media can
include, for example, optical or magnetic disks, mag-
neto-optical disks, and the like. Volatile media can
include dynamic memories, and the like. Transmission
media can include coaxial cables, copper wire, fiber
optics, and the like. Transmission media also can take
the form of acoustic, optical, electromagnetic waves,
and the like, such as those generated during radio
frequency (RF) communications, infrared (IR) data com-
munications, and the like. Common forms of computer-
readable media can include, for example, a floppy
disk, a flexible disk, hard disk, magnetic tape, any
other suitable magnetic medium, a CD-ROM, CDR, CD-RW,
DVD, DVD-ROM, DVD±RW, DVD±R, any other suitable opti-
cal medium, punch cards, paper tape, optical mark
sheets, any other suitable physical medium with pat-
16
5
10
15
20
25
30
35
terns of holes or other optically recognizable indi-
cia, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other
suitable memory chip or cartridge, a carrier wave or
any other suitable medium from which a computer can
read.
While the present inventions have been de-
scribed in connection with a number of exemplary em-
bodiments, and implementations, the present inventions
are not so limited, but rather cover various modifica-
tions, and equivalent arrangements, which fall within
the purview of prospective claims.
17
5
10
CLAIMS:
1. A method for determining motion information
of an object from a video signal, the method
comprising:
receiving the video signal, the video signal
comprising a plurality of frames representing
motion;
analysing the video signal to determine at least one
identifiable object in the plurality of frames of
the video signal; and
determining motion information of the object
orthogonal to the two dimensional planar motion of
the video signal by tracking changes in optical
properties of the at least one identifiable object
in the plurality of frames of the video signal.
2. The method according to claim 1, further
comprising:
analysing the video signal to tracktwo-dimensional
planar motion component of the object,
and wherein the determining comprises:
determining relative size of the at least one
identifiable object in the frames of the video
signal; and
calculating motion information of the object
orthogonal to the two-dimensional planar motion
component from the changes in the relative size of
the at least one identifiable object in the frames
of the video signal.
3. The method according to claim 1, wherein the
at least one identifiable object comprises at least
one light source.
4. The method according to claim 3, wherein the
determining comprises:
analysing the video signal to track two-dimensional
planar motion component of the object,
wherein the identifiable object is a light source
and the determining comprises:
18
5
10
15
20
25
30
35
determining motion in a direction orthogonal to the
two-dimensional planar motion component from
signals of the light source, the signals of the
light source having been modulated based on motion
of the object.
5. The method according to claim 3, wherein the
at least one identifiable object comprises a red light
source, a green light source and a blue light source,
and the determining comprises:
determining motion in a first direction from signals
of the red light source, the signals of the red
light source having been modulated based on motion
of the object in the first direction;
determining motion in a second direction from
signals of the green light source, the signals of
the green light source having been modulated based
on motion of the object in the second direction;
and
determining motion in a third direction from signals
of the blue light source, the signals of the blue
light source having been modulated based on motion
of the object in the third direction.
6. The method according to any of claims 1 - 5,
wherein the at least one identifiable object comprises
three light sources, and wherein the determining
comprises:
determining motion from signals of a first light
source, the signals of the first light source
having been modulated as a function of absolute
sum of acceleration;
determining a static reference signal from signals
of a second light source;
determining direction of the motion from signals of
a third light source; and
determining the three-dimentional motion based on
the decoded signals from the first, second and
third light sources.
19
5
10
15
20
25
30
35
7. The method according to any of claims 1 - 6,
further comprising:
determining motion in a direction orthogonal to the
two-dimensional planar motion component by
simultaneously recording motion data received by
means of radio transmission from the object.
8. The method according to claim 7, further
comprising:
combining the planar motion information from the
video signal with the motion data received by the
radio transmission, and
correcting and referencing the motion information
sent by the radio transmission.
9. The method according to any of claims 1 - 6,
further comprising:
determining motion in a direction orthogonal to the
two-dimensional planar motion component by
simultaneously recording motion data received by
means of wire transmission from the tracked
object.
10. The method according to claim 9, further
comprising:
combining the planar motion information from the
video signal with the motion data received by the
wire transmission, and
correcting and referencing the motion information
sent by the wire transmission.
11. A computer program comprising program code,
which when executed on a processor implement the
method of any of claims 1 – 10.
12. The computer program according to claim 8,
wherein the computer program is embodied on a
computer-readable medium.
13. An apparatus for determining motion
information of an object from a video signal, the
apparatus comprising:
a processor;
20
5
10
15
20
25
30
35
means for receiving the video signal; and
a memory comprising a computer program, which when
executed by the processor is configured to:
receive the video signal, the video signal
comprising a plurality of frames representing
motion;
analyse the video signal to determine at least one
identifiable object in the plurality of frames of
the video signal; and
determine motion information of the object
orthogonal to the two dimensional planar motion of
the video signal by tracking changes in optical
properties of the at least one identifiable object
in the plurality of frames of the video signal.
14. The apparatus according to claim 13, wherein
the processor is configured to:
analyse the video signal to tracktwo-dimensional
planar motion component of the object;
determine relative size of the at least one
identifiable object in the frames of the video
signal;
calculate motion information of the object
orthogonal to the two-dimensional planar motion
component from the changes in the relative size of
the at least one identifiable object in the frames
of the video signal.
15. The apparatus according to claim 13, wherein
the at least one identifiable object comprises at
least one light source.
16. The apparatus according to claim 15, wherein
the processor is configured to:
analyse the video signal to track two-dimensional
planar motion component of the object; and
determine motion in a direction orthogonal to the
two-dimensional planar motion component from
signals of the light source, the signals of the
21
5
10
15
20
25
30
35
light source having been modulated based on motion
of the object;
wherein the identifiable object is a light source
and the determining comprises:
17. The apparatus according to claim 15, wherein
the at least one identifiable object comprises a red
light source, a green light source and a blue light
source, and wherein the processor is configured to:
determine motion in a first direction from signals
of the red light source, the signals of the red
light source having been modulated based on motion
of the object in the first direction;
determine motion in a second direction from signals
of the green light source, the signals of the
green light source having been modulated based on
motion of the object in the second direction; and
determine motion in a third direction from signals
of the blue light source, the signals of the blue
light source having been modulated based on motion
of the object in the third direction.
18. The apparatus according to claim 13 - 17,
wherein the at least one identifiable object comprises
three light sources, and wherein the processor is
configured to:
determine motion from signals of a first light
source, the signals of the first light source
having been modulated as a function of absolute
sum of acceleration;
determine a static reference signal from signals of
a second light source;
determine direction of the motion from signals of a
third light source; and
determine the three-dimentional motion based on the
decoded signals from the first, second and third
light sources.
19. The apparatus according to any of claims 13 -
18, further comprising:
22
5
10
15
20
25
30
35
a receiver configured to receive a radio
transmission; wherein the processor is configured
to:
determine motion in a direction orthogonal to the
two-dimensional planar motion component by
simultaneously recording motion data received by
means of the radio transmission from the object.
20. The apparatus according to claim 19, wherein
the processor is configured to:
combine the planar motion information from the video
signal with the motion data received by the radio
transmission; and
correct and reference the motion information sent by
the radio transmission.
21. The apparatus according to any of claims 13 -
18, further comprising:
a receiver configured to receive a wire
transmission;wherein the processor is configured
to:
determine motion in a direction orthogonal to the
two-dimensional planar motion component by
simultaneously recording motion data received by
means of wire transmission from the tracked
object.
22. The apparatus according to claim 21, wherein
the processor is configured to:
combine the planar motion information from the video
signal with the motion data received by the wire
transmission; and
correct and reference the motion information sent by
the wire transmission.
23. A system for determining motion information
of an object from a video signal, the system
comprising:
a video camera;
at least one identifiable object attachable to a
target;
23
5
10
15
20
25
30
35
an apparatus comprising a processor; means for
receiving the video signal; and a memory
comprising a computer program, which when executed
by the processor is configured to:
receive the video signal, the video signal
comprising a plurality of frames representing
motion;
analyse the video signal to determine at least one
identifiable object in the plurality of frames of
the video signal; and
determine motion information of the object
orthogonal to the two dimensional planar motion of
the video signal by tracking changes in optical
properties of the at least one identifiable object
in the plurality of frames of the video signal.
24
5
10
15
ABSTRACT
The invention provides a method
for determining motion information of an
object from a video signal. The method
comprises: receiving the video signal,
the video signal comprising a plurality
of frames representing motion; analysing
the video signal to determine at least
one identifiable object in the plurality
of frames of the video signal; and
determining motion information of the
object orthogonal to the two dimensional
planar motion of the video signal by
tracking changes in optical properties
of the at least one identifiable object
in the plurality of frames of the video
signal.
(FIG. 1)
5
10
15
V I D E O S I G N A L
R E C E I V I N G T H E V I D E O S I G N A L , T H E V I D E OS I G N A L C O M P R I S I N G A P L U R A L I T Y O F F R A M E S
A N A L Y S I N G T H E V I D E O S I G N A L T O D E T E R M I N E A T L E A S T O N E I D E N T I F I A B L E O B J E C T
D E T E R M I N I N G M O T I O N I N F O R M A T I O N O F T H E O B J E C T O R T H O G O N A L T O T H E T W O D I M E N S I O N A LP L A N A R M O T I O N O F T H E V I D E O S I G N A L B Y T R A C K I N G C H A N G E S I N O P T I C A L P R O P E R T I E S O FT H E A T L E A S T O N E I D E N T I F I A B L E O B J E C T
1 0 0
1 0 2
1 0 4
F I G . 1
F I G . 2
2 0
2 2
2 4
F I G . 3 A
x , y , z1 1 1
F I G . 3 B
x , y , z2 2 2
3 0
3 0
F I G . 4 A
x , y1 1
F I G . 4 B
z 1
x , y2 2
z 2
4 04 2
4 04 2
F I G . 5 A
x , y1 1
F I G . 5 B
RBG
, z 1
x , y2 2
RBG
, z 2
5 0
5 2 5 4
5 0
5 2 5 4
V I D E OS I G N A LR E C E I V E R
P R O C E S S O R
M E M O R Y 6 2
6 4
6 6
6 0
F I G . 6
P R O C E S S I N GA P P A R A T U S
V I D E OC A M E R A
I D E N T I F I A B L EO B J E C T ( S )
7 0 7 2
7 4
F I G . 7