Department of Electrical Engineering
Department of Electrical and Computer Engineering
True 3D Image Manipulator
ELEC/CENG 499 – Design Project
Final Report
Dept. Electrical and Computer Engineering
University of Victoria
All rights reserved. This report may not be reproduced in whole or in part, by photocopy or other
means, without the permission of the author.
Prepared By: Fangzheng Song
Nelson Lee
Candace Chen
Yang Liu
Tristan Sturgess
V00700925
V00688530
V00723040
V00735027
V00684498
Supervisors: Dr. Kin Fun Li
Group No.: 16
Due Date: April 4, 2014
ELEC/CENG 499 – Final Report
ELEC/CENG 499 – Final Report
Abstract
The use of LEDs to create images has been widely used in modern society. Nowadays, large flat
screen advertisement can be seen everywhere, in terms of public information display and
business advertising, playing a crucial role of disseminating messages. However, ordinary LED
flat screen has limitation on dealing with 3D images due to the planar structure. With the
development of technology and the needs of advertising market, people can expect an application
of displaying 3D image so advertising companies can bring about more creative ad design. In
this case, the LED cube can easily meet the criteria of a 3D display. Compared with a LED
screen, the LED cube has a lower resolution, but its true 3D image creation ability is not
something a normal LED flat screen can achieve.
For the ELEC/CENG 499 project design course, our team will implement a continuation of our
ELEC 399 project. The 3D image manipulator is a system that displays a 3D image which can be
manipulated by intuitive hand gestures. Due to time and budget limitations, a very high
resolution LED cube would be very difficult to construct. However, this project will focus on
providing a proof of concept for the True 3D image manipulation technology. During the
demonstration day, many basic manipulations were shown to be done through an 8x8x8 LED
Cube. Other than the LED Cube the Leap Motion and Arduino are two of the other major
components in this project. The Leap motion is responsible for detecting hand motion and the
Arduino is responsible for transmitting the control signals to the LED cube. Overall, this report
will explain each part of the project, including project research, implementation plan, project
milestones, alternatives and detail project design. The team hopes readers could gain a deep
understanding on this true 3D image manipulator project and raise interest on true 3D image
technology.
ELEC/CENG 499 – Final Report
TABLE OF CONTENTS
1 Introduction ............................................................................................................................. 1
1.1 TIM-3D Team Members ................................................................................................. 2
1.2 Supervisor ....................................................................................................................... 2
2 Project Goals ........................................................................................................................... 2
2.1 3D Image Manipulator .................................................................................................... 3
2.2 3D Game ......................................................................................................................... 3
3 Project Research...................................................................................................................... 4
3.1 LED Cube Research ........................................................................................................ 4
3.2 Alternative 3D Displays .................................................................................................. 5
3.3 Arduino Research............................................................................................................ 5
3.4 Leap Motion Research .................................................................................................... 6
4 Early Project Plan Description ................................................................................................ 7
4.1 Project Implementation Strategy..................................................................................... 7
4.2 Milestones ....................................................................................................................... 8
4.3 Project Cost List .............................................................................................................. 9
4.4 Design Alternatives Comparison (Pros and Cons) ....................................................... 10
4.4.1 Cube – UI Connector – Java – Leap Motion ............................................................ 11
4.4.2 Cube – Arduino – Processing – Leap Motion ........................................................... 11
4.4.3 Cube – Arduino – Java – Leap Motion ..................................................................... 11
5 Project Detail Designs........................................................................................................... 12
5.1 LED Cube Initialization ................................................................................................ 12
5.2 Image Creation on Arduino .......................................................................................... 14
5.3 Arduino Interface .......................................................................................................... 17
5.4 Image Management ....................................................................................................... 18
5.5 Leap Motion Detection ................................................................................................. 18
5.6 3D Game Design ........................................................................................................... 18
5.6.1 Concepts of Pong 3D ................................................................................................ 19
5.6.2 Pong 3D Program ...................................................................................................... 20
6 Workload Distribution & Achievements .............................................................................. 21
ELEC/CENG 499 – Final Report
7 Website Design ..................................................................................................................... 22
8 Project Discussion ................................................................................................................. 23
9 Conclusion ............................................................................................................................ 24
10 APPENDIX A (Java Program) ............................................................................................. 25
11 APPENDIX B (Game Code) ................................................................................................ 15
12 APPENDIX C (Cube Construction) ....................................................................................... 8
13 REFFERNCES ........................................................................................................................ 5
Page : 1 of 25
ELEC/CENG 499 – Final Report
1 INTRODUCTION
True 3D effect has always been the ultimate goal of display technology. With the growing
popularity of pseudo-3D screens that can be found in movie theatres, TV’s, and mobile devices
there is growing pressure to make displays that are truly 3 dimensional. It is necessary to point
out that there are still some significant problems with this pseudo-3D technology. For example,
when the angle is changed between a person's eyes and the pseudo-3D screen the three
dimensional image will disappear. A truly 3D image will not be constrained by the same issues.
As the technology for 3D displays improve a new window of opportunity for 3D manipulation is
also emerging in the market.
It is our hope that we can create a cheap and effective way of initial prototyping that does not
require the user to spend lots of money on a solid prototype. Our solution to this problem is to
create a 3D display that will present the model and allows the user to interact with it using simple
hand gestures. We would also like to showcase the potential of this 3D image manipulator
project by rebooting some of the classic games such as Pong, Snakes, and/or Space Invaders, but
adding a third dimension.
The idea for this project came from the movie Iron Man 3, where Tony Stark was able to create a
3D projection of his suit and interact with that model. By using a similar concept, we propose to
create a True 3D Image Manipulator (TIM-3D) that can be used to manipulate a model by using
simple hand gestures. This would be done by displaying the model on an 8x8x8 LED Cube that
would be controlled by hand gestures detected with a Leap Motion. The LED cube will be
constructed by using 512 LED’s that are positioned in an 8x8x8 grid. This LED cube then
communicates to a computer that is programmed with a 3D image. By using the Leap Motion,
which is able to detect motion in its spatial grid, we will be able to manipulate the 3D image by
using intuitive hand gestures.
Page : 2 of 25
ELEC/CENG 499 – Final Report
1.1 TIM-3D Team Members
Nelson Lee - is a 4th year Electrical Engineering student taking a Biomedical option at the
University of Victoria. Nelson has done many coop’s from a number of companies such as the
Genome Science Center and Power Grid Specialist Corporation. Using his past experiences
Nelson has contributed his unique programing and organization skills to this project.
Fangzhen Song - is a 4rd year electrical engineering student. He is interested in electrical power
systems. He believes that the 3D image manipulator will be widely used for business
presentations, teaching, and advertisements. Working at AIM and BC Hydro provides him a
solid experience on project management and programming skills. This experience is very helpful
for his involvement in the design of the project. In his spare time, he enjoys watching soccer and
playing DOTA.
Wayne Liu - is a 4th year Computer Engineering student who has done two co-op terms as a
system administrator assistant. Wayne is good at system network, research, debug and analysis.
Wayne also creates the contribute ideas on the project.
Tristan Sturgess - is a 4th year Electrical Engineering student specializing in digital signal
processing. He has done a co-op as a software developer and a co-op developing VOIP
components for routers. He brought his programming skills and experience to the project.
Candace Chen - is a 4th year Electrical Engineering student who has done one co-op as a junior
process engineer at a research, development, and manufacturing establishment “Silian Sapphire
Corp”. Candace used her research and design specialties in contribution to the completion of
TIM-3D.
1.2 Supervisor
Dr. Kin Fun Li - the TIM-3D would like to thank Kin Fun Li, the Director of Computer
Engineering Program, for his supervision and patient.
2 PROJECT GOALS
It is our hope that we can create cheap and cost effective way of initial prototyping that does not
require the user to spend lots of money on a solid prototype. Our solution to this problem is to
create a 3D display that will present the model and allows the user to interact with it by using
Page : 3 of 25
ELEC/CENG 499 – Final Report
simple and intuitive hand gestures. We would also like to showcase the potential of this 3D
image manipulator by rebooting some classic games such as Pong, Snakes, and/or Space
Invaders, but adding a third dimension.
2.1 3D Image Manipulator
Our primary goal for the TIM-3D project is to manipulate a 3D image that is displayed in an
LED cube. This would be done by using an 8x8x8 LED Cube that would interface with a
computer that uses a product called Leap Motion. Using the Leap Motion, which is able to detect
motion in its spatial grid, we will be able to manipulate the 3D image using intuitive hand
gestures. This will enable the user to examine a model before they have it prototyped.
For this goal there are a large number of tasks that would need to be completed. The major
components of this goal will require the team to interface the LED cube to an Arduino
microcontroller, the Arduino microcontroller to a computer, and the computer to Leap Motion.
Once all of these connections have been made care will have to be taken to design the methods
of control and display of the model.
2.2 3D Game
Our secondary goal will be to make a 3D game that can be played on LED cube and controlled
by the Leap Motion hardware. Due to the limitations of the LED Cube and time constraints, it
will be very unlikely to produce a commercially viable product able to display high resolution
models. The 3D image manipulator is still a viable goal; however, the hardware required for high
resolution models is beyond our budget. Therefore, our goal of making some 3D games will
allow us to create a fun way of showcasing the potential of the 3D manipulation on the LED
cube. Since many old retro games use low resolution art already we would like to create one or
two games, such as Pong, Space Invaders, or Snake in a three dimensional grid.
Page : 4 of 25
ELEC/CENG 499 – Final Report
3 PROJECT RESEARCH
3.1 LED Cube Research
According to 399 project design, the initial plan is to build an 8x8x8 LED cube. However, when
the team started working on the cube, every team members realized that the workload of making
a cube was higher than anticipated. The detailed cube construction plans are explained in the 399
project report which has been included in Appendix B. It should be noted that the cube was
purchased instead of built by the team so that more time could be spent on the interface
development.
There are many LED cube products available in the market, especially in China. Multi-color
cubes also can be found in online stores. A multi-color cube has a similar size and structure as a
regular LED cube, but it uses multi-color LEDs that can display a full range of colors. On the
other hand, a multi-color LED cube is much more expensive than a single color one. Due to the
limited budget, TIM-3D still chooses a single color cube as the main 3D-image display. A
multicolor LED is also larger than a single color LED so the LEDs in the front of a cube
sometimes cover the LEDs behind them, which could affect the view of a 3D image. The figure
below is a photo of a multi-color cube; it’s fancy, but not feasible for this project.
Figure 3.1.1 – Multi-color LED Cube
Page : 5 of 25
ELEC/CENG 499 – Final Report
3.2 Alternative 3D Displays
We had a few options to choose from when deciding on a method of display. The two main
preferences were the 3D LED cube and a 3D image-reflecting prism. The cube provided a true
3D image representation, but places a limitation on the resolution based on the number of LED’s
used. A much sharper image could be produced within a prism using four reflective surfaces to
project a 3D image display. With a cleaner and a better resolution, we were hesitant in choosing
our method of display. In the end the LED cube was chosen for its lower cost and simpler
interface.
3.3 Arduino Research
The easiest way to send commands to the cube is using a microcontroller to send commands with
the I2C protocol. The microcontroller can then be used to translate commands from the
computer’s USB port to commands for the cube. However, not all types of micro-controller are
able to satisfy the LED cube. There are three major considerations when choosing a proper
microcontroller: compatibility with the LED cube, an open-source development platform, and
expansibility.
Figure 3.3.1 – Arduino Microcontroller
Page : 6 of 25
ELEC/CENG 499 – Final Report
The first requirement is the micro-controller support the I2C protocol to control the cube. The
cube purchased for the project has a mini-USB port, allowing users to design their own 3D
default animation through this port. Therefore, it is required that the microcontroller can get the
control of the cube through the mini-USB port so that the CPU keep sending commands to
manipulate the 3D image instantaneously. Another important requirement of the microcontroller
is programming. It is our hope that the micro-controller has its individual development platform.
Such a platform can make the programming work a lot more efficient. Open source platform
allows users research and edit their code and libraries a lot easier. It also provides an opportunity
of communicating with other developers, which is quite helpful when the programming work
encounter difficulties. Decent expansibility does not only stands for available open-source
libraries to simplify the code, but also illustrate the chip is able to interface with some other
graphic design software. The main reason of this feature so important is that the graphic
manipulation design work would be done easier by using professional design programs.
Based on the criteria shown above, Arduino is one of the most famous microcontrollers available.
The first advantage of the Arduino is that it has a built in bootloader which eliminates the need
for an expensive chip programmer. This feature makes the Arduino very convenient to setup and
program. Another important aspect of the Arduino is the CPU board can be connected to a
variety of interchangeable add-on modules. In this project, the LED cube is actually an add-on
module since the cube supports the I2C serial protocol, allowing the cube to receive data and
timer information through a single USB line from the Arduino.
The Arduino comes with a built in USB to serial converter and natively supports serial
communication. The 3D image manipulator design can be achieved using any PC software that
supports serial communication. These options will be explained in chapter 4.
3.4 Leap Motion Research
Leap Motion is a hand motion detector and a data collection device which sends hand motion
and hand position information to the computer. Based on commands from the Leap Motion the
Page : 7 of 25
ELEC/CENG 499 – Final Report
computer makes changes to the 3D image on the LED cube. Hand gestures such as swiping
forward, backward, up, down, left or right translate the image shown on the LED cube and
rotating the hand causes the image to rotate in kind. Unlike the Microsoft Kinect, which focuses
on body movements and actions, the Leap Motion focuses only on hand gestures which are all
that is needed for this project. In addition, the cost of Leap Motion is 65% lower than the
Microsoft Kinect. Another significant advantage of the Leap Motion is the free and open source
developer library it comes with which greatly simplifies the interface implementation.
4 EARLY PROJECT PLAN DESCRIPTION
4.1 Project Implementation Strategy
The project was implemented in pursuance of the ELEC 499 course schedule, ELEC 499 course
information and UVIC ECE Department Lab Safety Regulations, and abides by general
university policy and standards of professional behavior. The project cost will be included in this
report and will be reported to the project supervisor. The project strategy is composed of the
following components and sub-components, and their numerical order associated with time.
Component -1: Project Team Selection and Organizing
Sub-Component 1.1: Team meeting
Sub-Component 1.2: Workload Distribution
Component-2: Project Preparation
Sub-Component 2.1: Project equipment purchase
Sub-Component 2.2: Equipment assembly and testing
Sub-Component 2.3: Project research
Component-3: Project Design and Testing
Sub-Component 3.1: Design Alternatives
Sub-Component 3.2: LED Cube Interface
Sub-Component 3.3: Leap Motion Interface
Sub-Component 3.4: Game Design
Page : 8 of 25
ELEC/CENG 499 – Final Report
Sub-Component 3.5: Final Testing
Component-4: Other works and Documentation
Sub-Component 4.1: Monthly work-logs and milestones
Sub-Component 4.2: Presentation and Pamphlet design
Sub-Component 4.3: Final Report
From above, it is clearly seen that the entire project consists of 4 components, which are
according to the four milestones in the following project timeline figure. The first three
components include the major works required for this project. The design team spent
approximately one month for each of three components due to the three month duration of 499.
The last component was started on March 21 and was completed and will be submitted on April
4, 2014.
The third component in this implementation plan is the most important and involved the most
man-hours. Based on this component, every team member got their own assignments to do. Most
of the project work was programming which lead to each individual focusing on specific parts of
the project rather than working together to get one part done at a time. This working style was
decided on in the first team meeting. More importantly, the project plan was intended to
encourage everyone contribute to the project and avoid unbalanced workload distribution.
4.2 Milestones
The graphic below visualises the core milestones of the project and when they were
scheduled to take place.
Figure 4.2.1 – TIM-3D Milestones
Page : 9 of 25
ELEC/CENG 499 – Final Report
Comparing with the milestones and work logs submitted to the project supervisor, there are some
places with slight differences. The monthly milestones show the milestones and workloads for
each team member due to the working style the team decided on the reason actually is associated
with the team working style. Since each group member was doing their own parts for long
periods of time, it was more efficient to record the milestones for each person than it would have
been to record milestones for the whole team. Moreover, during the team meetings, every group
members could quickly know what other people did for the project.
4.3 Project Cost List
The table below shows the detail information of the equipment and material used and purchased in this 3D image manipulator design.
Table 4.3.1: Project Budget
Resources Required Prices
Leap-Motion(Donated by Team Member) $80.00
First LED Cube Kit (purchased during ELEC 399):
1. 3MM blue LED x550
2. 10MM LED x4
3. Mini USB slot x1
4. USB cable x1
5. DC power slot x1
6. 1 base circuit board
7. 2.54MM pin slot x1
8. 1k ohm resister x2
9. 10k ohm resister x2
10. TM1818 chip x4
11. 74HC138 chip x1
12. 2MM pin slot x1
13. 10UF capacitance x1
14. 0.1UF capacitance x1
15. 10 ohm resister x1
16. 4953 chip x4
$120.00
Second LED Cube kit (purchased during ELEC 399) 1. programmable circuit board
$120.00
Page : 10 of 25
ELEC/CENG 499 – Final Report
2. LED cube case
3. 8 8x8 LED slices
4. cube base
Arduino $30.00
USB to serial converter $20.00
USB to TTL converter $25.00
Breadboard and jumpers $15.00
Total: $410.00
4.4 Design Alternatives Comparison (Pros and Cons)
The previous section explained the details of the project implementation plan and schedule as
well as the project cost. In this alternative analysis we are examining the methods of
implementation that were considered and why each was or was not used. After completing the
project research it was decided that the Arduino and Java code method would be the best for the
TIM-3D project. Two alternative implementations were also considered for this project. The
following table shows advantages and disadvantages for each of these three implementations.
Table 4.4.1: Advantages and Disadvantages of Implementations
Implementation Advantages Disadvantages
Cube-USB to TTL
Converter – Java - Leap
Motion
-Cheapest Design
-Eliminates need to
program Arduino
-Cannot implement I2C
protocol on USB to TTL
connector
Cube-Arduino-Processing-
Leap Motion
-Fewer interfacing issues
-Better graphic design
-Open source library
-Require knowledge of
Arduino
-Require knowledge of
Processing
Cube – Arduino – Java –
Leap Motion
-Fewer interfacing issue
-Team familiar with Java
-Open source library
-Delay Problem
Page : 11 of 25
ELEC/CENG 499 – Final Report
4.4.1 Cube – UI Connector – Java – Leap Motion
Ideally the cube could be controlled directly from the computer. This would eliminate the
Arduino and the problems that come with it such as increased latency and complexity. It was
thought that a USB to TTL converter would be able to relay commands from a Java program to
the cube. The investigation into this alternative revealed that the USB to TTL converter could not
implement the I2C protocol required to control the cube. The lack of I2C protocol support made
this alternative impractical.
4.4.2 Cube – Arduino – Processing – Leap Motion
This alternative is very similar to the design that was chosen. The only difference in this
alternative is that the Processing program environment is used for 3D image manipulation
instead of using Java code. One of the nice things of developing 3D images through the
Processing environment is that the Processing code interfaces with the Arduino perfectly. There
are also several Leap Motion libraries available. Additionally, the Processing IDE is designed for
media art, electronic arts,c and visual design. This functionality applies directly to the
requirements of the TIM-3D project.
The Processing programming environment has the significant limitation that none of the team
members have solid experience with it. This situation results in the team having to spend time
learning the new programming environment which would be difficult in this 3-month-long
project. Team members Song and Wayne attempted to design and code 3D image manipulation
using the Processing IDE. Even though the final product does not use this alternative both Song
and Wayne believe that they could make this alternative work if given more time.
4.4.3 Cube – Arduino – Java – Leap Motion
This design alternative is the one selected for implementation. From the table above, it is
apparent that this design satisfies the most design criteria. The Arduino is able to control the
LED Cube and the Java programming environment allows us to establish a 3D coordinate system
and also design the 3D pong game. Comparing with UI connector method, this design reduces
the work on making up a solution of transmitting LED data to control the Cube. The Arduino can
Page : 12 of 25
ELEC/CENG 499 – Final Report
handle this part of work using open source libraries. Everyone on the team has also used Java
before, so it brings an easier developing environment for this project.
5 PROJECT DETAIL DESIGNS
As mentioned before in the main design milestones the chosen implementation is based on a Java
program and an Arduino. This Java program has the function of translating the hand position
data from the Leap Motion into commands and creating the 3D images accordingly. Then the
Arduino board is used to receive the image data and send it to the cube. The block diagram
below shows the entire system. Next, this report will concretely explain the design and function
of each block and how they interact with each other.
5.1 LED Cube Initialization
The 8x8x8 LED Cube used in this project is one of the products from Doyoung Studio. It mainly
includes two black mirror surface PCBs, 512 LEDs with pins, one microcontroller chip
(STC12C5A60S2), one mini USB port and a plastic cover. Overall, the cube is simple, cheap,
and reliable. The pictures below shows all the items included in the cube and the assembled one.
Figure 5.1.1 – LED Cube Components
Page : 13 of 25
ELEC/CENG 499 – Final Report
Figure 5.1.2 – Completed LED Cube
The cube assembling process didn’t cost a lot of time to team, but it played a crucial role in the
project. As the main display in 3D image manipulator, all the 3D effects and performance were
presented though this Cube. Therefore, in order to design and improve the performance, the first
step is to discover how the cube works and how to control the cube. Actually, Doyoung Studio
designs this cube to be able to work in two different modes: default mode and user mode. When
the cube is set up in default mode, it will keep displaying the animation which is saved in its
memory. In this mode, the cube won’t be interrupted unless the mode switch button is pressed.
The user mode requires the user to send 3D images to the cube; otherwise the cube won’t display
anything. Obviously the user mode is the mode chosen to be used in the project. By using this
mode, the LED cube can be used to display any kind of 3D image, such as rotation and
translation, by interfacing with the connected Arduino microcontroller.
The communication protocol of the LED cube is based on the I2C serial protocol. I2C uses only
two bidirectional open-drain lines, Serial Data Line (SDA) and Serial Clock Line (SCL). The
voltages used here are 5V. The I2C reference designed to use two bytes to finish one command.
One byte is address and the other byte is for data. The graph below is the I2C serial bus timing
diagram.
Page : 14 of 25
ELEC/CENG 499 – Final Report
Figure 5.1.3 – I2C Serial Bus Timing Diagram
More importantly, the LED cube provides the instruction sets for the user mode development.
There are three main instructions that were used in the project: select LED, lighten LED and
clean cube. The value of each instruction is in hexadecimal. For example, 0xC000 is the
instruction to clean the cube according to the instruction table below.
Figure 5.1.4 – Instruction Set for LED Cube
5.2 Image Creation on Arduino
If the entire project is viewed as a human body, the Arduino microcontroller would be the
nervous system. Similar to the nervous system, the Arduino plays the role of information
transmitter. The circuit connection between the Arduino and the LED cube is simple. Since the
Arduino fully support I2C serial bus, one single mini USB line enough for the connection to LED
cube. On the Arduino side, the USB line has to be modified in order to connect to Arduino.
Page : 15 of 25
ELEC/CENG 499 – Final Report
Basically, there are four slender lines inside an USB line: SDA line, SCL line, power line and
ground line. Depending on the Arduino program code, port 2 and port 3 are separately assigned
to SDA line and SCL line. The ground line of course connects to the grounding port on the
Arduino. The finished connection is shown below. The yellow line is SDA line and the green one
is SDL.
Figure 5.2.1 – Arduino Setup
The Arduino IDE is the software that the team used to program the Arduino chip. The IDE is
written in Java, and actually is derived from the IDE for the Processing programming language.
Arduino programs are written in C or C++; C++ was used for this project. The main function of
the Arduino code is to establish a scenario to receive a string of data from the Java program and
translate it to the cube's instruction set, and finally send these instructions to the cube. The code
is shown below.
#include "CUBE8.h"
int _SDA = 2;
int _SCL = 3;
Page : 16 of 25
ELEC/CENG 499 – Final Report
CUBE8 Cube(_SDA,_SCL);
unsigned int c;
boolean update_go;
unsigned int matrix[64];
void setup(){
Serial.begin(115200); // initialize serial communication
Cube.GetControl(); // initialize the cube
Cube.Display(0xc000);
update_go = true;
}
void loop(){
// check for new data
if(Serial.available() >= 2) {
unsigned int inAddr = Serial.read();
unsigned int inByte = Serial.read();
if(inAddr==64){ update_go = true;
}
if(inAddr>=0 && inAddr<63){
// add new data to the array
matrix[inAddr] = inByte;
}
}
if(update_go){
// update columns
for(int addr = 0; addr < 64 ; addr++){
Cube.Display(addr*(0x100) + matrix[addr]);
}
update_go = false;
}
Cube.Display(0xE006);
Page : 17 of 25
ELEC/CENG 499 – Final Report
}
From the code, it is can be seen that the library cube8.h is included in the code. This library code
is attached in Appendix D. In the cube8.h library, functions like Cube.Display(), Cube.
GetControl(), and Cube8 Cube(SDA, SDL) are defined. Just as with any other Arduino Code,
this code only needs to define two functions to make it run able. Setup() is the function run once
at the start of a program that initializes settings. Also, the loop() is the function called repeatedly
until the board powers off.
In the setup() function, two things are initialized: the serial read speed and the cube mode. The
serial speed is set to be 115200 which is the highest speed for Arduino. The reason behind is that
when the team was doing the final testing, approximately 1 second delay occurred between the
hand movement and image display. The highest transmitting speed (115200) was set up to
reduce the delay period. The function Cube.GetControl() is implemented to push the cube into
user mode so that the cube will display the Image created in the main loop.
In the Loop(), first of all the program is checking is there any data coming from the serial.read()
function. At least two bytes must be received at a time; the first byte indicates the column that is
to be updated and the second byte indicates the state of each LED in that column. This data is
then stored in an array so that it can be sent to the cube. The size of the array is 64 and it saves
the data for one frame of an animation. The last step is to tell the cube which LEDs should be
turned on and the last command, Cube.Display(0xE006), is to actually turn on these LEDs.
5.3 Arduino Interface
The Arduino interface is implemented using an open source library called RXTX. This library
defines a Java class that handles serial input and output using a standard stream API. Data is sent
to the Arduino in pairs of bytes with the first byte indicating the column that should be updated
and the second byte indicating the data that should be written to that column.
Page : 18 of 25
ELEC/CENG 499 – Final Report
5.4 Image Management
The image is stored in the Java program as a 3 dimensional binary matrix. This form makes it
simple to translate or rotate the image by moving each individual point. When the cube needs to
be updated the matrix can be read out and sent to the Arduino. Each binary column can be
treated as a byte and sent to the Arduino with its corresponding column number.
5.5 Leap Motion Detection
The Leap Motion offers development libraries that provide built in support for detecting several
gestures. Of interest for this project are the circle and swipe gestures. The original plan was to
use the swipe gesture to translate the image and the circle gesture to rotate the image. The
problem with this approach is that the Leap Motion only detects circle gestures in one plane. An
alternative approach was devised which used only the position of a hand above the Leap Motion
to manipulate the image.
The first step is to define a starting point for the hand. A fraction of a second after the Leap
Motion detects a hand the position of that hand is defined as the origin point. If the hand moves a
certain distance from the origin point then the image moves in the direction of the hand. Once the
image moves the hand must return to the origin point before the image can be moved again.
Rotation is handled is a similar manner by recording the rotation of the hand along each axis.
5.6 3D Game Design
For the project our team only had enough time to create one 3D game, which was a simple game
of pong. Pong is the classic table tennis sport game that is featured using simple two-dimensional
graphics. Players would use paddles to hit a ball back and forth in a playing field, and aim to
score points against their opponent. A point is earned when an opponent is unable to successfully
return the ball to that player. Pong 3D is your typical game of pong; however it utilizes the third
dimension. In Pong 3D the player is essentially playing against a wall, to keep the programming
simple. For this game the ball is allowed to bounce around in the three-dimensional confines of
the LED cube and a paddle controlled by the player reflects the ball away from the player
controlled end plane. For this section of the report we will describe the concepts needed to
Page : 19 of 25
ELEC/CENG 499 – Final Report
govern the game, discuss the major components of the code, and any difficulties encountered
during the programming.
5.6.1 Concepts of Pong 3D
Pong 3D consists of two major components which are the ball and the paddle. The ball will be
confined to bounce within the 8x8x8 grid of the LED cube, and will report its position to be
displayed in the cube. The paddle is confined to one of the outside planes and will receive its
position from the Leap Motion interface. In the most general sense this is the logic used in the
Pong 3D script.
Figure 5.6.1.1 – Pong 3D Logic
The ball is allowed to bounce around in the confines of the LED cube which is defined by the
limits of 0≤X≤8, 0≤Y≤8, Z≤8, with the origin at the corner near the axis legend (in the figure
above). Now if the ball’s Z direction component is less than zero, the player will lose the game
because it refers that a point has been scored against the player. The ball contains an X, Y, and Z
positional and directional components of its movement. When the ball hits one of the boundary
planes it will reverse its corresponding direction. For example, if the ball hits the YZ-plane it
Page : 20 of 25
ELEC/CENG 499 – Final Report
reverses the X components direction of the ball. It should also be noted that the ball is of size of
1x1x1 and will not be affected by the laws of gravity, to keep it simple.
The paddle is confined to the XY-plane where Z=0 and is allowed to move freely along that
plane ( 0≤X≤8, 0≤Y≤8, Z=0). The size of the paddle is a 2x2x1 rectangle that will reflect the ball
in the inverse Z direction. However, if the ball does not intersect with the paddle it will continue
outside the boundary and the player will lose the game. The paddle’s position will be defined by
the position data from the Leap Motion hardware.
5.6.2 Pong 3D Program
Pong 3D game was created using the Java programming language, and built with a number of
classes. The main classes used in this program are the shown in the figure below and it outlines
how each of those classes interacts with one another. The GameEngine.java class contains the
Leap Motion handlers that control the paddle in the game. This class also communicates with the
Paddle.java and Grid.java classes. The Paddle.java class updates the position of paddle to the
LED cube, and the size of that paddle. The Grid.java class completes a number of tasks that
checks if the ball has collided with one of the planes or paddle and continues to increment the
position of the ball. This class also checks to see if the ball has passed the paddle plane and a
point has been scored, so the game will be shut down. The Ball.java class takes the current
position of the ball and increments its direction according the path of travel. Please see Appendix
B for Pong 3D code.
Figure 5.6.2.1 - Major Classes Interactions in Pong 3D
Page : 21 of 25
ELEC/CENG 499 – Final Report
6 WORKLOAD DISTRIBUTION & ACHIEVEMENTS
In order to complete this project in an efficient way, the team built a work log to record each
period involved with the task. The whole project is divided into four stages.
First stage is initiating a group with five group members and continues pursuing the topic which
has been chosen in ELEC/CENG 399 during the first two weeks.
Second stage, preparation of the project including: Leap Motion open source coding
investigation, LED cube full functionality test, Arduino investigation, and Arduino and Cube
communication theory. When team finished planning our project in this stage, a brief
introduction was presented to our advisor and ELEC/CENG 499 classmates during the proposal
presentation. The topic contains the motivation of project, materials need, procedures and the
total project price cost.
Third stage, the whole group was divided by two small teams. Team A began to work on the new
LED cube construction, as well as installing the eight 8x8 LED slices on the cube base. In
addition, the team tested each slice of LED cube in order to make sure the LED pin connection,
soldering joints and LEDs functionality were working properly. Another task of team A was to
code the Arduino program to design an image and send image signals to the LED
cube. Meanwhile, team B worked on the game coding and communications configuration
between the Leap Motion and the Arduino.
The fourth stage was the last 3 weeks, two teams worked together to test the whole project's
functionality and work on the demonstration and final report.
The group clearly defined jobs to each group members and accomplished the cube as planned.
Most of the primary functionality was achieved; specifically the Leap Motion to Arduino
communication, the Arduino to LED cube communication, and the image manipulation.
However, the 3D game program still had some bugs that could not be fixed by the date of
demonstration. However, the primary goal of the project was achieved.
Page : 22 of 25
ELEC/CENG 499 – Final Report
Fangzheng Song: My contribution to the project mainly focus on the LED Cube interfacing with
Arduino, which includes researching LED cube documentation, researching Arduino library and
programming Arduino code. A portion of time was spent on the alternative of connecting
Arduino to Processing. Moreover, the last two week of the semester was spent on project report
and demo preparation.
Nelson Lee: My contributions for this project have been to oversee many of the different tasks
throughout this project. I was also primarily responsible for the Pong 3D code and aiding the
other team members with any of the other tasks (such as the Arduino to cube interface and
coding, webpage design). I have also spent many hours on the PR side of this project as well,
such as the Final Report, Milestones, Worklogs, and the many presentations.
Tristan: My primary focus was interfacing the Leap Motion with our program and interfacing
our program with the Arduino. I was also involved in interfacing the Arduino and the cube.
Candace Chen: My contribution to the project consisted of researching on 3D LED Cube and
alternative 3D image displays. We worked cooperatively weekly to design and talk about the
motivation of this project. A great deal of the time was spent designing and creating project
webpage and pamphlet. A portion of time was spent on LED cube functionality testing.
Remaining week of the semester was spent on completing the final project report.
Yang Liu: My contributions to the project consisted of researching on new LED cube, Arduino
circuit board, processing coding, website design and Leap Motion. About 50 hours were spent on
LED cube, Leap Motion and Arduino communication, 10 hours were spent on website design
and the website post, and 10 hours were spent on LED cube functionality test and construction.
7 WEBSITE DESIGN
The main theme of the group project website is for demonstrating the TIM-3D project; this
website includes “Home”, “Team”, “Gallery” and “Documents” main links. Dreamweaver 8 was
used for designing the website. Previously, team tried to use raw html to design the website,
however, none of team members had solid html coding experience. Dreamweaver 8 also has an
Page : 23 of 25
ELEC/CENG 499 – Final Report
html coding environment that made it easy to implement the website on different web browsers
and different operating systems. The built in web page templates were so that the web
components could be added or moved based on changes to the project during development.
On the main page, there was the project description part to introduce the project motivations,
basic theory and components. On the team page, there were general introductions of team
members and team photos. On the Gallery page, in this webpage, the photos of the Arduino Unit
Connected to Cube and Computer, Arduino Circuit Connection and Demonstration Day were
shown to help visitors to get a visual experience on the project. The Documents page listed all
worklogs, progress reports, milestones from the beginning of the project, and the final report.
These offered the details of how the project get start, process, develop and demonstrate.
The most useful function that was used to complete the webpage was the split screen function
which allowed us to view the webpage and the web page's code as well as being able to edit code
on one side, and see changes on the website on the other side immediately. The webpage zip
folder was then transferred to a domain to upload.
8 PROJECT DISCUSSION
The main concern with TIM-3D is the limitation on the resolution of the cube. Also, the
processing time of the cube created many restraints between the Arduino and Leap Motion
interfaces. The capabilities of Leap Motion exceeded the capability of the LED cube. Thus,
during demonstration day there was a delay in response to hand gestures being implemented on
the cube. It was not able to simulate real time hand movements. Instead, the demonstration had
to be implemented in steps, increments of motion up, down, left, right…etc. Expanding the size,
which will intensify the resolution, would do improvements to the cube. As well as upgrading
the cube’s controller to respond quicker to commands, it will need to compatible to the
capabilities of Leap Motion and the Arduino interface.
Good teamwork was achieved towards completion of this project. Due to the each member’s
distinctive schedules and commitments it was difficult to have many scheduled constructing
Page : 24 of 25
ELEC/CENG 499 – Final Report
conventions. Improvements could have been made to reduce the amount of work done with
limited amount of time. Generally, the project and each milestone were completed on schedule.
Work was being accomplished cooperatively and individually. All compositions were
documented and up to date as planned. Overall, the project goal of displayed the theory and idea
of a 3D image manipulator was achieved.
9 CONCLUSION
The project includes this final report as well as a webpage which includes all milestone, work log
and progress reports that can be found on www.ece.uvic.ca/~wayneliu. The entire project
consists of 4 components: project team selection and organizing, project preparation, project
design and testing, other works and documentations. Additionally, a simple 3D pong game was
created to demonstrate the functionality of TIM-3D using java programming language.
This project has successfully allowed us to create and understand the technology of 3D image
manipulation. It started with the researching of methods to display 3D images. Through the pros
and cons, the 3D LED cube was chosen. Further on, an Arduino (the most famous single-broad
microcontrollers in market) was chosen to act as the microcontroller in controlling the image
production of the LED cube. The development language of the Arduino, C, C++, and Java are
most convenient and compatible with software running on a computer. This device allowed
connection between the cube and a computer. Also connected to the computer is the hand motion
detector, Leap Motion. Leap Motion transfers the commands from the user’s hands to the
computer. TIM-3D was able to read front, back, up, down, pitch, and roll hand gestures. All
components chosen were based on size, functionality but mostly price due to low budget
restraints.
This project completed in an efficient and creative way. The whole group was divided into two
small teams that worked collectively with clearly defined jobs. The cube was accomplished as
planned, as well as pamphlets, web page and poster before the deadline.
Page : 25 of 25
ELEC/CENG 499 – Final Report
10 APPENDIX A (JAVA PROGRAM)
#include "CUBE8.h"
CUBE8::CUBE8(byte SDA,byte
SCL){
// TODO Auto-generated
constructor stub
_SDA = SDA;
_SCL = SCL;
pinMode(_SDA,OUTPUT);
pinMode(_SCL,OUTPUT);
ack = false;
}
void CUBE8::DELAY_U(unsigned
int a){
bool x = false;
while(a-- != 0)
{
delay(1);
x = ! x;
digitalWrite(_SCL,x);
}
}
void CUBE8::Display(unsigned
int data){
SendDeviceByte(data /
0x100,data % 0x100);
}
void CUBE8::I2cDelay(unsigned
int a){
while(a-- != 0)
asm("Nop");
}
void CUBE8::StartI2c(void){
I2cDelay(18);
pinMode(_SCL,INPUT);
while(!digitalRead(_SCL));
I2cDelay(9);
digitalWrite(_SDA,HIGH);
I2cDelay(9);
pinMode(_SCL,OUTPUT);
digitalWrite(_SCL,HIGH);
I2cDelay(9);
digitalWrite(_SDA,LOW);
I2cDelay(9);
digitalWrite(_SCL,LOW);
I2cDelay(4);
}
void CUBE8::I2cSendByte(byte
data){
byte i;
for(i=0;i<8;i++)
{
digitalWrite(_SDA,((data << i)
& 0x80));
I2cDelay(6);
Page : 2 of 15
ELEC/CENG 499 – Final Report
digitalWrite(_SCL,HIGH);
I2cDelay(8);
digitalWrite(_SCL,LOW);
}
I2cDelay(8);
digitalWrite(_SDA,HIGH);
I2cDelay(8);
digitalWrite(_SCL,HIGH);
I2cDelay(8);
pinMode(_SDA,INPUT);
ack = digitalRead(_SDA);
digitalWrite(_SCL,LOW);
I2cDelay(8);
pinMode(_SDA,OUTPUT);
}
void CUBE8::GetControl(void){
DELAY_U(1000);
digitalWrite(_SCL,HIGH);
}
void CUBE8::SendDeviceByte(byte
address,byte data){
StartI2c();
I2cSendByte(address);
I2cSendByte(data);
StopI2c();
}
void CUBE8::StopI2c(void){
digitalWrite(_SDA,LOW);
I2cDelay(9);
digitalWrite(_SCL,HIGH);
I2cDelay(9);
digitalWrite(_SDA,HIGH);
I2cDelay(9);
}
void CUBE8::SetAll(unsigned int
data){
return;
}
void CUBE8::Update(unsigned int
data[]){
return;
}
CUBE8::~CUBE8(){
}
//---------------------------//
#ifndef CUBE8_H_
#define CUBE8_H_
#include "Arduino.h"
class CUBE8 {
public:
CUBE8(byte SDA,byte SCL);
virtual ~CUBE8();
void GetControl(void);
void Display(unsigned int
data);
void SetAll(unsigned int
Page : 3 of 15
ELEC/CENG 499 – Final Report
data);
void Update(unsigned int
data[]);
private:
byte _SDA,_SCL;
bool ack;
void DELAY_U(unsigned int a);
void StartI2c(void);
void I2cDelay(unsigned int a);
void StopI2c(void);
void I2cSendByte(byte data);
void SendDeviceByte(byte
address,byte data);
};
#endif
/----------------------------//
import java.io.IOException;
import com.leapmotion.leap.*;
import
com.leapmotion.leap.Gesture.Sta
te;
import java.io.OutputStream;
import java.util.Scanner;
class Application {
public static void
main(String[] args) {
// Create a sample
listener and controller
DefListener listener =
new DefListener();
Controller controller =
new Controller();
// Have the sample
listener receive events from
the controller
controller.addListener(listener
);
// Keep this process
running until Enter is pressed
System.out.println("Press Enter
to quit...");
try {
System.in.read();
} catch (IOException e)
{
e.printStackTrace();
}
// Remove the sample
listener when done
controller.removeListener(liste
ner);
Page : 4 of 15
ELEC/CENG 499 – Final Report
SerialOutput
serPortUSB = new
SerialOutput();
serPortUSB.initialize();
Scanner in = new
Scanner(System.in);
int inVar = 0;
String inStr = "";
int[] output_buffer;
Matrix mat = new
Matrix();
inStr =
in.nextLine();
mat.makeTestBox();
output_buffer =
mat.getMatrix();
serPortUSB.updateAll(outpu
t_buffer);
do {
inStr =
in.nextLine();
mat.translate(1,-1);
output_buffer =
mat.getMatrix();
serPortUSB.updateAll(outpu
t_buffer);
} while
(inStr.compareTo("")==0);
serPortUSB.close();
}
}
//---------------------------//
import java.math.*;
public class Matrix {
private boolean[][][]
data;
private int size_x;
private int size_y;
private int size_z;
public Matrix (int x, int
y, int z) {
size_x = x;
size_y = y;
size_z = z;
Page : 5 of 15
ELEC/CENG 499 – Final Report
data = new
boolean[x][y][z];
for( int a=0; a<x;
a++ ){
for( int b=0;
b<y; b++ ){
for( int
c=0; c<z; c++ ){
data[a][b][c] = false;
}
}
}
}
public Matrix (int s) {
this(s, s, s);
}
public Matrix () {
this(8);
}
public int getColumn(int
x, int y) {
int sum = 0;
for( int z=0;
z<size_z; z++ ) {
if(
data[x][y][z] ) {
sum +=
(int) Math.pow(2, z);
}
}
return sum;
}
public int[] getMatrix() {
int[] data_out = new
int[64];
for( int a=0;
a<size_x; a++) {
for( int b=0;
b<size_y; b++) {
data_out[b+(a*size_y)] =
getColumn(a, b);
}
}
return data_out;
}
public void setAll(boolean
x) {
for( int a=0;
a<size_x; a++ ){
for( int b=0;
b<size_y; b++ ){
Page : 6 of 15
ELEC/CENG 499 – Final Report
for( int
c=0; c<size_z; c++ ){
data[a][b][c] = x;
}
}
}
}
public void updateBall(int
x, int y, int z) {
}
public void
updatePaddle(int x, int y) {
}
/* x-axis: 1
* y-axis: 2
* z-axis: 3
*
* n is the number of 45
degrees increments
*/
public void rotate(int
dir, int n) {
}
/* x-axis: 1
* y-axis: 2
* z-axis: 3
*
* n is the number of
increments
*/
public void translate(int
axis, int n) {
boolean[][][]
new_data = new
boolean[size_x][size_y][size_z]
;
switch(axis) {
case 1: // x-axis
for(int a=0;
a<size_y; a++) {
for(int
b=0; b<size_z; b++) {
for(int c=0; c<size_x;
c++) {
new_data[c][a][b] = false;
if((c-n)>=0 && (c-
n)<size_x) {
new_data[c][a][b] =
data[c-n][a][b];
Page : 7 of 15
ELEC/CENG 499 – Final Report
}
}
}
}
break;
case 2: // y-axis
break;
case 3: // z-axis
break;
default:
return;
}
data = new_data;
}
public void makeTestBox()
{
int[][][] box = {
{
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0}},
{
{0,0,0,0,0,0,0,0},
{0,1,1,1,1,0,0,0},
{0,1,0,0,1,0,0,0},
{0,1,0,0,1,0,0,0},
{0,1,0,0,1,0,0,0},
{0,1,1,1,1,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0}},
{
{0,0,0,0,0,0,0,0},
{0,1,0,0,1,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,1,0,0,1,0,0,0},
Page : 8 of 15
ELEC/CENG 499 – Final Report
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0}},
{
{0,0,0,0,0,0,0,0},
{0,1,1,1,1,0,0,0},
{0,1,0,0,1,0,0,0},
{0,1,0,0,1,0,0,0},
{0,1,0,0,1,0,0,0},
{0,1,1,1,1,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0}},
{
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0}},
{
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0}},
{
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
Page : 9 of 15
ELEC/CENG 499 – Final Report
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0}},
{
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0}}
};
for(int a=0; a<8;
a++) {
for(int b=0;
b<8; b++) {
for(int
c=0; c<8; c++) {
if(box[a][b][c]==1) {
data[a][b][c] = true;
}else{
data[a][b][c] = false;
}
}
}
}
}
}
//---------------------------//
import java.io.IOException;
import java.io.OutputStream;
import
gnu.io.CommPortIdentifier;
import gnu.io.SerialPort;
import gnu.io.SerialPortEvent;
import
gnu.io.SerialPortEventListener;
import java.util.Enumeration;
public class SerialOutput
implements
SerialPortEventListener {
SerialPort serialPort;
Page : 10 of 15
ELEC/CENG 499 – Final Report
/** The port we're normally
going to use. */
private static final
String PORT_NAME = "COM5";
public OutputStream
output;
/** Milliseconds to block
while waiting for port open */
private static final int
TIME_OUT = 2000;
/** Default bits per
second for COM port. */
private static final int
DATA_RATE = 115200;
public void initialize() {
CommPortIdentifier
portId = null;
Enumeration portEnum
=
CommPortIdentifier.getPortIdent
ifiers();
//First, Find an
instance of serial port as set
in PORT_NAME.
while
(portEnum.hasMoreElements()) {
CommPortIdentifier
currPortId =
(CommPortIdentifier)
portEnum.nextElement();
System.out.println(currPor
tId.getName());
if
(currPortId.getName().equals(PO
RT_NAME)) {
portId =
currPortId;
break;
}
}
if (portId == null)
{
System.out.println("Could
not find COM port.");
return;
}
try {
// open serial
port, and use class name for
the appName.
serialPort =
(SerialPort)
portId.open(this.getClass().get
Name(), TIME_OUT);
Page : 11 of 15
ELEC/CENG 499 – Final Report
// set port
parameters
serialPort.setSerialPortPa
rams(DATA_RATE,
SerialPort.DATABITS_8,
SerialPort.STOPBITS_1,
SerialPort.PARITY_NONE);
// open the
stream
output =
serialPort.getOutputStream();
} catch (Exception
e) {
System.err.println(e.toStr
ing());
}
}
/**
* This should be called
when you stop using the port.
* This will prevent port
locking on platforms like
Linux.
*/
public synchronized void
close() {
if (serialPort !=
null) {
serialPort.removeEventList
ener();
serialPort.close();
}
}
public synchronized void
serialEvent(SerialPortEvent
oEvent) {
}
public void updateAll(
int[] data ) {
try {
for( int i=0;
i<64; i++ ){
output.write(i);
output.write(data[i]);
}
output.write(64);
Page : 12 of 15
ELEC/CENG 499 – Final Report
output.write(0);
} catch (IOException
e) {
e.printStackTrace();
}
}
}
//---------------------------//
import java.lang.Math;
import com.leapmotion.leap.*;
import com.leapmotion.leap.Gesture.State;
class DefListener extends Listener {
public void onInit(Controller controller) {
System.out.println("Initialized");
}
public void onConnect(Controller
controller) {
System.out.println("Connected");
controller.enableGesture(Gesture.Type.TYP
E_SWIPE);
controller.enableGesture(Gesture.Type.TYP
E_CIRCLE);
controller.enableGesture(Gesture.Type.TYP
E_SCREEN_TAP);
controller.enableGesture(Gesture.Type.TYP
E_KEY_TAP);
}
public void onDisconnect(Controller
controller) {
//Note: not dispatched when running in
a debugger.
System.out.println("Disconnected");
}
public void onExit(Controller controller)
{
System.out.println("Exited");
}
public void onFrame(Controller controller)
{
// Get the most recent frame and report
some basic information
Frame frame = controller.frame();
System.out.println("Frame id: " +
frame.id()
Page : 13 of 15
ELEC/CENG 499 – Final Report
+ ", timestamp: " +
frame.timestamp()
+ ", hands: " +
frame.hands().count()
+ ", fingers: " +
frame.fingers().count()
+ ", tools: " +
frame.tools().count()
+ ", gestures " +
frame.gestures().count());
if (!frame.hands().isEmpty()) {
// Get the first hand
Hand hand = frame.hands().get(0);
// Check if the hand has any fingers
FingerList fingers = hand.fingers();
if (!fingers.isEmpty()) {
// Calculate the hand's average
finger tip position
Vector avgPos = Vector.zero();
for (Finger finger : fingers) {
avgPos =
avgPos.plus(finger.tipPosition());
}
avgPos =
avgPos.divide(fingers.count());
System.out.println("Hand has " +
fingers.count()
+ " fingers, average
finger tip position: " + avgPos);
}
// Get the hand's sphere radius and
palm position
System.out.println("Hand sphere
radius: " + hand.sphereRadius()
+ " mm, palm position: " +
hand.palmPosition());
// Get the hand's normal vector and
direction
Vector normal = hand.palmNormal();
Vector direction = hand.direction();
// Calculate the hand's pitch, roll, and
yaw angles
System.out.println("Hand pitch: " +
Math.toDegrees(direction.pitch()) + "
degrees, "
+ "roll: " +
Math.toDegrees(normal.roll()) + " degrees, "
+ "yaw: " +
Math.toDegrees(direction.yaw()) + "
degrees");
}
Page : 14 of 15
ELEC/CENG 499 – Final Report
GestureList gestures = frame.gestures();
for (int i = 0; i < gestures.count(); i++)
{
Gesture gesture = gestures.get(i);
switch (gesture.type()) {
case TYPE_CIRCLE:
CircleGesture circle = new
CircleGesture(gesture);
// Calculate clock direction
using the angle between circle normal and
pointable
String clockwiseness;
if
(circle.pointable().direction().angleTo(circle.
normal()) <= Math.PI/4) {
// Clockwise if angle is less
than 90 degrees
clockwiseness = "clockwise";
} else {
clockwiseness =
"counterclockwise";
}
// Calculate angle swept since
last frame
double sweptAngle = 0;
if (circle.state() !=
State.STATE_START) {
CircleGesture
previousUpdate = new
CircleGesture(controller.frame(1).gesture(ci
rcle.id()));
sweptAngle =
(circle.progress() -
previousUpdate.progress()) * 2 * Math.PI;
}
System.out.println("Circle id: "
+ circle.id()
+ ", " + circle.state()
+ ", progress: " +
circle.progress()
+ ", radius: " +
circle.radius()
+ ", angle: " +
Math.toDegrees(sweptAngle)
+ ", " + clockwiseness);
break;
case TYPE_SWIPE:
SwipeGesture swipe = new
SwipeGesture(gesture);
System.out.println("Swipe id: "
+ swipe.id()
+ ", " + swipe.state()
+ ", position: " +
swipe.position()
+ ", direction: " +
swipe.direction()
Page : 15 of 15
ELEC/CENG 499 – Final Report
+ ", speed: " +
swipe.speed());
break;
case TYPE_SCREEN_TAP:
ScreenTapGesture screenTap =
new ScreenTapGesture(gesture);
System.out.println("Screen Tap
id: " + screenTap.id()
+ ", " + screenTap.state()
+ ", position: " +
screenTap.position()
+ ", direction: " +
screenTap.direction());
break;
case TYPE_KEY_TAP:
KeyTapGesture keyTap = new
KeyTapGesture(gesture);
System.out.println("Key Tap id:
" + keyTap.id()
+ ", " + keyTap.state()
+ ", position: " +
keyTap.position()
+ ", direction: " +
keyTap.direction());
break;
default:
System.out.println("Unknown
gesture type.");
break;
}
}
if (!frame.hands().isEmpty()
|| !gestures.isEmpty()) {
System.out.println();
}
}
}
11 APPENDIX B (GAME CODE)
package pong; import pong.frontEnd.GameEngine; public class Main { public static void main(String[] args) { GameEngine engine = new GameEngine(); engine.run(); } }
//------------------------------------------------//
package pong.frontEnd; //import java.awt.Dimension; import javax.swing.JPanel; import pong.components.Component; import pong.components.Paddle; import pong.constants.Constants; import pong.util.Point2D; import java.awt.event.MouseEvent; import java.awt.event.MouseMotionListener; import java.util.ArrayList; /**
Page : 2 of 8
ELEC/CENG 499 – Final Report
* Gets the ball and paddle 3D location from the grid Class in the
* pong.frontEnd Package
*
* **Need to replace mouse listener with Leap Motion**
*/ public class GameEngine extends JPanel implements MouseMotionListener { protected Grid grid; protected ArrayList<Component> list; public GameEngine() { initInput(); initGame(); } //Starts the game and terminated loop when ball passes the paddle
public void run() { while (true) { //System.out.println("something"); stepBalls(); //ball=stepBalls(); try { Thread.sleep(Constants.DELAY); } catch (InterruptedException ex) {} } } protected void initGame() { this.grid = Grid.getInstance(); for (int i = 0; i < Constants.NUM_BALLS; ++i) grid.addBall(); } protected void initInput(){ this.addMouseMotionListener(this); } protected void stepBalls(){ grid.stepBalls(); }
public void mouseEntered(MouseEvent e) { } public void mouseExited(MouseEvent e) { } public void mouseDragged(MouseEvent e) { } public void mouseMoved(MouseEvent e) { Point2D point = new Point2D((int)e.getPoint().getX(), (int)e.getPoint().getY()); Paddle.getInstance().setPoint(point); } }
//------------------------------------------------//
package pong.frontEnd; import java.util.ArrayList; import pong.components.Ball; import pong.components.Component; import pong.components.Paddle; import pong.constants.Constants; import pong.util.CollisionDetector; import pong.util.Point2D; import pong.util.Point3D; public class Grid { private static Grid INSTANCE = new Grid();
protected ArrayList<Component> list; private Grid () { this.list = new ArrayList<Component>(); } public static Grid getInstance(){ return Grid.INSTANCE; }
Page : 3 of 8
ELEC/CENG 499 – Final Report
public void addBall() { list.add((Component) new Ball()); }
// Iterate through list stepping only the balls
public void stepBalls(){ Point3D currentPoint;
for(Component item: this.list){ if(item instanceof Ball) { ((Ball)item).step(); currentPoint = ((Ball) item).getPoint(); //Matrix.updateBall(currentPoint.getX(),currentPoint.getY(),currentPoint.getZ()) }
} //System.out.print(list); checkCollisions(); //return(list); } protected void checkCollisions(){ for(Component item: this.list){ if(item instanceof Ball){ Point3D currentPoint = ((Ball) item).getPoint();
System.out.printf("x: %d y: %d z: %d\n", currentPoint.getX(), currentPoint.getY(), currentPoint.getZ()); //System.out.print(currentPoint);
if(currentPoint.getZ() < 0){ //System.out.printf("Point<0 z:
%d\nGame Over\n",currentPoint.getZ()); throw new IllegalStateException("Game Over"); }
//System.out.print(CollisionDetector.distanceCompare(currentPoint.getZ(), Constants.SCREEN_DEPTH, 1)); if(CollisionDetector.distanceCompare(currentPoint.getZ(), Constants.SCREEN_DEPTH, 1)){ ((Ball) item).bounce(Constants.PLANE_Z); //System.out.printf("Collision at +Z=%d\n",currentPoint.getZ()); }
//Check -Z plane (Plane 1)(Paddle Plane) else if(CollisionDetector.distanceCompare(currentPoint.getZ(), 0, 0)){ //System.out.printf("Collision at -Z=%d\n",currentPoint.getZ());
//Check if intercepted by paddle
Point2D paddlePoint = Paddle.getInstance().getPoint(); //System.out.printf("Paddle X:%d | Y:%d\n",paddlePoint.getX(),paddlePoint.getY()); if (CollisionDetector.paddleDistanceCompare(currentPoint.getX(), paddlePoint.getX(), Constants.PADDLE_SIZE/2) && CollisionDetector.paddleDistanceCompare(currentPoint.getY(),
Page : 4 of 8
ELEC/CENG 499 – Final Report
paddlePoint.getY(), Constants.PADDLE_SIZE / 2)) { ((Ball) item).bounce(Constants.PLANE_Z); //System.out.println("Paddle Bounce"); }
} //Check +-Y plane (+Y=Plane 6)(-Y=Plane 5) if(CollisionDetector.distanceCompare(currentPoint.getY(), Constants.SCREEN_HEIGHT, 1) || CollisionDetector.distanceCompare(currentPoint.getY(), 0, 0)){ ((Ball) item).bounce(Constants.PLANE_Y); //System.out.printf("Collision at Y=%d\n",currentPoint.getY()); } //Check +-X plane (+X=Plane 4)(-X=Plane 3) if(CollisionDetector.distanceCompare(currentPoint.getX(), Constants.SCREEN_WIDTH, 1) || CollisionDetector.distanceCompare(currentPoint.getX(), 0, 0)){ ((Ball) item).bounce(Constants.PLANE_X); //System.out.printf("Collision at X=%d\n",currentPoint.getX()); } } //return(currentPoint); } } }
//------------------------------------------------// package pong.components; //import java.awt.Color; import java.awt.Graphics; import pong.constants.Constants; import pong.util.*; public class Ball extends Component{ protected Direction direction; protected Point3D point; public static int SIZE = Constants.MAX_BALL_RADIUS; public Ball (){ this.direction = new Direction(Constants.X_BALL_DIRECTION, Constants.Y_BALL_DIRECTION, Constants.Z_BALL_DIRECTION); this.point = new Point3D(Constants.MAX_BALL_RADIUS, Constants.MAX_BALL_RADIUS, Constants.MAX_BALL_RADIUS); }
//**Note: Returns a copy of the balls location**
public Point3D getPoint(){ return this.point.clone(); }
//**Note: Increments directional vectors**
public void step(){
Page : 5 of 8
ELEC/CENG 499 – Final Report
int x = point.getX(); //gets current x-direction
int y = point.getY(); //gets current y-direction
int z = point.getZ(); //gets current z-direction
point = new Point3D(x + this.direction.getA(), //determines new 3D point y + this.direction.getB(), z + this.direction.getC()); }
//**Note: x direction changes only if it hits a paddle**
public void bounce(int planeID){ if(planeID == Constants.PLANE_Z){ this.direction = new Direction(this.direction.getA(), this.direction.getB(), -this.direction.getC()); } else if(planeID == Constants.PLANE_X){ this.direction = new Direction(-this.direction.getA(), this.direction.getB(), this.direction.getC()); } else if(planeID == Constants.PLANE_Y){ this.direction = new Direction(this.direction.getA(), -this.direction.getB(), this.direction.getC()); } } @Override
public void draw(Graphics g) {
// TODO Auto-generated method stub } }
//------------------------------------------------// package pong.components; //import java.awt.Color; //import java.awt.Graphics; import pong.constants.Constants; import pong.util.Point2D; /**
* Components used to define the paddle
*/ public class Paddle { protected Point2D point; protected static Paddle thePaddle = new Paddle(); private Paddle() { this.point = new Point2D(Constants.SCREEN_WIDTH/2, Constants.SCREEN_HEIGHT/2); } public void setPoint(Point2D point){ Point2D p = this.point; //Matrix.updatePaddle(p.getX(), p.getY()); } public Point2D getPoint(){ return this.point; } public static Paddle getInstance(){ return Paddle.thePaddle; } }
//------------------------------------------------//
Page : 6 of 8
ELEC/CENG 499 – Final Report
package pong.util; /**
* Checks to see if the ball is out of bounds of the 8x8x8 grid and returns true if the
* ball is out of bound, and returns false if it is not. *
* distanceCompare() = Checks if ball is out of the +y,y0,+x,x0,+z planes
* paddleDistanceCompare() = Checks if ball is out of paddle plane (z0 plane) */ public class CollisionDetector { private CollisionDetector(){}
//Check ball position in respect to the boundary of the grid and return true if ball // is out of bounds and false if it is not //point - Position of ball //wall - Distance of wall //dir - Direction 0 = neg direction, 1 = pos direction
public static boolean distanceCompare(int point, int wall, int dir){
//System.out.println("Point : " + point + " | Wall: " + wall + "| dir:"+dir+"\n"); if((point>=wall) && (dir==1)){ //System.out.println("True 1"); return true; } if((point<=wall) && (dir==0)){ //System.out.println("True 2"); return true; }
//System.out.println("False"); return false; }
public static boolean paddleDistanceCompare(int point, int padPoint, int padSize){
//System.out.println("Point : " + point + " | PadPoint: " + padPoint + "| Pad:" + padSize + "\n"); if((point<=(padPoint+padSize)) && (point>=(padPoint-(padSize-1)))){ //System.out.println("True"); return true; } //System.out.println("False"); return false; } }
//------------------------------------------------// package pong.components; import java.awt.Graphics; public abstract class Component { public abstract void draw(Graphics g); }
//------------------------------------------------// package pong.util; /**
* Find and return the direction of the ball */ public class Direction { protected int a, b, c;
//Find current ball co-ordinates
Page : 7 of 8
ELEC/CENG 499 – Final Report
public Direction(int a, int b, int c){ this.a = a; this.b = b; this.c = c; } //Determing magnitude
public int getMagnitude(){ return (int)(Math.sqrt(a * a + b * b + c * c)); } public Direction clone(){ return new Direction(this.a, this.b, this.c); } //New direction
public String toString(){ return "3D-Vector: a = " + this.a
+ " | b = " + this.b
+ " | c = " + this.c; }
public int getA() { return a; }
public int getB() { return b; }
public int getC() { return c; } }
//------------------------------------------------//
package pong.util; /**
* Find and return a 3D point */ public class Point2D { protected int x;
protected int y; public Point2D(int x, int y){ this.x = x; this.y = y; } public int getX(){ return this.x; } public int getY(){ return this.y; } public Point2D clone(){ return new Point2D(this.x , this.y); } }
//------------------------------------------------// package pong.util; /**
* Find and return a 3D point */ public class Point3D {
private int x, y, z; public Point3D (int x, int y, int z) { this.x = x; this.y = y; this.z = z; } public int getX() { return x; } public int getY() { return y; } public int getZ() { return z; } public Point3D clone(){
Page : 8 of 8
ELEC/CENG 499 – Final Report
return new Point3D(this.x, this.y, this.z); } }
//------------------------------------------------// /**
* Config file for most constant values used in the game
*
* Constants: * -Delay
* -NumBalls
* -LED cube max parameters
* -Plane ID's
* -3D Directions
* -Ball Radius
* -Paddle size
*/ package pong.constants; public class Constants {
public static int DELAY = 100; public static int NUM_BALLS = 1;
//LED cube max parameters
public static int SCREEN_WIDTH = 10; public static int SCREEN_HEIGHT = 10; public static int SCREEN_DEPTH = 12; //Plane ID
public static int PLANE_Z = 0; public static int PLANE_X = 1; public static int PLANE_Y = 2;
//3D Directions
public static int X_BALL_DIRECTION = 1; public static int Y_BALL_DIRECTION = 1; public static int Z_BALL_DIRECTION = 1;
//Ball Radius
public static int MAX_BALL_RADIUS = 1;
//Paddle Constants
public static int PADDLE_SIZE = 2;
private Constants(){} }
12 APPENDIX C (CUBE CONSTRUCTION)
The 8x8x8 LED cube is the main display component of the whole project. The LED cube is
constructed by the 512 blue LEDs. Each LED has two leads, the longer is the positive lead, and
the shorter one the negative lead. The first steps was to bended all negative leads to 90o which
shows in the figure below.
Page : 2 of 6
ELEC/CENG 499 – Final Report
Figure AC.1 – Bent Negative Leads
Then, the first slice was built by 8x8 LEDs, in order to keep all LEDs in line. A template board
was used during construction. The template board had 64 equal space holes which 3 cm
separated apart. The diameter of each LED holes was 3 mm which was perfect to fit LED head
inside. After that, the first row of eight holes on template board was filled with LEDs. All
negative leads had to solder on next LED’s negative lead. As shown in Figure AC.1. Then,
another six rows were soldered. After the total eight rows LED done, all positive leads soldered
to the positive leads of following rows. The bended position was at approximately 1.5 mm away
from LED head, as shown in Figure AC.3.
Figure AC.2 – Soldering of an LED Row
Page : 3 of 6
ELEC/CENG 499 – Final Report
Figure AC.3 – Positive Lead Soldering
After the other seven slides were completed, all slides had to do on board test to ensure all LEDs
work as shown in Figure AC.4. The working voltage of LED is between 3.3~3.6V and working
current is between 0.02~0.03A. The purpose of testing after the slides completed is to find out
the faulty LEDs before the whole cube construction completed. Finally, all LED slides were
inserted into circuit board slots, and then the LED positive leads on last row of each LED slide
were inserted into the slots at the back circuit board. Due to the weak joints between LEDs,
faulty LEDs and nonprogrammable chip, the team decided to purchase a new programmable
LED cube materials. The new cube had 512 blue LEDs and one programmable external chip.
Comparing with the old cube, the new cube’s LED joints were stronger and the built on board
chip leads were perfect soldered. Basically, the team got a stable platform to design and test the
code. The completed cube can be seen in the figures below.
Page : 4 of 6
ELEC/CENG 499 – Final Report
Figure AC.4 – Testing First Slide of LED’s
Figure AC.5 – Completed LED Cube
Page : 5 of 6
ELEC/CENG 499 – Final Report
13 REFFERNCES
[1] What Arduino Can do. [online] [Cited: April 3, 2014.] http://arduino.cc/en/Main/Homepage
[2]STC12C5A60S2 series MCU Data Sheet. STCMCU. [online] [Cited: April 3, 2014] http://www.stcmcu.com/datasheet/stc/STC-AD-PDF/STC12C5A60S2-english.pdf
[3]LED Cube 8x8x8. Instructables. [Online]. [Cited: April 4, 2014] http://www.instructables.com/id/Led-Cube-8x8x8/
[4] John Boxall, Arduino Workshop, No Starch Press, May 2013.
[5] J.Eric and Robert Rothery, Design for Electrical and Computer Engineers, Wiley, 2002.
[6] Cube8. DoYoung Studio. [Online] [Cited: April 3, 2014] http://www.doyoung.net/works/CUBE8/index.html
[7] Cube8 User mode information sheet. DoYoung Studio. [online] [Cited: April 3, 2014] http://www.doyoung.net/works/CUBE8/CUBE8_U.pdf
[8] ATmega328. Atmel. [Online] [Cited: Dec 2, 2013.] http://www.atmel.com/devices/atmega328.aspx.
[9] Assembling Cube8. DoYoung Studio. [Online] [Cited: April 3, 2014] http://www.doyoung.net/articles/CUBE8/CUBE8_DYS.pdf
[10] Casey Reas and Ben Fry, Getting Started with Processing, Maker Media, 2010.
[11] LeapMotion Introduction [Online] [Cited: April 3, 2014] https://www.leapmotion.com/product
[12] Overview. A short introduction to the Processing software and projects from the communication [Online] [Cited: April 3, 2014] http://www.processing.org/overview/
Page : 6 of 6
ELEC/CENG 499 – Final Report
[13] Dreamweaver 8 introduction [Online] [Cited: April 3, 2014] http://www.adobe.com/support/documentation/en/dreamweaver/dw8/releasenotes.html