2014 - 2015 Texas State Electrical EngineeringTyler Lyssy, Kyle Paul, Alex Leon

PixelSense Interface

Problem Statement

• NASA requires an expandable interface for controlling and monitoring the different systems required for future space flight.

Project Overview

• Texas State University will develop a multiple user interface that uses not only touch based interaction, but also allows the user to place physical objects on the screen as an alternative form of interaction.

Design Requirements

• Must be able to allow up to four users to interact with the interface simultaneously.

• The system will show statuses and updates in an organized and orderly fashion.

• Must use tagged objects as input into the system.• The system will also sync with the Smart Watch to use for

input.• System must be designed to allow for easy portability to

NASA’s systems.• The System must be expandable.

Hardware

5

Protection Glass

LCD Glass Panelwith Integrated

Sensors

Optical Sheets

Backlight withVisible and IR LEDs

EmbeddedPC/Housing

PixelSenseProcessing

Samsung SUR40

Hardware

6

Hardware

Safety

8

• The main safety concern is the weight of the SUR40.

• It weighs roughly 80 lbs. and is difficult to move.• The aluminum stand it rests on allows it to lay flat

or at an angle.• Adjusting the viewing angle should be performed

by two people.• The aluminum frame also has some sharp edges

that could be potential hazards.

Ethical and Privacy Issues

• The interface would be used by NASA crew members, with its primary purpose is to increase efficiency, so no ethical issues are present.

• With further development, each user has the capability to hide certain information of themselves from the crew tab by using their password protected Smartwatch for privacy concerns.

Ethics and Privacy

Software

• The SUR40 is currently running windows 7.• Tried to install windows 8 with no success.• Microsoft surface 2 SDK. • Microsoft Visual Studio IDE for development.

Why?

• The SUR40 can detect objects placed on the screen

• This gives an additional level of interaction and could increase productivity.

11

How?

12

• The Samsung SUR40 uses special integrated PixelSense sensors and infrared LED’s to track tags.

• The tags have a predefined shape and 8-bit mapping scheme.

• 8 bit tag values allow for 256 different tags.• Each tag has a black infrared absorbent

background and a pattern of white infrared reflective circles.

Tag Dimensions

13

0.25 in.

0.16 in.

0.28 in.

0.14 in.

0

1

2

34

5

6

7

00010111 = 0x17

= Data bit circle; White = 1 Black = 0

= 3 outer circles, Used for orientation

= Center circle, used for location (X,Y)

Pros vs. Cons

14

Pro Con• Adds unique level of

interaction.• Wide variety of

applications.• 256 unique tags.• Methods for manipulating

the tags are already written by Microsoft.

• IR Sensors are very sensitive to ambient lighting.

• Tags must lay flat on surface.

• Limited to Surface 2 SDK and windows 7.

• PixelSense is no longer supported by Microsoft.

• Will not work for small objects.

15

Object Recognition

• We will be tracking two separate types of items parts and food.

• Each item will have its own tag ranging from value 0 - 255.

• Parts will range from 0 - 127 and food will range from 128 - 255.

• When a user place an item on the display a window will pop up displaying information for the item.

16

Parts

Picture of Parts interface

• The Part interface will show relevant information for the part such as the name, size, mass, and location that they are stored in.

• There will also be two buttons that will allow the user to add or remove and item from the inventory or watch a video on how the part is used/installed.

17

Food

Picture of food interface

• The Food interface will show specific information for the food item including the name, quantity, weight, and sodium values.

• This window will also have a set of buttons that will allow the user to add or remove the item from inventory or to view a video on how to prep the meal.

Design

Zones

1

32

4

Design

• When a user presses one of the four buttons in their respective zone a window will appear with one of four windows depending on which button is pressed.

• Once the window is up the user can toggle between the different views using the buttons.

• Only one window can be open at any time per user.• The window can be moved, rotated, and resized.• When the window is opened again it will reappear in its

respective zone.

Schedule

• The Schedule will show all of tasks currently scheduled for the day for all the crew members.

• Users will be able to add, remove, or modify tasks in later updates.

Picture of schedule interface

Systems

22

• The window will have selector buttons along the sides to choose which system to view. Once the user selects a system they will be able to monitor and manipulate system specific information.

• Systems will include C&DH, C&T, ECLS, EPS, GNC and Thermal.

• The visuals for the system will be place holders for the actual systems interfaces.

Picture of Systems interface

Crew

• The window while display the entire crew with basic non private medical information to help monitor crew health.

• Further development for the Crew window would allow the user to be able to log in using there smart watch to edit what information being displayed.

Picture of Crew interface

Vehicles

24

• The final button will display a window that will show vehicle specific information. The user can monitor vehicles and vehicle systems such as fuel levels, tire pressure levels, etc.

• The window will have a column of buttons along the side to choose which vehicle to monitor.

Picture of Vehicles interface

• To test the functionality of the interface, multiple tags were placed on the SUR40 as well as going through and testing every button that was available.

Validation Test Plan

26

Communications

• The PixelSense and SmartWatch will communicate with a WAMP (Windows Apache MySQL PHP) server.

• All of the the information for the systems, parts, food, etc. will be stored in a database on the server.

• The Server will be hosted on a machine provide by Texas State

• Protocols used for WAMP data transfer include HTTP and PHP.

Project Timeline

Preliminary Design Review

Place Holders For Vehicles and Systems Module completed

Integration with server completed

Completion of Parts and Food

Interface

Critical Design Review

Senior DesignDay

Install and Documentation

Turnover

2/18/15

3/11/15

3/18/15

4/1/15

4/18/15

5/8/15

5/15/15Feb. May

Crew and Schedule Module Completed

4/22/15

Turnover Documentation

28

• This will include a User Guide, the Requirements and ConOps (Concept of Operations).

• The User Guide will be in PDF format and will include diagrams demonstrating the different features of the interface.

Stretch Goals

29

• We would like to add an entertainment aspect to the interface, a game of some sort or activity that can promote teamwork and camaraderie as well as provide a release at the end of the day.

Outreach

• Texas State Senior Design Day (Dec. 2014 & May 2015)– Poster display which provided information about

the project.– Device setup for hands-on demonstration

• Texas State University Newspaper– A sectional piece describing the project.– Informational video created about the project

Cost Analysis

SUR40 ~$8,000 (current hardware on loan from NASA)

Microsoft Visual Studio

~$600 (Free with Texas State DreamSpark account)

Printing cost for tags

<$10 or ( Texas State has a Vinyl printer )

2 Surface Pro 3 tablets for

development~$1600

• In the construction of the interface, Texas State has created expandable interface for controlling and monitoring the different systems required for future space flight as well as utilizing tag recognition on objects for time efficiency and ease of access.

Conclusion