Single input device for cross-machine manipulation

Cross-machine, cross-platform desktop manipulation with a single set of input devices

Abstract

When working on multiple local computers, switching control across a few machines can be really painful. You either have to pick one machine and to remotely control all the others from there, or pick up another mouse and keyboard for each machine.

This presentation shows a solution to reduce control to a single set of devices and virtually no network connection between machines to be needed.

Contents1. Problems

2. Motivation

3. Idea

4. Graphical demo

5. Extend

6. Hardware

7. Encapsulation

8. Mechanism

9. Conclusions

1. Problems● Working on more machines at the same time

involves a pair of input devices for each one of them

● Switch hand between mice is counterintuitive

● File transfer between machines nearby proves to be quite tedious (especially cross-platform)

2. Motivation● Improve productivity by keeping hands on

the same mouse and keyboard

● Flawless and intuitive drag-and-drop file transfer and cross-machine copy-paste

● Minimize changes and additions needed for the system to work

2. Motivation● Why is this different from "remote desktop

control"?

○ No output overhead:● Doesn't stream the whole output from the "remote" machine

(including video)○ Faster:

● Remote-Control usually works over Internet which is a few orders of magnitude slower

○ Accessibility:● To improve speed, RC can run on LAN, but this involves

more configs

3. Idea● Usual input devices to work would be great,

but would involve complex machines config.

● So, empower the mouse○ Mouse = server○ Machines = clients

● The mouse is connected to all the computers and reasons about who it is currently controlling

4. Graphical demo

Pointer initially active on M1 (Machine1)

Graphical demo

Transition between displays of the same machine is naturally handled by the OS

Graphical demo

We want to be able to jump to the other machine

The right (configurable) edge triggers the jump

The pointer becomes inactive on M1 and active on M2

Now the mouse controls M2

M2 has left screen edge configured to trigger a jump

You've been waiting for something to happen to that folder on M1, right?

Can do drag-and-drop file transfer cross-machine(even cross-platform)

Drag it all the way into the edge of screen

Activates the jump and file transfer too(more details later)

File transfer operation (like the usual one) initiates

Can jump around from a machine to the other...

... just like the jump between screens on the same OS

5. Extend

● The system can be extended to a virtually unlimited number of machines

(For more than 4 neighbours to each machine, some other trigger could be used, instead of screen edges)

6. Hardware● Although all the system's complexity is in the

mouse, there is no innovation required○ most "expensive" mice have internal memory○ small microcontroller or microprocessor○ duplex communication through wireless or Bluetooth○ one separate communication channel for each

machine

● (The keyboard works similarly, so the discussion will be limited to the mouse device)

7. Encapsulation● Installation should be piece of cake:

○ Install driver and software○ Establish connection to mouse

■ Either insert one of the wireless receivers■ Either pair through Bluetooth

○ Configure machines layout (visually)

● The driver delegates everything, keeping the complexity as much as possible encapsulated inside the mouse

8. Mechanism● Mouse behaves normally● Screen edge is reached

○ Driver on the active machine (M-old) sends notification to mouse

● Mouse switches the output channel to the new active machine (M-new)○ M-old becomes inactive by default, being ignored○ Mouse movement and actions are received only by

M-new until a new jump event is encountered

8. Mechanism

Even file data is transferred: M1 -> Mouse -> M2

8. Mechanism (files)

● Pointer activates a screen edge on M1 while dragging a file○ Software driver on M1 notifies the mouse about the

jump and also requests to initiate file transfer○ Mouse switches output channel to M2 and notifies

driver on M2 about the file transfer● M2 accepts the transfer

○ M1 starts streaming data to mouse○ Mouse forwards data to M2○ On M2 additional file-system related changes are

applied

9. Conclusions● Usability gain comes with some additional

cost○ If the usual workflow involves more machines on the

same desk, it should be worth it

● The system could be extended such as windows and processes could be dragged across different machines○ Better parallelism results from this

Single input device for cross-machine manipulation

This presentation is meant to propose a solution to an unsolved problem at the moment (although many work-arounds had been implemented)

Iulius Curt© february 2013

Custom made illustrations © Iulius Curt