![Page 1: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/1.jpg)
Hi everyone, I’m Rong-Hao, and we are from National Taiwan university and Keio university.Today we are going to talk about FlexiBend, which is a shape-sensing strip that enables interactivity of multi-part, deformable fabrications.
![Page 2: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/2.jpg)
By installing the FlexiBend into a deformable fabrication, The user interactions can be sensed by tracking the shape of sensor.
![Page 3: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/3.jpg)
It also supports fabrications of multiple movable parts, such as widgets.
![Page 4: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/4.jpg)
Several previous works present techniques to enable interactivity of deformable and multi-part fabrications using mechanical, optical, and capacitive sensors.
![Page 5: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/5.jpg)
For these techniques, the hardest part for makers is installing the numerous sensors in correct ways.
![Page 6: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/6.jpg)
Installing single sensor unit is way easier.Possible methods such as active swept-frequency sensing
![Page 7: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/7.jpg)
passive auditory sensing
![Page 8: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/8.jpg)
and time-domain reflectometry sensing.
![Page 9: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/9.jpg)
However, they fail to capture high-dimensional deformations, say, they can only sense one operation at a time.
![Page 10: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/10.jpg)
Using single camera can track multiple movable part, such as Sauron,
![Page 11: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/11.jpg)
but the line-of-sight issues limit the physical design as well.
![Page 12: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/12.jpg)
For both reliable tracking and easy installation, our idea is using a single shape-sensing strip for tracking.
![Page 13: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/13.jpg)
By stuffing the sensing strip to a deformable fabrication,
![Page 14: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/14.jpg)
or a multi-part fabrication, the user interactions can be reliably tracked based on the sensing data. Therefore, we can turn deformable or multi-part fabrications into computer input devices.
![Page 15: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/15.jpg)
First of all, we attempted to develop thin, flexible shape-sensing strips for embedded uses in fabrications. The conventional solution, ShapeTape, turns out to be the possible candidate, but the resolution is not high enough.
![Page 16: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/16.jpg)
Therfore, we further explore other bend sensors, including flexSense, Flex, Bend micros.Finally we found that strain gauge can be an interesting one to try, because it’s the smallest one.
![Page 17: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/17.jpg)
A strain gauge is a thin and small bend sensor typically used for monitoring deformation in rigid structures.Deformation changes the electrical resistance of a strain gauge.
![Page 18: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/18.jpg)
To determine how deformation actually affects the resistance reading of strain gauges. We made six short flexible strips, each of which contains one strain gauge. Each one was fit to a set of plastic molds that described semicircles of 13 different radiuses.
![Page 19: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/19.jpg)
The experimental results show that the sensor readings are linearly related to the bend angles. So, we can move on fabricating a strain gauge array as a shape sensor.
![Page 20: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/20.jpg)
We implement a sixteen-link strain gauge array.
![Page 21: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/21.jpg)
and 3D printed the substrate using the flexible filament, NinjaFlex.
![Page 22: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/22.jpg)
The sensing data of all sixteen strain gauges were collected by a main board consisting of a micro-controller, a ADC, and two multiplexers, in a 45 fps refresh rate
![Page 23: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/23.jpg)
Because it’s resistive-based sensing, the power consumption is relatively low.
![Page 24: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/24.jpg)
For each frame, we capture the sensor data,
![Page 25: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/25.jpg)
subtract the background first
![Page 26: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/26.jpg)
Then, measure the deformation by offset data.
![Page 27: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/27.jpg)
We then obtain the curvature of sensor by mapping the sensor data to each sensor’s response curve, which is obtained from the calibration process. Please read the paper for the details.
![Page 28: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/28.jpg)
Based on the curvature of each sensor, we can construct the shape of FlexiBend.
![Page 29: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/29.jpg)
Now, we have a series of sixteen curvatures
![Page 30: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/30.jpg)
The first thing is to add the spaces between the curvatures. We already know the actual length of space when it is fabricated.
![Page 31: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/31.jpg)
Then, we turn each curvature into several control points, for example, four points.
![Page 32: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/32.jpg)
Finally, we can use NURBS to connect the control points as a resulting shape.
![Page 33: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/33.jpg)
To evaluate the performance of shape construction, we used a DI platform to capture the ground truth with the reconstructed shape.
![Page 34: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/34.jpg)
We tested 17 example shapes, and the result show that the mean position error of each joint is only 7 millimeter. Although the error is cumulated at the end point, the error is still less than 2 cm, that is sufficiently accurate for our application.
![Page 35: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/35.jpg)
To fabricate deformable object for FlexiBend, we 3D print the substrate using NinjaFlex as well. A grove is preserved on the model for installing the FlexiBend.
![Page 36: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/36.jpg)
By placing the FlexiBend into the grove, the shape is captured at the same time. After that, the user interactions can be easily detected by resolving the differences.
![Page 37: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/37.jpg)
For storytelling, a user can change the expression of a virtual seahorse displayed on the screen by bending the physical seahorse puppetry, making it looks confident and proud.
![Page 38: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/38.jpg)
Using common rigid filaments, such as PLA or ABS, can also fabricate multi-part widgets such as buttons.A widget consists of an entrance, a cavity, and an exit for the FlexiBend, and a joint is used to keep the green part movable.
![Page 39: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/39.jpg)
However, it’s not enough. Because the moving part affects the FlexiBend outside the widgets, the challenge turns out to be “how to isolate each widgets”.and that would makes the remaining parts of strain gauge reusable for other widgets.
![Page 40: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/40.jpg)
Our solution, is the locking mechanism.The landmarks on the entrance and exit of the case lock the gear-shape pattern on the edges of the FlexiBend.
![Page 41: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/41.jpg)
The locking mechanism fixed and isolated the segment of FlexiBend, that is loosely mounted within the widgets.Because the FlexiBend is loosely mounted, the green part is still movable.Additionally, the 3D printed physical spring can recover the FlexiBend to its original state.
![Page 42: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/42.jpg)
So that’s how a button works, note that the button is a pressure-sensitive, analog button.
![Page 43: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/43.jpg)
We also design switch, which has no physical spring but a lever-like structure instead
![Page 44: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/44.jpg)
A knob with a rollable handle that deforms the FlexiBend can be used as a dial.
![Page 45: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/45.jpg)
With the same idea, we made a slider as well.
![Page 46: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/46.jpg)
Our experiment result shows that a 3 centimeter slider can reliably distinguish 8 positions,
![Page 47: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/47.jpg)
Furthermore, it shows that the locking mechanism is effective,because the strain gauges outside the widget were not affected at all.
![Page 48: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/48.jpg)
To wrap up, we show how to turn a toy pistol fabrication into an input device for a first-person shooting game.The user installs a flexiBend into the toy pistol, which has a slider and a button.
![Page 49: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/49.jpg)
Then, the user records the states of a the input events, such as the button and the slider positions. Note that the operations on different widgets can be performed simultaneously.
![Page 50: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/50.jpg)
Then, the user can select the weapon by moving the slider, confirm by clicking the button,
![Page 51: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/51.jpg)
then fire and reload the bullets intuitively.
![Page 52: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/52.jpg)
By adding a motion sensor on a FlexiBend, it further supports 3D spatial operations such as wielding the knife.
![Page 53: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/53.jpg)
Also, from the raw data’s perspective, forming a FlexiBend into a spiral shape can sense subtle 3D operations, such as twisting, stretching, and bending it, although the 2D sensor does not know that its a spiral.
![Page 54: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/54.jpg)
Customizing a flexiBend by making it longer and denser allows for high-resolution and simultaneous detection on more widgets, or even sensing human body as SensorTape just did. Making FlexiBend using Flexible printed circuits also make a flexiBend thinner and more durable, because it can reduce the structure deficiencies of fabrication.
![Page 55: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/55.jpg)
Our prototype made by cooper tapes demonstrates this possibility.
![Page 56: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/56.jpg)
Conclusion. We presented FlexiBend, a novel shape-aware strip that brings interactivity to multi-part, deformable objects with ease.
![Page 57: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/57.jpg)
Because it is so easy to install and reusable, it can be an useful tool for iterative prototyping, because a user can easily optimize the 3D model design with the same flexiBend. Future work can also consider to provide modularized 3D modeling tools to accelerate the iterations of design and fabrication. It’s a promising area to further explore.
![Page 58: [UIST 2015] FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications](https://reader030.vdocuments.us/reader030/viewer/2022011721/587b4a171a28abff1a8b4877/html5/thumbnails/58.jpg)
Thanks for your attention, and I’m glad to take all questions.