advancing the ui — part 1: look, motion, and gestures

#SDC13

Advancing the UI – Part I:Look, Motion, and Gesture

Anna SchallerSr. Manager, Developer RelationsSamsung Media Solutions Center America



Creating Moments of Delight



What’s Next for the UI?

Mobile SDK– Look – Gestures– Motion– MultiWindow– Visual View

Part I

Part II

SAMSUNG DEVELOPERS CONFERENCE

Look



A “Look” at features

Writing Buddy– Hand writing recognition extension to UI elements

Air Button– Quick access menu for various arrays of data

Smart Clip– Clipboard capture with metadata recognition

Pointer Icon– Extends icon for hover pointer to user-defined

images



Architecturally speaking……

View System modified to support extensions to Android UI elements– New default behaviors (Writing Buddy enabled on all

editable fields)– New packaged library with added functionality to enhance

your UI

APPLICATION FRAMEWORK

Activity Man-ager

Content Provider

Window Man-ager

View System Slook Impl

Telephony Man-ager

Resource Man-ager

Notification Man-ager

…..



Look implementation

SlookJAR

APPLICATION FRAMEWORK

View System

APPLICATION

SlookImplement

MyBeautifulApp

WritingBuddyImpl AirButtonImpl SmartClipImpl HoverPointIconImplSlookImpl

SlookWritingBuddy SlookAirButton SlookSmartClip SlookHoverPointIconSlook

Application

SDK Module

Implement

1.Request(Call Public API)2.Check Feature3.Check version

5.Request(Call Internal API)

7.Response

6.Response



Samsung Mobile SDK Libraries

New APIs to check for presence and properties of technology



@Overridepublic void onCreate(Bundle savedInstanceState) {

Slook slook = new Slook(); try { slook.initialize(this); } catch (SsdkUnsupportedException e) {

// notify user device does not support return;

} if (slook.isFeatureEnabled(Slook.AIRBUTTON) if (slook.isFeatureEnabled(Slook.SPEN_HOVER_ICON) if (slook.isFeatureEnabled(Slook.SMARTCLIP) if (slook.isFeatureEnabled(Slook.WRITINGBUDDY) {

Slook<class> variable = new Slook<class> }

Implementing Look Features



Writing Buddy

On by default for editable fields (EditText, DatePicker) Disable system wide with Direct pen input turned off Disable on app-by-app basis through look library



Writing Buddy Extensions

SLookWritingBuddy class– 2 constructors: ViewGroup and EditText

Support for text and images– 2 Listeners with callbacks



Writing Buddy Styles

EditText (text only) (default)

EditText (text and image toggle)

ViewGroup



SlookWritingBuddy mWritingBuddy;EditText editInput;ImageView mOutputImageView;

public void onCreate(. . . ) {

editInput = (EditText) findViewById(R.id.edit_input);mOutputImageView = (ImageView) findViewById(R.id.image_output);

mWritingBuddy = new SlookWritingBuddy(editInput);

mWritingBuddy.setImageWritingListener( new SlookWritingBuddy.ImageWritingListener() {

public void onImageReceived(Bitmap arg0) {mOutputImageView.setBackground(

new BitmapDrawable(getResources(), arg0));

}});

Implementing Writing Buddy – Text and Image



public void onCreate(. . . ) { FrameLayout fl = (FrameLayout) findViewById(R.id.input);

mOutputTextView = (TextView) findViewById(R.id.text_output); mWritingBuddy = new SlookWritingBuddy(fl);

mWritingBuddy.setTextWritingListener(new SlookWritingBuddy.TextWritingListener() {

public void onTextReceived(CharSequence arg0) { mOutputTextView.setText(arg0); } });

<LinearLayout><FrameLayout>

<SomeView></FrameLayout><TextView>

</LinearLayout>

Implementing Writing Buddy – ViewGroup



// Turn recognition on for numeric or alphanumericmType = SlookWritingBuddy.TYPE_EDITOR_NUMBER;mType = SlookWritingBuddy.TYPE_EDITOR_TEXT;mWritingBuddy.setEditorType(mType);

// enable or disable Writing BuddymWritingBuddy.setEnabled(mIsEnabled); // true or false

Writing Buddy Miscellaneous Features



Air Button

Activated with S Pen hover + side button Defaults to “Air Command” system wide Override behavior inside app

– com.samsung.android.sdk.look.airbutton Implemented with an ArrayAdapter

– Array of “data” attached to view– Array data can be text, images, contacts, video, audio



Data Types

Images

ContactsMenus



protected void onCreate(Bundle savedInstanceState) {setContentView(R.layout.activity_airbutton_simple);

Button btnText = (Button) findViewById(R.id.btn_text); createTextListWidgetFromView(btnText);} public SlookAirButton createTextListWidgetFromView(View v) { SlookAirButton airButtonWidget = new SlookAirButton(

v, // ButtongetAdapterStringList(), // Adapter SlookAirButton.UI_TYPE_LIST); // Data type

airButtonWidget.setItemSelectListener(mCallback); airButtonWidget.setPosition(0, 50); return airButtonWidget;}

Air Button -- Text



public SlookAirButtonAdapter getAdapterStringList() {ArrayList<AirButtonItem> stringList =

new ArrayList<AirButtonItem>();

stringList.add(new AirButtonItem(null, “Come here at 5:00", null));

stringList.add(new AirButtonItem(null, "Why?", null));stringList.add(new AirButtonItem(null,

"I wanna watch movie", null));stringList.add(new AirButtonItem(null,

"Ok. No problem", null));

return new SlookAirButtonAdapter(stringList);}

Creating the list of Air Button Strings



new SlookAirButton(View parentview, SlookAirButtonAdapter adapter,int uiType)

where uiType is: SlookAirButton.UI_TYPE_LIST // use with text and media arrays SlookAirButton.UI_TYPE_MENU // use with menu arrays

// ArrayAdapter ItemsAirButtonItem(Drawable image, // R.drawable.image or null,

String description, // Text or nullObject data) // null

new AirButtonItem(null, “text”, null) // text onlynew AirButtonItem(R.drawable.png, “text”, null) // image and text

Data Types



get/setGravity(int g) // within viewSlookAirButton.GRAVITY_LEFTSlookAirButton.GRAVITY_RIGHTSlookAirButton.GRAVITY_TOPSlookAirButton.GRAVITY_BOTTOM

get/setDirection(int d) // for adapterSlookAirButton.DIRECTION_UPPERSlookAirButton.DIRECTION_LOWERSlookAirButton.DIRECTION_RIGHTSlookAirButton.DIRECTION_LEFT

get/setPosition(int x, int y) // coordinates

Positioning the List

y

x



AirButton -- Gravity versus Direction



// lets user scroll through listssetScrollEnabled(boolean enable)

// animates the elements in the list when it openssetBounceEffectEnabled(boolean enable)

// animates the list when hovering over elementsstart/stopAnimationIcon(MotionEvent event)

ACTION_HOVER_ENTER | ACTION_HOVER_EXIT

More . . .



Smart Clip



Smart Clip Features

Default behavior– Captures the text and images in ImageView and TextView

Extensions with Look– Meta-data can now be included with the image clip– Meta-data includes URLs, strings, as well as the launch

intent for your activity– Meta data can be Static or Dynamic



public void onCreate(. . .) {

TextView tv = (TextView) findViewById(R.id.text_static);SlookSmartClip sc = new SlookSmartClip(tv);sc.clearAllMetaTag();

sc.addMetaTag(new SlookSmartClipMetaTag (SlookSmartClipMetaTag.TAG_TYPE_URL,

"http://www.samsung.com"));

sc.addMetaTag(new SlookSmartClipMetaTag (SlookSmartClipMetaTag.TAG_TYPE_PLAIN_TEXT, "This is android textview."));

}

public void someRoutine() {sendData();

}

Simple Use Case -- Scrapbook



public void sendData() { Intent intent = new Intent(); intent.setComponent(new

ComponentName("com.samsung.android.app.pinboard", "com.samsung.android.app.pinboard.ui.PinboardActivity")); try { startActivity(intent); } catch (ActivityNotFoundException e) { Toast.makeText(SmartClipActivity.this, ”Scrapbook application is not installed.”,

Toast.LENGTH_SHORT).show(); }}

Simple Use Case – Scrapbook (con’t)



Pointer Icon

Writing Buddy icon overrides pointer icon– Cannot redefine hover icon for these

fields without disabling writing buddy

Set pointer icon for the View you want it to appear with – Can be ViewGroup or View



Pointer Icon – Adding in codepublic class MainActivity extends Activity {

SlookPointerIcon mPointerIcon = new SlookPointerIcon();

@Overrideprotected void onCreate(Bundle savedInstanceState) {

super.onCreate(savedInstanceState);setContentView(R.layout.activity_main);

RelativeLayout panel = (RelativeLayout) findViewById(R.id.container);

mPointerIcon.setHoverIcon (panel, getResources().

getDrawable(R.drawable.x_pointer_icon));

}

SAMSUNG DEVELOPERS CONFERENCE

LOOK Demo

SAMSUNG DEVELOPERS CONFERENCE

Gesture



What is Gesture?

Gesture allows you to use events generated by user’s hand movements in your application

Recognizes hand movements using data from the gesture sensor to the right of the device speaker

Requirements:– Gesture sensor– Android 4.3 (Jelly Bean API level

18) or higher



Gesture Package



public void onCreate() {private Sgesture mGesture;private SgestureHand mHand;

mGesture = new Sgesture();

try {mGesture.initialize(this);

} catch (IllegalArgumentException e) {//Error Handling

} catch (SsdkUnsupportedException e) {//Error Handling

}

if(mGesture.isFeatureEnabled(Sgesture.TYPE_HAND_PRIMITIVE)){mHand = new SgestureHand(Looper.getMainLooper(), mGesture);mHand.start(Sgesture.TYPE_HAND_PRIMITIVE, mChangeListener);

}

}

Implementation -- Check for support of feature



private final SgestureHand.ChangeListener mChangeListener = new SgestureHand.ChangeListener() {

@Overridepublic void onChanged(Info info) {

Log.d(TAG, “Hand : angle = “ + info.getAngle());// do something with data

}};

public void onStop() {mGestureHand.stop();

}

Register Listener

0º / DOWN

180º / UP

90º / RIGHT270º / LEFT



Reporting Angles

SAMSUNG DEVELOPERS CONFERENCE

Motion



What is Motion?

Package allows you to take advantage of movement data coming in from sensors for both “calling” motion and “forward” motion (walking or running)



“Calling” motion

Requirements:– Proximity sensor, acceleration sensor,

and a gyro sensor– Android 4.3 (Jelly Bean API level 18) or

higher

Recognizes the motion of bringing device up to your ear When you read a message on your device and bring the

device up to your ear, the call motion event is triggered, and your application calls the phone number that sent the message.



“Forward” motion

Pedometer data generated when walking/running with device (steps, calories, speed, distance, and step status)

Requirements:– Acceleration sensor, pressure sensor– If no pressure sensor, number of

up/down steps is not recognized– Android 4.3 (Jelly Bean API level 18) or

higher



Motion Library

Call data

Forward data



Implementation -- Check for support of featurepublic void onCreate() {

private Smotion mMotion;private SmotionCall mCall; // SmotionPedometer

mMotion = new Smotion();

try {mMotion.initialize(this);

} catch (IllegalArgumentException e) {//Error Handling

} catch (SsdkUnsupportedException e) {//Error Handling

}

if(mMotion.isFeatureEnabled(Smotion.TYPE_CALL)){ // TYPE_PEDOMETER mCall = new SmotionCall(Looper.getMainLooper(),mMotion);

mCall.start(Smotion.TYPE_CALL, mChangeListener);

}}



private SmotionCall.ChangeListener mChangeListener = new SmotionCall.ChangeListener() {

@Overridepublic void onChanged(Info info) {

int position = info.getCallPosition(); switch (position){

case SmotionCall.POSITION_LEFT:Log.d(TAG, "Call motion happened! : left");break;

case SmotionCall.POSITION_RIGHT:Log.d(TAG, "Call motion happened! :

Right");break;

default:break;

}}

};

Register Listener – “Call” Motion



private SmotionPedometer.ChangeListener mChangeListener = new SmotionPedometer.ChangeListener() {

@Overridepublic void onChanged(Info info) { 

Log.d(TAG, “Pedometer : calorie = “ + info.getCalorie());Log.d(TAG, “Pedometer : distance = “ +

info.getDistance());Log.d(TAG, “Pedometer : speed = “ + info.getSpeed());Log.d(TAG, “Pedometer : step = “ +

info.getCount(SmotionPedometer.Info.COUNT_TOTAL));Log.d(TAG, “Pedometer : status = “ + info.getStatus());

}};

Register Listener – “Forward” Motion



Sensing the data

SDK

SensorHub

Pedometerhappened 1 step 1 step 1 step 1 step 1 step 1 step 1 step 1 step 1 step

…

…

…Screen off

Application

…



Getting the latest Pedometer data

Screen : OFF

Screen : ONgetInfo()

Application SDK

The latest pedometer info.

getInfo()

null

SDK just knows the pedometer information

before screen is off.It can’t know whether the data is the latest

data or not.So in this case it returns

null.

mPedometer.getInfo(); mPedometer.updateInfo();

Screen : OFF

Application SDK

updateInfo()

The latestpedometer info

SensorHub

Request

The latest dataYou can receive the latest pedometer info through ChangeListener though

screen is off.



What’s next…….

Advancing the UI -- Part II: MultiWindow and Visual View– Tuesday 9:00– California West

Thank You!



License NoticesExcept where noted, sample source code written by Samsung Electronics Co. Ltd and provided to you is licensed as described below. Copyright © 2010-2013, Samsung Electronics Co. Ltd. All rights reserved except as otherwise explicitly indicated. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials

provided with the distribution. Neither the name of the Samsung Electronics Co. Ltd nor the names of its contributors may be used to endorse or promote products derived from this software without specific

prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE

IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Other source code displayed in this presentation may be offered under other licenses. Apache 2.0 Copyright © 2010, Android Open Source Project. All rights reserved unless otherwise explicitly indicated. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Creative Commons 3.0 Attribution License Portions of this presentation are reproduced from work created and shared by Google (http://code.google.com/policies.html) and used according to terms described in the Creative Commons 3.0 Attribution License (http://creativecommons.org/licenses/by/3.0/).