mobile device development camera and sensors dr.yingliang ma
TRANSCRIPT
UFCFX5-15-3Mobile Device Development
Agenda
Android Camera Overview
Simple android app using camera
Using Camera API
Using
UFCFX5-15-3Mobile Device Development
Camera Specifications 1. Camera resolutions (image sensors)
2. Optical Zoom or Digital Zoom
3. Xenon flash or LED flash
4. Video capture resolutions (4K??)
5. Face recognition
6. Low-light performance
7. Anti-shake performance
UFCFX5-15-3Mobile Device Development
Camera Quality variation 1. Camera resolutions
2. With or without flash
3. No face-forward camera
4. No advance features
UFCFX5-15-3Mobile Device Development
Simple Camera App
Using existing android camera application in our application
Intent intent = new Intent(android.provider.MediaStore.ACTION_IMAGE_CAPTURE);
1 ACTION_IMAGE_CAPTURE_SECURE
It returns the image captured from the camera , when the device is secured
2 ACTION_VIDEO_CAPTURE
It calls the existing video application in android to capture video
3 EXTRA_SCREEN_ORIENTATION
It is used to set the orientation of the screen to vertical or landscape
4 EXTRA_FULL_SCREEN
It is used to control the user interface of the ViewImage
UFCFX5-15-3Mobile Device Development
UI Design<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">
<ImageView
android:id="@+id/imageView1"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:contentDescription="@string/hello_world"
android:src="@drawable/ic_launcher" />
<TextView
android:id="@+id/textView1"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_alignParentTop="true"
android:layout_alignRight="@+id/imageView1"
android:text="@string/tap"
android:textAppearance="?android:attr/textAppearanceLarge" />
</RelativeLayout>
UFCFX5-15-3Mobile Device Development
String definition
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="app_name">Camera</string>
<string name="action_settings">Settings</string>
<string name="hello_world">Hello world!</string>
<string name="tap">Tap the image to open the camera!!</string>
</resources>
UFCFX5-15-3Mobile Device Development
Main Activitypublic class MainActivity extends Activity {
ImageView imgFavorite;
@Override
protected void onCreate(Bundle savedInstanceState) {
imgFavorite.setOnClickListener(new OnClickListener() {
@Override
public void onClick(View v) {open(); }
});
}
public void open(){
Intent intent = new Intent(android.provider.MediaStore.ACTION_IMAGE_CAPTURE);
startActivityForResult(intent, 0);
}
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
// TODO Auto-generated method stub
super.onActivityResult(requestCode, resultCode, data);
Bitmap bp = (Bitmap) data.getExtras().get("data");
imgFavorite.setImageBitmap(bp);
}
}
UFCFX5-15-3Mobile Device Development
Camera API app
We will be using the camera API to integrate the camera in our application
Class Description
Camera It is used to control the camera and take images or capture video from the camera
SurfaceView This class is used to present a live camera preview to the user.
UFCFX5-15-3Mobile Device Development
UI Design<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="fill_parent"
android:layout_height="fill_parent"
android:orientation="horizontal" >
<LinearLayout
android:layout_width="wrap_content"
android:layout_height="match_parent"
android:layout_weight="0.30"
android:orientation="vertical" >
<FrameLayout
android:id="@+id/camera_preview"
android:layout_width="fill_parent"
android:layout_height="400dp" />
<Button
android:id="@+id/button_capture"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:onClick="snapIt"
android:text="@string/Capture" />
<ImageView android:id="@+id/imageView1" android:layout_width="fill_parent" android:layout_height="fill_parent" android:scaleType="fitXY" android:src="@drawable/ic_launcher" />
</LinearLayout>
</LinearLayout>
UFCFX5-15-3Mobile Device Development
Android Manifest file<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.camera1"
android:versionCode="1"
android:versionName="1.0" >
<uses-sdk
android:minSdkVersion="8"
android:targetSdkVersion="17" />
<uses-permission android:name="android.permission.CAMERA"/>
<uses-feature android:name="android.hardware.camera" />
<uses-feature android:name="android.hardware.camera.autofocus" />
<application
android:icon="@drawable/ic_launcher"
android:label="@string/app_name" >
<activity
android:name="com.example.camera1.MainActivity"
android:label="@string/app_name"
android:configChanges="orientation"
android:screenOrientation="portrait">
UFCFX5-15-3Mobile Device Development
Structure
MainActivity ShowCamer
a
Preview windows
Main UIFrameLayout AddView
PictureCallbackTake photo
UFCFX5-15-3Mobile Device Development
Show Camera Activitypublic class ShowCamera extends SurfaceView implements SurfaceHolder.Callback {
private SurfaceHolder holdMe;
private Camera theCamera;
public ShowCamera(Context context,Camera camera) {
super(context);
theCamera = camera;
holdMe = getHolder();
holdMe.addCallback(this);
}
@Override
public void surfaceCreated(SurfaceHolder holder) {
try {
theCamera.setPreviewDisplay(holder);
theCamera.startPreview();
} catch (IOException e) {
}
}
@Override
public void surfaceDestroyed(SurfaceHolder arg0) {
}
}
UFCFX5-15-3Mobile Device Development
Main Activity
public class MainActivity extends Activity {
private Camera cameraObject;
private ShowCamera showCamera;
private ImageView pic;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
pic = (ImageView)findViewById(R.id.imageView1);
cameraObject = isCameraAvailiable();
showCamera = new ShowCamera(this, cameraObject);
FrameLayout preview = (FrameLayout) findViewById(R.id.camera_preview);
preview.addView(showCamera);
}
UFCFX5-15-3Mobile Device Development
Camera thread
private PictureCallback capturedIt = new PictureCallback() {
@Override
public void onPictureTaken(byte[] data, Camera camera) {
Bitmap bitmap = BitmapFactory.decodeByteArray(data , 0, data .length);
if(bitmap==null){
Toast.makeText(getApplicationContext(), "not taken", Toast.LENGTH_SHORT).show();
}
else
{
Toast.makeText(getApplicationContext(), "taken", Toast.LENGTH_SHORT).show();
pic.setImageBitmap(bitmap);
}
cameraObject.startPreview();
}
};
UFCFX5-15-3Mobile Device Development
Save the picture Put SD card write permission in android Manifest file<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
In PictureCallback function Toast.makeText(getApplicationContext(), "taken", Toast.LENGTH_SHORT).show();
pic.setImageBitmap(bitmap);
String path = Environment.getExternalStorageDirectory().toString();
String filename = path + "/SpyApp"+String.format("%d.jpg", System.currentTimeMillis());
FileOutputStream outStream = null;
try {
outStream = new FileOutputStream(filename);
outStream.write(data);
outStream.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
}
UFCFX5-15-3Mobile Device Development
See the picture files Download File Manage from Google Play Store
UFCFX5-15-3Mobile Device Development
Record audio
If you want to have more control over audio recording you could use the MediaRecorder.
UFCFX5-15-3Mobile Device Development
Example codes
Set audio permission in android Manifest file<uses-permission android:name="android.permission.RECORD_AUDIO"/>
MediaRecorder recorder = new MediaRecorder();
recorder.setAudioSource(MediaRecorder.AudioSource.MIC);
recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);
recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);
recorder.setOutputFile(PATH_NAME);
recorder.prepare();
recorder.start(); // Recording is now started
...
recorder.stop();
recorder.reset(); // You can reuse the object by going back to setAudioSource() step
recorder.release(); // Now the object cannot be reused
UFCFX5-15-3Mobile Device Development
Add them to MainActivity
Set audio permission in android Manifest file<uses-permission android:name="android.permission.RECORD_AUDIO"/>
protected void onCreate(Bundle savedInstanceState) {
…..
///record audio
String path = Environment.getExternalStorageDirectory().toString();
String filename = path + "/SpyAudio"+String.format("%d.3gp", System.currentTimeMillis());
recorder = new MediaRecorder();
recorder.setAudioSource(MediaRecorder.AudioSource.MIC);
recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);
recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);
recorder.setOutputFile(filename);
try {
recorder.prepare();
} catch (IllegalStateException e) {e.printStackTrace();}
recorder.start(); // Recording is now started
UFCFX5-15-3Mobile Device Development
Stop recording@Override
public boolean onKeyDown(int keyCode, KeyEvent event) {
//Handle the back button
if(keyCode == KeyEvent.KEYCODE_BACK) {
recorder.stop();
recorder.reset(); // You can reuse the object by going back to setAudioSource() step
recorder.release(); // Now the object cannot be reused
return true;
}
else
return super.onKeyDown(keyCode, event);
}
UFCFX5-15-3Mobile Device Development
Use Sensor in Android App
Motion sensors
These sensors measure acceleration forces and rotational forces along three axes. This category includes accelerometers, gravity sensors, gyroscopes, and rotational vector sensors.
Environmental sensors
These sensors measure various environmental parameters, such as ambient air temperature and pressure, illumination, and humidity. This category includes barometers, photometers, and thermometers.
Position sensors
These sensors measure the physical position of a device. This category includes orientation sensors and magnetometers.
UFCFX5-15-3Mobile Device Development
Use Sensor in Android App
You can access sensors available on the device and acquire raw sensor data by using the Android sensor framework.
Determine which sensors are available on a device.Determine an individual sensor's capabilities, such as its maximum range, manufacturer, power requirements, and resolution.Acquire raw sensor data and define the minimum rate at which you acquire sensor data.Register and unregister sensor event listeners that monitor sensor changes.
UFCFX5-15-3Mobile Device Development
Sensor availablity
Few Android-powered devices have every type of sensor.
For example, most handset devices and tablets have an accelerometer and a magnetometer, but fewer devices have barometers or thermometers.
Also, a device can have more than one sensor of a given type. For example, a device can have two gravity sensors, each one having a different range.
Identifying Sensors and Sensor Capabilities
private SensorManager mSensorManager;
mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE);List<Sensor> deviceSensors = mSensorManager.getSensorList(Sensor.TYPE_ALL);
UFCFX5-15-3Mobile Device Development
Monitoring Sensor Eventspublic class SensorActivity extends Activity implements SensorEventListener {
private SensorManager mSensorManager;
private Sensor mLight;
@Override
public final void onSensorChanged(SensorEvent event) {
// The light sensor returns a single value.
// Many sensors return 3 values, one for each axis.
float lux = event.values[0];
// Do something with this sensor value.
}
}
UFCFX5-15-3Mobile Device Development
Handling Different Sensor Configurations
private SensorManager mSensorManager;
...
mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE);
if (mSensorManager.getDefaultSensor(Sensor.TYPE_PRESSURE) != null){
// Success! There's a pressure sensor.
}
else {
// Failure! No pressure sensor.
}
Using Google Play filters to target specific sensor configurations
<uses-feature android:name="android.hardware.sensor.accelerometer"
android:required="true" />
UFCFX5-15-3Mobile Device Development
Best Practices for Accessing and Using Sensors
Unregister sensor listeners
Don't test your code on the emulator
Don't block the onSensorChanged() method (do as little as possible within the onSensorChanged(SensorEvent) method as sensor data can change at a high rate)
Avoid using deprecated methods or sensor types
Verify sensors before you use them
Choose sensor delays carefully
UFCFX5-15-3Mobile Device Development
Using the Proximity Sensorprivate SensorManager mSensorManager;
private Sensor mSensor;
...
mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE);
mSensor = mSensorManager.getDefaultSensor(Sensor.TYPE_PROXIMITY);
@Override
public final void onSensorChanged(SensorEvent event) {
float distance = event.values[0];
// Do something with this sensor data.
}
UFCFX5-15-3Mobile Device Development
Using the Orientation Sensor
The orientation sensor lets you monitor the position of a device relative to the earth's frame of reference (specifically, magnetic north).
private SensorManager mSensorManager;
private Sensor mSensor;
...
mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE);
mSensor = mSensorManager.getDefaultSensor(Sensor.TYPE_ORIENTATION);
@Override
public void onSensorChanged(SensorEvent event) {
float azimuth_angle = event.values[0];
float pitch_angle = event.values[1];
float roll_angle = event.values[2];
// Do something with these orientation angles.
}
UFCFX5-15-3Mobile Device Development
Using the Step Counter Sensor
The step counter sensor provides the number of steps taken by the user since the last reboot while the sensor was activated. The step counter has more latency (up to 10 seconds) but more accuracy than the step detector sensor.
private SensorManager mSensorManager;
private Sensor mSensor;
...
mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE);
mSensor = mSensorManager.getDefaultSensor(Sensor.TYPE_STEP_COUNTER);
UFCFX5-15-3Mobile Device Development
Environment SensorsUnlike most motion sensors and position sensors, which return a multi-dimensional array of sensor values for each SensorEvent, environment sensors return a single sensor value for each data event.
TYPE_AMBIENT_TEMPERATURE event.values[0] °C Ambient air temperature.
TYPE_LIGHT event.values[0] lx Illuminance.
TYPE_PRESSURE event.values[0 hPa or mbar Ambient air pressure.
TYPE_RELATIVE_HUMIDITY event.values[0] % Ambient relative humidity.
TYPE_TEMPERATURE event.values[0] °C Device temperature.