MediaQ1 Introduction

MediaQisanonlinemediamanagementframeworkthatprovideservicestocollect,organizeandshareuser-generatedmobilevideos/imageswithautomaticallytaggedgeo-spatialmetadata.Itcouldbeusedindisasterresponsesituationsuchasearthquakeorfireplace.Thegovernmentandanyhelpingassociationunitcangetthefirstin-placelooktohelpmaketheaccuratedecision.Consideringthebroadcastofnetworkincrowdedplace,afterusergeneratesvideo/photo,MediaQalwaysuploadthesmall-sizemetadatafirstwhichcontaincapturetime,cameralocation,trajectoryandviewingdirection.Thenasanyoneinterestedinthatvideo/photo,weuseGoogleCloudMessagetomanagethecontentqueryfromotheruserandsendnotificationtothecontentowner.Therefore,thegoalofourfinalprojectistodevelopsuchfeaturetogetherwithphotographyuploadinginAndroidphones.

Figure1MediaQHomeScreen

2 BasicFunctions2.1 Recordavideoandtakeapicture

Userscanrecordavideoandtakeapicturelikeothercameraapp.Whiledoingit,ourappsavesthemetadatasuchasGPS,Compassdata,Internetinformationanddeviceinformationinthebackgroundside.

Figure2.Screenshotofcamera2.2 Automaticmetadatauploading

Whenuserscreatecontentbyrecordingavideoortakingapicture,metadataassociatedwithitisautomaticallyuploadedtotheserver.Contentitselfisonlyuploadedwhenusertriggersittoavoidunwantednetwork/datausageforuploadinglargefiles(metadatafilesizeisverysmall,atmostafewkBs).Westoredthedeviceid,userId,andlocationinformationandsoontobothshowvideoorphotoiconsintheMediaQwebsiteandidentifythefileholderlateron.2.3 Uploadvideosandpictures

Whenuserspushtheuploadbutton,thepictureandvideocontentsareuploadedtotheserverincludingtheirmetadata.2.4 ViewTranjectory

Whilerecordingavideoandtakingapicture,ourappsavesthepathdescribedbyanandroiddevicemovingbyusingGPSinformation.Userscanseethepathongooglemaps.2.5 TasksInquiriesandPendingTasks

Userscanasksometaskstotheserver.Theservercanreceivetheuser’srequestsandgivesometaskslikequeststousers.Afterreceivingthetasks,usercanuploadthevideosandpicturestakenfromthespecificplacewhichtheservergave.

2.6 PushnotificationsSincemetadataisautomaticallyuploaded,serverknowswhereusersrecorded

videos/photos,eveniftheydidn’tuploadthecontent.Theservercansendoutpushnotificationstotheclientstorequestthattheyuploadspecificmedia.2.7 Minorfunctions

Userscanviewlistofrecordedvideos,changethemood,previewthevideos,seewheretheyweretaken,uploadthem,deletethem,viewthetrajectoryongooglemaps,andetc,

Figure3.PendingTasks

Figure4.Videopreview/upload/delete

3 Details3.1 OverallApplication

Figure5.MediaQArchitecture

TherearethreepartitionofthisoverallsystemshowninFigure5.Androidapplication,GCM

serveranddataserver.AndroidapplicationuploadmetadatatodataserverandalsocommunicatewithGCMserver.AnotificationsentbyGCMservernoticeusertouploadtherecordedvideo/photocontent.Formorespecific,theMediaQapplicationnotonlyuploadthemetadata,theclientalsoregistertheirdeviceIDonserver.Thentheserverkeeptrackallclient'sIDanduseittodistinguisheachdevice.Sowhenuserwanttoparticipatewithvideowhichisnotuploadedyet,theserverisabletosentthenotificationtothecontentownerthroughGooglecloudmessage,askingthemuploadtherealdata.

3.2 AnalyzingaJsonfile(Thedatasendingtotheserver)

{ "format_version":"0.4", "video_id":"1lwpl9ky2c21o_2016_3_23_Videotake_1458763979183.mp4", "owner_properties":{ "tid_type":"email", "id":"geovideo@example.com" }, "device_properties":{ "SIM_id":"1lwpl9ky2c21o", "OS":"android", "OS_version":"5.0", "firmware_version":"5.0", "model_name":"SM-N900K", "camera_direction":180 }, "UploadType":"request", //request:taskinquiries,personal:personalupload "TaskId":"4341029c-8095-4e", "TaskInquiryId":"f001dd75-49ec-4b", "TaskLat":"null", "TaskLong":"null", "UserId":"2832bc0e-cfc8-48", "Username":"user190", "TaskType":"video", "keywords":[ ], "LocationType":"outdoor", "LevelOfVideoPrivacy":"Public", "Mood":2, "UploadDate":"03/23/2016/08:13:07/PM/UTC-07:00", "RecordDate":"03/23/2016/08:12:59/PM/UTC-07:00", "sensor_data":[ { "location_array_timestamp_lat_long_accuracy":[ //GPSdata [1458763979926,03/23/2016/08:12:59/PM/UTC-07:00,0.000000,0.000000,0.000000,0.0], [1458763979960,03/23/2016/08:12:59/PM/UTC-07:00,34.019416,-118.288382,17.08,22.092] ] }, { "sensor_array_timestamp_x_y_z":[ //Compassdata [1458763979926,03/23/2016/08:12:59/PM/UTC-07:00,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0], [1458763980155,03/23/2016/08:13:00/PM/UTC-07:00,220.60396,-56.323544,343.79306,252.4256,-

1.5382746,5.2923827,8.271368,92.8,-121.7,-30.7,-0.0892159,-0.004261058,0.021837922], [1458763980329,03/23/2016/08:13:00/PM/UTC-07:00,217.96803,-57.671223,341.94247,254.1888,-

1.638831,5.0266266,8.353969,92.8,-121.1,-31.4,-0.090281166,-0.09853696,-0.0061252704], [1458763980532,03/23/2016/08:13:00/PM/UTC-07:00,217.61182,-61.34031,342.43127,255.716,-

1.5532384,4.9057193,9.414599,92.8,-119.9,-31.4,-0.1536644,-0.24021713,-0.09773801] ] } ,{ "wifi_array_timestamp_BSSID_level_frequency":[ [1458763979659,"00:24:6c:b4:1e:19",-58,"5745"] ] }]}

3.2.1 Location_array_timestamp_lat_long_accruacy(1)Timestamp(2)Recordingdate(3)Latitude(4)Longitude(5)Altitude(6)Accuracy

3.2.2 Sensor_array_timestamp_x_y_z

(1)Timestamp(2)Recordingdate(3)Orientationz(manipulatedazimuth)(4)Orientationx(manipulatedpitch)(5)Orientationy(manipulatedroll)(6)Light(7)Accelerometerx(8)Accelerometery(9)Accelerometerz(10)MagneticFieldx(11)MagneticFieldy(12)MagneticFieldz(13)Gyroscopex(14)Gyroscopey(15)Gyroscopez

3.3 Pushnotificationstorequestuploadingofmediacontents

Figure6.flowofgettingGCMRegistrationID

Toidentifyeachintent,weretrieveGCMregistrationIDbycommunicatewithGCMserverasFigure6shown.InthebeginningwegetthegoogleAPIkeyatGoogleDevelopersConsole.WiththeAPIkey,MediaQapplicationcommunicatewithgooglecloudmessageandgetRegistrationID.Finally,theclientshareRegIDwithserverandpostnotificationtoGCM.

4 Theroleoffiles4.1 Serverupload[_pictures].php

Ourapplicationsupportbothpictureandvideorecordinganduploading.AfterreceivednotificationfromGCM,wedeployedthisAPItouploadthevideo/photocontenttoMediaQDatabaseserver.uploadstatus[_picture].php

Toensureweknowthefilecontentormetadataiscorrectlystoredintheserver,especiallyallthemetadataisuploadedsuccessfullyontheserver,werequestthefilestatusthroughuploadstatus[_picture].phpanditreturnsusboththemetadatalength,andcontentlengthoftherequestedfile.Ifbothlengthmatchesthelocalfilelengths,wemarkedthisfileascompletedintheclientapplication.Otherwise,ifthemetadatasizeislessthelocalfile,weuploadedmetadata.Thisisaservicetohelpclientapplicationsknowthefilestatusintheserver,andclientwilltrytouploadmetadatawheneveritcanwhenserverdoesnotacquirethemetadatabecauseofsomeinterruptionofinternetorserverbreakdownatthemomentthatuserjusttakesvideosorphotos.GCMDeviceRegistration.php

Onthedataserver,therearenotonlystoredmediacontentandmetadata,butalsoserverservicesrunonit.OneoftheseisGCMdeviceregistration.

Foreveryintent,GooglewillassignauniqueIDwhichisdependsontheApplication.Inthisphpprocess,wecollectanyidenticalinformationfromdevice,userandregistrationIDgettingfromgoogleserverandproducethemtostoreonMediaQserverforlaterGCMnotificationusage.

4.2 Client(Android)DashboardActivity

FirstscreenshownasFigure7isthepageuserseesafterloggingin.Inthispage,usercanaccessTaskInquiries,PendingTasksandvideos/photoscollection.Whilethispageload,itimmediatelyretrievesregistrationtokenfromGCMserverandsendingdevicetokentoMediaQdataserver,toregisterforpushnotificationsservice.

Figure7.DashboardpageGeoVidDashboardActivity

AsFigure8shown,thedashboardsupportusertoaccessthatcontentrecordpageandnavigatestolistofvideos/photos.Thisdashboardisalsoautomaticallyswitchedtorightafteruserjusttakenavideo/photos.

Morespecific,thevideotakencontentalsocontainthetrajectoryandthedirectionofcamera.Soit’sgeo-spatialmediacontent.

Figure8.DataCollectionincludingVideoandPhoto

GeoVidActivityThisistheactivitythatshowalistofvideo/photocontents.Itdisplaysalistofcontents,

eachwithitsinformation,screenshotpicture,takendateandtimeandtheuploadstatus.Eachelementconnectstoitsvideo/photo.Whenauserselectsanitem,thatcorrespondvideo/photoisreadyforuploadingbyoneclickonitandchoose“upload”.GeoVidDialog

Thisdialogmenuactivitywillbeshownwhenuserselectsoneofvideos/photosinGeoVidActivity.GeoVidActivityretrievethecontentidandpopupandialog.Fromhere,usercanupload,changemood,preview,etc.

Figure9.Dialogtomakechangeofanyvideointhelist

GeoRecorderActivityThisactivityallowstheusertotakeapictureforanewvideo/photocontent.Itutilizesthe

Androidphone’scameratotakeapictureorvideoandsavesitontothephoto’slocalstoragepriortosendingthemetadatatotheMediaQdataserver.Afterfinishing,theactivityflowreturntoGeoVidActivity.GeoVideo

ThisserviceEncapsulatestheAndroidCameraAPIs,andtherecordingofmetadata(GPS,sensors)alongwithrecordingvideo.Afterthethevideo/phototaken,themetadatawillbeproducedandwillbeuploadtoMediaQdatabaseserverafteritbycallingUploadService.UploadService

Thisserviceisusedtouploadmetadataand/orcontenttotheserver.Inourapplicationthemetadataisimmediatelyuploadafterthevideo/photocontenthasbeentakenandthecontentwillbeaskedtouploadwhentheuserisnotifiedbyaninquiryanduploadthevideo/photocontent.Andtheseuploadingprocessisservedwiththisactivity.Weuploadfileschuckby

chuck,foreachchuck,theUploadServiceretriesatmaximal10timestoavoidsomesuddeninternetinterruptionstoppinguploading.MetadataTimedUploadService

Thisserviceisusedtotryuploadingmetadataforfileswhichhaven’thadmetadatauploadedyet.Incaseoffailure,retriesafter2minutes,foramaximumof10times.GcmBroadcastReceiver

ThisserviceisusedtoreceiveGCMmessages(i.e.pushnotificationsfromserver).LaunchesGcmMessageHandler.TheapplicationperiodicallysubscribestheGCMservertocheckifanynewinquiryhasbeensentwhichthesubjectcontentowneristheselfdevice.Iftheserverrespondswithnewvideo/photocontentinquiryandthisdeviceanduseristhecontentowner,theuserwillbenotifiedviaGooglecloudmessagingandthenotificationwillinformthedetailinformationofthedesirevideo/photocontent,filenameashelpingusertouploadassignedcontent.GcmMessageHandler

HandlesGCMmessagesbyshowingnotificationinthesystemnotificationbarwiththemessagethatwasreceivedfromtheserver.Iftheserverrespondswithnewvideo/photocontentinquiryandthisdeviceanduseristhecontentowner,therewillbeanmessageshownatnotificationbarandclearlyinformtheaskedfiletobeupload.

Figure10.ReceiveGCMNotification

5 Classdiagram

Figure11.Theflowofprocess

6 Futureworks6.1 GCMnotification

Fornow,theGCMnotificationonlyshowwhichfileisrequestedtoupload.Inthefuture,itmightdirectlylinktotheuploadingactivityofspecificfileforusertouploadthefile.6.2 GeoRecorderActivity

Fornow,whenrecordingavideoandtakingapicture,wegetrotationsensordatafromtheandroiddevice.Whilerotatingthedevicetorecordavideoforlandscapeorportraitmode,somelagsoccur.Itmightbefixedbyusingorientationsensordatainsteadofrotationsensordata.6.3 Compassdata

Intheserverside,onlyanazimuthvalue(orientation[0])isusedamongthe15compassdata.Wewillthinkabouthowremainingdatawilluse.6.4 Resolution

Currently,theandroiddevicesendsthephotoandvideocontentswiththehighestresolutiontotheserver.Wewillanalyzewhataretheprosandconsontheresolutionandmakebuttonswhichuserscanchoosetheresolution.