Connecting the Real World With Your Mobile!

… or how to ask “what’s that?”

Geoff Ballingerhttp://www.mobileacuity.comhttp://blog.geoffballinger.co.uk

me

us

• based on work at Edinburgh University

• spun out in 2006 to commercialise mobile visual search

• initial commercial marketing work with leading brands and agencies

• now focussed on visual search for products

Edinburgh

more us

Your mobile is:

• the little bit of the “internet” in your pocket wherever you are all the time

• becoming a bridge between the “internet” and things around you in the real world

e.g. a retailer’s mobile app ...

• Tesco “Groceries”• Launched yesterday• Top lifestyle app on

iPhone!• Built by Ribot• Great integration!

• BUT hard to find products!

e.g. price comparison ...

• “How much can I save by not impulse purchasing now?”

• quickly/easily identify the product• identify the specific product• discrete to use in store!

e.g. augmented reality ...

• augments reality - overlay digital information onto the real world

• typically on a camera viewfinder view• can be hard to produce a believable

view

• e.g. Layar• e.g. theodolite app• e.g. cityguides etc

e.g. brand marketing ...

• mobile becoming a serious option• often combined with traditional

media• needs to be inclusive• mustn’t be off-putting

“real people”

• As Aral said in the keynote – “stay naive”

• Real users are typically not like us ...• What is the user there to do?• Bar is set higher in mobile – less

tolerant.

• Objective is to build apps/services that real people want to and can use!

Confidence is everything!

1D barcodes

• On most products.• UPC/EAN codes.

• BUT ...• Designed for laser

scan – not imaging.• Some retailers

have private systems.

• Not user friendly!

how

• able to overlay info in viewfinder?• platform capable of “scanning”?• close focus capability?• distortion across the frame?• inter-pixel effects• binarised vs “fuzzy” reader

readers

pic2shop / Vision Smarts

Google ZXing

( )

Sccope

Sccope demo

2D barcodes

Patent No. 6,999,573 31st Jan 2006

x

x

Visual Search

But ...not every product has a barcode!

“What’s that?” ...

Visual Search

Visual Search

Who’s who

PixTo

Google

• Google Goggles – general visual search

• Google Shopper – visual price comparison

• UI issue – what does the scanline mean?

Amazon

• Amazon Remembers – snap a picture of the item

• recognition of 2D media products

• mechanical Turk style “recognition” of all other items

MA Demo

Visual AR

• Specific and accurate placement of digital info into the video stream.

marketing

recognition

• simplest use• replaces SMS keyword response

zoning

• activation of Vodafone’s Champions League sponsorship

• take penalties by taking a picture of where you want to aim the ball

• send in by MMS to see whether you score or not

• true user interaction

video

• MMS darts• YouRockstar

Compass and GPS

• location and orientation to deduce view

• (accelerometer for finer movements)• many good AR and map based

examples on iPhone and Android

Layar AR

but ...

... in Father Ted’s immortal words:

(Father Ted is demonstrating some plastic toy cows to Dougal)

“OK, one last time. These are small... but the ones out there are far away. Small... far away... ah forget it!”

tags

• Some objects in the world have RFID tags

• e.g. TOTeM “Remember Me”with Oxfam

• NFC – tags + trust

• c.f. swatch skipass and Oyster features

Summary

• Linking the internet into the real world via your mobile using:– barcodes– visual recognition– compass and GPS– tags

• The Internet of Things?

What we can provide!

• class leading 1D barcode reader• visual indexing and search over

databases of up to 10 million 2D images

• 3D indexing and visual search working now and will be launched in 2011

Thanks for listening!

Geoff Ballingergeoff.ballinger@mobileacuity.com

http://www.mobileacuity.comhttp://blog.geoffballinger.co.uk

@geoffballinger