The Road to ARToolKit 6

The Road to ARToolKit 6ARToolKit for the Next GenerationWally YoungTechnical Product Manager, ARToolKit

Whats that?

My name is Wally Young, and Im the product manager for ARToolKit at DAQRI.Ive been at DAQRI a little over 3 and a half years, and ever since Philip Lamb and Ben Vaughan joined us last year with ARToolKit, Ive been working with them.

I guess first is What is ARToolKit?Thank you for asking little clipart arrow.ARToolKit is an open source tracking & recognition SDK.

And for most of us here, ARToolKit is an AR SDK that needs no introduction.Its one of the oldest and most downloaded AR SDKs on the market.

Its been around since 1999.And has had over 1,000,000 downloads.

Runs on everything (list platforms).Works with see-through displays (like the Epson Moverio, DSH).Plays well with stereo rendering (for those of you with DK1s or HTC Vives).

We sstarted with square fiducials.Added later added image tracking.

And what we have today- ARToolKit 5- is the natural evolution of these features.And by evolution I mean time and effort.

1999First public demonstration @ SIGGRAPH.2001 v1, with square tracking released open source.2001 ARToolworks Incorporated, begins selling Pro version.2004NFT tracking integrated in ARToolKit.2004v2 released open source.2008-10iPhone 3g Support, Open Source Innovation2011-14Android support added, further extending platform support.2015Acquired by DAQRI, pro version open sourced....The History of ARToolKit...so what have we been up to?

ARToolKit has had a very long history. It all began with Hiro Kato now of the Nara Institute of Science and Technology, then at the University of Washingtons HIT Lab.

There have been numerous spin-offs of ARToolKit, incuding:FLARToolKit, Which brought ARToolKit to Flash / Action ScriptNyARToolKit, Which brough ARToolKit to C#, and has had a lot of popularity in Japan, in particular.JSARToolKit, for JavascriptSLARToolKit for Silverlight

2015-16

2015 has been a great year for us. Weve been very busy.ON THE TECH FRONT:Weve replaced SURF with FREAK in our keypoint matching library.-> This made ARToolKit completely free-to-use, with no license or patent complications.Weve been integrated into the Unreal Engine - twice!-> One of these is for sale on the Unreal store, and I believe the other one is planning on being free.Added to OSVRRecompiled ARToolKit 5 with emscripten for Javscript and we now run in the browser.-> Thats also free and open source on our github as JSARToolKit5, please go check it out.

IN EVANGELISM AND EDUCATION:We were at Laval virtual and ismar 2015- evangelizing open source and talking with some brilliant computer scientistsWeve given summer schools at U. South Australia-> We saw some really cool examples come out of that around collaborative play.And Telkom University in Indonesia-> We saw a lot of really cool AR story books here.Weve been at hackatons in Dublin.And weve even given classes at CSM and MIT.

ARToolKit 5, by the NumbersPro Versions Open Sourced on May 13 201535,000+Visitors Per Month to artoolkit.org71,000+SDK Downloads235ARToolKit 5 Forks88ARToolKit for Unity Forks1240Members of ARToolKit Forum

We announced ARToolKit 5 was coming to open source last year right here at AWE. Since then, lets see how much activity weve had

Why Open Source?If we want the AR community to grow, price (and license) should not be a barrier to entry.We want to encourage innovation in AR circles, and share transparently.We want to raise awareness and understanding of AR, and how it works.

If we want the AR community to grow, price (and license) should not be a barrier to entry.As youve seen, weve been doing this since 1999- This is a well-understood problem, and people deserve the solution to that problem to be available to use.

Encourage innovation in AR circles.Were trying to solve these problems in a transparent way- everything down to the algorithms under the hood are available to read, use, reuse, and learn from.Furthermore, with the LGPLv3 license, any advances that are made to ARToolKit are meant to come back to the community, and make it better for everyone.

We dont want to give you a black box. And we dont want research done on top of black boxes. Knowledge is meant to be shared.

We also want to help people understand augmented reality. For this crowd, its obviously less necessary- but the more people who understand AR, and at the core, how it works, the average users awareness goes up, and we can move beyond the how and why, and spend more time on the what- and come closer to AR Everywhere.

New Users, New Devices, New Tech

Long before smartphones ARToolKit first ran on the desktop. We added capabilities that werent there before and enabled people to do amazing new things.

Then we were one of the first AR SDKs on smartphones- heres ARToolKit running on an iPhone 3g or 3gs.ARToolKit has been seen running on Symbian as early as 2005.

Today, we find ourselves with whole new classes of devices, and a whole new user.We have handheld devices, some whose screens are larger than laptops, with GPUs that blow older phones, and even some older desktops out of the water.We have wearables which try understand the world around you, and compete against the REAL WORLD for video update rate.And we believe that our users, too, now come with a whole new set of expectations.

ARToolKit for the Next Generation

What if you could start over? What if you could take everything youve learned, and do it again?

Thats what were doing with ARToolKit 6.

ARToolKit 6 is a fast and modern open source tracking and recognition SDK which enables computers to see and understand more in the environment around them. ARToolKit 6 represents a push for AR everywhere by targeting a new epoch of researchers, educators, and developers to empower their research, teach others about AR, and share what they create and learn.

When were launching, were on Linux, Android, OS X, and iOS. Shortly after, well bring it to Windows, as well, to take part in the growing convertible tablet market.

But thats it? All of these words and losing one platform? (Windows phone, for those of us playing the home game.)

Not exactly...

New Image TrackerNew for ARToolKit 6. Like v5.3, it uses FREAK features, but is a ground-up new development from DAQRI, using approaches such as optical flow alongside template matching for more robust tracking.1999. Handbook of Computer Vision and Applications with Cdrom (1st ed.). Bernd Jahne, Peter Geissler, and Horst Haussecker (Eds.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

We have a brand new image tracker that is A-MA-ZING.Just like our previous tracker, we start with FREAK for initialization- but then we use techniques like optical flow alongside traditional template matching. Frankly, weve never tracked better.

Not only that, image training has gotten significantly easier. There's a lot more to go into for this feature, and a lot more to announce.But instead of passing your jpeg to a command line tool, and calculating the effective DPIs- in the easiest case, you can just pass it an image, and it will start tracking on-the-fly.

Heres a side-by-side of ARToolKit 5 (left), and the tracker thats going in ARToolKit 6 (right).

You can see pretty clearly that most of the jitter youve seen is pretty much gone, unless you go reeeeeealllllyyyy reeeeeeaaaallllyyyyy close to what youre trying to track.

By the way- This content is from ALTERRA, a brand new experience by DAQRI, powered by ARToolKit.For those of you that are Unity devs- This is our version of Battle Bots- a way to show you the power of ARToolKit, and what you can do with it.It will be released open-source and shipping with ARToolKit 6.

And were hoping to build a story together with the open source community around this.

If you go to our booth, you can see this app running live on both ARToolKit 5, and a very early alpha of ARToolKit 6, and see the difference for yourselves.

For those of you that took the time to come to this talk (thank you), well also be giving you a first edition ALTERRA comic book, which will work with the ALTERRA app upon release.

Planar TrackerUsing the same algorithms in our image tracker, we can also track unprepared planes. Just start the tracker and go!

This is something Im really excited about - Weve also got a planar tracker.

Opening up the camera feed, ARToolKit 6 begins to look for a plane.Once it has found a plane, it begins to track that plane, allowing you to place content in the world around you.

I really believe this is a step in a new direction for AR Developers-Content, whether it starts off of an image or not, is free to be in the world around you, and not just tied to a marker.

This is from a very early version of Crayola Color Alive- probably about a year and a half to two years old nowrunning on an iPad mini.

Needless to say- you will see substantial improvements to this tech come release.

BY THE WAY- These two tracking options arent the limit of ARToolKit 6. Were designed to be a true toolkit- and extending ARToolKit 6 to encompass new tracking technologies is exactly what this effort is meant to do. Whether its ARToolKit fiducials, or a new SLAM implementation, or something completely different.

New Unity PluginCompletely new developer experience, designed to make going for 0-to-AR faster and easier than ever before.

Were doing a complete overhaul of our Unity plugin.

Gone are the days of worrying about making sure if you named that marker correctly,Or worrying about what parts you need to setup vs what ARToolKit will set up for you.

A lot of these feedback has come from the community forums and emails, but additionally, everytime we taught an ARToolKit class or were at a hackathon, we received constant user feedback, and it became pretty apparent where users were having a frustrating time.

Were trying to be as transparent as possible to what the developer cares about,And taking care of the rest, in order to get developers into AR as seamlessly as possible.

Commitment to Open SourceARToolKit 6 will continue our tradition of being completely free and open source.You do not need to purchase a license.You do not need to register for an account.You do not need to phone home with our SDK.You do not need to give away what you build with it.

When we released ARToolKit 5 open source, we chose to release it LGPLv3. I still think this is the right type of license to release with, but there was definitely some confusion-With LGPLv3, anything you build on top of ARToolKit your app, an integration framework (say, for the Unreal Engine), whatever- really- is still yours.Weve even included a clause for static linking to avoid license issues on iOS, and the infectious nature of GPL-style licenses.Now- say you were developing on iOS version 12, or something new... And theres a brand new video capture libray (and AVFoundation is deprecatead), and you want to support that. Thats modifying the SDK, and that is what you would have to open source with our license.We did this because the core tech, the guts of ARToolKit, will get better for everybody, in a rising tide raises all ships fashion.Im happy to say that our license with ARToolKit 6 will be equally as permissive, if not more permissive. That means [click, continue reading slide].

COMING THIS FALL

When?

Wally YoungTechnical Product Manager, ARToolKitwally@artoolkit.org@wallyyoungmusicARToolKit Homehttp://www.artoolkit.orgCommunityhttp://www.artoolkit.org/communityDocumentationhttp://www.artoolkit.org/documentationARToolKit 5 SDK Repohttps://github.com/artoolkit/artoolkit5ARToolKit 5 for Unity Repohttps://github.com/artoolkit/arunity5DAQRI Homehttp://www.daqri.com4D Expohttp://www.4dexpo.com

Thank you!Follow us on Twitter!@artoolkit_@DAQRI