Technical introduction to the deeper parts of SailfishOS, a Qt5-Wayland based mobile OS

by Carsten Munk @ FOSDEM 2014, Embedded devroom

About me & the company I work for

● I'm (Carsten Munk/Stskeeps) a Chief Research Engineer at Jolla● Jolla was born in 2011 out of passion of its founders towards

open innovation in the mobile space. ● Currently we are a team of 90+ people and have our offices in

Helsinki and Tampere, Finland and Hong Kong.● We believe that the best way of creating something meaningful

is DIT, doing it together. ● Since the beginning, the community has been at the heart of our

idea generation, technology development and distribution.

We built a deviceDisplay

Ample 4.5" IPS qHD display , 5-point multi-touch w/ Gorilla 2™ Glass

Camera

8 megapixel AF camera with LED flash  ,

2 megapixel front-facing camera

Memory & CPU & Miscellaneous

16GB storage, 1GB RAM, MicroSD slot, Dual-core Qualcomm 1.4ghz,

Proximity, Accelerometer, Gyro, E-compass, Ambient light sensors

Talk time & battery

9 / 10 hours (GSM / 3G approx.)  , User-replaceable battery

Connectivity: GSM/3G/4G LTE

Dimensions: h/w/thickness 131 mm/68 mm/9.9 mm 

Weight 141 g 

Price 399 € (less if you pay attention to this talk)

* List of supported countries and operators will be published later.

and if you have Wi-Fi you can:

● ssh -p 2023 nemo monster.tspre.org● Password: fosdem2014

and look around on the device for the duration of the talk.

ps aux; rpm -qa | sort recommended.

Device tinkerability● Sorry! No factory images due to 3rd party restrictions.● But we provide a recovery mode where you can revert to factory

state or otherwise recover.● Standard USB 'fastboot' protocol bootloader ● Ability to unlock bootloader and flash own kernels (as of 1.0.3.8) & to

lock your device pretty tight with a device lock code.● Full root by enabling developer mode (comes with Terminal). Kernel

source code available; WLAN module closed source (prima); but later open source version is working fine.

● Some of this may void your warranty. Power comes with responsibility. If you break it, you get to keep both pieces.

We made a mobile OS

But what does that entail specifically?

● System essentials: systemd, coreutils, bash, gzip, util-linux, file, eglibc, glib2, curl, sqlite, openssl, OpenSSH, Linaro GCC, btrfs..

● Multimedia: GStreamer, OpenMAX, PulseAudio, Grilo, Tracker

● Communications: ConnMan, oFono (+RIL), Telepathy, BlueZ, GeoClue

● PIM: KCalCore, QtPim● Software management: RPM, PackageKit, Libzypp

And the list goes on..● Graphics: Wayland 1.1.0 (and no X11)

Qt 5.1, OpenGL ES 2.0, EGL, QtWayland, libhybris

● Input methods: Maliit Framework● UI technology: QtQuick2.0/QML● Browser engines: Gecko and QtWebKit● Build and development: Scratchbox2,

QtCreator, Open Build Service●

Development of SailfishOS 1.0

● New technology: libhybris, leverage existing Android hardware adaptations.

● Drop X11 and focus on modern UI but don't reinvent the wheel. Use Wayland (see why next)

● Develop UI with existing working Wayland stack (Mesa/LLVMpipe on VirtualBox/X86) while we build the HW adaptation

● Use QtCompositor to fulfill the wildest dreams of our designers.

● The result you can see in the following slides: a productised (not a mockup – end-users use it as their daily devices) future proof mobile platform.

And with that stack we built a beautiful UI

With Wayland, Qt5, QtQuick2 & QML

And with that stack we built a beautiful UI

And with that stack we built a beautiful UI

Contributing to SailfishOS

There's many forms of contributions beyond

code:● Ideas, bug reports,

enhancement/feature requests, challenges, constructive criticism, translations, hugs, artwork, community support, virtual keyboard layouts, events, visions, hardware hacks, .. etc

Co-creation with Jolla and SailfishOS at together.jolla.com

The skunkworks

What we contribute: ● Everything except:

● Jolla/SailfishOS artwork/trademark and/or Look and feel (this pretty much means UI)

● 3rd party closed source software● Contributions written using NDA'ed materials● Contributions requiring copyright transfer

● Unless you get permission, of course.● This pretty much means that for a very large

part of our effort, it goes straight into existing open source projects such as Mer, Qt, Nemo, libhybris, etc.

SailfishOS architecture● http://releases.merproject.org/~carst

en/niceview.png●

How to contribute to SailfishOS Core and middleware

● http://github.com/nemomobile

● https://review.merproject.org

● http://gitweb.merproject.org

● https://github.com/nemomobile-packages

● https://github.com/mer-packages/

● https://github.com/sailfish-sdk

● #nemomobile #mer #sailfishos on irc.freenode.net

● http://piratepad.net/SailfishOSContributionAndDevelopment

SailfishOS technology

Libhybris“Hubris (/ˈhjuːbrɪs/, also hybris, from ancient Greek ὕβρις), means extreme pride or arrogance. Hubris often indicates a loss of contact with reality and an overestimation of one's own competence, accomplishments or capabilities, especially when the person exhibiting it is in a position of power”

Why libhybris?● I accidentally ported the Android/bionic linker

to a glibc environment. And it worked.● Which is pretty crazy, a loss of all technical

sanity and contact with reality. ● Solves one major problem for alternative non-

Android mobile OS'es though: We had really no hardware to put them on if we wanted modern HW capabilities.

A quick note on dynamic linkers

● “Dynamic loading is a mechanism by which a computer program can, at run time, load a library [dlopen] (or other binary) into memory, retrieve the addresses of functions [dlsym] and variables contained in the library, execute those functions [function pointers] or access those variables, and unload the library from memory. Unlike static linking and loadtime linking, this mechanism allows a computer program to startup in the absence of these libraries, to discover available libraries, and to potentially gain additional functionality” --wikipedia

Initial experiment● Built bionic linker for glibc. Rename its

dlopen/dlsym/dlclose to android_*● Disable all dependency loading● Build a small .so against a bionic sysroot: int

foo(int a, int b) { return 1; }● handle = android_dlopen(“foo.so”, ...); fp =

android_dlsym(handle, “foo”); if ((*fp)(1,2) == 1) { printf(“yay!\n”);}

It gets crazier..● It seemed ridiculous to load glibc and

bionic side by side in a process's address space.

● So that's what I did.● And it worked*

* for the most part

Couple of problems*● Two different ways of using Thread Local Storage● Pthread implementation differences● Errno● -mfloat-abi=hard vs -mfloat-abi=softfp● So we decided to patch bionic.● /dev/log/log_main✗Problems may in this case mean nightmares

You can then imagine the possibilities

● android_dlopen(“libEGL.so”)● myeglCreateWindowSurfaceFP =

android_dlsym(“eglCreateWindowSurface”); *myeglCreateWindowSurface(....)

● It meant we could build (glibc) libEGL.so and libGLESv2.so wrappers that accessed the Android ones.

● And the list goes on:● Gralloc, OpenGL ES 1.1/2.0, NFC, Hardware HAL, OpenCL,

SurfaceFlinger, OpenMAX, Camera, Hwcomposer,...

Wayland on libhybris

EGL on Android● eglCreateWindowSurface parameter

takes an ANativeWindow● ANativeWindow contains hooks like

queueBuffer, dequeueBuffer, querying/setting sizes, transforms, crop

● Most important is the ANativeWindowBuffer which contains an Android native handle

Native handles and gralloc

● Allocation of graphic buffers happen through the aptly named gralloc

● Native handles are practically file descriptors and integer values

● You can share file descriptors between processes (fd passing)

● You can register another process' buffer in your process through gralloc

Wayland on libhybris● If you can make your EGL stack speak Wayland

(ANativeWindow), share buffers (fd passing) and use the buffer as a texture (EGL_ANDROID_image_native_buffer), you can have a GPU accelerated Wayland compositor.

● So that was quite easy to do.● No devices with requirements for server-side

buffers seen so far.● Server-side buffers may be useful for graphics

sharing (wallpaper, icons..)

Libhybris today● Was open sourced according to Jolla open

source policy.● A project with participants from

Jolla/SailfishOS, Intel/Tizen, Canonical/Ubuntu, OpenWebOS and many others.

● Tested on many different Android versions, SoCs and architectures.

● http://github.com/libhybris/libhybris● #libhybris on irc.freenode.net

And one more thing..Jolla.com webshop discount code for developers and community members for a Jolla

● Use “FOSDEM2014” discount code (40 EUR)● One customer can use the coupon just once● Discount applies only to one phone in cart● Offer can't be combined with any other offer● Coupon validity from 29.1.2014. until 9.2.2014

Thank you for listening● http://www.jolla.com● http://www.sailfishos.org● @JollaHQ on twitter● https://joindiaspora.com/u/jolla ● #jollamobile and #sailfishos on irc.freenode.net● SailfishOS community round-table at 16.30 in

H.3227/3228● Community dinner @ À la Mort Subite, rue

Montagne-aux-Herbes Potagères at 19:30