master class: protocol interoperability

MASTER CLASS INTEROPERABILITY

JULIEN KERIHUEL / @jkerihuel | 23/01/2015

PROTOCOLS PLUGFEST EUROPE 2015 | PRESENTATION'S TITLE (UPPERCASE) 2 of 79 2 of 79

INTEROPERABILITY: THROUGH DCE/RPC HISTORY

PROTOCOLS PLUGFEST EUROPE 2015 | PRESENTATION'S TITLE (UPPERCASE) 3 of 79

WHAT IS DCE/RPC?

● DCE for Distributed Computing Environment● RPC for Remote Procedure Calls

“ This system allows programmers to write distributed software as if it were all working on the same computer, without having to worry about the underlying network code.”

Source: wikipedia


HISTORY OF DCE/RPC

● Idea of RPC in 1976 [RFC-707]

● OSF DCE/RPC Request for Technology– OSF DCE 1.1 Reference implementation published in 1993 (250K LOC, OSF 1.0 BSD-compatible license)– OSF DCE 1.2.2 on top of DCE/RPC 1.1 in 1996 (3.5 millions LOC, price per copy)

● Released under LGPL in 2005: FreeDCE implementation in Linux 2.4.X and 2.6.X


THE BIRTH OF MSRPC

● A Question of money● Lead to significant DCE/RPC improvements

– Transport over Named Pipes (NCACN_NP)– NTLM / GSSAPI– UNICODE support– IDL extensions


WHAT IS INTEROPERABILITY?

“The ability of making systems and organizations work together: inter-operate”

Compatibility vs de facto Standard vs Interoperability

Source: wikipedia


COMPATIBILITY DIFFERENT FROM INTEROPERABILITY

When two systems communicate together

Image: Camille Moulin


DE FACTO STANDARD DIFFERENT FROM INTEROPERABILITY

When dominant position lead other actors to be compatible with it

Master (1) / Slave (n) relationship

Open Standard / Closed Standard



INTEROPERABILITY

N to N communication – not dependent of a 3rd party vendor

“Everybody talks the same language without restrictions”



THE PATH TO INTEROPERABILITY

● Free Software is the link from standard to interoperability● Open Specifications



INTEROPERABILITY AND OPENESS

● Closed protocols provide security by obfuscation● It is bad practice● Remember Blaster or Sasser worms?



PROTOCOL ANALYSIS: DEFINITION/TECHNIQUES


PROTOCOL ANALYSIS DEFINITION

● It is the core methodology involved in Samba and OpenChange● Unique requirements are:

– Network packet analyzer– Methodology and Commitment

● Different from reverse engineering because no binary analysis or decompilation



FRENCH CAFE TECHNIQUE

Robert Doisneau, 1966

THE CONCEPT



SambaXP 2005

THE REALITY



1. LISTEN AND REPEAT




2. SWEAR WORDS AND ERROR PACKETS





3. WRITE A PROTOCOL SCANNER

1. Append additional blank bytes until the server changes its response in some way

2. Change additional bytes until server accepts it

3. Figure out what the command does





3. WRITE A PROTOCOL SCANNER

4. DIFFERENTIAL TECHNIQUE

1. Discover interactions between different command words

2. Change custom server until it behaves like real one


FRENCH WAITER TECHNIQUE

Switch roles and analyze customer's reaction



1. SALT COFFEE

1. Silently drink it?

2. Ask for another one?

3. Swear and Leave?

Salt shaker on white background" by Dubravko Sorić SoraZG on Flickr - http://www.flickr.com/photos/11939863@N08/3793288383/in/photostream/.



1. SALT COFFEE

2. IGNORE CUSTOMER CALLS

1. How long does he wait?

2. Call manager or other waiter?

3. Move to the café next door?

Image: photostock / FreeDigitalPhotos.net



1. SALT COFFEE

2. IGNORE CUSTOMER CALLS

3. PRETEND YOU DON'T UNDERSTAND FRENCH

1. How many tries before he leaves?

2. Does the client order in another language?


MIDL DECOMPILER

● Muddle: first MIDL decompiler ● Written by Matt Chapmann in 2000● Controversial and imperfect technique● Process RPC Service's executable to return IDL file● IDL - Interface Description Language


INTEROPERABILITY BY EXAMPLE


THE EXCHANGE LOGON CASE

Let's run a network analysis session on Exchange MAPI Logon

and figure out the IDL of the RPC call


NETWORK DATA REPRESENTATION

● Used by DCE/RPC and MSRPC for data marshalling/unmarshalling● Transfer syntax● Provides primitives and complex types● Little-endian based (least significant bit first)● Alignment enforced● http://pubs.opengroup.org/onlinepubs/9629399/chap14.htm


EXCHANGE LOGON: SDK & CAPTURE

1. Write a atomic program using MAPI SDK that logs onto Exchange Server

2. Capture the traffic with Wireshark and export stub data


EXCHANGE LOGON: HEXDUMP


EXCHANGE LOGON: HEXDUMP AND PROCESSING$ hexdump C <pkt>

00000000 00 00 00 00 83 f2 c7 15 74 66 15 47 99 33 ac 9f |........tf.G.3..|00000010 fd bf 24 08 00 00 00 00 6f 00 00 00 00 00 06 00 |..$.....o.......|00000020 67 00 67 00 c6 a5 5b a5 a5 a4 a9 a1 a5 a5 a5 a5 |g.g...[.........|00000030 a5 a5 f6 a5 8a ca 98 e3 cc d7 d6 d1 85 ea d7 c2 |................|00000040 c4 cb cc df c4 d1 cc ca cb 8a ca d0 98 e3 cc d7 |................|00000050 d6 d1 85 e4 c1 c8 cc cb cc d6 d1 d7 c4 d1 cc d3 |................|00000060 c0 85 e2 d7 ca d0 d5 8a c6 cb 98 f7 c0 c6 cc d5 |................|00000070 cc c0 cb d1 d6 8a c6 cb 98 e4 c1 c8 cc cb cc d6 |................|00000080 d1 d7 c4 d1 ca d7 a5 5a 5a 5a 5a 00 6f 00 00 00 |.......ZZZZ.o...|00000090 07 80 00 00 30 00 00 00 00 00 06 00 28 00 28 00 |....0.......(.(.|000000a0 ad a5 a4 a4 a4 a5 a7 a5 b5 a5 a4 a9 d9 a5 a5 a5 |................|000000b0 a5 a5 a5 a5 a4 a5 a5 a5 b5 a5 a4 a9 d8 a5 a5 a5 |................|000000c0 d8 a5 a5 a5 a4 a5 a5 a5 30 00 00 00 08 10 00 00 |........0.......|

handle: struct policy_handle handle_type : 0x00000000 (0) uuid : 15c7f283-6674-4715-9933-ac9ffdbf2408


EXCHANGE LOGON: Generate IDL with unknown fields[public,noprint] MAPISTATUS EcDoRpcExt2(

[in] policy_handle *handle,[in] uint32 *unknown1,[in, size_is(unknown3)]uint8 unknown2[],[in] uint32 unknown3,[in] uint32 *unknown4,[in, size_is(unknown6)]uint8 unknown5[],[in] uint32 unknown6,[in] uint32 *unknown7,

);

$ hexdump C <pkt>00000000 00 00 00 00 83 f2 c7 15 74 66 15 47 99 33 ac 9f |........tf.G.3..|00000010 fd bf 24 08 00 00 00 00 6f 00 00 00 00 00 06 00 |..$.....o.......|00000020 67 00 67 00 c6 a5 5b a5 a5 a4 a9 a1 a5 a5 a5 a5 |g.g...[.........|00000030 a5 a5 f6 a5 8a ca 98 e3 cc d7 d6 d1 85 ea d7 c2 |................|00000040 c4 cb cc df c4 d1 cc ca cb 8a ca d0 98 e3 cc d7 |................|00000050 d6 d1 85 e4 c1 c8 cc cb cc d6 d1 d7 c4 d1 cc d3 |................|00000060 c0 85 e2 d7 ca d0 d5 8a c6 cb 98 f7 c0 c6 cc d5 |................|00000070 cc c0 cb d1 d6 8a c6 cb 98 e4 c1 c8 cc cb cc d6 |................|00000080 d1 d7 c4 d1 ca d7 a5 5a 5a 5a 5a 00 6f 00 00 00 |.......ZZZZ.o...|00000090 07 80 00 00 30 00 00 00 00 00 06 00 28 00 28 00 |....0.......(.(.|000000a0 ad a5 a4 a4 a4 a5 a7 a5 b5 a5 a4 a9 d9 a5 a5 a5 |................|000000b0 a5 a5 a5 a5 a4 a5 a5 a5 b5 a5 a4 a9 d8 a5 a5 a5 |................|000000c0 d8 a5 a5 a5 a4 a5 a5 a5 30 00 00 00 08 10 00 00 |........0.......|




);



EXCHANGE LOGON: the blob mistery

● With our IDL we don't have much info about Logon● The solution may lie in the blobs we reasonably identified



EXCHANGE LOGON: deeper underground

● Can we identify another structure within the blob?



● Strange … no 0x0● Very strange … lots of 0xa5● Very strange … 0x5a is the reverse of 0xa5




● XOR 0xA5




KEEP CALM AND DO NOT PANIC


THE RDESKTOP CASE

● Free client for RDP (Remote Desktop Protocol) used by Terminal Services Server● Developed by Matt Chapmann

Problem: RDP 4 & 5 communication are encrypted

WORKAROUND?


THE RDESKTOP CASE

Encrypted login packet BUT unencrypted transfer of other packets

$ rdesktop -hrdesktop: A Remote Desktop Protocol client.Version 1.7.1. Copyright (C) 1999-2011 Matthew Chapman et al.See http://www.rdesktop.org/ for more information.

Usage: rdesktop [options] server[:port][...] -e: disable encryption (French TS) -E: disable encryption from client to server -m: do not send motion events


THE OPENCHANGE CASE


2003: THE STARTING POINT - EPITECH

ZERO TO ONE EXPERIMENT


2005: THE MENTOR

ANDREW TRIDGELL (TRIDGE)FOUNDER OF THE SAMBA PROJECT


2006: THE HAPPY UNDERLINE

3 Decembre 2006:

Exchange Account created with OpenChange Server


2007: FIRST OPENCHANGE CLIENT LIBRARY RELEASE

● LibMAPI-0.2 MAILOOK Release @ SAMBAXP 2007

● GNOME Evolution Plugin based on libmapi released


2008: Open Specifications Program

Microsoft releases documentation on Exchange protocols


2008-2010: WAN ACCELERATION AND MAPIPROXY ERA

● Lay the foundations of current server architecture ● MITM Proxy && Stackable modules


2010-2012: First Storage Back Development for Server

● SOGo backend● Technology Preview● Backend developed in objective-C● Plugs onto services like Dovecot, Postfix, MySQL


2013-2015: Zentyal & Pivot

● Joined Zentyal as CTO● Development of OpenChange technology● Took the lead on openchange-sogo backend development● Hard learning curve for new openchange developers


2014-2015: Soften the Learning Curve

● Objectives:● Write OpenChange backends very quickly● Abstract MAPI complexity to the maximum

●Technology:● Python-C gateway● REST API


DEVELOPER PROGRAM


OBJECTIVES

● technical contest opened to open source developers, and aiming at challenging your innovation skills.

● manipulate Exchange protocols data to twist Microsoft Outlook graphical interface and make it do unexpected and amazing things.

● Leverage the Python-C gateway and REST-API to start very quickly


HOW DOES IT WORK?

● Decide to work alone or join a team● Register on http://www.protocolsplugfest.com/europe/developer-program/● Write your backend● Publish it on github repository under the terms of GPLv3 or later● 30th March 2015: deadline for submission● 5th April 2015: Finalist teams announced

http://www.protocolsplugfest.com/europe/developer-program/


PRICES

● INDUSTRY PASS TO PROTOCOLS PLUGFEST● 3 DAYS STANCE TO ATTEND PROTOCOLS PLUGFEST● 20 MINUTES TO PRESENT THE PROJECT IN THE EVENT MAIN TRACK


POSSIBLE PROJECT TOPICS

Social mediaBig dataStorage


PREPARE DEVELOPMENT ENVIRONMENT

1. Clone openchange-docker repository:

1. git clone https://github.com/openchange/docker

2. Clone openchange repo with Python-C/REST API support

1. git clone https://github.com/openchange/openchange

2. cd openchange

3. git fetch && git checkout jkerihuel/mapistore-python

3. Install docker 1.4.1

https://github.com/openchange/openchange


PREPARE DEVELOPMENT ENVIRONMENT

DEMO


QUESTIONS?

http://www.protocolsplugfest.com@plugfestcon

http://www.protocolsplugfest.com/

master class: protocol interoperability

Engineering