![Page 1: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/1.jpg)
Building a maintainable bi-directional cross platform protocol
Pavel DovbushWilliam Lewis@
![Page 2: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/2.jpg)
BackgroundSolution and ImplementationExamples
![Page 3: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/3.jpg)
● Application Programming Interface
● Operations● Inputs● Outputs● Type definitions
● Independent of implementation
API
API
![Page 4: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/4.jpg)
![Page 5: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/5.jpg)
RequirementsPerfect Client Server API:
● Just works – Magic! :)
● Flexible● Extensible● Testable● Maintainable● Platform and language neutral● Focused on features, not bytes over the wire
Requirements
![Page 6: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/6.jpg)
Overview● Encoding ● Message exchange● Data access
data access
Server
exchange data accessexchange
Client
serialise
deserialise
Overview
![Page 7: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/7.jpg)
REST+JSON
PHP array
Server
URI JS ObjectAjax
Client
JSON
URL
REST + JSON
![Page 8: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/8.jpg)
REST+JSON problems
Message exchange:● Client: HTTP request/response model● Server: URI-based, config on web- or app- server
Data access:● No canonical definition● No versioning● Duplicate implementations and configuration
REST + JSON
Encoding:● Server: JSON● Client: URLEncode
![Page 9: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/9.jpg)
Badoo APIs
Badoo APIs
Core API
Mobile API
???
![Page 10: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/10.jpg)
BackgroundSolution and ImplementationExamples
![Page 11: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/11.jpg)
Protocol implementations
SOAP
Protocol Values
![Page 12: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/12.jpg)
![Page 13: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/13.jpg)
Protocol values
● Protocol description● Encoding● Data access● Message exchange● Versioning
Protocol
![Page 14: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/14.jpg)
Protobuf + Own RPC
Generated class
Server
Events Generated classEvents
Client
Protobuf
Protobuf
Description, versioning
Usage
![Page 15: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/15.jpg)
Google Protocol Buffers
● Interface description language● Internal representation● Language support
○ v2.3 plugin support ● Encoding and network efficiency
Protcol values
![Page 16: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/16.jpg)
Interface description language
Protobuf is self-describing - descriptor.proto
Interface description language
● enum● message● field● service● option
![Page 17: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/17.jpg)
Interface description languageLabel Type Name Number
optionalrequiredrepeated
boolstring
messageenumfloatint32
+ more numeric
field_name = 1;
required string user_name = 1;optional uint32 age = 2;
Interface description language
![Page 18: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/18.jpg)
Interface description language
enum Role {ADMIN = 1;USER = 2;
}
message User {required string name = 1;repeated string nickname = 2;optional uint32 age = 3;required Role role = 4;
}
Interface description language
![Page 19: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/19.jpg)
Protobuf usage
Encoding:● Only binary
Message exchange:● Simple RPC
Data access:● No support for PHP● No support for JS
Protobuf compiler plugins
![Page 20: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/20.jpg)
Protobuf v2.3.0 compiler plugins
● Brilliant internal architecture● Very simple plugin system● Can generate any code in any language● IDL is completely separated from serialization part
● gist with example
Protobuf complier plugins
![Page 21: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/21.jpg)
Encoding
Encoding
Core API
Mobile APINative binary
serializationCustom JSON serialization
![Page 22: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/22.jpg)
Performance
Performance
![Page 23: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/23.jpg)
Code auto-generation
● WebSite DEVEL - On any request if ‘.proto’ file is newer than generated code - regenerate
● Mobile DEVEL - Grunt task - regenerate on file change
● PRODUCTION - generate before deploy
Code auto-generation
![Page 24: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/24.jpg)
Message exchange
service SearchService {rpc Search (SearchRequest) returns (SearchResponse);}
● Too simple for a complex application● We need a wrapper for every Request/Response● Anytime responses
Message exchange
![Page 25: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/25.jpg)
new RPC(request_type, parameter) .on(response_type, callback) .on([type1, type2], callback) .request();
Message exchange (two-way RPC)
RPC.any.on(type3, callback);
request_type & response_type are values of enum MessageTypeparameter & response are Protobuf messages
function callback(err, /** Type1 */ response1) {}
Message exchange
![Page 26: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/26.jpg)
Versioning
Old clients ignore:● new fields● unsupported commands
Migrating to new protocol version:● cause compilation error on field removal
Versioning
![Page 27: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/27.jpg)
Take aways● Protocol defined in one place● Code uses data-access classes● Validation● Encoding can vary● Flexible message exchange● Versioning
● Any part can be changed without affecting anything else
Summary
![Page 28: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/28.jpg)
BackgroundSolution and ImplementationExamples
![Page 29: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/29.jpg)
Building a service
Example
![Page 30: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/30.jpg)
Example
![Page 31: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/31.jpg)
Building a service
● Location search○ Client query: city name○ Server response: list of cities
(ID, name, lat, long)
● Client notification○ Anytime server response: user message
Example
![Page 32: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/32.jpg)
Protobuf definition
enum MessageType { // body: CityQuerySERVER_SEARCH_CITIES = 1;
// body: CitiesCLIENT_FOUND_CITIES = 2;
// body : ClientNotificationCLIENT_NOTIFICATION = 3;
}
message ClientNotification {required string id = 1;optional string title = 2;optional string message = 3;
}
message CityQuery {optional string name = 1;
}
message Cities {repeated City cities = 1;
}
message City {required int32 id = 1;required string name = 2;optional double longitude = 3;optional double latitude = 4;
}
Example protobuf
![Page 33: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/33.jpg)
Protobuf definition
message RPCMessage {required int32 version = 1;optional int32 message_id = 2;repeated MessageBody body = 3;
}
message MessageBody {required MessageType message_type = 1;optional CityQuery city_query = 2;optional Cities cities = 3;optional ClientNotification client_notification = 4;
}
Example protobuf
![Page 34: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/34.jpg)
Generated classes examplesdefine(['GPB/gpb2'], function(/** $gpb */$gpb) {var Protocol = $gpb.namespace('Demo');/** * CityQuery * @class {Protocol.CityQuery} * @extends {$gpb.Message} */var CityQuery = Protocol.CityQuery = function() {
$gpb.Message.apply(this, arguments);};$gpb.extend(CityQuery, $gpb.Message);CityQuery.prototype.$gpb = 'Demo.CityQuery';CityQuery.prototype._descriptor = {"fields": {"name": {"type": 9, "number": 1, "label": 1}}};
return CityQuery;
});
define(['GPB/gpb2'], function(/** $gpb */$gpb) {var Protocol = $gpb.namespace('Demo');/** * CityQuery * @class {Protocol.CityQuery} * @extends {$gpb.Message} */var CityQuery = Protocol.CityQuery = function() {
$gpb.Message.apply(this, arguments);};$gpb.extend(CityQuery, $gpb.Message);CityQuery.prototype.$gpb = 'Demo.CityQuery';CityQuery.prototype._descriptor = {"fields": {"name": {"type": 9, "number": 1, "label": 1}}};
return CityQuery;
});
define(['GPB/gpb2'], function(/** $gpb */$gpb) {var Protocol = $gpb.namespace('Demo');/** * CityQuery * @class {Protocol.CityQuery} * @extends {$gpb.Message} */var CityQuery = Protocol.CityQuery = function() {
$gpb.Message.apply(this, arguments);};$gpb.extend(CityQuery, $gpb.Message);CityQuery.prototype.$gpb = 'Demo.CityQuery';CityQuery.prototype._descriptor = {"fields": {"name": {"type": 9, "number": 1, "label": 1}}};
return CityQuery;
});
Generated class
![Page 35: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/35.jpg)
Generated classes examples<?php
namespace GPBJS\Demo;
class CityQuery extends \GPBJSBase\Message{ protected static $name = 'Demo.CityQuery'; protected static $fields = array( 'name' => array('type' => 'string', 'optional' => true, 'repeatable' => false, 'hash' =>
false, 'raw' => False, 'is_enum' => false, 'is_message' => false), );
public function setName($value) { $this->_setFieldValue('name', $value); return $this; } public function getName() { return $this->_getFieldValue('name'); }}
<?php
namespace GPBJS\Demo;
class CityQuery extends \GPBJSBase\Message{ protected static $name = 'Demo.CityQuery'; protected static $fields = array( 'name' => array('type' => 'string', 'optional' => true, 'repeatable' => false, 'hash' =>
false, 'raw' => False, 'is_enum' => false, 'is_message' => false), );
public function setName($value) { $this->_setFieldValue('name', $value); return $this; } public function getName() { return $this->_getFieldValue('name'); }}
<?php
namespace GPBJS\Demo;
class CityQuery extends \GPBJSBase\Message{ protected static $name = 'Demo.CityQuery'; protected static $fields = array( 'name' => array('type' => 'string', 'optional' => true, 'repeatable' => false,
'hash' => false, 'raw' => False, 'is_enum' => false, 'is_message' => false), );
public function setName($value) { $this->_setFieldValue('name', $value); return $this; } public function getName() { return $this->_getFieldValue('name'); }}
Generated class
![Page 36: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/36.jpg)
RPC city query example
new RPC(Protocol.MessageType.SERVER_SEARCH_CITIES, cityQuery).on(Protocol.MessageType.CLIENT_CITIES, onCities).on(Protocol.MessageType.CLIENT_NOTIFICATION, onNotification).request();
function onCities (err, /** Protocol.Cities */ cities) { if (err) { /* error handling */ return; }
for (var city in cities.getCities()) { cityListView.update(city.getId(), city.getName()); }}
function onNotification ( err, /** Protocol.ClientNotification */ notification) {alert(notification.getTitle() + ‘\n’ + notification.getMessage());
}Example usage
![Page 37: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/37.jpg)
RPC city query examplevar cityQuery = new Protocol.CityQuery().setName(‘london’);new RPC(Protocol.MessageType.SERVER_SEARCH_CITIES, cityQuery)
.on(Protocol.MessageType.CLIENT_CITIES, onCities)
.on(Protocol.MessageType.CLIENT_NOTIFICATION, onNotification)
.request();
function onCities (err, /** Protocol.Cities */ cities) { if (err) { /* error handling */ return; }
for (var city in cities.getCities()) { cityListView.update(city.getId(), city.getName()); }}
function onNotification ( err, /** Protocol.ClientNotification */ notification) {alert(notification.getTitle() + ‘\n’ + notification.getMessage());
}Example usage
![Page 38: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/38.jpg)
RPC city query examplevar cityQuery = new Protocol.CityQuery().setName(‘london’);new RPC(Protocol.MessageType.SERVER_SEARCH_CITIES, cityQuery)
.on(Protocol.MessageType.CLIENT_CITIES, onCities)
.on(Protocol.MessageType.CLIENT_NOTIFICATION, onNotification)
.request();
function onCities (err, /** Protocol.Cities */ cities) { if (err) { /* error handling */ return; }
for (var city in cities.getCities()) { cityListView.update(city.getId(), city.getName()); }}
function onNotification ( err, /** Protocol.ClientNotification */ notification) {alert(notification.getTitle() + ‘\n’ + notification.getMessage());
}Example usage
![Page 39: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/39.jpg)
RPC city query examplevar cityQuery = new Protocol.CityQuery().setName(‘london’);new RPC(Protocol.MessageType.SERVER_SEARCH_CITIES, cityQuery)
.on(Protocol.MessageType.CLIENT_CITIES, onCities)
.on(Protocol.MessageType.CLIENT_NOTIFICATION, onNotification)
.request();
function onCities (err, /** Protocol.Cities */ cities) { if (err) { /* error handling */ return; }
for (var city in cities.getCities()) { cityListView.update(city.getId(), city.getName()); }}
function onNotification ( err, /** Protocol.ClientNotification */ notification) {alert(notification.getTitle() + ‘\n’ + notification.getMessage());
}Example usage
![Page 40: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/40.jpg)
RPC city query examplevar cityQuery = new Protocol.CityQuery().setName(‘london’);new RPC(Protocol.MessageType.SERVER_SEARCH_CITIES, cityQuery)
.on(Protocol.MessageType.CLIENT_CITIES, onCities)
.on(Protocol.MessageType.CLIENT_NOTIFICATION, onNotification)
.request();
function onCities (err, /** Protocol.Cities */ cities) { if (err) { /* error handling */ return; }
for (var city in cities.getCities()) { cityListView.update(city.getId(), city.getName()); }}
function onNotification ( err, /** Protocol.ClientNotification */ notification) {alert(notification.getTitle() + ‘\n’ + notification.getMessage());
}Example usage
![Page 41: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/41.jpg)
RPC city query examplevar cityQuery = new Protocol.CityQuery().setName(‘london’);new RPC(Protocol.MessageType.SERVER_SEARCH_CITIES, cityQuery)
.on(Protocol.MessageType.CLIENT_CITIES, onCities)
.on(Protocol.MessageType.CLIENT_NOTIFICATION, onNotification)
.request();
function onCities (err, /** Protocol.Cities */ cities) { if (err) { /* error handling */ return; }
for (var city in cities.getCities()) { cityListView.update(city.getId(), city.getName()); }}
function onNotification ( err, /** Protocol.ClientNotification */ notification) {alert(notification.getTitle() + ‘\n’ + notification.getMessage());
}Example usage
![Page 42: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/42.jpg)
RPC city query examplevar cityQuery = new Protocol.CityQuery().setName(‘london’);new RPC(Protocol.MessageType.SERVER_SEARCH_CITIES, cityQuery)
.on(Protocol.MessageType.CLIENT_CITIES, onCities)
.on(Protocol.MessageType.CLIENT_NOTIFICATION, onNotification)
.request();
function onCities (err, /** Protocol.Cities */ cities) { if (err) { /* error handling */ return; }
for (var city in cities.getCities()) { cityListView.update(city.getId(), city.getName()); }}
function onNotification ( err, /** Protocol.ClientNotification */ notification) {alert(notification.getTitle() + ‘\n’ + notification.getMessage());
}Example usage
![Page 43: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/43.jpg)
RPC anytime response example
RPC.any.on(Protocol.MessageType.CLIENT_NOTIFICATION, onNotification);
function onNotification ( err, /** Protocol.ClientNotification */ notification) {alert(notification.getTitle() + ‘\n’ + notification.getMessage());
}
Example usage
![Page 44: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/44.jpg)
Building a maintainable bi-directional cross platform protocol
● REST + JSON● Protocol values
○ Interface description language○ Encoding and performance○ Protobuf compiler plugins○ Code auto-generation○ Message exchange (two-way RPC)○ Versioning
● Examples
Summary
![Page 45: Building a maintainable bi-directional cross platform protocol](https://reader034.vdocuments.us/reader034/viewer/2022052315/555e5ca3d8b42a8e4c8b5558/html5/thumbnails/45.jpg)
Thanks! Questions?
Pavel Dovbush <[email protected]>
William Lewis <[email protected]>@netproteus
Slides: techblog.badoo.com
Thanks:Google team for Protobuf itselfAndrey Nigmatulin <[email protected]> for bringing GPB in Badoo in 2009