cmf: a pain in the f @ phpday 05-14-2011

Long story short, a true tale about an hopeless DevTeam

CMF: a pain in the F

David Funaro & Alessandro NadalinMay, 14 2011

domenica 15 maggio 2011

let us introduce

The Problem™


There comes the boss< there's a new project


There comes the boss

< seems easy, let's use Joomla!


Don't open that door!


The 80/20 dilemma

CMS


80% is cool at all


but as

the requirements grows


(you always knew that)


the project becomes a mess


There comes the boss< can I haz fotonic menthal scanner in the website



< can't do boss Joomla! is not meant...


There comes the boss< STFU NOOB DO THAT OR BLA BLA BLA


what if



< seems complex,no Joomla this time,

let's use a framework!


There comes the boss< STFU NOOB WE SHOULD GO LIVE NEXT WEEK BLA BLA


The 20/80 dilemma

Framework


20% is cool at all


the 80%is reallyfunny


but has a cost

Time To Market


again, what if



< seems complex,no Joomla this time,

when should it go live?


There comes the boss< salesguy just told the client tomorrow



< ok, let's use a CMF


unfortunately, there is no stableCMF in the PHP ecosystem


so it has a cost

initial development


investment=

know how (team) + quality (framework) +

velocity (that customers pretend)

=

CMF


"Our" CMF


Should be able to manage a site editing pages with

different elements.

Target: End User/Site admin

Easily, like a CMS.


Target: Developer

should be able to plug the CMF with NO PAIN,using the kick-ass framework at the bottom of the CMF.


2things


Pages

• main structure of a web site • organized as a tree• a way to express our contents• usually composed by a series of

containers ( that we'll refer to as boxes )


Boxes


Boxes

The way to separate single elements of a page

Loosely coupled HMVC structures


Entities Pages


Entities

Boxes {Bridge}

Pages

Boxes let entities be shown inside webpages


Boxes

We dispose of different types of boxes:• Common• DSL


Common Box

• Simple o Input field o Textareao Videoo Imageo File o Richtext

• Compositeo Input field + imageo input field + image + richetexo textarea + video


what about the DSL?


Domain Specific Language

DSL = ! (Website 1 Website 2 )


Event

News Users

Books Review

Page

Widget

Event

News

UsersCars

Contract

Widget

Page

Example: Identify DSL


News

Page

Widget


News

Page

Widget

Web Elements = CMF Entities


Cars

Contract

Books

Review

News

Page

Widget



Cars

Contract

Books

Review

News

Page

Widget

DSL



back to the CMF


CMF Stack

• Framework: symfony, for example• CMF entities: Site, Pages, Menus, Users ...• DSL entities: cars, contracts (for Hertz)• Common Boxes : Textarea, image, video, ...• Dsl boxes: Cars image gallery


Common Boxes + DSL Boxes=

The main tool for the developer to build a page with the CMF


Example : page & box compositionm

ost s

old

prod

uct

Richtext

Main product Image



ost s

old

prod

uct

Richtext

Main product Image

Dsl Boxes



ost s

old

prod

uct

Richtext

Main product Image

Dsl Boxes

Common Boxes


The Page-Box Binding Storage

Data can change its structure fastly, according to the domain model.

We need a technology supporting this fast change.

Since we have N box type, to compose a page we shouldlook for a few tables.

We need a technology supporting this kind of lookups.


so,How will we

manage the data?


RDBMS, of course


because different data always have

the same need


one size fits all


We tend to think at the way we should store data

not how we're gonna use it


and we think that the

shit-it-always-happens migration

will be easy


also if that column is useful for one and one only row


among milions


so we know we're gonna facea few well-known problems...


Data Mapping

Data Structure RDBMS

we have to find a way to re-organize the data-structre to insert our tree inside a table.But is really different! Tree find a node in O(nlogn)

id

name

parent_id


Data Mapping


is better than before, but isn't the best way


Data Mapping


... and NOW??????

?


The Alien

BUT you always find

something that as a different Behaviour

spend more time to normalize your models


Unpredictable Tomorrow

User Story evolve continous

Worse than the Alien


EAVformerly known as:

normalization WTF?!?!?!

or:how to do 70k joins to retrieve an entity of you model


Overnormalized

user_id | address_id

user_id | phone_id

user_id | mail_id

...

...

...

...

...

LEFT JOIN

LEFT JOIN

LEFT JOIN


Overnormalized

Users ListLEFT JOIN

LEFT JOIN

LEFT JOIN

LEFT JOINLEFT JOIN

LEFT JOINLEFT JOIN


Overnormalized

LEFT JOIN

LEFT JOIN

LEFT JOINLEFT JOIN

LEFT JOIN

LEFT JOIN

LEFT JOINLEFT JOINLEFT JOIN

LEFT JOINLEFT JOIN

LEFT JOIN

LEFT JOIN

LEFT JOINLEFT JOIN

LEFT JOIN

LEFT JOINLEFT JOIN

What if you need to show only a single primary address?

Users List


back to the CMF


Bind Page with Boxes

Foreing Key -> TextareaBox


Bind Page with Boxes

Foreing Key -> ?

AND NOW ???


Mantain the RI#1


Not Mantain the RI#2


Maintain or not Maintain the RI; that is the

question


we have to choose


have you ever thought

"there should beanother way!"

?


Can someone help us ?


NoSQLlets try

world


• Written in: C/C++ • Main point: Blazing fast• Protocol: Telnet-like• Disk-backed in-memory database, • but since 2.0, it can swap to disk. • Master-slave replication • Simple keys and values, • but complex operations like

ZREVRANGEBYSCORE • INCR• Values can be set to expire (as in a cache)

$ redis-cli set mykey "my binary safe value" => OK $ redis-cli get mykey => my binary safe value


Best used: • rapidly changing data with

a foreseeable database size (should fit mostly in memory)

For example: • analytics• real-time data collection• caching• MQ• log

http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis


very fast


very fast

Object != key->value


Not reliable

very fast



Not reliable

very fast


Can I haz otha nozql?


> j = { name : 'david', surname : 'funaro' };

{ "name" : "david", "surname" : "funaro" }

> db.things.save(t);

> db.things.find();

{ "_id" : ObjectId("4c2209f9f3924d31102bd84a"), "name" : "mongo" }

• Written in: C++ • Main point: Retains some friendly properties

of SQL• Protocol: Custom, binary (BSON) • Master/slave replication • speaks JavaScript• Built-in sharding• Performance over features • After crash, it needs to repair tables


Best used:• you need dynamic queries• you prefer to define indexes,

not map/reduce functions• you need good performance on

a big DB

For example: • all things that you would do

with MySQL or PostgreSQL, but having predefined columns really holds you back



Document Oriented


Document Oriented

Schema Free


Document Oriented

Schema Free

Navigate Object


• Written in: Erlang • Main point: DB consistency, ease of use• Protocol: HTTP, some say REST• Bi-directional replication => master-master• conflict detection ( 409 Conflict )• MVCC: write operations do not block reads • Previous versions of documents are available • Crash-only (reliable) design • Needs compacting from time to time • Views: embedded map/reduce• jQuery library included

curl -d '{"keys":["bar"]}' -X POST http://host:port/foo/_all_docs?include_docs=true


http://host

http://host

Best used: • accumulating occasionally-

changing data• pre-defined queries• versioning is king

For example: • CRM/CMS systems• API



Reliable


Document Oriented

Reliable


Document Oriented

Schema Free

Reliable


Document Oriented

Schema Free

Navigate Object

Reliable


Why do we so hardlyneed a GraphDBlike OrientDB?


"GraphDBs don't avoid relations but they embrace them in a way that they are not a computational problem anymore, by making them explicit instead of implicit through joins."

Claudio Martella( http://blog.acaro.org/entry/somebody-is-going-to-hate-me-nosparql )


Easy joins

select from users where city.country.continent = 'Oceania'


Same link semantics,different objects

select links from menu >> 28:1, 17:5


Same link semantics,different objects

select from [28:1, 17:5]

28:1

ExternalLinkurl: http://www.google.com

17:5

ProductLinkid: 4


28:1

ExternalLinkurl: http://www.google.com

17:5

ProductLinkid: 4

$link->render()


• Written in: Java• Main point: Graph• Protocol: Binary or HTTP• ACID• Disk-persisted or in-memory,• Cluster• 150k inserts per second• Schema-less, *-full, *-mixed• Speaks JSON• Integrated GUI• Documental• SQL commands

curl -X PUT '127.0.0.1:2480/document/demo/1:2' --data "{'@rid': '1:2'}"


Reliable


Graph - Documental

Reliable


Graph - Documental

Schema-*

Reliable


Graph - Documental

Schema-*

Reliable

Easy traversal


Graph - Documental

Schema-*

Reliable

Easy traversal

SQL-friendly


Graph - Documental

Schema-*

Reliable

Easy traversal

SQL-friendly

UberFast ( binary protocol )


Graph - Documental

Schema-*

Reliable

Easy traversal

SQL-friendly

UberFast ( binary protocol )

Universal ( HTTP protocol )


is so cool


but has a problem...


ver 1.0


rc1


but has a couple problems...


JAVA


but somebody


started writing the binary-protocol binding

https://github.com/AntonTerekhov/OrientDB-PHP( beta, 28 April 2010 )


and others


wrote the HTTP one

https://github.com/odino/Orient


$driver = new Orient\Http\Client\Curl();$orient = new Orient\Foundation\Binding( $driver, '127.0.0.1', '2480', 'admin', 'admin', 'demo');

$response = $orient->query("select from fellas where any() traverse(0,-1) ( @rid = 1:1 )");

$output = json_decode($response->getBody());foreach ($output->result as $friend){ echo $friend->name;

// other fun...}


you can start playing with a pure

GraphDB now


High-level PHP Component?


No


but we know that ;)


that's why in DNSEE we will build a basic

Object Graph Mapperfor OrientDB


...because...


hey, you remember about our CMF?


codename:


codename:

ConGoW


codename:

contents gone wild

( don't ask why )

ConGoW


ConGoW


• soon the real kickstart

ConGoW


• soon the real kickstart• 10 devs at the starting point

ConGoW


• soon the real kickstart• 10 devs at the starting point• OSS on GitHub

ConGoW


• soon the real kickstart• 10 devs at the starting point• OSS on GitHub• Stack

ConGoW



oSymfony2

ConGoW



oSymfony2oOrient

ConGoW



oSymfony2oOrientoDoctrine2

ConGoW


"My only friend, the end"



David Funaro@ingdavidinohttp://davidfunaro.com




Alessandro Nadalin@_odino_

http://odino.org




Alessandro Nadalin@_odino_

http://odino.org

http://joind.in/talk/view/3008


Creditshttp://www.flickr.com/photos/pagedooley/5313217918/sizes/l/in/photostream/

http://www.flickr.com/photos/zoetnet/5520594473/sizes/l/in/photostream/http://www.flickr.com/photos/mhxbhd/3962410821/sizes/o/in/photostream/

http://www.flickr.com/photos/labyrinthx/1955594336/sizes/z/in/photostream/http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

http://www.flickr.com/photos/salim/19426192/sizes/o/in/photostream/http://www.flickr.com/photos/mybluevan/3780363712/sizes/l/in/photostream/

http://farm4.static.flickr.com/3064/3086258014_f1925639e0.jpg


cmf: a pain in the f @ phpday 05-14-2011

Technology