pistonhead's use of mongodb for analytics
TRANSCRIPT
Haymarket Media Group
Building a reporting and analytics suite for dealers and administrators
Pete Dignan - Platform Director@RacingPete
PistonHeadsPistonHeadsStarted in 1998Original social networkAlways irreverentClassifiedsNewsForums
Classifieds140,000 cars6.5 million users100 million page views9 minute average visit durationMore 17k+ vehicles than competitors
Vision - PistonHub
“To create a stock insights and trading screen, providing secure access to the statistics behind the classifieds, and opening up the true performance of our
classifieds to Dealer Groups, Individual Dealers and Internal Staff”
1st Problem - View Data
2nd Problem - Stock and Classifieds Data
MongoDB
Advert Data
Category
Make/Model
History
Ad Status
MySQL
Dealer Groups
Individual Dealers
3rd Problem - Lead Data
PistonHub Architecture Solution
Abstract Data Aggregator
Google Data API
HTTP/SOAP
.Net Driver
ADO .NetAPI AngularJS App
{ "_id" : ObjectId("564a9892ad2e970fac7f6fff"), "CDRId" : "ac61259e35bacc174f3e9ff0afd4c0c58ff6ffff", "NTSNum" : "01234567890", "FromNum" : "0771234567", "ToNum" : "01234567891", "DialledNum" : "+44771234567", "ConnectTime" : ISODate("2015-11-11T02:18:14Z"), "CallDuration" : "3", "TalkTime" : "3", "TimeToAnswer" : "1975", "DisconnectText" : "Answered", "DisconnectType" : "Answered", "Recorded" : "N", "CallId" : "i1447208272.153117.xxxx", "LoadDate" : ISODate("2015-11-11T03:00:01.177Z")}
{ "CallDate" : ISODate("2015-11-10T22:27:00Z"), "CallDuration" : NumberLong(3), "CalledTrackingNumber" : "01234567890", "CallerNumber" : "0771234567", "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "RingTime" : NumberLong(1975), "Status" : "Answered", "_id" : ObjectId("564a9892ad2e970fac7f6fff")}
{ "DealerAltPhone" : "01234567891", "DealerGroupId" : 0, "DealerGroupIdStr" : "0", "DealerGroupName" : null, "DealerId" : 61056992, "DealerIdStr" : "61056992", "DealerName" : "Dealer LTD", "DealerPhone" : "01234567890", "TrackingAltPhone" : null, "TrackingPhone" : "01234567890", "_id" : ObjectId("555ef398e816135315639fff")}
Raw Gamma Data Raw Dealer Data
Phone Data
Abstract Data Aggregator
{ "CallDate" : ISODate("2015-11-10T22:27:00Z"), "CallDuration" : NumberLong(3), "CalledTrackingNumber" : "01234567890", "CallerNumber" : "0771234567", "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "RingTime" : NumberLong(1975), "Status" : "Answered", "_id" : ObjectId("564a9892ad2e970fac7f6fff")}
Phone Data
Abstract Data Aggregator
{ "CallDate" : ISODate("2015-11-10T22:27:00Z"), "CallDuration" : NumberLong(3), "CalledTrackingNumber" : "01234567890", "CallerNumber" : "0771234567", "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "RingTime" : NumberLong(1975), "Status" : "Answered", "_id" : ObjectId("564a9892ad2e970fac7f6fff")}
{ "CallDate" : ISODate("2015-11-10T22:27:00Z"), "CallDuration" : NumberLong(3), "CalledTrackingNumber" : "01234567890", "CallerNumber" : "0771234567", "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "RingTime" : NumberLong(1975), "Status" : "Answered", "_id" : ObjectId("564a9892ad2e970fac7f6fff")}
{ "CallDate" : ISODate("2015-11-10T22:27:00Z"), "CallDuration" : NumberLong(3), "CalledTrackingNumber" : "01234567890", "CallerNumber" : "0771234567", "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "RingTime" : NumberLong(1975), "Status" : "Answered", "_id" : ObjectId("564a9892ad2e970fac7f6fff")}
View Data
Email Data
Stock Data
{ "AdViews" : 119, "CallLeads" : 0, "Calls" : 1, "Date" : ISODate("2015-11-10T00:00:00Z"), "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "DeepLinkClicks" : 1, "EmailLeads" : 1, "Emails" : 1, "NormalizedDealerName" : "DEALER LTD", "Phone" : "01234567890", "Stock" : 2, "TrackingPhone" : "0771234567", "WebsiteClicks" : 14, "_id" : "61xxxx_2015-10-10"}
Dealer Stats
The Dashboard
The Dashboard
The Dashboard
The Dashboard
The Dashboard
The Dashboard
The Dashboard
Conclusions
Pipeline aggregation is super fast to collate data…
… and run queries for AngularJS
Storing day by day data means easy to do custom date formats
Dealer feedback has been very positive on the extra insights
Sales teams have used the system to proactively help dealers
Wish we were using 3.2
Questions?