what we have learned about api scaling throughout messente's international sms traffic growth
TRANSCRIPT
What we have learned about
API scaling throughout Messente's
international SMS traffic growth
Build Working Product• SMS Sending API
• Delivery Report API
• Backend services
• Public website
• Admin interface
22,5
45
67,5
90
2011 2012 2013 2014 2015 2016
SMS Sending API$ curl 'https://messente.com/api/send_sms/' \-d 'user=hnoac48nta938ynvas039n04972' \-d 'api_key=023fjhwlekhf023ihf2poi3hf23' \-d 'from=Test' \-d 'to=%2B37251916062' \-d 'text=Hi+There!' \-d 'dlr_url=http%3A%2F%2Fmyshop.com%2Fmessente-dlr%2F'
OK slkdlskndflkdnflksdnflksdnl 0.056
ERROR 101ERROR 102....
Delivery Report Requesthttp://myshop.com/messente-dlr/?message_unique_id=slkdlskndflkdnflksdnflksdnl&status=DELIVERED
SMS Sending API• connect to database
• validate account credentials
• validate sender name
• determine country and operator
• calculate the message price
• account crediting
• unique message ID generation
• save all this data to database
SMS Sending API• connect to database
• validate account credentials
• validate sender name
• determine country and operator of the number
• determine the message price
• account crediting
• unique message ID generation
• save all this data to database
SMS Sending API$ curl 'https://messente.com/api/send_sms/' \-d 'user=hnoac48nta938ynvas039n04972' \-d 'api_key=023fjhwlekhf023ihf2poi3hf23' \-d 'from=Test' \-d 'to=%2B37251916062' \-d 'text=Hi+There!' \-d 'dlr_url=http%3A%2F%2Fmyshop.com%2Fmessente-dlr%2F'
OK slkdlskndflkdnflksdnflksdnl 0.056
bottleneck
“Build now. Scale later.”
SMS Sending API - v2$ curl 'https://api2.messente.com/send_sms/' \-d 'username=hnoac48nta938ynvas039n04972' \-d 'password=023fjhwlekhf023ihf2poi3hf23' \-d 'from=Test' \-d 'to=%2B37251916062' \-d 'text=Hi+There!' \-d 'dlr_url=http%3A%2F%2Fmyshop.com%2Fmessente-dlr%2F'
OK slkdlskndflkdnflksdnflksdnl
bottleneck
Delivery Report Requesthttp://myshop.com/messente-dlr/?message_unique_id=slkdlskndflkdnflksdnflksdnl&stat=DELIVRD&err=000
error parameter added
SMS Sending API - v2• extremely fast API response
• handle hundreds of req/sec
• return error codes asynchronously
• offloaded processing to backend service
SMS Processing FlowClient
Messente HTTP API
Operator
Messages Queue
Backend Service
Backend Service - v2• determine the country &
operator
• check the roaming and portability information from HLR service
• determine the route
• calculate the price of the message
• credit account
• check for number in blacklist
• verify sender name
• auto-convert text if required
• update statistics
• save message to database
Backend Service - v2• determine the country &
operator
• check the roaming and portability information from HLR service
• determine the route
• calculate the price of the message
• credit account
• check for number in blacklist
• verify sender name
• auto-convert text if required
• update statistics
• save message to database
SMS Processing FlowClient
Messente HTTP API
Operator
Messages Queue
Backend Daemon
SMS Processing FlowClient
Messente HTTP API
Backend Daemon
Operator
Messages Queue
Backend DaemonBackend Daemon
SMS Processing FlowClient
Messente HTTP API
Backend Daemon
Operator
Messages Queue
Backend DaemonBackend DaemonBackend Daemon
Backend Daemon
Backend DaemonBackend Daemon
Backend Daemon
Backend Daemon
SMS Processing FlowClient
Messente HTTP API
Operator
Messages Queue
SMS Processing Pipeline
SMS Processing PipelineSMS
Determine Operator & MNP
Pricing & Account Crediting
Validator
Send SMS to Operator
Statistics
SMS Processing PipelineSMS
Determine Operator & MNP(Multi-threaded HLR service)
Pricing & Account Crediting
Validator
Send SMS to Operator
Statistics
Handling Database Size Growth
Multiple GB of data added every month
Handling Database Size Growth
SMS DLR
Handling Database Size Growth
SMS_2016_01 DLR_2016_01
SMS_2015_12 DLR_2015_12
SMS_2015_11 DLR_2015_11
Handling Database Size Growth
SMS DLR
ARCHIVE_2016_01
ARCHIVE_2015_12
ARCHIVE_2015_11
Backup Servers• Multiple API nodes
• api2.messente.com
• api3.messente.com
• Database Replication
• Server monitoring
Quick Takeaways• Don't be afraid to scale later
• API will evolve - be prepared to handle different versions
• Always have a backup