is your api naked 9.2.09
TRANSCRIPT
-
8/8/2019 Is Your API Naked 9.2.09
1/30
Is yo ur API naked?10 roadmap considerations for product and engineeringmanagers with open APIs
http://www.sonoasystems.com/http://www.sonoasystems.com/mailto:[email protected]:[email protected]:[email protected]://www.sonoasystems.com/ -
8/8/2019 Is Your API Naked 9.2.09
2/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 2 July 21, 2009
This whitepaper is a compilation of posts (and posts in waiting) from ourAPI technology
blog series Is your API Naked? 10 considerations for your API roadmap.
This series is put together based on our collective experience talking to hundreds ofproduct and engineering managers on issues around operating an open API.
Why naked in the title? Once a customer of ours said most open APIs start out aslittle more than raw, naked, features exposed to developers.. and there is a big gapbetween a feature and a full-fledged service..
So this paper is about the capabilities around the API that might be needed toproductize, monetize, or operationalize an API-based service. These include
issues around traffic management, protecting your organization and customers, scalingperformance, monetization, community and more.
So if you own an open API roadmap, these short pieces are meant to give you aspectrum of questions to think about for your own roadmap across different areas whereyour API might need covering up.
And it goes without saying wed love to talk to you about howSonoa API managementcan help with any of these.
Cheers,The Sonoa [email protected]
http://blog.sonoasystems.com/http://blog.sonoasystems.com/http://blog.sonoasystems.com/http://blog.sonoasystems.com/http://blog.sonoasystems.com/detail/is_your_api_naked_10_api_roadmap_considerations_part_1_visibility/http://blog.sonoasystems.com/detail/is_your_api_naked_10_api_roadmap_considerations_part_1_visibility/http://blog.sonoasystems.com/detail/is_your_api_naked_10_api_roadmap_considerations_part_1_visibility/http://www.sonoasystems.com/solutions/managementhttp://www.sonoasystems.com/solutions/managementhttp://www.sonoasystems.com/solutions/managementhttp://www.sonoasystems.com/solutions/managementhttp://blog.sonoasystems.com/detail/is_your_api_naked_10_api_roadmap_considerations_part_1_visibility/http://blog.sonoasystems.com/http://blog.sonoasystems.com/ -
8/8/2019 Is Your API Naked 9.2.09
3/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 3 July 21, 2009
Table of Contents
1. Is your API naked? (API Visibility) ............................................................................ 4
2. Dont have a meltdown: API Traffic Management ....................................................... 6
3. Do you need API keys? API Identity, Authorization, & Authentication ........................ 8
4. Dont roll your own: API Security Recommendations ................................................ 12
5. Indecent Exposure: Data Protection .......................................................................... 15
6. One size doesnt fit all: API Versioning and Mediation .............................................. 187. Welcome Aboard: API User Management ................................................................ 20
8. If you build it will they come?: API Community and Audience .................................. 22
9. Show me the money: Monetization ........................................................................... 25
10.Turn up the volume: Scalability Concerns ................................................................ 27
-
8/8/2019 Is Your API Naked 9.2.09
4/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 4 July 21, 2009
1. Is your API naked? (API Visibil ity)We're always surprised how almost every company we talk to will
acknowledge how little they know about their API traffic and usage.
We see most folks sifting through web server logs to understandusage of the API as a product. As the API becomes more strategicand you want to make the case for more investment - this gets morepainful.
This happens a lot when an API starts as an experiment. As we once heard a customerexplain most APIs start out as little more than raw naked features. APIs are oftenlaunched by the 'ask for forgiveness, not permission' types (you know who you are :-).Things like metrics or analytics are back burner until an API either gets off the ground or
doesn't.
But most APIs usually end up getting more important more quickly than expected, andas a product and engineering manager you may start asking:
Who is using the API and how much are they using? How many clients, apps, developers are out there?
How do they break down by type, region?
How does usage map to existing customer or partner organizations. Or how dodevelopers map to applications map to customers? (This can be tough with onlykey or IP based tracking.
What parts of the API and service are they using? You probably use webanalytics- at some point you need this for APIs as well to see where to invest.
How does traffic breakdown between your own products and 3rd party products?(If they use the same API.)
What the aggregate and per developer/app/customer transaction and data
volumes?
How fast and 'good' is your service? What latency are customers experiencing, by developer, customer, region, or
operation?
Where are errors and user experienced bugs happening and how often?
-
8/8/2019 Is Your API Naked 9.2.09
5/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 5 July 21, 2009
How is the API delivering vs. any formal SLA have agreed to or paid for?
How can you find out if a customer is having a problem (before they call you)?
How is the API usage impacting the rest of the platform or web products that alsouse the same API?
Can you quickly trap and debug based on a specific message? Based on what isin a cache?
How does the API drive the business? Who are the top priority customers? Developers? Partners? Who should you
call to up sell to a higher service tier or do a deal with?
What do you need to show general management to make product strategy (ortactical) decisions?
Will you need to create audit trails or metering reports for partners that arepaying for API access?
Do you need to create metrics based on a certain data value in the payload?(Such as a specific product SKU)
What is the cost of the data that you are serving up? (if you are licensing thisdata)
Are you in line with all the compliance standards that IT enforces for the on-premise apps?
These questions become increasingly important when opening a service as an API delivery times are critical when customers, contract terms, and compliance regulationscome into play. Knowing these metrics and analytics and how they map to the businesswill help you be get ahead of the demands of customers and partners, and are criticalwhen you need to make the business case for an APIs to executives.
Wed love to hear what you think, and next up is traffic management.
(and thanks toEspartaandNaked Cowboy for the pic)
http://www.flickr.com/photos/esparta/http://www.flickr.com/photos/esparta/http://www.flickr.com/photos/esparta/http://www.flickr.com/photos/esparta/3631859653/http://www.flickr.com/photos/esparta/3631859653/http://www.flickr.com/photos/esparta/3631859653/http://www.flickr.com/photos/esparta/3631859653/http://www.flickr.com/photos/esparta/ -
8/8/2019 Is Your API Naked 9.2.09
6/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 6 July 21, 2009
2. Dont have a meltdown: API TrafficManagementCustomers come up with far better sound-bites than we do. One asked
us if we could protect them from back-end melt down. They had opened an API forcustomers, but it was the same one their Web apps was using, so bursts of traffic fromcustomer apps had their Web product running hot and not in a good way.
A back-end melt down will lead to a front-end melt down in a hurry. That's a major riskto the business.
It's also one reason whytraffic management is usually one of the first managementfeatures added to an open API. But usually there are both technical and businessreasons for this.
Technical folks want throttling, or rate-limiting, to regulate traffic flow. Your customer'sdevelopers might inadvertently write some inefficient code. Someone might try todownload all your data through your API. Rarely you might get some malicious use.Rate limiting or transaction throttling (X transactions per second usually) can be like acircuit breaker that protects your own apps and keeps your Web and API customershappy.
Business managers may need to measure consumption against a daily quota. Thismight be to find growing customers to upsell, or keep track of customer data usage(especially if you are paying for that data). Or they may want to create different or
custom 'service tiers. '
Pitfalls to avoid wi th Rate limits and quotasWe see a few API providers substitute quotas for rate limits telling developers theycan have X transactions per day. But just measuring against daily quotas can leaveyour back end exposed if you get a short burst of traffic. And 'per second ' rate limitsaren't a great way to create tiers of business-level consumption. You might need both.
Or, we see customers that define an SLA only on paper and say call us if you need
more. We had a SaaS customer that started this way but then found it was tough totell important customers they needed to manage their API usage. They eventuallyfound a good way to ease into the traffic management discussion - by first providing thecustomer with usage reports. (Look how all your requests come in Monday at 9 am, welove you but can you help us?)
And you may not consider all requests equal you may find that some requests containmany transactions or big messages, so a simple X requests per second rate limitmight not be enough.
-
8/8/2019 Is Your API Naked 9.2.09
7/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 7 July 21, 2009
So consider these questions for your open API roadmap:
What kinds of rate limi ting do you need?
Do you need to rate limit by developer, app, or customer organization?
Can you rate limit a particular client (key and IP address)?
Are all messages the same or do you need to adjust based on message size, orrecords per message, or bandwidth?
How do you throttle differently for your own web or internal apps vs. API traffic?
Do you need to buffer or queue requests?
Does your business need quotas on API usage? Do you need to keep track of daily or monthly usage to measure business
consumption?
Do you need to define different consumption levels for different service tiers?
If someone goes over the quota, what is the best business recourse? Call themup and upsell them? Or shut them off?
If you are paying for the data are you measuring this for billing and pricing?
How do you monitor and respond to traffic management? How will you know when someone goes over a rate limit or quota?
Are quota or rate limit overages a trend or a 'one time thing'?
Can you define flexible actions? (I.e. drop requests, do nothing, send an alert?)
Can you provide customers with data so they can help by metering themselves?
What about bad s tuff like unauthorized access, Denial-of-Service, or XML specificattacks?More on that in the next couple sections Security and Data protection
-
8/8/2019 Is Your API Naked 9.2.09
8/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 8 July 21, 2009
3. Do you need API keys? API Identity,Author ization, & AuthenticationWeve seen very few API providers with a completely open API almost allemploy at least one of these:
Identity - who is making an API request? Authentication - are they really are who they say they are?
Authorization are they allowed to do what they are trying to do?
Do you need them all? Maybe not. Some APIs only need only to establish identity and
dont really need to authenticate or authorize.
Compare the Google Maps and Twitter API Identity vs. AuthenticationTake Yahoo and Google maps they are fairly open. They want to know who you arebut they arent concerned what address you are looking up. So they use an API key toestablish identity, but dont authenticate or authorize. So if you use someone elses APIkey, its not good but not a serious security breach.
The API key lets them identify (most likely) who is making a API call so they can limit onthe number of requests you can make. Identity is important here to keep service volume
under control.
Then take Twitters API - open for looking up public information about a user, but otheroperations require authentication. So Twitter supports both username/passwordauthentication as well asOAuth. Twitter also has authorization checks in its code, sothat you cannot tweet on behalf of another user without either their password or anOAuth key to their account. This is an example of an API that implements bothauthenticationand authorization.
The API Key do you need one?API keys originated with the first public web services, like Yahoo and Google APIs. Thedevelopers wanted to have some way to establish identitywithout having the complexityof actually authenticating users with a password so, they came up with the API key,which is often a UUID or unique string. If the API key doesnt grant access to verysensitive data, it might not be critical to keep secret, so this use of the API key is easyfor the consumers of the API to use however they invoke the API.
http://en.wikipedia.org/wiki/OAuthhttp://en.wikipedia.org/wiki/OAuthhttp://en.wikipedia.org/wiki/OAuthhttp://en.wikipedia.org/wiki/OAuth -
8/8/2019 Is Your API Naked 9.2.09
9/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 9 July 21, 2009
An API key gives the API provider a way to (most of the time) know the identity of eachcaller to maintain a log and establish quotas by user(see the last section).
Usernames and Passwords again, see Twit terWith more sensitive data a simple, API key is not enough, unless you take measures toensure users keep the key secret. An alternative is username/password authentication,like the authentication scheme supported by the vast majority of secure web sites.
Its easiest to use HTTP Basic authentication that most websites use. The advantageof using this technology is that nearly all clients and servers support it. There is nospecial processing required, as long as the caller takes reasonable precautions to keepthe password secret.
Twitter simplifies things for their users by using usernames and passwords for APIauthentication. Especially with human users. Every time a user starts a Twitter client, iteither prompts for the username and password to use, or it fetches them from the disk(where it is somehow scrambled or encrypted where possible). So here it makes a lot ofsense to have the same username / password for the Twitter API that it used for theweb site.
Usernames and passwords also work well for application-to-applicationcommunications. The trick - the password must be stored securely, and if its beingused by a server, where do you store it? If you are running an application server thatuses a database, you already have solved this same problem, because the database
usually requires a password too. Better application server platforms include acredential mapper that can be used to store such passwords relatively securely.
Session based Authentication cumbersome wi th RESTful APIsLots of APIs support session-based authentication. In these schemes, the user first hasto call a login method which takes the username and password as input and returns aunique session key. The user must include the session key in each request, and calllogout when they are done. This way, authentication is kept to a single pair of API callsand everything else is based on the session.
This type of authentication maps very naturally to a web site, because users are used tologging in when they start working with a particular site and logging out when theyare done.
However, session-based authentication is much more complex when associated with anAPI. It requires that the API client keep track of state, and depending on the type ofclient that can be anything from painful to impossible. Session-based authentication,
http://blog.sonoasystems.com/detail/dont_have_a_meltdown_api_traffic_management/http://blog.sonoasystems.com/detail/dont_have_a_meltdown_api_traffic_management/http://blog.sonoasystems.com/detail/dont_have_a_meltdown_api_traffic_management/http://blog.sonoasystems.com/detail/dont_have_a_meltdown_api_traffic_management/ -
8/8/2019 Is Your API Naked 9.2.09
10/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 10 July 21, 2009
among other things, makes your API less RESTful - an API client cant just make oneHTTP request, but now a minimum of three.
The Role of OAuthToday, many APIs also support OAuth authentication. OAuth was designed to solve theapplication-to-application security problem. The idea of OAuth is that it gives the user ofa service or web site a way to conditionally grant access to another application. OAuthmakes it possible for a human user to individually grant other APIs or sites access totheir account without sharing their actual password. It works by giving a token to eachAPI or site that will access the account, which the user may revoke at any time theywish.For instance, if web site FooBar.com wants access to the Twitter API on behalf of JohnSmith, then OAuth specifies the protocol that FooBar.com must go through to get anOAuth token for the Twitter API. Part of this process requires John Smith to log in to hisTwitter account using his normal username and password (which, in the OAuth protocol,are never seen by FooBar.com) and authorize FooBar to access his account. The resultis that FooBar.com will have an OAuth token that gives it access to John Smithsaccount. If, later on, John Smith decides he no longer trusts FooBar.com, he has theoption to revoke that OAuth token without affecting his regular password or any otheraccounts.
This process makes OAuth the ideal for communication from one application to another for instance, allowing MySpace to post photos to your Twitter account withoutrequiring that you enter your Twitter password every time you want to do it but it can
be used for any kind of API communications as well.
Two-Way SSL, X.509, SAML, WS-Security...Once we leave the world of plain HTTP we encounter many other ways ofauthentication, from SAML, X.509 certificates and two-way SSL, which are based onsecure distribution of public keys to individual clients, to the various WS-Securityspecifications, which build upon SOAP to provide... well, just about everything.
An API that will primarily be used by enterprise customers that is, big IT
organizations might consider these other authentication mechanisms such as an.X.509 certificate or SAML for more assurance over authentication process than a simpleusername/password. Also, a large enterprise may have an easier time accessing anAPI written to the SOAP standard because those tools can import a WSDL file andgenerate an API client in a few minutes.
The idea is to know your audience. If your audience is a fan of SOAP and WSDL,
-
8/8/2019 Is Your API Naked 9.2.09
11/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 11 July 21, 2009
then consider some of the more SOAP-y authentication mechanisms like SAML and thevarious WS-Security specifications. (And if you have no idea what Im talking about,then your audience probably isnt in that category!)
Rolling Your OwnIn between OAuth, HTTP Basic, and the basic API key are many alternatives. It seemsthat there are as many otherAPI authentication schemes as there are APIs. AmazonWeb Services, Facebook, and some Google APIs, for instance, are some big APIs thatcombine an API key with both public and secret data, usually through some sort ofencryption code, to generate a secure token for each request.
The issue every new authentication scheme requires API clients to implement it. Onthe other hand, OAuth and HTTP Basic authentication are already supported by many
tools. The big guys may be able to get away with defining their own authenticationstandards but its tough for every API to do things its own way.
SSLMost authentication parameters are useless, or even dangerous, without SSL. Forinstance, in HTTP Basic authentication the API must be able to see the password theclient used, so the password is encoded but not encrypted in a format called base64. Its easy turn this back into the real password. The antidote to this is to use SSL toencrypt everything sent between client and server. This is usually easy to do and does
not add as much of a performance penalty as people often think. (WithSonoasproducts, we see a 10-15 percent drop in throughput when we add SSL encryption.)
Some security schemes, such as OAuth, are designed to be resistant to eavesdropping.For instance, OAuth includes a nonce on each request, so that even if a clientintercepts an OAuth token on one request, they cannot use it on the next. Still, there islikely other sensitive information in the request as well.
In general, if your API does not use SSL it also potentially exposes everything that yourAPI does.
http://www.sonoasystems.com/technologyhttp://www.sonoasystems.com/technologyhttp://www.sonoasystems.com/technologyhttp://www.sonoasystems.com/technologyhttp://www.sonoasystems.com/technologyhttp://www.sonoasystems.com/technology -
8/8/2019 Is Your API Naked 9.2.09
12/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 12 July 21, 2009
4. Dont roll your own: API SecurityRecommendationsBased on the previous points, heres a short summary of recommendations:
Use API Keys for non-sensitive data (only):
If you have an open API - one that exposes data youd make public on the Internetanyway - consider issuing non-sensitive API keys. These are easy to manipulate andstill give you a way to identify users. Armed with an API key, you have the option ofestablishing a quota for your API, or at least monitoring usage by user. Without one, allyou have to go on is an IP address, and those are a lot less reliable than you might
think. (Why dont web sites issue web site keys? Because they have cookies.)
For example, the Yahoo Maps Geocoding API issues API keys so it can track its usersand establish a quota, but the data that it returns is not sensitive so its not critical tokeep the key secret.
Use username / password authentication for APIs based on web sites:
First, if your API is based on a web site, support username/password authentication.That way, users can access your site using the API the same way that they access it
using their web browser. For example, the Twitter API supports username/passwordauthentication, so when you access it using a Twitter API client like Spaz or TweetDeckyou simply enter the same password you use when you use the twitter.com web site.
However, if you do this, you may want to avoid session-based authentication,especially if you want people to be able to write API clients in lots of environments. It isvery simple to use curl, for instance, to grab some data from the Twitter API because ituses HTTP Basic authentication. It is a lot more complex if I instead have to call login,and extract a session ID from some cookie or header, and then pass that to the real APIcall I want to make...
OAuth for server-to-server APIs that are based on web sites:
If your API is based on a web site (so you already have a username / password for eachaccount) and the API will also be used by other sites, then support OAuth in addition tousername / password authentication. This gives users a lot of peace of mind, becausethey can authorize another site to access their sensitive data without giving that sitetheir password permanently.
-
8/8/2019 Is Your API Naked 9.2.09
13/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 13 July 21, 2009
SSL for everything sensitive:
Unless your API has only open and non-sensitive data, support SSL, and consider evenenforcing it (that is, redirect any API traffic on the non-SSL port to the SSL port). Itmakes other authentication schemes more secure, and keeps your users private APIdata from prying eyes and its not all that hard to do.
Dont roll your own:
If the above rules dont apply to you, then keep looking between OAuth, OpenID,SAML, HTTP authentication, and WS-Security, there are a lot of authenticationschemes, and each has its pros and cons.
So over the last 3 entries based on what weve seen in the past couple years - hereare some suggested questions you may want to ask when putting together your APIsecurity roadmap:
How valuable is the data exposed by your API? If its not that valuable, or if you plan to make it as open as possible, is an API
enough to uniquely identify users?
If it is valuable, can you reuse username and password scheme for each user?
Are you using SSL? Many authentication schemes are vulnerable without it.
What other schemes and websites wi ll your API and users want to integrate with? If your API will be called programmatically by other APIs, or if your API is linked
to another web site that requires authentication, have you considered OAuth?
If you have username/password authentication, have you considered OpenID?
Can you make authorization decisions in a central place?
What other expectations might your customers have? If your customers are enterprise customers, would they feel better about SAML
-
8/8/2019 Is Your API Naked 9.2.09
14/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 14 July 21, 2009
or X.509 certificates?
Can you change or support more than one your authentication approach fordiverse enterprise customers?
Do you have an existing internal security infrastructure that you need your API tointeract with?
Up next: API Data Protection
-
8/8/2019 Is Your API Naked 9.2.09
15/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 15 July 21, 2009
5. Indecent Exposure: Data Protection None of us ever want to see accidental customer data exposure through
our APIs. Especially likeParisexperienced last year (an API auth issue).
Last time we talked about starting with strong practices inidentity,authentication, and authorization. But another important aspect of APImanagement is data protection - specifically:
Encryption protecting your API data from eavesdropping
Threat detection ensuring client API requests wont cause back-end problems
Data masking preventing inadvertently data exposure in API responses.
First things first - protecting sensitive data requires knowing what and where it is, andhow sensitive it is to you and your customers.Be aware of the various regulations andbest practices for Social Security numbers, credit card numbers, home addresses,birthdays, and such.
Once you have identified which data is sensitive, you can think about how to bestprotect it in your API.EncryptionEncryption is a basic technology that must be part of any API with sensitive data. Somerelevant technologies include SSL and XML encryption.For most APIs, the most critical encryption mechanism is SSL. It works on everyplatform, and the overhead is minimal (about 5% through our ownAPI gateway productin benchmark tests.) Using SSL to encrypt any sensitive data is the least your APIshould do.Another alternative is to encrypt either all or part of each message using XMLEncryption. (This is a W3C standard implemented by many SOA products.) However,
this requires you and your clients to manage public/private key pairs, so deploying thistechnology can be more complex and it has a larger performance impact. However,XML Encryption is tremendously useful when its important to manage sensitive databehind the API. For instance, if API data must not only be transmitted securely over theInternet, but also stored in internal systems in encrypted form on a disk or in adatabase. Otherwise, stick with SSL.Threat Detection
http://valleywag.gawker.com/5012543/paris-hilton-lindsay-lohan-private-pics-exposed-by-yahoo-hackhttp://valleywag.gawker.com/5012543/paris-hilton-lindsay-lohan-private-pics-exposed-by-yahoo-hackhttp://valleywag.gawker.com/5012543/paris-hilton-lindsay-lohan-private-pics-exposed-by-yahoo-hackhttp://blog.sonoasystems.com/detail/do_you_need_api_keys_api_identity_vs._authorization/http://blog.sonoasystems.com/detail/do_you_need_api_keys_api_identity_vs._authorization/http://blog.sonoasystems.com/detail/do_you_need_api_keys_api_identity_vs._authorization/http://blog.sonoasystems.com/detail/do_you_need_api_keys_api_identity_vs._authorization/http://www.sonoasystems.com/solutions/managementhttp://www.sonoasystems.com/solutions/managementhttp://www.sonoasystems.com/solutions/managementhttp://www.sonoasystems.com/solutions/managementhttp://blog.sonoasystems.com/detail/do_you_need_api_keys_api_identity_vs._authorization/http://blog.sonoasystems.com/detail/do_you_need_api_keys_api_identity_vs._authorization/http://valleywag.gawker.com/5012543/paris-hilton-lindsay-lohan-private-pics-exposed-by-yahoo-hack -
8/8/2019 Is Your API Naked 9.2.09
16/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 16 July 21, 2009
Any server that receives data over the Internet is subject to attack. Some attacks aremore specific to an API and merit additional consideration.The first is SQL injection. This attack takes advantage of internal systems that constructdatabase queries using string concatenation. If theres a way to take data from the clientand paste it inside a database query, then there may be a way to compromise thesystem using SQL injection. The best way to prevent SQL injection is to code the back-end systems so that an SQL injection attempt is not possible. But its also important tostop SQL injection attempts before they get to the back end.The second is an XML attack - taking advantage of the flexibility of XML by constructinga document that could cause a problem for a back-end system. For example, causingthe XML software to try to allocate more memory than is available. Or, an XMLdocument that is nested many levels deep, or with extremely large entity names or
comments. A simple check to ensure that XML or JSON is well-formed can saveresources on the back end that would otherwise be devoted to generating and loggingerror messages.These attacks arent always intentional. Ever used an API like StAX to construct anXML document, but forgotten to add all the end tags? An invalid XML document thatappears to be nested very deep can cause problems for the back end servers, or atleast tie up more CPU and memory resources.Fortunately, there are many products, including SonoaServiceNet, that contain pre-defined policies to defend against these types of attacks.Data MaskingEncryption is critical to keep sensitive data private, But in some cases, it may makesense to try and re-use internal services and data.
However, you might need to screen or mask- some private data for the API. Thismeans using an XML or JSON transformation to either remove or obfuscate certain datafrom an API response.While this technique must be used with care there may be cases in which only certain
API users are authorized to see certain information. For example, there might be an APIcall that returns a user record with all the details when the user him or herself calls thisAPI, but only limited data when a customer service reps access the user record usingthe API.
In that case, you could implement this by building only one version of the API on a backend server and adding a data transformation rule that removes the users home addressif the request is coming from a CSRs account. If you have many services, you might
http://www.sonoasystems.com/technologyhttp://www.sonoasystems.com/technologyhttp://www.sonoasystems.com/technologyhttp://www.sonoasystems.com/technology -
8/8/2019 Is Your API Naked 9.2.09
17/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 17 July 21, 2009
consider having a common layer that performs these types of transformations especially if you may need to add certain data fields as well as masking fields. (moreon that in the next section on API Mediation).
Possible recommendations for your API roadmap Use SSL when the API includes sensitive data, or if the authentication
mechanism in use does not include an encryption component. (HTTP basicauthentication, for instance, allows an eavesdropper to intercept the passwordunless SSL is used; OAuth does not.)
Always defend against SQL injection, either in the back end server, at the edgeof the network, or both.
If your API accepts XML input via HTTP POST or some other way, then defendagainst the many types of XML attacks. These include large inputs, payloads orattachments, header bombs, replay attacks, message tampering and more.
Consider using data masking in a common transformation layer if your back endservers may return some data that should not be given out to all users of the API.
(and thanks tocarbonnycfor the photo)
http://www.flickr.com/photos/carbonnyc/57280104http://www.flickr.com/photos/carbonnyc/57280104http://www.flickr.com/photos/carbonnyc/57280104http://www.flickr.com/photos/carbonnyc/57280104 -
8/8/2019 Is Your API Naked 9.2.09
18/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 18 July 21, 2009
6. One size doesnt fit all: API Versioning and Mediation
TrueCredit.com tellsa storyof calculating they would needthousands of IP addresses for all the different versions andflavors of their open API - to account for different variations andversions needed by partners.
Even if you have a one sized fits all API for starters - you mightneed to be able to transform data, mediate terms or customizeSLAs without coding each change or a creating a new version of
the API. Reasons could include:
Protocol needs - We have a SaaS customer with a REST API that had animportant deal on the table with a bank. The Bank insisted on a SOAP API withWS-Security. Some SOAP shops want to offer a RESTful API because itseasier for developers to work with. And we see lots of needs to transformbetween different syntaxes of SOAP, REST, or REST/JSON the list goes on.
Monetization You might want to sell a premium version of your one size fits allfree API. For example, a search API provider wanted to do a BD deal with itsfree API and needed to insert extra data the partner wanted to pay for.
Standardization If you have a lot of APIs built by different teams one dayyou might want to enforce some consistency. A customer of ours grew from 1API to 40, and needed to add some standard fields to each API without needing
to coordinate 10 teams to write code.
Versioning - Ever used an API where you get an email every month asking you toupgrade to a new version? TrueCredit wanted to provide API upgrades tocustomers that needed it while holding the API fixed for everybody else longer, toreduce versioning headaches.
So you may need to figure out how you to provide and manage different flavors orversions of the same API or mediate (or transform) API content and syntax.
Alternatives might be to (a) support multiple APIs (painful),(b) hold off as long aspossible and push back on customers to snap to a one size fits all model (more
painful), or (c ) create a mediation capability or layer that can transform betweendifferent shapes of the API protocol, data, version, credentials, etc.
(And going back to TrueCredits story at the top, this is what led them to think about anAPI gatewayfor mediation, caching, load balancing, and more.)
So ask if and when any of these issues might apply to your roadmap:
Will you need to mediate protocols?
http://www.sonoasystems.com/customer-success/truecredit-api-infrastructurehttp://www.sonoasystems.com/customer-success/truecredit-api-infrastructurehttp://www.sonoasystems.com/customer-success/truecredit-api-infrastructurehttp://www.sonoasystems.com/technologyhttp://www.sonoasystems.com/technologyhttp://www.sonoasystems.com/technologyhttp://www.sonoasystems.com/customer-success/truecredit-api-infrastructure -
8/8/2019 Is Your API Naked 9.2.09
19/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 19 July 21, 2009
Do you anticipate needing to offer more than one protocol or a different protocolto what you have now? (SOAP for enterprise customers? REST or JSON fordeveloper adoption? )
Do you anticipate needing to map across different security or credentialschemes? (ex: from simple HTTP auth to WS-Security)
Do you currently write a lot of code to transform between different styles of aparticular protocol (SOAP 1.1 vs. 1.2, etc.)
How important will it be to reduce the number of APIs versions for developmentand maintenance?
Version management How often will you need to release upgrades to the API? What is your process
for asking customers to upgrade and how long will it take to sunset a version?
If you offer more than one API, do you have a need to standardize elements ofthe API (header or payload)? Do different teams need to do this or does it makesense to put this capability at one point?
Message enrichment, clipping
Will you ever need to enrich an API for a particular customer or class of servicewith more data or functionality? (such as a big customer that licenses moredata..)
Will you need to remove or clip certain fields for certain customers or classes ofservice?
How fast will you need to turnaround these requests for the business vs. yourdev or product cycle?
A mediation layer (see more flavorshere) can be important to handle complexity so youcan focus development on business specific API capabilities..
(and thanks tocollinandersonfor the photo)
http://www.sonoasystems.com/product-matrixhttp://www.sonoasystems.com/product-matrixhttp://www.sonoasystems.com/product-matrixhttp://www.flickr.com/photos/collinanderson/3082868600/sizes/s/http://www.flickr.com/photos/collinanderson/3082868600/sizes/s/http://www.flickr.com/photos/collinanderson/3082868600/sizes/s/http://www.flickr.com/photos/collinanderson/3082868600/sizes/s/http://www.sonoasystems.com/product-matrix -
8/8/2019 Is Your API Naked 9.2.09
20/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 20 July 21, 2009
7. Welcome Aboard: API User ManagementJust like you have processes to add and maintain user accounts with
your website, you'll need the same for your API.
Making it fast and easy for developers to quickly get API accountdetails squared away critical to reduce barriers to adoption. And good
user management can pay off by reducing your support costs and increasing customersatisfaction.
This includes setting developers up with an account and any authentication credentialsthey need to work with your API. But you'll also need UI and processes to add, maintain(such as reset) and revoke or delete these accounts.
You may also need to take into account how to hand out keys or OAuth keys, and evenconsider an approval process, if you are going that route.
Also, consider whether you can make things simpler by defining rights around standarduser profile. For example, bronze customers have access to certain APIs or requestsvs. gold customers.
Finally, consider any other information you can give developers that will make it fasterand easier to work with you - to get their questions answered without calling you. Forexample, usage statistics and alerts on versioning upgrades and outages.
Do you need to start from scratch?Before building or buying this, consider if you might be able to extend some otherexisting user management UI and processes to work for developers requesting accessto your API.
Key to this decision is - how much integration do you need into your existing businessprocesses? Do you need to create accounts in your CRM system, and make sure youhave enough developer information to map developers to applications, and customers?
Consider the following user management questions for your API roadmap:
For on-boarding developers Do you already have a way to manage user accounts that you plan to re-use for
your API? Or have you considered it?
If you have, do you also wish to offer OAuth keys?
-
8/8/2019 Is Your API Naked 9.2.09
21/30
-
8/8/2019 Is Your API Naked 9.2.09
22/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 22 July 21, 2009
8. If you build it wil l they come?: APICommunity and AudienceYou've built a great API, and have all the operational basescovered. But it doesn't matter if nobody knows about it.
Awareness and adoption are likely a big reason you might open an API. So if youthink of it like throwing a party - you need to make sure people know it's happening,guests show up, have a great time, and make it an interesting.
(And while many APIs are private such as a SaaS API only for your partners youstill need to do a lot of this.)
Developers, developers, developersDevelopers make or break any 'openness' strategy. They are your greatest source of
feedback, your best advocates, your toughest critics. If you keep your eye on makingdevelopers successful you are building on a very strong foundation.
But first you must have a cool product.I once sawDave McCluregive an super developer programs presentationa while back.Really agreed with his key point it all starts with a great product and we'll write aseparate entry on this.
Developer CommunityOne of the best ways to make developers successful is to put them in touch with each
other so that they can share information, tips and tricks, and creativity in working withyour API that you may never figure out on your own. That's why some of the mostsuccessful software companies have the most vibrant developer communities.
So you need to put down the tools that help developers work together and discuss andshare knowledge about your APIs. Blogs, wikis with documentation, newsgroups,venues for code examples are all important. One of the best tools for your developersare code samples that they can incorporate in their own apps, so having a place for not
just your own sample apps and widgets but also for developers share their own iscritical.
Plug into other developer communities!While building your own community can be important, there are lots of existing, largedeveloper communities that you should plug into right away. You can sign up for thesefor free. They depend on and are all set up to distribute and promote your great content(your API and samples) These might include:
Vendor communities likeMicrosoft's MSDNor Google's COMMUNITY works.These guys might be your partners, but if they are not it might not matter plug
http://video.google.com/videoplay?docid=8913084255008000794http://video.google.com/videoplay?docid=8913084255008000794http://500hats.typepad.com/http://500hats.typepad.com/http://500hats.typepad.com/http://www.slideshare.net/dmc500hats/creating-successful-developer-programshttp://www.slideshare.net/dmc500hats/creating-successful-developer-programshttp://msdn.microsoft.com/en-us/default.aspxhttp://msdn.microsoft.com/en-us/default.aspxhttp://msdn.microsoft.com/en-us/default.aspxhttp://msdn.microsoft.com/en-us/default.aspxhttp://www.slideshare.net/dmc500hats/creating-successful-developer-programshttp://500hats.typepad.com/http://video.google.com/videoplay?docid=8913084255008000794 -
8/8/2019 Is Your API Naked 9.2.09
23/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 23 July 21, 2009
in anyway. Even if you compete or do not build on those platforms there will bedevelopers that work with those platforms that use your APIs)
Platform communities, such as RubyOnRails, or iPhone
Independent communities likeStackoverflow. These existing communities haveaudience and depend on great content from API providers and developers.
API Directories and communities - like ProgrammableWeb. These are wheredevelopers looking for great APIs and information about them come.
Don't forget about AudienceThink of launching an API like throwing a party. Sure, you need a great venue, food,
and music. But how do you make sure you get the word out to make sure lot of guestsshow up? We see lots of API strategies that focus on the community tools, but toolsdoes not equal a community. If you can't find an *audience* , it's like a party with greatcatering that nobody knows about your.
A big part of this is making sure your API is highly ranked in Google searches onanything releated to API + your business . There might be 10 other APIs in yourvertical space you need to be near the top of the list. This is another reason whybeing on all those other community sites, directories and discussion boards is soimportant, as they give you the links that raise your Google score that get you found,that get you used, that get you found, etc.
You can't outsource passionShould you outsource community? Be careful. The best communties have a 'hotcore' of very active members and the ringleader is usually an incredibly passionateevangelist that knows each of these folks personally, and is obsessed with tracking andimmediately responding to any question or issue, or leaping on any opportunity to talkabout how cool your API is.
Can you outsource that passion? We recommend not find one dedicated, maniacalevangelist in your organization to nurture community.
Finally, consider evangelizing your developers if they are doing great work with your
API or platform, give them props and make them stars.
Your Roadmap: Audience and CommunitySo if you are opening and API, think about the following for your community roadmapand don't forget about Audience
http://stackoverflow.com/http://stackoverflow.com/http://stackoverflow.com/http://stackoverflow.com/ -
8/8/2019 Is Your API Naked 9.2.09
24/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 24 July 21, 2009
Community tools Do you have formal documentation? Can you put it on a wiki so that developers
can edit, add, and comment?
Do you have code samples on how to use the API?
Do you have a place for developers to put their own code samples and showcasetheir own work and sample apps? (widgets, mobile apps, etc.)
Are code libraries needed for important platforms? Should these be opensource?
Do you have a blog for best practices and a way to get in touch with developerson important changes (such as API version updates?)
Audience and distribution Can you get awareness and istribution through existing developer communities,
such as any vendor (MS, Google IBM), Platform (Ruby, iPhone), Independent,Directories (programmableweb)
What Web marketing or SEO/SEM (Search Engine Optimization or SearchEngine Marketing (Adwords) can you do to make sure developers find you whensearching for API + your type of content or business.
Community support and toolsCommunitymanagement
Should you have a dedicated full-time employee to drive community andevangelize your product and best developers?
Are there any offline events or meetups you should be at?
How can you recognize and promote your hardcore community members? Doyou have an evanglist that knows these folks personally?
What about adding and managing developers?More on that in a future section User Management
-
8/8/2019 Is Your API Naked 9.2.09
25/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 25 July 21, 2009
9. Show me the money: MonetizationAt some point most API product managers are on the hook todemonstrate how the API results in downstream revenue.
And even if you never intend to charge for your API, you may be surprised byunexpected opportunities.
I used to be responsible for a number of free open, take-it-or-leave-it APIs at a largeWeb portal. After a few months, the team was surprised to get a ton of calls from bigcompanies that wanted to pay (a lot) for use of the API within their commercial products.
The catch they expected that the API could support some of the basic business termsof any commercial deal. For example could the API provider:
Meter service and give them a bill every month?
Enforce a special rate limit give them an SLA (service level agreement)?
Insert extra data fields if they were willing to pay for more licensed content?
We also needed some other basics:
SOX compliant audit logs for each deal.
Reporting on the cost of the data served by the API. (because we were licensingit from another company)
The means to enforce similar terms on a package of multiple APIs together inone deal
Unfortunately, all of these capabilities needed to be coded as they came up and wecouldnt lock down most of these deals in time.
So even if yourAPI business modelis already covered and baked in your API youmay want at least the option to support a slightly tailored or customized relationship.You may need to quickly tailor the APIs syntax, protocol, priority, or content to a specificopportunity.
Once you have partners using an API, your business managers and partners will putyour team under a lot of pressure - each day a policy change goes unmade, dollars arelost.
So consider if youll ever need to abstract these policies the difference between theAPI content and features that and the commercial business and operational terms ofeach deal in an a separate policy layer that you can manage quickly andindependently of development cycles. Or to make a single change across multiple
http://blog.sonoasystems.com/detail/api_business_models_and_monetization/http://blog.sonoasystems.com/detail/api_business_models_and_monetization/http://blog.sonoasystems.com/detail/api_business_models_and_monetization/http://blog.sonoasystems.com/detail/api_business_models_and_monetization/ -
8/8/2019 Is Your API Naked 9.2.09
26/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 26 July 21, 2009
APIs.
Therefore, some questions to ask for your API roadmap might include:
General business and partner model questions: How is your business and revenue model supported by your API? (are you
looking for distribution and awareness only? Does the API drive a monetizedtransaction? Will you ever charge for pay by the API drink?)
How are your costs reflected in your API? Do you pay for any of the data you areserving up? If so, how do you keep track of this for the business and partner?
Will large partners or deals surface where your team will need to change the APIcontent, SLA, protocol or content? Is there a partner that might want to payenough and who is large enough to drive your team to move a way from one
size fits all?
Will you need to create service tiers?
Enforcing unique business and operational terms How will you meter service like a utility, so that you can bill partners and report
data costs?
What regulatory compliance will you need to support? Do you need SOX-compliant audit trails by partner? HIPPA? PCI?
How would you create and enforce a partner specific SLA, rate limit, or offerpriority access to a priority partner?
If the partner wants any change in the API protocol or content do you need tosupport a different API code-base? Is there a way to create a transformationlayer to handle and scale this?
Do you need to buffer up or queue incoming requests?
-
8/8/2019 Is Your API Naked 9.2.09
27/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 27 July 21, 2009
10. Turn up the volume: API Scalabil ityOur discussion of APIs so far has focused on aspects like security,visibility, and data protection. We've been leaving out a very importantconcept, though - how do you make your API scale?
The term "scale" means different things to different people, so let's narrow it down. Thequestion is, what are you going to do as your traffic increases? Do you have a plan tohandle 10, 100, or 10,000 times more traffic than your API is receiving today?
The truth is that solving this problem at the truly high end can require fundamentalchanges to your architecture and code. The kind of engineering required to run an APIthat accepts a few requests per second is very different from what's required to scale tothe size of Facebook or Twitter. However, the chances of any particular API needing to
scale to that level, of course, are pretty slim..
Writing about all the dimensions of scalability and how to achieve them is a subject for apretty long book. But here are a few specific things to think about:
Caching
Rate limiting and threat protection
Offloading expensive processing
CachingCaching is a huge part of any scaling strategy. Whole products and web sites are builtaround caching as a fundamental architectural concept. Caching works because even inan API, usually much of the data that's returned is read. A good caching strategy candecrease latency by huge percentages, and improves throughput by taking load offexpensive back-end servers and databases.
Typically, caching today is done in a few different places. Caching between theapplication server and database using a product like Coherence or memcached helpsby reducing the number of database queries needed to serve an API request. Additional
caching inside the application server code can further decrease throughput by making itpossible to reassemble parts of an API response from pre-cached parts. For instance,the response to a typical Twitter API call consists of an arrangement of many individualsnippets of XML, each of which may have been cached to reduce the overhead neededto fetch data from the database and convert it into XML.
HTTP and CDN caching
Caching at the HTTP level is also very common. The HTTP protocol supports several
-
8/8/2019 Is Your API Naked 9.2.09
28/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 28 July 21, 2009
headers that enable caching by proxy servers and of course by the web browser itself.This type of caching works fine for API calls. However, since it's based only on theHTTP request, it doesn't work for everything. Imagine a SOAP API, for instance, whichincludes an HTTP POST body describing the request - HTTP-only proxies cannot cache
the response because they can't look inside the request and see what needs to becached.
Similarly, CDNs like Akamai work by caching pieces of content all over the Internet sothat users will receive it from a server near them. Caching strategies like this are greatfor big files that don't change often, but by design they are poor at data that changesmore than every day or so.
Caching API responses
With APIs, it's also possible to add caching in between, by caching entire APIresponses. For instance, imagine a "getAccount" method on an API. Software likeSonoa ServiceNet can look at the parameters to the request and use them as a "key" tothe cache. If a response already exists in the cache, ServiceNet simply returns theexact response from the back-end server from its in-memory cache. This cuts out all thelatency to make an HTTP request to the application server, process the request, querythe database, assemble the response, and so on, and unlike a cache that works only atthe HTTP level, it can be configured to understand the semantics of the API so that itcaches effectively and correctly. There are also ways to purge cache itemsprogrammatically, so that the "updateAccount" API method ensures that the next call to"getAccount" won't return stale data.
In other words, adding a intelligent caching layer between the client of an API and the
back end application server adds more caching options that can increase scaling anddecrease latency even further than some alternatives because this cache is closer tothe API client. Plus, due to the magic of HTTP, XML and/or JSON, it's possible for thiscaching to be performed without any changes to the API client or server.
This means that a highly scalable web site could use caching in different tiers, each withits own contribution to overall API performance:
A cache like Coherence or memcached may be used between the applicationserver and database to reduce database load.
A similar cache may also be used from within the application server to reduce the
overhead of assembling API responses.
A proxy like Sonoa ServiceNet can cache complete API responses, reducingload on the application server tier.
A CDN like Akamai can provide an extra caching layer for large files that do notchange often.
-
8/8/2019 Is Your API Naked 9.2.09
29/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Page 29 July 21, 2009
Rate Limi ting and Threat ProtectionAnother aspect of scaling is just keeping unnecessary traffic away from your applicationservers and databases. Some of the techniques that we've discussed previously, suchas rate limits and threat protection, apply here as well.
For instance, an API's performance can drop precipitously if a client, on purpose or byaccident, sends too much traffic. A rate limit helps a lot here!
Bad requests can kill API performance too. XML threats, which we discussed in the lastepisode, are one example of a way that a bad request from a client can causeperformance problems or even a crash on the server side. It's a lot easier to maintainscalability if you can stop these kinds of problems before they can hurt your servers.
OffloadingFinally, consider the things that you can offload from your application server tier. Themore you can offload to more efficient platforms, the less load your application servershave to handle. Plus, the more things you can offload, the simpler those applicationservers and their applications become, which means they're easier to manage andeasier to scale.
For example:
SSL. Load balancers and ADCs like F5 and NetScaler products, not to mentionweb service proxies like Sonoa ServiceNet, can process SSL more efficientlythan most application servers.
HTTP Connections. Those same products are highly optimized to handle tens ofthousands of simultaneous connections from HTTP clients, and operate asmaller pool of connections to the back-end application servers. OffloadingHTTP connection handling to another tier can free up a lot of server resources.
Authentication. If you perform authentication, a proxy like Sonoa ServiceNet canhandle all your authentication for you, freeing your application servers to worryonly about properly-authenticated requests. And if you're using SOAP, a product
like ServiceNet can process many of the SOAP headers, such as WS-Securityheaders for authentication, then remove them so that the application serverdoesn't even need to see them.
Validation. If your API depends on XML input, it may run more efficiently if it onlyaccepts valid XML requests. Turning on XML schema validation can hurtperformance of most application servers - products like ServiceNet can do it moreefficiently.
-
8/8/2019 Is Your API Naked 9.2.09
30/30
Is y our API Naked? 10 API Roadmap Con siderat ions
Key Questions to ask for your roadmap:
What kind of volume are you expecting?
Are you prepared if you get 10, 100, or 10,000 times that amount of volume withlittle warning?
Do you have a way to shut a user off if they consume too much volume?
Do you have a way to control API traffic in case you are unable to handle thevolume (see Traffic Management)
Are your back end servers capable of handling tens of thousands of concurrentconnections?
Are your back end services cacheable? Do you have a cache that you can use toreduce response times?
Are you monitoring response times and tracking them to gauge customersatisfaction?