Tuesday, October 23, 2012

Job posting: Vordel API Gateway administration at Blackhawk Network in California

Blackhawk Network, a subsidiary of Safeway Inc located in Pleasanton, California, has just posted an open position of Senior Security Systems Engineer on LinkedIn. The role includes substantial Vordel expertise, so readers of this blog might be interested.

The job description is:
This role is vital to the ongoing operations, development and maintenance of key systems at Blackhawk Network. The primary role is the project management and leadership of the projects, maintenance, and operations of our primary Application Gateway, Vordel running both as software components, and as appliances. You will play a key role operating and improving current environments, as well as defining future environments for API gateways and key management. A talent for process improvement, customer focus, operational excellence, and working with multiple teams will help you excel in this role.
If this sounds like a position for you, check it out on LinkedIn at the URL above. 

Pro tip: Running Vordel API Gateway analytics as a Windows service

[ Update: Axway acquired Vordel in 2012 and the new name for the Vordel Gateway is the Axway API Gateway ]

If you've downloaded the Vordel API Gateway v7.1 from the Vordel website and want to configure it to run analytics as a Windows service, it's quite simple to do this. The command is:

installservice.bat "C:\Vordel-7.1.0\analytics" "analytics" "7.1" analytics.xml

Where "analytics" is the name you want to give the service, and "7.1" is the version number. The installservice.bat script can be found under the /win32/bin folder of the API Server installation.

Once you run this, hey presto, you'll see Vordel API Gateway analytics in your list of Windows services:

Note: Ensure you run the installservice.bat with the permissions to create a Windows Service, otherwise you will see a "Access is denied. . Could not open service control manager" error.

Video: API Analytics in the Vordel API Server v7.1

This week I am down in Dallas visiting Vordel customers. Today I'm visiting a customer which uses the analytics component of the Vordel API Server to analyse the usage of its APIs. They are upgrading to the new Vordel v7.1 and are excited about the new functionality provided for API analytics. I've put together a video, below, of the new functionality in Vordel v7.1 API analytics. This includes the generation of PDF reports right from the web interface itself, and the ability to drill into traffic by dragging and selecting it.


Monday, October 22, 2012

At the API Strategy and Practice Conference next week in New York City

Next Thursday (Nov 1) in New York City I'm speaking at the API Strategy and Practice conference.

First up, I am speaking on API Security and Stability at 3pm. Following that, I'm on a panel with the other speakers on the same topic: Paul Madsen of Ping Identity and Travis Reeder of Iron.io. It will be interesting to talk about the take-up of OAuth 2.0, and also some of the interesting deployments we are seeing for mobile payments APIs (which of course need security and stability, it goes without saying).

Next up is the API Infrastructure Providers panel at 5.10pm. This panel also has Paul Madsen of Ping Identity, this time with Andy Thurai of Intel, someone (unnamed so far) from SOA Software, Steve Willmott of 3Scale, Chris Haddad of WSO2, Evgeny Popov of APIphany, and me.

Vordel is a Gold Sponsor of the API Strategy and Practice conference and we'll be exhibiting on both the Thursday and Friday, and giving away a Google Nexus 7 tablet (complete with NFC so you can use it with mobile payments APIs :-) ). Looking forward to seeing you there!

UPDATE: This conference has been postponed due to Hurricane Sandy. New dates will be announced. 

Pro Tip: Upgrading a previous configuration to Vordel API Server 7.1

You can now download the Vordel API Server v7.1 right off the Vordel website. It comes with handy QuickStart samples to walk you through API registration and management. 

But what if you have an older Vordel installation and you want to upgrade this configuration to run it in the Vordel API Server v7.1? This is easily done using the upgradeconfig script. The upgradeconfig.bat script can be found in the \win32\bin folder of a Vordel 7.1 installation. There is a similar script provided with the Linux and Solaris versions. 

Here is how I upgraded a Vordel v6.3.0 configuration:

upgradeconfig.bat -d C:\Vordel\Vordel630\vordelgateway -o NewConfig

As you can see, I used the -d parameter to point the script to my older configuration. Then I used the -o parameter to point to the output folder where I want the new, updated, configuration to be placed. 

Once I upgraded the configuration, I can then replace an existing /conf/fed folder with the contents of my updated configuration (being sure to rename the old /conf/fed to something else). 

And voila, all my older configuration is updated to the brand new Vordel v7.1. 

Congratulations to Isis on its mobile payments launch today

Isis launched its NFC-enabled mobile payments in Austin and Salt Lake City today, and it's the subject of an interesting thread on the Vordel Community LinkedIn group.

If you want to get started with Isis mobile payments today, Isis has posted these helpful steps:

How to Get Started With the Isis Mobile Wallet on October 22: 

1. Visit an AT&T Mobility, T-Mobile USA or Verizon Wireless retail store in Austin or Salt Lake City to receive a secure element SIM and ensure your handset model is Isis Ready 

2. Download the Isis Mobile Wallet application from Google Play

3. Load an eligible American Express, Capital One or Chase credit card into the Isis Mobile Wallet or get started with the Isis Cash™ card that comes in every Wallet. 

4. Get out and get tapping at any of the hundreds of locations across Austin and Salt Lake City where contactless payments are accepted, a full list of which can be found at www.paywithisis.com

I've an NFC-enabled tablet and I'm thinking about a trip to Austin or Salt Lake City to try out Isis :-)

Wednesday, October 10, 2012

How to configure "blocking logging" in the Vordel API Server

"Blocking logging" is a common requirement for APIs, but what does it mean? In order to understand blocking logging, let's take a step back. If you're serving out an API to your clients, you usually want to ensure that all access is logged. This means that you can generate Analytics reports on your API usage, like this one:

Analytics depends on the presence of a database to record the API accesses. But what if the database is down, or otherwise not available? What do you want to happen then? There are two possible answers, depending on the API. You may want to:

a) Let the traffic through, so that it is monitored in the "Traffic" tab of the Vordel Manager web interface, written to the trace file, but just not written to the database for Analytics, or
b) Block the traffic, because you want to make absolutely sure that all traffic is recorded in the Analytics database. [note, even without the database, it is still recorded in the trace files and in the "Traffic" table of the Vordel Manager web interface].

Policy Studio makes it easy to configure either of these two scenarios. In each of the filters, you many notice a "Next" button:

If you click this button, you see the logging configuration screen. You have a lot of flexibility about how to log, including logging for tools like Splunk, as I explain here.

Notice the checkbox marked "Abort policy processing on database log error". This is where you can choose to continue processing even if the logging database is unavailable. It's where you decide whether you want scenario (a) or (b) above. All filters have this option.

Note that the trace and the "Traffic" table in Vordel Monitor are completely separate from the database logging. Database logging is for Analytics (i.e. reports over time on API usage, including PDF generation).

Webinar on how to Securely Exchange Healthcare Information in a Cloud & Mobile World

The webinar I recorded last week with Jason Cardinal from Perficient is now online. Jason provides many interesting metrics around healthcare IT, including the implications of the new healthcare laws here in the US. We look at some practical case studies in detail. We also look at mobile app security for healthcare, and how the Vordel API Server can address these issues.

Tuesday, October 2, 2012

Throttling API traffic based on client and API, with SLAs

[ Update: Axway acquired Vordel in 2012 and the new name for the Vordel Gateway is the Axway API Gateway ]

A neat feature provided by the Vordel API Gateway is the ability to limit Web Service or API access based on client identity. This is implemented by a throttle. The throttle is a filter within the Vordel API Gateway which applies traffic rate-limiting. It is extremely flexible. Let's look at how it works.

In the screenshot below, I've configured a throttle to limit API requests to 1000 per second. Notice that I have left the "Track per key" box unchecked. This means that across the board I am limiting traffic to my API to 1000 requests per second. It could be that a single client sends 1000 requests, then no more traffic from any client is accepted for the rest of that second. This configuration is often used when you know that the API can only take a certain amount of traffic [Tip: To find out the ceiling for an API's accepted traffic, before it falls over, you can use the "SR" free load-generation tool which comes with Vordel's products].

If you want to apply the throttle to a particular API, you can map it to that API based on the path (e.g. to "/myAPI"). Or, you could put it into a policy which you set as the Default Request Policy. To do this, simply right-click on the policy and choose "Set as Default Request Policy".

But what if you wanted to limit clients to only 10 requests per second? In this case, now you would use the "Track per key" checkbox. In the example below, I've checked "Track per key". I have chosen the default "Key value", which is the client IP address. This means that the throttle is limiting client access to 10 API requests per client IP address.

Sometimes it is too broad to limit based on IP address. Consider a company where clients come in through the same IP address for each branch office. In that case, you could choose authentication.subject.id as the key instead. Then, traffic is throttled based on authenticated client, even if some clients come in through the same IP address. Again, simple by right-clicking and selected "Set as default request policy", you can set this as the policy across-the-board for all your APIs. Or, you could just apply it to one API.

But what if you wanted to combine the rules, so you are saying "I want to limit traffic to my API to 1000 requests per second, but any given client can only do 10 requests per second". You can do this by simply chaining your throttles. In the policy below, I have chained the across-the-board 1000-requests-per-second throttle and the per-client 10-requests-per-second throttle:

In the screenshot above, you can see that I'm first applying the across-the-board 1000-requests-per-second throttle. If this passes, the flow follows the green arrow, then I apply the  per-client 10-requests-per-second throttle. But when message number 1001 arrives within the one-second timescale, the flow goes down the red arrow, to an Alert filter which I have placed there. You may notice also that I'm applying an SLA (Service Level Agreement) to the API itself, so that if it's running slower than expected, an alert is also raised. The screenshot above shows a very common API deployment scenario, combining throttling and SLA with alerting.

We can go further still. What if different clients have different limits, even to the same API? In this case, the throttling limits are often read from a database, or other store. I could have an SLA with each client where they are allowed a certain amount of API usage per second. Let's say I am authenticating the client based on a UserID in the query-string used to access the API. In that case, I could look up the throttling limit in the database using the API Server's "Read from or Write to Database" filter, like this:

The SQL could also take into account the client's group membership, as part of the SELECT statement. In the Advanced settings for this "Read from or write to database" filter, I've chosen that the attributes returned from the database are read into attributes called client.x where x is the name of the database table column name. I have a column called "SLA", so that means that the value will be read into an attribute called client.SLA. So, here is how I then enforce the limit, again using a throttle, but in this case I am using the value I've read from the database.

Note in the screenshot above that I'm using two wildcards. The first is the actual client ID (which, in this case, is sent in a querystring, but I could also have used authentication.subject.id) and the SLA limit I have read from the database.

Hopefully this blog post explains all of the powerful things you can do with the throttle in the Vordel API Gateway. Happy API throttling! :-)

Monday, October 1, 2012

SOAP-to-REST with Parameterized URLs and content-based routing using the Vordel API Gateway

[ Update: Axway acquired Vordel in 2012 and the new name for the Vordel Gateway is the Axway API Gateway ]

The world is moving from SOAP to REST. Many applications continue to send old-style SOAP messages. Therefore, a very common usage of the Vordel API Gateway is to convert from legacy SOAP messages into REST API calls. This provides the best of both worlds: (1) Existing SOAP apps continue to work and don't have to be re-coded, and (2) going forward, REST APIs are used instead.

I've put together an example of this below. In this case, I'm using the common Parlay-X format which is SOAP-based, and I'm converting to the GSMA OneAPI, which is a REST API. The example I'm using is the common "amountCharging" service, which, here in Vordel, we often encounter with our telecoms customers [Side note: Here is some information about how BT, 3 UK, Telefonica, and TIM Brasil are using Vordel with their telecoms services]. 

Here's an example of a legacy SOAP message to our Parlay-X API:

<?xml version="1.0" encoding="UTF-8" standalone="no"?> <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body> <par2:chargeAmount xmlns:par2="http://www.csapi.org/schema/parlayx/payment/amount_charging/v2_1/local"> <!-- Element must appear exactly once --> <par2:endUserIdentifier>6175551234</par2:endUserIdentifier> <!-- Element must appear exactly once --> <par2:charge> <!-- Element must appear exactly once --> <description>Game Currency</description> <!-- Zero or one occurrences --> <currency>USD</currency> <!-- Zero or one occurrences --> <amount>25.0</amount> <!-- Zero or one occurrences --> <code>a</code> </par2:charge> <!-- Element must appear exactly once --> <par2:referenceCode>a</par2:referenceCode> </par2:chargeAmount> </soap:Body> </soap:Envelope>

You can see above that this is a charge for some "Game Currency", of the amount of 25 US Dollars. 

We want to convert this to a REST API call like this:


You might notice, in the example, that the "endUserIdentifier" value in the SOAP message must be mapped into part of the URL used by the REST API. This is sometimes called a "Parameterized URL", or REST API parameterization.

Let's see how this is configured at the Vordel (now Axway) API Gateway...

The first step is to register the Parlay-X WSDL. I am doing this within the API Gateway's Repository, which I am accessing using the Policy Studio (Under "API Server" -> "Business Services", then right-clicking on the group I want to import the service into, and selecting "Register Web Service"). 

I am choosing the "chargeAmount" operation, as shown above, by selecting it. I simply follow through the rest of the steps as default, and for the final step, I'm choosing to deploy under a group called "Default Services", which has a number of HTTP listeners, and an SSL listener, as shown below:

The API Gateway will automatically create the mapping from the inbound path to the policy for our Parlay-X service. 

Finally we hit "Deploy" to deploy our new SOAP service to the API Gateway. 

The next step is to map from the SOAP message to the REST API call. Let's see how this is done.

Here is the WSDL for the virtualized Parlay-X service at the API Gateway. I load this up into the free Vordel (now Axway) API Tester tool (or SOAPUI) in order to generate a sample SOAP request to this Parlay-X service. 

I then save this sample request as a file called "SampleSOAPMessage.xml". We will use this in order to tell the API Gateway how to look up the "EndUserIdentifier" value from the SOAP now.

When I registered the Parlay-X WSDL, a sample "handler" was created for me. I am going to delete this, and configure my own handler, which reads the value from the SOAP and dynamically constructs the REST API requests based on the "EndUserIdentifier" value. So first let's delete the default handler:

Next, I drag in a "Set Service Context" into my empty canvas, and I choose the WSDL for the Parlay-X service I just imported. What I am doing here is telling the API Server (and Vordel API Analytics) the name of the SOAP service, so that this shows up in monitoring and Analytics. 

Remember to select "Set as start" for this filter. Otherwise it will continue to be grayed out and will not run. Here I am setting it as start:

In the next step, I am setting Schema Validation for my incoming message. Note that, if we don't want to schema-validate the message, you can skip this step. I put a green arrow from the "Set Service Context" filter into the "Schema Validation" filter. 

Next, I am using a "Retrieve from message" filter to select the value of the EndUserIdentifier field from the XML. As you can see below, I am using the XPath Wizard in Policy Studio with the sample message which I made from SOAPbox (or SOAPui if you prefer). I select the place where the EndUserIdentifer is to be found in the message, and I set that it will be placed into an attribute (variable) called "EndUserIdentifier". 

Next, I am using a "Switch" filter to switch based on the content of that "EndUserIdentifier" variable. You can see that I'm saying "If it starts with '617', use routing policy 'Route to Px Service 1", "If it starts with '212', use routing policy 'Route to Px Service 2", etc. This Switch filter is very powerful.

At this point your policy should look like this next screenshot:

So let's look at one of the routing policies which we are calling from our Switch filter. This is a routing policy which is creating a REST API request. It consists of four simple steps. 

Step one is to rewrite the URL, taking into account the EndUserIdentifier value which we read in from the SOAP message. You can see it below with the syntax ${EndUserIdentifier}

Step two is to set the verb to "GET". Because the incoming SOAP message was sent using a POST verb, but we want to use GET to connect to our REST API, I am using a "Set HTTP Verb" filter to convert the verb to "GET".

Step three is to use a "Static Router" filter to set the hostname and port of my REST API server which I want to route to:

Step four is to place a "Connection" filter after the "Static Router". I accept all the default for the "Connection" filter. Note that if I was connecting over SSL to the API, I'd have to make sure I checked the boxes for the certificates used at the endpoint, so that they are trusted.

And that's it! What we achieved here is:

1) Converting from a legacy SOAP request to a REST API call
2) Parameterization of a URL, based on a value read from a SOAP message
3) Conditional routing based on the value from a message (content-based routing).

If you'd like to see more "recipes" like this, check out the tutorials on the Vordel website.

Free Vordel API Workshop in San Francisco this Wednesday

I'm co-leading a Vordel API Workshop with my colleague Wayne in San Francisco this Wednesday. There is still time to register, if you want to go along. Attendance is free, and you don't need to have registered to Oracle Open World (which is running at the same time) to attend. Here is the registration page for the Vordel API Workshop.

We'll be covering how to:

- Create new REST APIs in front of legacy SOAP services - map from the old to the new - Cloud Mashups with Salesforce
- Modernize legacy SOAP web services and convert to REST and JSON APIs
- Find out "Who's been using my APIs?" with real-time API usage, and investigate trends over time.
- Use OAuth 2.0
- Enable usage quotas in order to monetize your APIs, with the "Freemium" model

I'll be showing how an Android app accesses APIs through the Vordel API Server, and then continuing the theme, we're giving away an Android tablet (Google's Nexus 7). You'll also get a Vordel "API First" t-shirt.

See you there :-)