Scheduling reports on API usage

One really neat feature in the Vordel Application Gateway is the ability to schedule reports on API and Web Service usage. You can schedule reports to run on a regular basis, and have the results emailed to the user in PDF format, just like Google Analytics. These reports include summary values at the top (for example, the number of requests, SLA breaches, alerts triggered, and unique clients in a specified week) followed by a table of APIs and Services, and their aggregated usage data (for example, the number of requests on each API or Web Service).

It is quite simple to configure this with the Vordel Application Gateway. Just right-click the Listeners -> Vordel Reporter node in the Policy Studio tree, and follow the configuration steps which are listed here on the Vordel Extranet: https://extranet.vordel.com/documentation2/VG6/common/tutorials/reporter_scheduled_reports.html [ If you need a logon for the Vordel Extranet, contact Vordel at info@vordel.com ].

Gateways and Load Balancers

A very common question is "How do Application Gateways work with Load Balancers"?

To answer this question, it is useful to divide up the load balancing requirement into "in front of" (ingress) and "behind" (egress) the Gateways.

In front of the Gateways (ingress), if there is already a load-balancer in place, then it's natural to deploy the Gateways in Active-Active configuration behind it. If the reporting or routing requires the source IP address, then load balancer can place this into the X-Forwarded header and the Gateway consumes this.

Policy Director is used to send the same configuration to multiple Gateways in an Active-Active cluster (and if there is a hot-standby, then to it also).

The distributed caching across the Gateways is used to keep state (eg message counts for throttling across Gateways in a cluster).

If there is no load-balancer in place, then it's simple to setup a cluster of Gateways with a VIP (Virtual IP address) shared across them. Policy Director is used to keep the configuration in sync.

Behind the Gateways (egress), you can use the Gateway's inbuilt load-balancing. This can be Round-Robin across backend servers, or weighted on response time (Gateways will then favor faster backend servers). So there is no need for a Load Balancer behind the Gateway. Load-balancing in the Gateway is configured within a "Remote Host" in Policy Studio.

Who manages Application Gateways?

Because Application Gateways touch on a number of areas of functionality, including networking and security, a very common question is "Who manages an Application Gateway?".

To answer the question, take a step back and think about what an Application Gateway is. An Application Gateway takes tasks such as application integration and application security and it moves them into a piece of network infrastructure (which may be virtual or physical). By moving these tasks out of applications and onto the network, you make the tasks run faster, and easier to manage since they are now decoupled from the applications themselves.

So, breaking it down, there are two things going on here: (1) moving integration and security tasks onto network infrastructure, and (2) managing the Gateway as a piece of network infrastructure. It follows logically that there are two distinct roles involved here: (1) The person configuring services and policies on the Gateway, and (2) the person operationally managing the Gateway. Person (1) knows about APIs, apps, and policies. Person (2) does not need to know anything about APIs (much less know anything about REST or XML) but knows all about SNMP, Splunk, load-balancers, and NAT.

This is why Vordel provides two distinct training courses, aimed at these two different people. For person (1), who will be registering APIs and SOAP services on the Gateway and applying policies to them, we provide the Vordel Certified Systems Engineer (VCSE) training. This involves registering APIs (via their URLs), choosing policies to apply to them, and pushing the policies out to Gateways. It includes SOAP and REST. For person (2), we provide the Vordel Certified Network Administrator (VCNA) training. This explains how to monitor the "speeds and feed" of a Gateway, where alerts can be sent, how to monitor it via network admin tools such as HP Arcsight, how to deploy additional virtual appliances in a VMware ESX environment, etc.


The separation of duties carries over to the Web-based interfaces used to manage the Vordel Application Gateway. In the case of the person managing APIs and policies, they see dashboards such as the one shown above. Here I see the traffic being passed to three APIs: CDYNE's EmailVerify service, SalesForce itself, and SalesForce's login service, for a two minute period on January 9th 2012.


In the screenshot above, we see what the Network Administrator sees. Rather than seeing information about SalesForce APIs, they instead see the network administration view of the Application Gateway. We see, for example, here the RAID Array Status for disks in the Application Gateway. Also, as you can see, system logs, networking, and server status can be monitored in this way. All of the same information can be sent to a product like HP Arcsight so that the network administrator does not even need to look at Vordel tooling.

In summary, there are two distinct types of people who manage Application Gateways on an ongoing basis, and this maps to Vordel's two distinct training courses, and to the different Web interfaces we provide for these people. All of this is geared towards moving application integration application security out of applications and onto the network (virtual or physical).

Kin Lane's API Management Service Provider Roundup for 2011 covers Vordel

Vordel is featured in Kin Lane's First Dimension of API Management Service Providers. It's a really good round-up of the current state of play, as we start 2012.

Kin mentions that:

One thing to know about these API management service providers is, well...they are API management service providers. To my knowledge they don’t actually deploy your API for you, they help you build a strategy, and manage the API. But you still need to rely on other tools and in-house resources to deliver your API.

This is where I'd disagree slightly. In the case of an API Gateway, it can also be used as a service delivery platform to deliver an API in front of existing enterprise system systems. This is because API Gateways include many adapters to, for example, various message queues, TIBCO EMS/Rendezvous, databases, and FTP/SFTP. These are used to deliver a REST API interface in front of an enterprise system which does not natively support REST, not to mention JSON, OAuth, or OpenID. This can be as simple as delivering a REST API interface in front of an existing SOAP service within the enterprise, to mapping from REST into messages placed on the message queue, through to mapping from "lightweight" identity mechanisms suitable for APIs (such as OpenID or OAuth) through to enterprise identity management (e.g. mapping from Google OpenID to Oracle OAM). So, an API Gateway usually does not only manage the API, it actually delivers the API too.

Mapping from Google login, with OpenID, to Oracle Access Manager login, with obsso cookie

For a demo recently, I configured the Vordel Gateway to map from a Google account logon to an Oracle Access Manager obsoo cookie. This was to enable users to use their Google account to log into a local enterprise service, using the Vordel Gateway to do the mapping to the Oracle Access Manager login (since Oracle Access Manager is what is protecting the enterprise service).

Here is the policy in action. You can see that the Vordel Gateway creates a hyperlink to Google, to ask the user to select their account. If you look at the status bar in the screenshot below, you can see the OpenID string generated by the Vordel Gateway:

When I click the link, I am asked which Google account I want to use to log into the service. Google shows me the two accounts which I am currently logged into:

Once I choose the Google Account, I see that the Vordel Gateway has generated an Oracle Access Manager obsso cookie, which is visible in Firefox:


If you're interested in getting a copy of the policies I used for this, contact Vordel at info@vordel.com . For more information, including videos, about Vordel Gateway interop with various Oracle products, check out: http://www.vordel.com/oracle/

Returning JSON fault information to JQuery-based API clients

Since many APIs are called by JavaScript libraries like JQuery, it's convenient to return fault information as JSON so that it can be easily read by the client. The object is then parsed to retrieve the reason for the fault. Some APIs make this easy, such as the Rackspace Cloud Identity API which allow faults to be returned as JSON or XML. But in many cases, you have to laboriously configure this JSON conversiona yourself [For example, in the case of WCF, Iain Mitchell has a good blog post about how it can be made to return JSON formatted faults for consumption by JQuery.]

A Gateway serves an important purpose here, by providing fine-grained information on why clients are blocked. A Gateway can XML to clients which expect XML responses, and JSON to REST API clients which expect JSON. Let's see how you can use the Vordel Gateway to take a policy which returns an XML fault (in this case a SOAP Fault) and converts it automatically to JSON...

To start this, I fired up Vordel Policy Studio and connected to my Dev API Gateway. I then dragged in a Policy Shortcut to a policy called "JSON Fault". Next I right-clicked on it and choose "Set as Fault Handler", as shown below:

This means that if any of the filters in the chain return false, my policy gets called. Next let's look at this policy. You can see below that it takes a regular SOAP Fault and converts it automatically to JSON:

When we look at it in action, in the API Gateway's Real-Time Monitoring, we see the familiar "Path through the policies" on the left, and on the right you can see the JSON which is returned to the JQuery client:


The API Gateway's Real-Time Monitoring shows what is happening now for your APIs and services. But in order to look at what has happened in the past, we open the API Gateway's audit trail. Here you can see the opening page, and I click on the "Audit Trail" button:

On the Audit Trail, I run a report for the API requests blocked in the last hour by my API Gateway. Here I see a bunch, and I click on one to see more detail.

When I click on the API request which was blocked, I see the exact reason why it was blocked in the "Path through the Policy". I can see below that it was the authentication which failed, the same reason which was passed back to the JQuery client.

So in summary, it's quite straightforward to return JSON fault information to clients, and view fault information in the Audit Trail, using the Vordel Gateway. To get more info, or your own copy of the Vordel Gateway, contact us on info@vordel.com

How to change the default alert email subject line

First How-To of 2012 and it's via Niall Commiskey, who posted a useful tip which applies also to the Vordel Gateway: How to change the default alert email subject line.

Testing HTTP Authentication to a Web API

It's natural to concentrate on the New New Thing, but in the case of authentication web APIs, based on HTTP, there are options. There is certainly HMAC authentication for APIs, as shown in this Vordel Gateway case study. But remember that HTTP authentication still exists, and can be used for authentication to an API. If you want to test this, you can pick up the free Vordel SOAPbox tool which, although it has "SOAP" in its name, can be used to test REST APIs also. Here is how you do this:

Firstly click up on the title bar for your API call:



You'll now see the "Request Settings" dialog. Notice that it's a GET request (not a POST in the case of SOAP). I then choose the lower "Security" tab and the upper "HTTP Authentication" tab, and configure my parameters there. Notice also that mutual SSL and Kerberos are options too.


Happy testing!

NFC - a "hand wavy" technology that may succeed

In the deluge of tech prediction lists which come out this time of year, one commonality stands out: NFC. NFC stands for Near Field Communication. Here you can see it at Number 3 on CNN's list for 2012. Like Microsoft Kinect or the Nintendo Wii, it is one of those technologies you can literally describe using "hand-waving". This is because the most highly cited usage for NFC is to enable you to wave your phone at a payment terminal and pay for goods.

Although mobile payments is the most commonly mentioned case for NFC, it has many other applications. For example, Transport for London plans to use NFC cards with its Oyster readers. And, back at Halloween, Mark Diodati from Gartner published a blog post (and Smiths reference in the title?) about the usage of NFC to literally open doors. The description of the experiment, where students used either a modified phone case or a MicroSD card with an attached antenna, reminded me of the early attempts to place digital certificates and private keys onto mobile phones via battery case appendages, circa 2000, in an ill-fated attempt to "PKI-enable" phones. That was ahead of its time, but here I think NFC is about to happen. 2012 may not be the "year of NFC" but it will be the year it continues its adoption, as gradually more and more people have an NFC-enabled phone in their hands.

As full disclosure, one of the large NFC players uses Vordel Gateways in its infrastructure. So I guess I have some other interest in NFC taking off. But, really it is one of those technologies which just makes too much sense to not take off in the coming years.

Checking against a CRL from a Mutual SSL connection

A common scenario for an Application Gateway is to perform mutual SSL and then check the client certificate against a Certificate Revocation List (CRL).

Here is a simple circuit which check a Certificate Revocation List following SSL on the Vordel Gateway:


First things first

Firstly, it's important to setup an HTTPS listener on the Gateway which is requiring that a client sends a mutual SSL certificate to it. To do this, in Policy Studio, look under "Gateway", the "Listeners", and then your Gateway instance and then a particular Services group (by default there is one there called "Default Services"). You can right-click here, on your Services group, and choose "Add Interface" and then "HTTPS".

Here you choose your port for SSL (by default this is 443, but ensure that nothing is already running on that port or else when you try to deploy, the Gateway will detect this and tell you). Under the "Mutual Authentication" tab choose the option to "Require Client Certificates". In my example I have set the "Maximum Depth of Certificate Chain" to "3". Now ensure that you check the CA Certificates of all certs which will be trusted to be sent by clients. It's important to note that here you are checking the CA Certificates. If you have 1,000 clients who are sending certs all signed by the same CA, clearly it would be infeasible to check 1,000 boxes, one for each client. Instead, the trust is at the CA level.

[ Note: If you want to ensure that clients can only access your APIs or services over SSL, then simply delete the existing plain HTTP entry under the Services Group. Or, create a new "HTTP Services", only put an HTTPS listener in there, and then the only way to access the paths in that group is through SSL].

Next, right-click on your Services in Policy Studio again and this time choose "Add Relative Path". For my example, I am adding "/SSL" and it links to the policy I've made.

Step by Step

So, now let's look at the policy, step by step:

SSL filter. This filter is from the "Authentication" group in Policy Studio. The purpose of this filter is to force SSL authentication and to retrieve the certificate which is presented by the client. If you hover the mouse over this filter, you can see that one of the attributes it creates is a "certificate" attribute, which is the certificate presented by the client.

Check certificate against CRL filter. This is a "CRL (Dynamic)" filter. It is from the "Certificate" group in Policy Studio. I have configured the URL to "http://localhost:8080/GetCRL/MyCRLName.crl" . This means that the CRL is being served up by the Gateway (which is also listening on port 8080) using a Static Content Provider. The Static Content Provider is essentially a Web Server, which serves up the content of a directory. In this case, I map it to a folder on the same machine as the Gateway. I then place the CRL in there, and, under the services entry under "Listeners" in Policy Studio, I right-clicked and choose "Add Static Content Provider", and then pointed it at this directory. You can verify it works, once you deploy this, by simply pulling the CRL down through the browser (point your browser to: http://GatewayAddress:8080/GetCRL/MyCRLName.crl).Note that you must not have anything listening on the "/" path, since that takes precedence over the Static Content Provider here.

"Certificate is Valid" and "Certificate is not Valid" filters. These are simply "Set Message" filters, which return a message of "Certificate is Valid" or "Certificate is not valid", depending on the outcome of the CRL check filter above them in the decision tree. I choose the content-type of "text/html" since I am testing this with a browser.

"Return HTTP 200" filter. This is simply a "Reflect" filter, which returns the message I've set ("Certificate is valid" or "Certificate is not valid") with a HTTP response code of 200.

Testing this

In order to test this, I imported a client certificate into the "Personal" group of certificates in Internet Explorer. For some reason, the certificates configuration in Internet Explorer is under the "Content" tab (not the "Security" tab as you might expect) in Tools->Internet Options. This is where I clicked "Import" and imported a certificate to test. I then opened up https://GatewayAddress/SSL in the browser and presented my certificate for authentication. I then received the "Certificate is Valid" response from the Gateway, and in the Gateway's Real-Time Monitoring (port 8090) I could see the successful path through the CRL-checking policy.

Leveraging Cloud Computing for the Financial Services Sector - Bob's Guide

I've written an article on Leveraging Cloud Computing for the Financial Services Sector for Bob's Guide in London.

Some key points are brokering:

One option to achieve this is to use a broker architecture that ensures an organisation does not hard-wire its infrastructure into a single cloud provider. This approach enables organisations to easily choose between cloud service providers and manages the subtle differences between the various providers, in terms of pricing, how an organisation connects to them and how it manages the associated keys and so on.

And tokenization:

Many IT managers have been wary of cloud computing because of the prospect of private and sensitive data being sent up to third-party cloud services. However, with the development of technologies enabling the tokenization of data, there is now an opportunity to replace any private information with a random token that can be sent to the cloud service provider, and then swapped back with the real data when retrieved. In this way, the private data is never sent to the cloud provider. Usage of tokenization will certainly help the broader adoption of cloud computing within the financial services industry.

Check out the whole article here.

Identity propagation from the Vordel Gateway with Oracle IdM through to Oracle OSB

I've put together a diagram showing one of the scenarios where the Vordel Gateway operates with various Oracle Identity Management products. The scenario, which is very common, is Identity Propagation. If a client is authenticated at the Gateway, it's usually important to propagate their identity right through to the app server tier, because otherwise all requests may appear to simply come from the Gateway. It's also important for audit trail reasons (you need identity information available if you want to keep a trail of who has accessed which service).

One of the underlying technologies used for this is SAML, and you can seem more information about how we do it in this blog post I wrote after setting this Vordel-OSB interop up myself. You can follow these instructions to setup the identity propagation scenario.

In the diagram below, the Vordel Gateway is working with a number of Oracle IM products (OAM, OES, OVD, OWSM - see a naming pattern there? ;=) ) to provide end-to-end identity propagation from the edge of the network through to the app server.

For more information about Vordel Gateway interop with many Oracle products, check out http://www.vordel.com/oracle