Adopt the OASIS Open Data Protocol to consistently publish and consume REST services.

In the article "Implement a Service-Oriented Architecture with REST APIs," I covered some basics about implementing a service-oriented architecture using REST APIs. In general, because the REST methodology doesn't have many standards regarding how to pass parameters, how to format the result data, etc., the implementation of these APIs can vary widely from vendor to vendor and developer to developer.

Originally started by Microsoft, the Open Data Protocol (OData) was devised in part as a means of having a standard input and output so that REST APIs based on OData would be consistent in their implementation. This standardization offers many benefits, including the ability to have a generic tool set connect to an OData source and consume the data. Just as ODBC paved the way as a standard for database access, OData is striving to be a popular standard for REST. OData's slogan is "The Protocol for REST APIs."

ODBC was great because it allowed client/server developers a standard way to access various database servers (such as DB2, SQL Server, Oracle, etc.) using the same coding practices. Because it was standardized and could access database metadata, various tools could interrogate the database server and build a list of available tables, columns, and other database objects to present to the user. The drawbacks with ODBC are that it's primarily Windows desktop- and server-based (with ports available on Linux), each database manufacturer needs to provide a driver, and the driver needs to be installed on each machine requiring database access. That's quite a bit of deployment maintenance.

With REST, ODBC's problems go away in that no special "middleware" providers/drivers are required and that use of Internet standards makes REST platform- and OS-independent. Building on REST's and ODBC's benefits, OData gives developers a standard way to "program" against an API, which increases productivity.

To illustrate some OData concepts, I'll be using the sample test services conveniently offered on OData.org to demonstrate some OData highlights (as I perceive they'd be of interest to the IBM i community). This article will merely be an introduction to OData, but interested parties can pursue much more at OData.org.

Excel and Other Tools

To start, I'll defer some boring technical explanations in favor of showing how OData can be used by Excel. Microsoft Query (built into Excel) was one of the early tools that allowed users to access database data with ODBC. Excel 2013 allows users to directly query data from an OData service with similar benefits.

To access data from an OData API in Excel 2013, there are a few simple steps to follow: choose the Data ribbon > "From Other Sources" > From "OData Data Feed" as shown in Figure 1.

Figure 1: In Excel 2013, select an OData Data Feed as a source to import data directly from an OData API.

When prompted for the location of the OData data feed, simply paste in the URI to the sample Northwind Summary of Sales by Years test service:

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years

This sample OData resource returns a list of orders, their ship dates, and their subtotals—similar to a database query against an ERP orders table.

Excel gives the option to put in credentials, but the test service does not require them, so simply hit Next. You will be prompted to check the checkbox next to the Summary_of_Sales_by_Years table and then click Finish. The last Excel prompt allows you to place the OData feed as a table, pivot table, etc. at a specified location in the worksheet. Shown in Figure 2 is a partial result set with the OData feed formatted as a table (the Excel default):

Figure 2: This abridged Excel data table was created directly from the Summary_of_Sales_by_Years OData feed.

This simple demonstration illustrates that OData APIs can offer data and logic to be consumed by a client similar to how ODBC clients access data and logic from a database server. But unlike ODBC, a driver installation is not required, knowledge of a particular database's data types and conventions is not required, and it can theoretically be accessed by any modern device or platform.

Incidentally, Excel 2010 and 2013 can also access OData and many other data feed types via the Power Query add-on.

In addition to Excel, many other tools and products can produce and/or consume an OData feed, including these:

• SharePoint 2010 and later
• Microsoft Azure
• IBM WebSphere
• SQL Server Reporting Services (reading data via the XML data source wrapped with a special query, producing an AtomPub feed with SQL Server 2008 R2 and later)
• SQL Server 2012 and later Integration Services (using an OData source)
• SAP NetWeaver Gateway
• Microsoft Dynamics NAV ERP
• eBay

As for OData-capable development environments:

• Publish using .NET applications using WCF
• Publish or consume using .NET applications created with LightSwitch
• Publish or consume using Java applications using OData4j (a work in progress) or Apache Olingo.
• Any applications that can take advantage of ODBC, OLE DB, or JDBC using a tool from RSSBUS that puts an ODBC (or OLE DB using ODBC)/JDBC wrapper around an OData feed
• Others listed on the OData website, including options for iOS and Python

As applications become "service-oriented," it's easy to see the advantage of being able to share data among multiple products and programming languages with a standard protocol.

Keep in mind there are several versions of OData (with v4 as the latest) and you'll need to check with your tool/application provider to find out what version(s) are supported. As noted, Microsoft is the big supporter of OData but other major companies have contributed to the standard too.

The Ins and Outs of an OData Service

Back to the technical explanations, I'll describe here a little bit about how OData works.

URI Structure for Accessing an OData Service

The OData standard has laid out a specific URI syntax to support many features, as shown in Figure 3.

Figure 3: This is the anatomy of an OData URI (from odata.org).

This URI structure should be familiar to web developers, particularly those working with HTTP-based services. The Query Options portion of the URI is where OData's features shine. Shown below are several examples of useful things that can be done with the OData URI, although this list certainly is not exhaustive. And the good news is you can paste these URIs in your browser to see how they work. Oddly enough, I'd stay away from Internet Explorer in these examples in favor of Chrome or Opera, where it's easier to see the raw data returned from the service.

URI Examples

Starting with the same base URI used in the Excel example above, pasting this in a browser will return a summary of all orders in the OData test system (with results returned as XML):

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years

Building on the base URI, the OData specification offers several options for using the URI to filter, sort, compute, return a count, etc. To get a feel for OData's features, paste the next several URIs into your browser as well.

This next URI uses a filter query option to obtain all sales with a subtotal equal to a specific value of 654.06 (the M is used to indicate that the number is a decimal literal):

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years?$filter=Subtotal eq 654.06M

Of course, specifying the text $filter=Subtotal eq 654.06M is responsible for creating the filter. For another feature example, this URI selects only the top 10 sales entries, sorted by subtotal in descending order:

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years?$orderby=Subtotal desc&$top=10

When the host OData service supports it, client-side paging can even be implemented with the use of $skip and $top. (It is important to note that not every OData API is required to implement all of these featurers.)

In this next example, the first 50 rows of the result set are skipped and only 10 rows are returned, while $orderby ensures consistency in the rows returned.

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years?$skip=50&$top=10&$orderby=Subtotal desc

If you want to know how many rows will be returned by a service (which is useful when the client-side UI pages data), use the $count system query option:

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years/$count

One other notable feature is the $metadata operation that can be used to retrieve the metadata provided by an OData service root's resources:

http://services.odata.org/V3/Northwind/Northwind.svc/$metadata

Consider an excerpt from the metadata associated with the Summary_of_Sales_By_Quarter resource:

<EntityType Name="Summary_of_Sales_by_Quarter">

<Key>

<PropertyRef Name="OrderID"/>

</Key>

<Property Name="ShippedDate" Type="Edm.DateTime"/>

<Property Name="OrderID" Type="Edm.Int32" Nullable="false"/>

<Property Name="Subtotal" Type="Edm.Decimal" Precision="19" Scale="4"/>

</EntityType>

It's easy to discern from this XML that three pieces of information are returned by the resource, including ShippedDate (DateTime), OrderId (Int32), and Subtotal (Dec(19,4)). Also the PropertyRef named OrderId indicates that a specific piece of information related to a single order id can be obtained by passing the order id in the URI as follows (using order id 10250 as an example):

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years(10250)

This URI will return the sales only for the requested order id. If a resource's unique identifier is a string (such as an item number), it can be requested using a string value delimited by single quote. Each OData service is expected to have a unique identifier.

Further, the value of a specific property can be requested (in this case Subtotal). When the path segment contains $value, the raw value is returned (without XML or JSON formatting). This URI returns the plain value 1552.6000, the subtotal value of order id 10250:

http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years(10250)/Subtotal/$value

One important thing to note is that the names in these queries are case-sensitive (Subtotal is not the same as SUBTOTAL). These subsequent URI examples can be modified to use the other resources available by this OData test service, such as querying the products that have a unit of measure ending with ml (milliliter):

http://services.odata.org/V3/Northwind/Northwind.svc/Products?$filter=endswith(QuantityPerUnit,'ml')

Or query suppliers with a postal code starting with 99:

http://services.odata.org/V3/Northwind/Northwind.svc/Suppliers?$filter=startswith(PostalCode,'99')

For more information on OData URIs, including how to do computations, see the following links:

http://msdn.microsoft.com/en-us/library/ff478141.aspx

http://www.odata.org/documentation/odata-version-2-0/uri-conventions/

OData Responses: AtomPub (XML) and JSON

OData supports returning a response as the XML-based Atom Publishing Protocol (aka AtomPub) or JSON. Both of these formats are popular today and can be produced or consumed on the IBM i as discussed here. I'll show a few examples of the output formats in the next section(s). If you're unfamiliar with AtomPub, it’s the same XML you’ve seen show up in your browser if you've been reviewing the URI examples.

OData and CRUD Operations

OData is also geared for CRUD (CREATE, READ, UPDATE, DELETE) operations using HTTP verbs like DELETE (delete), POST (insert), PUT/PATCH (update), etc. This standardization gives clients a consistent way of updating data via an OData service.

An MSDN Blog titled oData and JSON payload examples shows a few examples of using JSON to perform CRUD operations using the Northwind sample OData service. Although you can't easily run these in a browser, the blog shows that it's quite easy to create the appropriate HTTP request programmatically.

This is an HTTP DELETE example from the blog that requests Product Id 200 be removed.

DELETE http://services.odata.org/V3/(S(ettihtez1pypsghekhjamb1u))/OData/OData.svc/Products(200) HTTP/1.1

DataServiceVersion: 1.0;NetFx

MaxDataServiceVersion: 3.0;NetFx

Accept: application/json;odata=minimalmetadata

Host: services.odata.org

The random portion of the URL (after the V3) is temporarily assigned for the test service. Likewise, another JSON example is reproduced here using an HTTP POST to create a new product (with data supplied as JSON):

POST http://services.odata.org/v3/(S(ettihtez1pypsghekhjamb1u))/odata/odata.svc/Products HTTP/1.1

DataServiceVersion: 3.0;NetFx

MaxDataServiceVersion: 3.0;NetFx

Content-Type: application/json;odata=minimalmetadata

Accept: application/json;odata=minimalmetadata

Accept-Charset: UTF-8

Host: services.odata.org

Content-Length: 180

Expect: 100-continue

{

   "odata.type":"ODataDemo.Product",

   "Description":null,

   "DiscontinuedDate":null,

   "ID":200,

   "Name":"My new product from leo",

   "Price":"5.6",

   "Rating":0,

   "ReleaseDate":"0001-01-01T00:00:00"

}

Of course, these data modification operations are possible only when the OData API supports it. (Otherwise, the danger of linking updateable ERP tables within a Microsoft Access database will be lived all over again!)

In summary, when supported by the service provider, OData queries come with the same benefits of database queries (available metadata; versatility with filtering, ordering, updating; and strongly typed results) with the advantage that OData, because it uses standards, doesn't require database-specific middleware.

DB2 Query to "Manually" Parse Atompub

Finally, let's not forget that good ol' DB2 can consume an OData source that returns AtomPub, which is, after all, XML. The following DB2 for i example query uses the new HTTP function HTTPGETBLOB to retrieve and parse the AtomPub data from the Summary_Of_Sales_By_Years test OData source (the same URI used in the Excel example).

Notice that the second parameter of HTTPGETBLOB is a request header containing an Accept key/value pair. This is required in order for the server to return a response. Note: DB2 HTTP functions require IBM i 7.1 with database group PTF level 23.  Finally, the XMLTABLE table function is used to shred the AtomPub XML returned from the service.

WITH XML_DATA AS (

Select XMLPARSE(DOCUMENT SYSTOOLS.HTTPGETBLOB('http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years',

'<httpHeader>

<header name="Accept" value="text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" />

<header name="Accept-Encoding" value="gzip" />

</httpHeader>')) AS ODataSummary

From SYSIBM.SYSDUMMY1

)

SELECT Shred.*

FROM XML_DATA,XMLTABLE(

XMLNAMESPACES(

'http://schemas.microsoft.com/ado/2007/08/dataservices' AS "d",

'http://schemas.microsoft.com/ado/2007/08/dataservices/metadata' AS "m",

'http://www.w3.org/2005/Atom' AS "a",

'http://services.odata.org/V3/Northwind/Northwind.svc' AS "base"),

'$doc/a:feed/a:entry/a:content/m:properties' PASSING XML_DATA.ODataSummary AS "doc"

COLUMNS

/* On i 7.2 you can use TIMESTAMP(0) */

ShippedDate TIMESTAMP     PATH 'd:ShippedDate',

OrderId     INT          PATH 'd:OrderID',

SubTotal   DECIMAL(19,4) PATH 'd:Subtotal'

) AS Shred

;

The results from this query are the same as the results shown from Excel in Figure 2. Basically, OData-aware tools examine the metadata and perform the above type of transformation automatically so the developer doesn't have to do the parsing!

The SQL query can be modified so that the service returns JSON instead of AtomPub XML. This can be done by simply changing the request header's "Accept" value:

WITH JSON_DATA AS (

Select SYSTOOLS.HTTPGETCLOB('http://services.odata.org/V3/Northwind/Northwind.svc/Summary_of_Sales_by_Years',

'<httpHeader>

<header name="Accept" value="application/json" />

<header name="Accept-Encoding" value="gzip" />

</httpHeader>') AS ODataSummary

From SYSIBM.SYSDUMMY1

)

SELECT *

FROM JSON_DATA

The abridged JSON result looks like this:

{"odata.metadata":

"http://services.odata.org/V3/Northwind/Northwind.svc/$metadata#Summary_of_Sales_by_Years",

"value":

[{"ShippedDate":"1996-07-16T00:00:00","OrderID":10248,"Subtotal":"440.0000"},

{"ShippedDate":"1996-07-10T00:00:00","OrderID":10249,"Subtotal":"1863.4000"},

{"ShippedDate":"1996-07-12T00:00:00","OrderID":10250,"Subtotal":"1552.6000"},

{"ShippedDate":"1996-07-15T00:00:00","OrderID":10251,"Subtotal":"654.0600"},

{"ShippedDate":"1996-07-11T00:00:00","OrderID":10252,"Subtotal":"3597.9000"},

{"ShippedDate":"1996-07-16T00:00:00","OrderID":10253,"Subtotal":"1444.8000"},

{"ShippedDate":"1996-07-23T00:00:00","OrderID":10254,"Subtotal":"556.6200"},

{"ShippedDate":"1996-07-15T00:00:00","OrderID":10255,"Subtotal":"2490.5000"},

{"ShippedDate":"1996-07-17T00:00:00","OrderID":10256,"Subtotal":"517.8000"}

etc.

Although probably not extremely useful with traditional IBM i languages such as RPG or COBOL, JSON is a frequently used format for client-side (i.e., browser) scripting in web-based applications. This is because the above result can be transformed into a JavaScript object that can be bound to controls on the web page, among other things.

Conclusion

OData provides a standard feature-rich protocol for REST APIs. These features include complex filtering, querying, sorting, paging, CRUD, and metadata retrieval—to name a few. IBM i and other shops can benefit by adopting the OData protocol when creating REST APIs and consuming them.

References

Implement a Service-Oriented Architecture with REST APIs

JSON and XML Conversion in DB2 for i

Rev Up "i" Reporting with SQL Server 2008 Reporting Services