Promoting the safe and effective use of powered access
Data Services REST API makes use of pagination on HTTP calls to limit the amount of data that can be requested. A lot of the time, when you're making calls to the API, there is a potential to return a heavy payload of results. For that reason, we paginate the results to make sure responses can be handled efficiently.
As an example a call into the licence endpoint could result in hundreds of thousands of records being returned without the paginiation limits in place to reduce unnecessary load on the servers.
Rather than that, the API has a maximum limit on what can be called, but it is best practice to set your own PageSize on each request so that you know what to expect and control the further calls to get the required records.
Working on the principle familiar with SQL users of LIMIT and OFFSET we can reduce the load on the servers by sending these parameters through the URI.
Adding ?PageSize=20, our version of LIMIT, after the URI of any call to the API will set the per page limit to 20. The returned data will provide details of total (record count) and links to first, self, previous, next, and last pages. These links (HAL links) can be used to make subsequent calls for records within an app to build the required dataset, or passed to the end user as navigation properties within the UI.
The following example shows how to retrieve the second page of 20 results;
https://ds.dev.ipaf.org/licence/v1.1/licence?PageSize=20&Page=2The JSON Hypertext Application Language (HAL) is a standard which establishes conventions for expressing hyperlinks with JSON [RFC4627]. HAL is a generic media type with which APIs can be developed and exposed as series of links. With each call to the Data Services API the links are generated to be consumed by the app/service calling for the data. When accessing a large dataset this will provide enough information to make subsequent calls for further data. These can also be used to form navigation properties with the UI of an app.
Below is an example of a full set of HAL _links from the licence endpoint. The self, next, prev, first and last links all have different URIs that can be used by the consuming app for navigation or further data calls.
{
"_links": {
"self": {
"href": "https://ds.dev.ipaf.org/licence/v1.1/licence?PageSize=20&Page=3"
},
"next": {
"href": "https://ds.dev.ipaf.org/licence/v1.1/licence?PageSize=20&Page=4"
},
"prev": {
"href": "https://ds.dev.ipaf.org/licence/v1.1/licence?PageSize=20&Page=2"
},
"first": {
"href": "https://ds.dev.ipaf.org/licence/v1.1/licence?PageSize=20&Page=1"
},
"last": {
"href": "https://ds.dev.ipaf.org/licence/v1.1/licence?PageSize=20&Page=33"
}
},
"total": "659"
}
Along with the HAL links there can be addtional data added to the response called 'meta information'. In the above example at the bottom of the JSON response there is a "total": "659" which indicates the total count of all underlying records. This addition of meta information offers a way to get record counts without having to consume large payloads or make heavy server calls.
As part of the Data Service API there are usual methods to filter results through URI query parameters being passed in the HTTP request. The following example sends three query parameters with the URI - LicenceStatus=Active, DistributionStatus=Pending and DateIssuedTo=2021-05-18.
GET/ https://ds.dev.ipaf.org/licence/v1.1/licence?licencestatus=active&distributionstatus=pending&dateissuedto=2021-05-18Within each endpoint there are numerous fields which are used for filtering the requests, the best place to find out which filters are used is via the endpoint Swagger definitions. As an example, the GET a list of Licence call accepts filters in the form of the following;
DistributionStatus LicenceStatus DateIssuedFrom DateIssuedTo ApplicationDateFrom ApplicationDateTo CourseDateFrom CourseDateTo DistributionStatusDateFrom DistributionStatusDateTo Grades Instructors Page PageSize As part of the query parameters it is possible to request that only a small selection of fields are returned in the response, for example;
GET/ https://ds.dev.ipaf.org/Licence/v1.1/Licence?Mode=All&Fields=surname%2Cforename%2CgradeThis would return a collection of licences from the Licence endpoint with only Surname, Forename and Grade in the response - please note the URL encoded commas (%2C) are required as delimiters.
In order to set a specific order when making calls to the API the OrderBy query param must be added to the URI. For example;
GET/ https://ds.dev.ipaf.org/Licence/v1.1/Licence?OrderBy=-surnamePlease note the minus notation before surname in the URI. This controls sort order, as in ?OrderBy=-surname sign will be descending and ?OrderBy=surname will be ascending.
For date fields throughout the API we are adhering to the ISO 8601 Date formatting. You may have noticed in examples we use filter dates in the format of yyyy-MM-dd. The purpose of using this standard is to provide an unambiguous and well-defined method of representing dates and times, so as to avoid misinterpretation of numeric representations of dates and times, particularly when data is transferred between countries with different conventions for writing numeric dates and times. We are storing dates in UTC so timezone offset will be required.
Versioning helps us iterate faster and prevents invalid requests from hitting updated endpoints. When we move a version of the API into production after testing in our development environment, it will be effectively frozen at that point, barring any necessary fixes.
This allows developers time to explore new API versions in their own development environments, while their live apps and services are maintained through the stability of older API versions. The versioning number is shown in the URI which allows browser explorability of the resources across versions and to aid a simpler developer experience.
Changes and alterations on the API schema will always be necessary to aid better functionalilty and take advantage of opportunities that arise. The versioning method we use gives us the opportunity to manage the changes effectively while maintaining service stability.
Future API versions will be announced with change logs, documentation and developer guides. Deprecation of older API versions will be announced with a long lead time schedule in order for developers to make necessary changes to their app/service.