The Hosted GET connector lets you expose Data Source rows from your own systems, for direct access by your app.
The key advantage of the HostedGET connector is that you can control the data rows being exposed to the app, letting you filter rows down to the individual app user level as desired. For ERP, CRM and other enterprise systems this is important as the security/permissions around “who gets to see what” is critical.A secondary benefit is that your data does not need to be pushed our platform, so any data sovereignty concerns you may have are resolved by this connector.
One final benefit is that Data Sources that run off of Hosted GET connectors are not subject to the standard 50,000 row limit, since the rows are delivered directly to the device.
|
NOTE: There are still performance limits on the app side when the rows become large – this varies across Android and IOS, and also depends heavily on the device specifications. |
A rule of thumb, performance is affected by the overall file size of the data source more than the number of rows.
For example 5,000 rows containing say 100k of text data per row can be expected to run slower than 50,000 rows which have a total of 1k of data per row (e.g. columns containing say just numbers and a few short text strings).
Additionally use of “heavy” data sources in many screens that can be navigated sequentially by the user will result in a compounding performance decrease, as those big data sources can get progressively accumulated in device memory.
So with Hosted GET, you really need to test on the app to find what is the best row count and screen mix for your specific data set.
It is assumed that you have software development resources to create the web service needed by the Hosted GET connector.
If not, contact our professional services team to give you a hand 🙂
Setting Up a GET Connector
For test purposes, we recommend you use a REST API development service like – https://www.mockable.io
- Create a Data Source as normal or select an existing Data Source for use.
- On the Rows page of your Data Source, manually define and organise your Data Source’s columns to match the output of your web service.
The order of the columns must match up with what your web service returns in order for any column references to work.
If you change column orders on your web service response at any stage, you will need to update the columns on the Rows page to match. - On the Settings page of your Data Source, make sure that you set the External Id field with some value for identifying this Data Source.
Remember this External Id value, since you will need to it for when you work through the GET web service details later in this document. - Also on the Settings page, add a Hosted GET connector via the “Add Connector” button.
Specify the web address of your web service endpoint in the provided URL field.
You can also set a global DataSource GET url on your organisation and specify that the connector use it – this allows reuse of the same service endpoint across multiple Data Sources.
How The GET Connector Works
When the app discovers that a Data Source has a HostedGET connector, it performs a HTTP GET request against the target web service URL.
The GET request contains the External Ids of the Data Source’s being requested along with other identifying information.
Included in the request is your account Integration Key (useful for authenticating the request), along with the logged in app user’s email address (useful for filtering the rows based on the user’s access).
The app will issue GETs to your service whenever it attempts a data synchronisation.
Requests are issued by the app in an itemised manner – one request per Data Source.
IMPORTANT: We strongly recommend that you expose your web service endpoint as SSL secured HTTPS to ensure that app requests to your service are protected.
NOTE: Attached to the bottom of this document is a C# file that outlines the request and response structures as simple classes.
Even if you are not using C#, reviewing the properties and structure in code form can be very helpful in getting started with your own Hosted GET service.
GET Request
GET requests are issued by the app to your service, passing the values below as query string parameters.
Most of the settings for the integration with your Hosted GET API can be obtained from the Organisation Setup screen. This includes API Integration Key and Provider ID (Customer ID).
The parameters passed on the request are as follows:
|
Input Name |
Data Type |
Required |
Description |
|
ids |
String |
Yes |
Comma separated list of External Ids for each Data Source that the app wants to synchronise. |
|
providerId |
Integer |
Yes |
Your unique Provider Id found on your Organisation Setup page (My Account -> Organisation Setup)
|
|
integrationkey |
String |
Yes |
Your unique integration key found on your Organisation Setup page (My Account -> Organisation Setup). |
|
|
String |
Yes |
Email address of the logged in app user making the request. |
| deviceid | GUID | Yes | The unique identifier of this user’s device as registered on our platform. Useful for tracking which device the request originated from. |
| lastupdated | DateTime (YYYY-MM-DDTHH:MI:SS) |
Yes | The last time that the app checked for Data Source updates. Use this to implement incremental row updates, passing back just the new/updated and delete rows since the time the app checked in. This is recommended for large Data Sources to save server resources and prevent exhausting mobile data allocations. You can also use this value to determine whether there is anything at all to update since the last time the app pinged your service. The first time the app checks for rows, the LastUpdated value will be at least 1 year in the past. |
| lastupdateds | String | No | Comma separated list of DateTime values in the format “YYYY-MM-DDTHH:MI:SS” that represent the last time the app requested an update for the corresponding data source. The values are listed in the same order as the external ids in the Ids parameter.
Each date in the list represents the last time that the app checked for Data Source updates for the specific data source. Use this to implement incremental row updates, passing back just the new/updated and delete rows since the time the app checked in. This is recommended for large Data Sources to save server resources and prevent exhausting mobile data allocations. You can also use these values to determine whether there is anything at all to update since the last time the app pinged your service. If nothing has changed for the related data sources, then you can simply return a response with no DataSources property defined. The first time the app checks for rows for a specific data source, the related last updated value will be left blank in its position in the comma separated list. We suggest that you return the full set of data in the Rows property of your response in this case. |
GET Response
 |
IMPORTANT: Your web service MUST return a response in the following structure. Failure to comply with this format will result in failure to download rows on your app. Your response MUST be in JSON format. |
We also support gzip and deflate compression on responses.
This is recommended for large data sets, since text data compresses very well and thus will conserve mobile data allocations and massively reduce download time.
Our support for compression is identified through the “Accept-Encoding” header being set by the app on its REST requests.
Your web service/server should encode the entire response stream using gzip or deflate, which will then be recognised by our app code.
This is usually an automatic behaviour at the web server level, though you should check that your server configuration has enabled compression for JSON content.
In terms of how to apply response encoding, this is beyond our support scope – you’ll need to confirm this with your software developers.
If you are using Microsoft IIS as your web server, then this article may help in terms of checking your compression settings.
This service may also help with identifying if your server response is indeed being compressed.
Rows Response (<DRowsResponse>)
|
Input Name |
Data Type |
Required |
Description |
|
DataSources |
Collection of DataSource |
No |
Collection containing the DataSources that have been requested. If nothing has changed since the LastUpdated received from the app request, then you can simply return a response with no DataSources property defined or, if returning multiple data sources, then no Rows, NewRows and DeletedRows properties set on those you do not wish to update. |
|
LastUpdated |
DateTime (YYYY-MM-DDTHH:MI:SS) |
Yes | The last time that the rows for these Data Sources were updated. Generally we recommend that this should be the current date and time on your host server at the time of processing the request.
This value is stored by the app and sent exactly as received in the LastUpdated and LastUpdateds fields of the next GET request made by the app. This gives a central point of time for incremental updates in particular, since the app will echo your server times rather than use its own unreliable device time.The “T” in the required format is a delimiter. |
Data Source (<DataSource>)
Below is an example of a functioning GET Request URL, populated from hidden fields where the integration key; providerID; datasource name are dynamically populated.
https://www.************.com/api/users?email={{userEmail}}&integrationkey={{integrationKey}}&providerid={{companyProviderID}}&ids={{dataSourceExternalIdsCommaSeparated}}
|
Input Name |
Data Type |
Required |
Description |
|
Id |
String |
Yes |
The External Id of the DataSource for which the associated Row data applies. |
|
Rows |
Collection of Row |
No |
A collection of Rows that will completely replace all current Data Source rows on app. |
|
NewRows |
Collection of Row |
No |
A collection of Rows that should be added/updated onto the existing Data Source rows on the app. |
|
DeletedRows |
Collection of Row |
No |
A collection of Row Items that should be deleted from the existing Data Source rows on the app. |
Row (<Row>)
Each Row is a collection of Val (<Val>) items, which are the individual column values for that Row.
Every Row must contain at least two Vals and must have as many Vals as there are Headers (columns).
The first Val should be the unique identifier or key that will be used as the answer value should this Data Source row be selected in a Form. The first Val is also critical as the key for performing incremental deletes through the DeletedRows field mentioned above.
The second Val should be the default displayable label text of the row. You can override this default by configuring the Display Options field in the Data Source “Settings” page.
|
Input Name |
Data Type |
Required |
Description |
|
Val |
String |
Yes |
A column value for this row, can contain any string value. If you want this column to display an icon/image in the mobile app, then specify the Val as a http url to the image in question. |
If the GET is unsuccessful, your web service should return a 400 level HTTP status.
API Usage Example
Given that this connector is REST based, you can test your web service directly via your web browser using a REST plugin like the Postman plugin for Google Chrome.
In this example, we assume you have set the “Target GET URL” on your Hosted GET Connector to be https://yourdomain.com/xx/rowsearch
We also assume that the Data Source has an External Id of “JOBS”
GET Request (made by the app, with [email protected] as the logged in user)
GET https://yourdomain.com/xx/[email protected]&integrationkey=xxxxxxxxxxxx&providerid=1&ids=JOBS,JOBS-DISS&lastupdated=2013-02-12T14:24:09
GET Response
{
"DataSources":[
{
"Id":"JOBS",
"Rows":[
[
"10011",
"10011 - Test Generic Project",
"TEST",
"12345AA",
"19 Nov 2013",
"12 Somewhere Road"
],
[
"ZDS654",
"ZDS654 - Maintenance Costs to Excavator",
"TEST",
"",
"27 Nov 2013",
"24 Somewhere Else Road"
],
[
"ZDF662",
"ZDF662 - Maintenance Costs for Hitachi Excavator",
"TEST2",
"",
"28 Nov 2013",
"182 Another Place, Somewhere"
]
],
"NewRows":[
],
"DeletedRows":[
]
},
{
"Id":"JOB-DISS",
"Rows":[
[
"10011-0201",
"Project Management",
"10011",
"Y"
],
[
"10011-0202",
"Quality Control",
"10011",
"Y"
],
[
"10011-0251",
"Electrical Design",
"10011",
"Y"
]
],
"NewRows":[
],
"DeletedRows":[
]
}
],
"LastUpdated":"2014-03-11T15:45:50.4818098+10:00"
}Example Code
Attached to the bottom of this document is a C# file that outlines the request and response structures as simple classes.
Even if you are not using C#, reviewing the properties and structure in code form can be very helpful in getting started with your own Hosted GET service.
Troubleshooting / Testing
Once you have set up a Hosted GET connector on your Data Source, the Rows page for that Data Source will automatically attempt to pull back rows from the given GET url.
The Rows page makes the same request as the app would, using your current logged in user as part of the request.
This is a quick and easy way to see if your Hosted GET web service is understanding the GET request and is returning rows in the correct format.
What to do when a Hosted GET works on my IOS app, but not on Android?
This is almost certainly due to the way that Android treats SSL certificates – essentially Android is more strict about SSL certificate chains.
SSL certificate chains involve the primary SSL certificate for your web service endpoint domain, along with any intermediate certificates issued by your Certificate Authority.
If you attempt to connect to a web address that features an incomplete SSL chain, then Android will fail the connection.
This is likely what is happening with requests by your Android devices to your Hosted GET web domain.
To test what we mean, attempt to browse to your Hosted GET service url from your default Android web browser.
You will likely get a connection error with text such as “the certificate authority is invalid”.
You can further verify your SSL configuration using a testing service like SSL Labs:
https://www.ssllabs.com/ssltest/
If your SSL installation has issues, you will see warnings such as “chain incomplete” on the SSL Labs test results.
To resolve the issue, you need to ensure that the server running the Hosted GET web service has the intermediate certificates from your Certificate Authority installed.
For example if you use GoDaddy, the best place to start would be here:
https://support.godaddy.com/help/article/868/what-is-an-intermediate-certificate?countrysite=au
Once you have the correct certificate chain in place, the Hosted GET data sources will sync to your Android devices, since they can now connect to your web service.
You can confirm the certificate chain by simply checking the Hosted GET url in your Android web browser – it should no longer show certificate errors when resolved.