IATI Validator Public API: user requirements feedback by 15 March

Dear technical team, as the company who has supplied the IATI secretariat with the IATI validator I like to point out that in the original ToR the request has been to come up with a web interface for validation. We would by the way appreciate the credentials for our build being mentioned on the current IATI validator page as it was included before. We welcome the discussion on building an API that was initiated after the technical stocktake and will provide feedback on the draft Validator API contract. 

Hi IATI Technical Team thanks a lot for sharing this and for the opportunity to engage! It's very welcome to have this kind of document.

So that API users can have further certainty and expectations in terms of service, perhaps it would be worth setting some other expectations in the contract, e.g.:

• what is the target level of service uptime, resolution of major outages/bugs, etc.
• how long will particular API routes remain available for maintained or unmaintained / "as is" (e.g. what is the deprecation notice period)

This could help potential users make decisions around potentially integrating with this tool.

I think the document itself looks great. Though I can appreciate you might want to keep the scope small to start with, here are a few thoughts on a some potential additions that I think would be very useful:

• Should there also be a set of additional endpoints to get:
  • a list of all files (including e.g. registry ID, hash, last validation date) - presumably paginated
  • a list of files currently in the queue awaiting processing
  • a list of files by publisher - again, presumably paginated
  • a list of files by various types of critical error^: e.g. couldn't be processed at all (e.g. they wouldn't download); failed schema validation
• I think it's really important to be able to access files by the Registry package ID (e.g. worldbank-bd rather than a hash).

(^) maybe eventually it would be nice to display files by all the various types of errors in the validator but I think this would be a good and important place to start. It's also in line with one of the findings in the feedback mechanisms report:

https://iatistandard.org/en/news/improving-iati-data-quality-feedback-l…

Echoing Mark’s comments, this sort of document is really helpful – thanks for providing it.

The report route seems okay to me. I assume `id` in the request parameters refers to the registry package ID, e.g. fcdo-af.

I have some questions about the validate route. Specifically, about the fact it is synchronous:

1. The validator’s web interface is currently asynchronous – I click “validate” and then arrive at a loading page, where I wait for my uploaded file to be processed. (The previous public validator processed files synchronously.) Is the plan to change the web interface too, so that it also processes files synchronously? (That would allow the web interface to use this public facing API.)

2. Are there potentially problems with making this route synchronous, given the validator may receive simultaneous requests to process large XML files? E.g. is there a risk that operations may time out, or the server will be overloaded? I suppose I’m wondering about the justification for making this asynchronous in the current web interface, and why those reasons might not still hold true.

3. Could making this route synchronous limit future additions to the validation report? I.e. it might be desirable to add extra validation checks in the future, but this could be problematic to do synchronously if those extra checks are computationally expensive.

4. Will an XML file size limit be imposed? If so, what will that limit be? I have a feeling there used to be a recommended maximum file size for IATI XML, but I’m not able to find it in current documentation.

Thanks in advance,

In addition to the remarks of Mark and Andy, I would suggest adding response times as a non-functional requirement in the contract. E.g. reporting API requests return the validation report with maximum 10 seconds after the request is being made, Validating requests return the results within 30 seconds after the request is being made for XML files smaller than 10Mb, within 60 seconds for files smaller than 20Mb and for larger files on best effort basis. Of course these requirement need to be realistically achievable given the current performance of the IATI data validator.

Also something should be said i.m.o. on the number of concurrent users the API should be able to handle without exceeding the response time targets.

Thanks to everyone for engaging in the request for feedback on the  draft Validator API. This  consultation was specifically about the API contract and whether it meets user requirements, rather than the implementation of the work.  We are pleased to see that based on the comments there are no objections and agreement that the draft API contract meets current user needs. We will proceed now with the implementation of the work.

In response to some of your specific comments:

Mark Brough  

• what is the target level of service uptime, resolution of major outages/bugs, etc.

These are sensible questions and we will have that level of detail at a later stage, when we develop the non-functional requirements and SLA for this. Once we get to that stage we will update the community.

• how long will particular API routes remain available for maintained or unmaintained / "as is" (e.g. what is the deprecation notice period)

Similar to the point above,  the routes will be available and as standard practice any deprecation period will be specified in advance.

• Though I can appreciate you might want to keep the scope small to start with, here are a few thoughts on a some potential additions that I think would be very useful

Thanks for these. There are some useful suggestions and we will be happy to engage at a later date once the initial API contract has been delivered. One point about some of the suggestions is that some endpoints are available by the Registry API, so we want to ensure there is no duplication. The Validator does pull the list of organisations/publishers by using the IATI Registry publisher id. 

• The report route seems okay to me. I assume `id` in the request parameters refers to the registry package ID, e.g. fcdo-af.

Yes, though we will make provision for the file to be accessed by either the package ID or the hash. 

Andy Lulham

Thanks for your questions above.  They will be really helpful for the next phase of this work, which is implementation. Our Developers will be working on delivering and ensuring that it works as expected.

To your specific point about IATI file size recommendation, when organisations add their datasets to the Registry,  they can  see the below message.

“All files should be in XML following the IATI standard. Note that datasets are not hosted on this site, but by the publisher of the data. Please ensure files are less than 40MB.”

Herman van Loon

Thanks for the points above-- they will be really useful when we move towards implementation and develop non-functional requirements. We will be load testing during the implementation phase to understand and ensure adequate performance. Also, do note that this API will be delivered in the next version of the Validator, and the infrastructure and architecture will be designed to allow the API contract to be adequately implemented.  

Would it also be possible to support requests to http://validator.iatistandard.org/?url=[dataset URL] ?

That’s the format used by the previous iteration of the validator, and in use e.g. in the “validator” links on the dashboard: http://dashboard.iatistandard.org/validation.html

These links are now broken – it would be good to reinstate them somehow.