[OAI-implementers] Re: [oai-alpha] Re: Help!

[Thomas G. Habing]

Hi all,

Just thought I would add to the discussion re. 400 errors versus empty
results.

Our repository currently has 14 cases which return 400 errors (although
based on the previous messages, I am thinking of adding some more):

1) If there is no verb parameter at all:
	400 Bad Request ('verb' is a required parameter)

2) An unknown verb is encountered:
	400 Bad Request('" & verb & "' is not a valid verb)

GetRecord
---------
3) If there is no identifier:
	400 Bad Request ('identifier' is a required parameter)

4) If there is no metadaPrefix:
	400 Bad Request ('metadataPrefix' is a required parameter)

ListIdentifiers
---------------
5) If there is a resumptionToken plus other parameters:
	400 Bad Request ('resumptionToken' must be the only parameter)

6) If until is not a valid date:
	400 Bad Request ('" & untilDateStr & "' is not a valid date)

7) If from is not a valid date:
	400 Bad Request ('" & fromDateStr & "' is not a valid date)

8) An unrecognized resumptionToken is sent:
	400 Bad Request ('" & server.urlencode(resumptionToken)& "' is not a valid
	resumptionToken)

ListRecords
-----------
9) If there is no metadaPrefix:
	400 Bad Request ('metadataPrefix' is a required parameter)

10) If there is a resumptionToken plus other parameters:
	400 Bad Request ('resumptionToken' must be the only parameter)

11) If until is not a valid date:
	400 Bad Request ('" & untilDateStr & "' is not a valid date)

12) If from is not a valid date:
	400 Bad Request ('" & fromDateStr & "' is not a valid date)

13) An unrecognized resumptionToken is sent:
	400 Bad Request ('" & server.urlencode(resumptionToken)& "' is not a valid
	resumptionToken)

ListSets
--------
14) A resumption token is sent (our ListSets doesn't implement this):
	400 Bad Request ('" & server.urlencode(resumptionToken) & "' is not a valid
	resumptionToken)

Weighing in on the current controversy, I agree with Simeon that malformed
arguments should probably result in 400s, but correctly formed arguments
that just can't be supported by the repository should result in empty XML. 
Based on this I am considering adding (in addition to the above):

1) checks that an 'identifier' is a valid URI (but not necessarily using the
OAI scheme),

2) that 'metadataPrefix' conforms to the given regular expression

3) that 'set' conforms to the given regular expression

4) possibly, that the the dates 'from' <= 'until'?

Another issue relating to errors is the addition of non-standard parameters.
For example, there might be a community of providers and harvesters which
decide to extend the protocol, possibly by adding non-standard parameters to
some of the verbs. For example, adding a 'creator' param to the ListRecords
verb so that only records with a given creator are returned.  The question
is, should providers return a 400 if they encounter an unknown parameter,
or, assuming all else is correct, should they just ignore the extra param,
and process the request as usual.  In other words, should communities be
allowed to extend the protocol with the expectation that providers who do
not support the extension will degrade gracefully, and continue to honor
their requests, simply ignoring the extensions?  Should providers that do
this be allowed to register as conformant to the OAI spec?

Note that I have no intention of doing this with our system, but since it
just occurred to me I thought I should bring it up.

Regards,
	Tom

-- 
Thomas G. Habing
Research Programmer, Digital Library Initiative
University of Illinois at Urbana-Champaign
052 Grainger Engineering Library, MC-274
thabing@uiuc.edu, (217) 244-7809


Hussein Suleman wrote:
> 
> hi
> 
> from the perspective of someone writing conformance tests, i am probably
> going to use a 2-fold certification scheme, possibly calling the levels
> "correct" and "robust" ... where correct implies that the repository
> responds properly to correctly formatted requests and robust meaning
> that the repository responds properly to incorrect requests ...
> 
> so, if your repository responds properly to all the verbs (with all the
> correct parameter combinations) your repository will be "correct"
> 
> and, if you can handle erroneous requests gracefully, where illegal
> combinations of arguments generate 400's and illegal values of arguments
> generate either 400's or valid XML, then your implementation is "robust"
> and you can add your archive to the repository explorer as a
> demonstration archive.
> 
> i think since we didnt rigorously define what illegal parameter values
> are, we need to be prepared to accept both 400's and empty responses as
> correct since the interpretation is going to be archive-specific.
> 
> [disclaimer: of course this is not how it works at the moment - right
> now the definitions are still fuzzy but i will change those as soon as i
> see agreement emerging from this discussion.]
> 
> one more concern i have is about list sizes... validation is a wonderful
> feature to have but more validation = more requests = more responses =
> more record metadata ... for illustration, if i wrote a "good"
> validation test (which, mind you i have not because of this exact issue)
> for "ListRecords" i would need at least 4 metadata formats (default,
> non-default, bad, missing), 8 date pairs (valid from, valid until, valid
> both, from>until, invalid from, invalid until, invalid both, missing), 3
> sets (valid, invalid, missing), 3 resumptionTokens (valid, invalid,
> missing), etc ...
> 
> so without going into more combinatorial explosion math, its obvious
> that testing is a delicate balance between correctness and denial of
> service :)) ... i would like to provide more correctness testing and the
> greatest obstacle to this is massive ListRecords/ListIdentifiers
> responses from some repositories ... so, a plea to all out there: flow
> control is a wonderful feature - lets all use it :)
> 
> ttfn
> 
> hussein
> 
> --
> =========================================================================
> hussein suleman -- hussein@vt.edu -- vt cs --
> http://purl.org/net/hussein
> =========================================================================
> _______________________________________________
> OAI-implementers mailing list
> OAI-implementers@oaisrv.nsdl.cornell.edu
> http://oaisrv.nsdl.cornell.edu/mailman/listinfo/oai-implementers