[OAI-implementers] Better resumption mechanism - more importa nt than ever!

['Alan Kent']

On Tue, Mar 05, 2002 at 02:40:58PM -0500, Xiaoming Liu wrote:
> I guess this is a major issue that harvester should follow a certain
> policy. In current implementation in Arc, for each failed request, the
> harvester will try at most three times using the same http request...

The problem I have with this is in the current OAI spec as I read
it, this is not a safe thing to do. OAI does not mandate that
resumptionToken's change. That is, resumptionToken *could* be
a session id, with no cursor information in it. Each request
using the same resumptionToken therefore is permitted to return
the next N records.

So I could implement a data provider where the above algorithm is
wrong and will silently deliver incorrect results.

Does OAI 2.0 say that resumptionToken's must be unique within
a download? And that reusing an old resumptionToken must be
supported (or rejected with an error)? If not guaranteed by
the spec, then I would not want to write a harvester relying
on it. I would rather spend the effort and get the spec right
rather than having to come to agreements with individual data
providers.

One approach discussed was of course to add something to the
Identify response allowing servers to advertise 'its safe
to reuse resumptionTokens'. More in a later mail.

(I am not proposing anything in this mail - just saying I believe
the above retry algorithm is theoretically unsafe.)

Alan