[OAI-general] FW: [OAI-implementers] Open Archives Initiative Protocol for Meta
data Harvesting Version 2 news
Young,Jeff
jyoung@oclc.org
Tue, 5 Feb 2002 12:27:06 -0500
Sorry Walter, I forgot to copy the listserv in my response.
-----Original Message-----
From: Young,Jeff
Sent: Tuesday, February 05, 2002 12:25 PM
To: 'Walter Underwood'
Subject: RE: [OAI-implementers] Open Archives Initiative Protocol for
Metadata Harvesting Version 2 news
> From: Walter Underwood [mailto:wunder@inktomi.com]
> Sent: Monday, February 04, 2002 5:11 PM
> This is surprisingly close to my suggested list approach. If this was
> changed to have the client to send the cursor and the number
> of records
> requested, then the resumption token is no longer needed.
> Think of it as
> moving the database cursor from the server the client. Offloading the
> state. This approach is proven in high-load applications
> like LDAP and
> HTTP search engines.
Tim Brody raised a point in a different thread that hasn't been addressed
yet:
I would be interested to know how resumptionTokens can be avoided,
as RT is both flow-control (which can be replaced by start-maxrows
requests), but also state information (i.e. which records are to be
returned). If different sections of the same query are requested,
without state being maintained, surely there is a risk that some
records may be missed in the overlap? (or are you presuming that all
records are added, and returned, sequentially?)
Of course, resumptionTokens don't guarantee that an arbitrary data provider
will return a complete set of results. They merely provide a mechanism to
make it possible. Without such a guarantee, harvesters are obliged to
periodically reharvest the entire repository if they want to pick up those
missed items.
Jeff