FW: [OAI-implementers] Open Archives Initiative Protocol for Meta data Harvesting Version 2 news

[Simeon Warner]

[Admin point first: Can we stop this duplicate posting to oai-general? 
The description of oai-general is "The OAI list will keep you informed 
about OAI-related activities." I don't think these discussions fall 
within that.]


My main objection to including an option for harvesters to specify the
maximum number of records they wish to get in a reply is that this will
force ALL repositories to implement resumptionTokens.  Currently, small
repsotiories (say a few thousand records) can happily ignore that part of
the spec.

One of the primary objectives of OAI is to provide a low barrier to
adoption, particularly on the repository side. I therefore think that
requiring this would go against our objectives, even if it is cleaner in
some ways.

Cheers,
Simeon.


On Wed, 6 Feb 2002, Young,Jeff wrote:
> Imagine a resultset with 1 million records served in 1000 record chunks.
> During the course of the harvest 10 records get deleted from the repository.
> Since the stateless solution relies on the position of a cursor, the
> client's view of the cursor may be as many as 10 records beyond the server's
> view and thus records will be missed. Using resumptionTokens, however, I can
> maintain a consistent cursor between client and server.
> 
> Also, I'd like to use OAI for internal operations within our organization.
> Under those circumstances, I can make assumptions about the OAI
> server/harvester such as records will never vanish and instead will be
> flagged as deleted. With millions records in our repository, I'd like to
> avoid a complete reharvest wherever possible. I don't believe reharvests can
> be avoided using stateless harvesting the way they can with stateful
> harvesting.
> 
> Jeff