FW: [OAI-implementers] Open Archives Initiative Protocol for Meta data Harvesting Version 2 news

[Simeon Warner]

On Fri, 8 Feb 2002, Xiaoming Liu wrote:
> Because the datastamp is always increasing in OAI, I think  Alan's
> method (high resolution date stamps and results is ordered by
> time) will also work, but not necessarily monatomically, if the DP could
> return all records of a specific datestamp in one reply. But it did
> put some dangers to harvester as Walter suggested, if suddenly DP creates
> 10K records with same datestamp, it has to return them in one response, it
> quite possibly will break the harvester.

This is pretty much the solution I adopted for arXiv. The resumptionToken 
is essentially a query to resume the list and where possible this follows
day granularity. However, we have a few days with large numbers of updates
(migrations) so to cope with this I add an index within the day.
--
Simeon