[OAI-implementers] Using dates other than metadata record creation date for data provider "from" and "until" searches

[Michael Nelson]

Hi Lisa,

The OAI-PMH is pretty explicit about not doing what you're asking.  The
OAI-PMH datestamps are about when the record was added (or updated) in the
repository.  The publication date (or discovery date, or ...) is reflected
in the metadata you are exporting.  People that harvest your metadata will
index the publciation date and throw the OAI-PMH datestamps away -- they
exist only to support harvesting.

Here is a scenario on why you don't want to set your OAI-PMH datestamps to
your publication date.

- 2008-04-11, you initially publish 500,000 records.  They have OAI-PMH
datestamps of 1985-01-01 through 2008-04-11.

- 2008-04-12, I discover your site for the first time and do a baseline
harvest.  I get all your metadata records (since I haven't harvested you
before).  My harvester decides to revist your site every week and ask
what's new.

- 2008-04-14, you publish 100,000 more records with datestamps of
1981-01-01 through 1984-12-31.

- 2008-04-19, I return to your site and say:

?verb=ListRecords&metadatPrefix=abc&from=2008-04-12

Now my harvester will miss your 100,000 new records since they were added
"earlier" in time and not later.

In summary, be honest about your datestamps or people will miss your
updates.  Harvesters / service providers won't confuse datestamps with
publication dates.

regards,

Michael

On Thu, 10 Apr 2008, Lisa M. Brooks wrote:

> Hello - We're very close to launching our data provider. Before we do I have
> a question about date-stamps.
>
> I understand that the "from" and "until" dates used to request metadata
> records refer to the date that the metadata record was created. We are an
> archive of research works that date back to the 1980s (we will definitely
> get even older works into our archive as we move forward). To my mind it
> would be more helpful to folks if our record date-stamps reflect the date
> the research work in question was first published.
>
> My concern is that we introduce our repository and harvesters don't get the
> gist of the temporal scope of our collection because everything is
> date-stamped en masse with the date that we generate our metadata records
> (which, with luck, will be this Saturday).
>
> I hope I'm making sense! Just want to know if this is a big no-no, or if
> there are things to consider before doing something like this. Appreciate
> the insight of list participants.
>
> Thanks for reading -
> ~Lisa
>
> Lisa M. Brooks
> IssueLab - bringing nonprofit research into focus
> lisa at issuelab.org
> 773-649-1790
> http://www.issuelab.org
>

----
Michael L. Nelson mln at cs.odu.edu http://www.cs.odu.edu/~mln/
Dept of Computer Science, Old Dominion University, Norfolk VA 23529
+1 757 683 6393 +1 757 683 4900 (f)