[OAI-implementers] Using dates other than metadata record creation date for data provider "from" and "until" searches

[Simeon Warner]

Hi Lisa,

I think the one exception is already pointed out in Michael's reply, but I'll
make it explicit: BEFORE your data provider is public you can use historical 
datestamps to populate. Harvesters will be expecting this for the initial 
baseline harvest.

However, as Michael very nicely illustrates, after the data provider has been 
made public all updates/additions must be refelected with the time of 
update/addition in the datestamp to avoid missing updates.

Cheers,
Simeon


On Fri, Apr 11, 2008 at 08:32:59AM -0400, Michael Nelson wrote:
> Hi Lisa,
> 
> The OAI-PMH is pretty explicit about not doing what you're asking.  The
> OAI-PMH datestamps are about when the record was added (or updated) in the
> repository.  The publication date (or discovery date, or ...) is reflected
> in the metadata you are exporting.  People that harvest your metadata will
> index the publciation date and throw the OAI-PMH datestamps away -- they
> exist only to support harvesting.
> 
> Here is a scenario on why you don't want to set your OAI-PMH datestamps to
> your publication date.
> 
> - 2008-04-11, you initially publish 500,000 records.  They have OAI-PMH
> datestamps of 1985-01-01 through 2008-04-11.
> 
> - 2008-04-12, I discover your site for the first time and do a baseline
> harvest.  I get all your metadata records (since I haven't harvested you
> before).  My harvester decides to revist your site every week and ask
> what's new.
> 
> - 2008-04-14, you publish 100,000 more records with datestamps of
> 1981-01-01 through 1984-12-31.
> 
> - 2008-04-19, I return to your site and say:
> 
> ?verb=ListRecords&metadatPrefix=abc&from=2008-04-12
> 
> Now my harvester will miss your 100,000 new records since they were added
> "earlier" in time and not later.
> 
> In summary, be honest about your datestamps or people will miss your
> updates.  Harvesters / service providers won't confuse datestamps with
> publication dates.
> 
> regards,
> 
> Michael
> 
> On Thu, 10 Apr 2008, Lisa M. Brooks wrote:
> 
> > Hello - We're very close to launching our data provider. Before we do I have
> > a question about date-stamps.
> >
> > I understand that the "from" and "until" dates used to request metadata
> > records refer to the date that the metadata record was created. We are an
> > archive of research works that date back to the 1980s (we will definitely
> > get even older works into our archive as we move forward). To my mind it
> > would be more helpful to folks if our record date-stamps reflect the date
> > the research work in question was first published.
> >
> > My concern is that we introduce our repository and harvesters don't get the
> > gist of the temporal scope of our collection because everything is
> > date-stamped en masse with the date that we generate our metadata records
> > (which, with luck, will be this Saturday).
> >
> > I hope I'm making sense! Just want to know if this is a big no-no, or if
> > there are things to consider before doing something like this. Appreciate
> > the insight of list participants.
> >
> > Thanks for reading -
> > ~Lisa
> >
> > Lisa M. Brooks
> > IssueLab - bringing nonprofit research into focus
> > lisa at issuelab.org
> > 773-649-1790
> > http://www.issuelab.org
> >
> 
> ----
> Michael L. Nelson mln at cs.odu.edu http://www.cs.odu.edu/~mln/
> Dept of Computer Science, Old Dominion University, Norfolk VA 23529
> +1 757 683 6393 +1 757 683 4900 (f)
> 
> _______________________________________________
> OAI-implementers mailing list
> List information, archives, preferences and to unsubscribe:
> http://www.openarchives.org/mailman/listinfo/oai-implementers