<br><br><div><span class="gmail_quote">2007/11/26, <a href="mailto:oai-implementers-request@openarchives.org">oai-implementers-request@openarchives.org</a> <<a href="mailto:oai-implementers-request@openarchives.org">oai-implementers-request@openarchives.org
</a>>:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Send OAI-implementers mailing list submissions to<br> <a href="mailto:oai-implementers@openarchives.org">
oai-implementers@openarchives.org</a><br><br>To subscribe or unsubscribe via the World Wide Web, visit<br> <a href="http://www.openarchives.org/mailman/listinfo/oai-implementers">http://www.openarchives.org/mailman/listinfo/oai-implementers
</a><br>or, via email, send a message with subject or body 'help' to<br> <a href="mailto:oai-implementers-request@openarchives.org">oai-implementers-request@openarchives.org</a><br><br>You can reach the person managing the list at
<br> <a href="mailto:oai-implementers-owner@openarchives.org">oai-implementers-owner@openarchives.org</a><br><br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of OAI-implementers digest..."
<br><br><br>Today's Topics:<br><br> 1. Re: Do you have instruction for ad hoc harvesters? (Caroline Arms)<br> 2. Re: Do you have instruction for ad hoc harvesters? (Ed Summers)<br><br><br>----------------------------------------------------------------------
<br><br>Message: 1<br>Date: Sun, 25 Nov 2007 12:29:29 -0500<br>From: "Caroline Arms" <<a href="mailto:caar@loc.gov">caar@loc.gov</a>><br>Subject: Re: [OAI-implementers] Do you have instruction for ad hoc<br>
harvesters?<br>To: <<a href="mailto:oai-implementers@openarchives.org">oai-implementers@openarchives.org</a>><br>Message-ID: <<a href="mailto:47496AA90200003500029730@ntgwgate.loc.gov">47496AA90200003500029730@ntgwgate.loc.gov
</a>><br>Content-Type: text/plain; charset=US-ASCII<br><br>Conal,<br><br>Thanks
for your suggestion about hiding the resumption token complexity from a
user via an HTML front end. Unfortunately, it's not that
simple for a variety of reasons, one of which is scale. A
popular set has almost 200,000 records. When the harvester
has to use the resumption token process, that can be done without
affecting performance on the applications that are on the same server
and using the same data. We don't want to create a short cut
that causes problems for our regular users and harvesters, just to help
a few requesters (who are already looking for records to batchload into
a local database and are assumed to be savvy about data wrangling in
their local environment) through steps that get them the records they
want in the format they want.<br><br> But thanks again for your suggestion.<br><br> Caroline Arms <a href="mailto:caar@loc.gov">caar@loc.gov</a><br><br>>>> Conal Tuohy <<a href="mailto:conal.tuohy@vuw.ac.nz">
conal.tuohy@vuw.ac.nz</a>> 11/22/07 9:19 PM >>><br>On Thu, 2007-11-22 at 12:38 -0500, Caroline Arms wrote:<br>>
At the Library of Congress we quite often get requests for the records
for a collection of digitized historical materials from entities
outside the library or digital repository
community. Typically, these are organizations
that want to integrate a collection of photographs into an internal
system for a particular project. An example would be the
production team for a TV documentary assembling an internal collection
of records and images relevant to the topic to use as the basis for
selection for use in the production and tracking of associated
workflow. When pointed at the OAI site, they are mystified.<br>><br>>
I am wondering whether we can create a quick how-to document tailored
to this particular task that makes no assumptions about the technology
at the other end.<br>><br>> Does anyone have or know of a brief
introduction aimed at someone who only needs to know enough about
OAI-PMH to get the records for an entire set given its setSpec (having
to deal with resumption tokens), but may need to be told soem other
things, such as:<br>> * they will have to
understand enough about the semantics of the metadata formats available
to select the right metadata prefix<br>>
* they will probably need XML tools to transform the records
into something compatible with their local system<br>> * etc., etc.<br>><br>>
If you have something like this written that has been used
successfully, and would be prepared to share it, we would love to see
it.<br><br>This is perhaps a bit tangential, but might be worth considering...<br><br>I've seen some OAI repositories which serve up their content with<br><Xxml-stylesheet?> processing instructions referring to XSLT stylesheets
<br>which convert the XML into nice HTML pages which provide a nice friendly<br>user interface. You don't know you're dealing with an OAI server at all!<br>These XML processing instructions are ignored by real OAI harvesters,
<br>but they are respected by ordinary web browsers, and they could provide<br>users with a simple way to select metadata formats, navigate through<br>sets, follow resumption tokens, etc, etc.<br><br>For instance, here's one here:
<br><a href="http://info-uri.info/registry/OAIHandler?verb=ListRecords&metadataPrefix=oai_dc">http://info-uri.info/registry/OAIHandler?verb=ListRecords&metadataPrefix=oai_dc</a><br><br>Cheers<br><br>Conal<br><br>--
<br>Conal Tuohy<br>New Zealand Electronic Text Centre<br><a href="http://www.nzetc.org">www.nzetc.org</a><br><br><br>_______________________________________________<br>OAI-implementers mailing list<br>List information, archives, preferences and to unsubscribe:
<br><a href="http://www.openarchives.org/mailman/listinfo/oai-implementers">http://www.openarchives.org/mailman/listinfo/oai-implementers</a><br><br><br><br><br><br>------------------------------<br><br>Message: 2<br>Date: Mon, 26 Nov 2007 10:19:57 -0500
<br>From: "Ed Summers" <<a href="mailto:ehs@pobox.com">ehs@pobox.com</a>><br>Subject: Re: [OAI-implementers] Do you have instruction for ad hoc<br> harvesters?<br>To: <a href="mailto:oai-implementers@openarchives.org">
oai-implementers@openarchives.org</a><br>Message-ID:<br> <<a href="mailto:f032cc060711260719se702e46j474873fd6d589ff@mail.gmail.com">f032cc060711260719se702e46j474873fd6d589ff@mail.gmail.com</a>><br>Content-Type: text/plain; charset=ISO-8859-1
<br><br>I wrote an article for Ariadne a few years ago that aimed to<br>demonstrate how to easily harvest records from an OAI-PMH provider<br>with Perl [1]. The examples used a Perl module called<br>Net::OAI::Harvester which supports the 6 verbs, and adds a couple
<br>more: listAllRecords and listAllIdentifiers which automatically handle<br>resumption tokens.<br><br>That being said, perhaps you could make a static repository [2]<br>available? You could periodically generate a static file from your
<br>dynamic service and make the big file available as needed. Just out of<br>curiosity has anyone written a program that will generate a static<br>repository given the base URL for a data provider? It seems like a<br>generally useful utility that may have already been written.
<br><br>//Ed<br><br>[1] <a href="http://www.ariadne.ac.uk/issue38/summers/">http://www.ariadne.ac.uk/issue38/summers/</a><br>[2] <a href="http://www.openarchives.org/OAI/2.0/guidelines-static-repository.htm">http://www.openarchives.org/OAI/2.0/guidelines-static-repository.htm
</a><br><br><br><br>------------------------------<br><br>_______________________________________________<br>OAI-implementers mailing list<br><a href="mailto:OAI-implementers@openarchives.org">OAI-implementers@openarchives.org
</a><br><a href="http://www.openarchives.org/mailman/listinfo/oai-implementers">http://www.openarchives.org/mailman/listinfo/oai-implementers</a><br><br><br>End of OAI-implementers Digest, Vol 38, Issue 7<br>***********************************************
<br></blockquote></div><br>