[OAI-general] System Architecture
Young,Jeff
jyoung@oclc.org
Fri, 14 Feb 2003 09:38:21 -0500
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.
------_=_NextPart_001_01C2D436.B8DF86D9
Content-Type: text/plain;
charset="iso-8859-1"
Amy,
DSpace is a wonderful product, but OAI-compliance is a very small part of
what DSpace is about. If you merely want to add OAI capability to a system
that already exists, DSpace would be overkill and would probably make your
job much harder. Because DSpace is an fairly elaborate system, using it as a
model for your own architecture probably wouldn't be appropriate either.
OAI-compliance can most easily be achieved by downloading one of the OAI
implementations found on http://www.openarchives.org/tools/tools.html
<http://www.openarchives.org/tools/tools.html> . This page is a little
misleading, though, because many of the tools listed are much more than mere
implementations of the OAI protocol (e.g. DSpace).
I'm only familiar with two of the implementations that are available. If
you're most comfortable working in Perl, the VTOAI OAI-PMH2 PERL
implementation.seems to be popular
(http://www.dlib.vt.edu/projects/OAI/software/vtoai/vtoai.html
<http://www.dlib.vt.edu/projects/OAI/software/vtoai/vtoai.html> .) If you're
most comfortable using Java, I would suggest OAICat (
http://www.oclc.org/research/software/oai/cat.shtm
<http://www.oclc.org/research/software/oai/cat.shtm> ) which runs under a
J2EE-compliant servlet engine (such as Apache's Tomcat). I am the author of
the OAICat and would be glad to help you get it set up for your needs. BTW,
DSpace uses the OAICat implementation to provide its OAI repository
functionality.
OAICat can be customized to work with any database engine that supports a
Java API. Implementations are available for JDBC as well as a simple
file-based system. Other databases can be accommodated by implementing a set
of abstract Java classes. Implementations are also available for Newton and
the open-source Pears database engines which are used in OCLC's SiteSearch
product.
BTW, if you have an unusually large repository (or even if you don't) you
may even want to consider using the open-source Pears database (
http://www.oclc.org/research/software/pears/
<http://www.oclc.org/research/software/pears/> ). I use it for our XTCat
repository ( http://alcme.oclc.org/xtcat/ <http://alcme.oclc.org/xtcat/> )
which contains over 4.2 million records.
Hope this helps.
Jeff
-----Original Message-----
From: Hatfield, Amy J [mailto:ajhatfie@iupui.edu]
Sent: Friday, February 14, 2003 8:50 AM
To: oai-general@oaisrv.nsdl.cornell.edu
Subject: [OAI-general] System Architecture
Hello -
I am currently exploring our institution's ability to create an
institutional repository that is OAI compliant. I feel I have a fair
understanding of the standards, functionality, etc. involved - and am
leaning toward DSpace as a model - but there is one aspect that I don't see
discussed very much... system architecture!
I would be grateful if folks would share with me some architecture
configurations they have developed. I am most interested in the storage
aspect. I have looked at jukebox technology as a storage backend, but am
not sure about the retrieval aspect. We also have a super computer with
lots of storage capacity - but it is not designed to handle small files, but
rather large datasets. Any information you would like to share will be most
appreciated.
Feel free to respond directly to me if you would rather not post to the
list.
Kind regards,
Amy
Amy Jo Hatfield, Systems Librarian
Ruth Lilly Medical Library
Educational Technology
(317)278-8402
975 West Walnut Street
IB-100 Room 102A
Indianapolis, IN 46202-5121
------_=_NextPart_001_01C2D436.B8DF86D9
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns=3D"http://www.w3.org/TR/REC-html40" xmlns:o =3D=20
"urn:schemas-microsoft-com:office:office" xmlns:w =3D=20
"urn:schemas-microsoft-com:office:word" xmlns:st1 =3D=20
"urn:schemas-microsoft-com:office:smarttags"><HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">
<META content=3DWord.Document name=3DProgId>
<META content=3D"MSHTML 6.00.2800.1106" name=3DGENERATOR>
<META content=3D"Microsoft Word 10" name=3DOriginator><LINK=20
href=3D"cid:filelist.xml@01C2D406.0D8CF8B0" =
rel=3DFile-List><o:SmartTagType=20
name=3D"PostalCode"=20
namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"></o:SmartTag=
Type><o:SmartTagType=20
name=3D"State"=20
namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"></o:SmartTag=
Type><o:SmartTagType=20
name=3D"City"=20
namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"></o:SmartTag=
Type><o:SmartTagType=20
name=3D"place"=20
namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"></o:SmartTag=
Type><o:SmartTagType=20
name=3D"Street"=20
namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"></o:SmartTag=
Type><o:SmartTagType=20
name=3D"address"=20
namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"></o:SmartTag=
Type><!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:DoNotRelyOnCSS/>
</o:OfficeDocumentSettings>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:WordDocument>
<w:SpellingState>Clean</w:SpellingState>
<w:GrammarState>Clean</w:GrammarState>
<w:DocumentKind>DocumentEmail</w:DocumentKind>
<w:EnvelopeVis/>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
</w:Compatibility>
<w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel>
</w:WordDocument>
</xml><![endif]--><!--[if !mso]>
<STYLE>st1\:* {
BEHAVIOR: url(#default#ieooui)
}
</STYLE>
<![endif]-->
<STYLE>@font-face {
font-family: Comic Sans MS;
}
@page Section1 {size: 8.5in 11.0in; margin: 1.0in 1.25in 1.0in 1.25in; =
mso-header-margin: .5in; mso-footer-margin: .5in; mso-paper-source: 0; =
}
P.MsoNormal {
FONT-SIZE: 10pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Comic Sans MS"; =
mso-style-parent: ""; mso-pagination: widow-orphan; =
mso-fareast-font-family: "Times New Roman"; mso-bidi-font-family: Arial
}
LI.MsoNormal {
FONT-SIZE: 10pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Comic Sans MS"; =
mso-style-parent: ""; mso-pagination: widow-orphan; =
mso-fareast-font-family: "Times New Roman"; mso-bidi-font-family: Arial
}
DIV.MsoNormal {
FONT-SIZE: 10pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Comic Sans MS"; =
mso-style-parent: ""; mso-pagination: widow-orphan; =
mso-fareast-font-family: "Times New Roman"; mso-bidi-font-family: Arial
}
A:link {
COLOR: blue; TEXT-DECORATION: underline; text-underline: single
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline; text-underline: single
}
A:visited {
COLOR: purple; TEXT-DECORATION: underline; text-underline: single
}
SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline; text-underline: single
}
SPAN.EmailStyle17 {
COLOR: windowtext; FONT-FAMILY: Arial; mso-bidi-font-family: Arial; =
mso-style-type: personal-compose; mso-style-noshow: yes; =
mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt; =
mso-ascii-font-family: Arial; mso-hansi-font-family: Arial
}
SPAN.SpellE {
mso-style-name: ""; mso-spl-e: yes
}
DIV.Section1 {
page: Section1
}
</STYLE>
<!--[if gte mso 10]>
<style>
/* Style Definitions */=20
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.0pt;
font-family:"Times New Roman";}
</style>
<![endif]--></HEAD>
<BODY lang=3DEN-US style=3D"tab-interval: .5in" vLink=3Dpurple =
link=3Dblue>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff=20
size=3D2>Amy,</FONT></SPAN></DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff=20
size=3D2></FONT></SPAN> </DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff =
size=3D2>DSpace is a=20
wonderful product, but OAI-compliance is a very small part of what =
DSpace is=20
about. If you merely want to add OAI capability to a system that =
already exists,=20
DSpace would be overkill and would probably make your job much harder. =
Because=20
DSpace is an fairly elaborate system, using it as a model for your own=20
architecture probably wouldn't be appropriate =
either.</FONT></SPAN></DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff=20
size=3D2></FONT></SPAN> </DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff =
size=3D2>OAI-compliance=20
can most easily be achieved by downloading one of the OAI =
implementations found=20
on <A=20
href=3D"http://www.openarchives.org/tools/tools.html">http://www.openarc=
hives.org/tools/tools.html</A>.=20
This page is a little misleading, though, because many of the =
tools listed=20
are much more than mere implementations of the OAI protocol (e.g.=20
DSpace).</FONT></SPAN></DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff=20
size=3D2></FONT></SPAN> </DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff =
size=3D2>I'm only familiar=20
with two of the implementations that are available. If you're most =
comfortable=20
working in Perl, the VTOAI OAI-PMH2 PERL implementation.seems to be=20
popular <A=20
href=3D"http://www.dlib.vt.edu/projects/OAI/software/vtoai/vtoai.html">(=
http://www.dlib.vt.edu/projects/OAI/software/vtoai/vtoai.html</A>.)=20
If you're most comfortable using Java, I would suggest OAICat (<A=20
href=3D"http://www.oclc.org/research/software/oai/cat.shtm">http://www.o=
clc.org/research/software/oai/cat.shtm</A>)=20
which runs under a J2EE-compliant servlet engine (such as Apache's =
Tomcat). I am the author of the OAICat and would be glad to help =
you get it=20
set up for your needs. BTW, DSpace uses the OAICat implementation to =
provide its=20
OAI repository functionality.</FONT></SPAN></DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff=20
size=3D2></FONT></SPAN> </DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff =
size=3D2>OAICat can be=20
customized to work with any database engine that supports a Java API.=20
Implementations are available for JDBC as well as a simple file-based =
system.=20
Other databases can be accommodated by implementing a set of abstract =
Java=20
classes. Implementations are also available for Newton and the =
open-source Pears=20
database engines which are used in OCLC's SiteSearch=20
product.</FONT></SPAN></DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff=20
size=3D2></FONT></SPAN> </DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff =
size=3D2>BTW, if you have=20
an unusually large repository (or even if you don't) you may even want =
to=20
consider using the open-source Pears database (<A=20
href=3D"http://www.oclc.org/research/software/pears/">http://www.oclc.or=
g/research/software/pears/</A>).=20
I use it for our XTCat repository (<A=20
href=3D"http://alcme.oclc.org/xtcat/">http://alcme.oclc.org/xtcat/</A>) =
which=20
contains over 4.2 million records.</FONT></SPAN></DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff=20
size=3D2></FONT></SPAN> </DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff =
size=3D2>Hope this=20
helps.</FONT></SPAN></DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff=20
size=3D2></FONT></SPAN> </DIV>
<DIV><SPAN class=3D235290214-14022003><FONT color=3D#0000ff=20
size=3D2>Jeff</FONT></SPAN></DIV>
<BLOCKQUOTE dir=3Dltr=20
style=3D"PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px =
solid; MARGIN-RIGHT: 0px">
<DIV class=3DOutlookMessageHeader dir=3Dltr align=3Dleft><FONT =
face=3DTahoma=20
size=3D2>-----Original Message-----<BR><B>From:</B> Hatfield, Amy J=20
[mailto:ajhatfie@iupui.edu]<BR><B>Sent:</B> Friday, February 14, 2003 =
8:50=20
AM<BR><B>To:</B> =
oai-general@oaisrv.nsdl.cornell.edu<BR><B>Subject:</B>=20
[OAI-general] System Architecture<BR><BR></FONT></DIV>
<DIV class=3DSection1>
<P class=3DMsoNormal><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: Arial">Hello=20
-<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: Arial">I am currently =
exploring our=20
institution's ability to create an institutional repository that is =
OAI=20
compliant.<SPAN style=3D"mso-spacerun: yes"> </SPAN>I feel I =
have a fair=20
understanding of the standards, functionality, etc. involved - and am =
leaning=20
toward DSpace as a model - but there is one aspect that I don't see =
discussed=20
very much...<SPAN style=3D"mso-spacerun: yes"> </SPAN>system=20
architecture!<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: Arial">I would be grateful if =
folks would=20
share with me some architecture configurations they have =
developed.<SPAN=20
style=3D"mso-spacerun: yes"> </SPAN>I am most interested in the =
storage=20
aspect.<SPAN style=3D"mso-spacerun: yes"> </SPAN>I have looked =
at jukebox=20
technology as a storage backend, but am not sure about the retrieval=20
aspect.<SPAN style=3D"mso-spacerun: yes"> </SPAN>We also have a =
super=20
computer with lots of storage capacity - but it is not designed to =
handle=20
small files, but rather large datasets.<SPAN style=3D"mso-spacerun: =
yes"> =20
</SPAN>Any information you would like to share will be most=20
appreciated.<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: Arial">Feel free to respond =
directly to=20
me if you would rather not post to the =
list.<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: Arial">Kind=20
regards,<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial">Amy<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3DArial size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; FONT-FAMILY: =
Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3D"Comic Sans MS" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; mso-no-proof: yes">Amy Jo =
Hatfield, Systems=20
Librarian<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3D"Comic Sans MS" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; mso-no-proof: yes">Ruth Lilly Medical=20
Library<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3D"Comic Sans MS" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; mso-no-proof: yes">Educational=20
Technology<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3D"Comic Sans MS" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; mso-no-proof: =
yes">(317)278-8402<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><st1:Street><st1:address><FONT face=3D"Comic =
Sans MS"=20
size=3D2><SPAN style=3D"FONT-SIZE: 10pt; mso-no-proof: yes">975 West =
Walnut=20
Street</SPAN></FONT></st1:address></st1:Street><SPAN=20
style=3D"mso-no-proof: yes"><o:p></o:p></SPAN></P>
<P class=3DMsoNormal><FONT face=3D"Comic Sans MS" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; mso-no-proof: yes">IB-100 Room=20
102A<o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><st1:place><st1:City><FONT face=3D"Comic Sans =
MS"=20
size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; mso-no-proof: =
yes">Indianapolis</SPAN></FONT></st1:City><SPAN=20
style=3D"mso-no-proof: yes">, </SPAN><st1:State><SPAN=20
style=3D"mso-no-proof: yes">IN</SPAN></st1:State><SPAN=20
style=3D"mso-no-proof: yes"> </SPAN><st1:PostalCode><SPAN=20
style=3D"mso-no-proof: =
yes">46202-5121</SPAN></st1:PostalCode></st1:place><SPAN=20
style=3D"mso-no-proof: yes"><o:p></o:p></SPAN></P>
<P class=3DMsoNormal><FONT face=3D"Comic Sans MS" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; mso-no-proof: =
yes"> <o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3D"Comic Sans MS" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; mso-no-proof: =
yes"> <o:p></o:p></SPAN></FONT></P>
<P class=3DMsoNormal><FONT face=3D"Comic Sans MS" size=3D2><SPAN=20
style=3D"FONT-SIZE: 10pt; mso-no-proof: =
yes"> </SPAN><o:p></o:p></FONT></P>
<P class=3DMsoNormal><FONT face=3D"Comic Sans MS" size=3D2><SPAN=20
style=3D"FONT-SIZE: =
10pt"><o:p> </o:p></SPAN></FONT></P></DIV></BLOCKQUOTE></BODY></HTM=
L>
------_=_NextPart_001_01C2D436.B8DF86D9--