org.dspace.content.packager

Class AbstractInternalMETSIngester

java.lang.Object

org.dspace.content.packager.AbstractPackageIngester

org.dspace.content.packager.AbstractInternalMETSIngester

All Implemented Interfaces:

org.dspace.content.packager.PackageIngester

Direct Known Subclasses:

InternalDSpaceAIPIngester

public abstract class AbstractInternalMETSIngester
extends org.dspace.content.packager.AbstractPackageIngester

Base class for package ingester of METS (Metadata Encoding & Transmission Standard) Packages.
See http://www.loc.gov/standards/mets/.

This is a generic packager framework intended to be subclassed to create ingesters for more specific METS "profiles". METS is an abstract and flexible framework that can encompass many different kinds of metadata and inner package structures.

Package Parameters:

• validate -- true/false attempt to schema-validate the METS manifest.
• manifestOnly -- package consists only of a manifest document.
• ignoreHandle -- true/false, ignore AIP's idea of handle when ingesting.
• ignoreParent -- true/false, ignore AIP's idea of parent when ingesting.

Configuration Properties:

• mets.CONFIGNAME.ingest.preserveManifest - if true, the METS manifest itself is preserved in a bitstream named mets.xml in the METADATA bundle. If it is false (the default), the manifest is discarded after ingestion.
• mets.CONFIGNAME.ingest.manifestBitstreamFormat - short name of the bitstream format to apply to the manifest; MUST be specified when preserveManifest is true.
• mets.default.ingest.crosswalk.MD_SEC_NAME = PLUGIN_NAME Establishes a default crosswalk plugin for the given type of metadata in a METS mdSec (e.g. "DC", "MODS"). The plugin may be either a stream or XML-oriented ingestion crosswalk. Subclasses can override the default mapping with their own, substituting their configurationName for "default" in the configuration property key above.
• mets.CONFIGNAME.ingest.useCollectionTemplate - if true, when an item is created, use the collection template. If it is false (the default), any existing collection template is ignored.

Version:

$Revision$

Author:

Larry Stone, Tim Donohue

See Also:

METSManifest, AbstractPackageIngester, PackageIngester

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	AbstractInternalMETSIngester.MdrefManager An instance of ZipMdrefManager holds the state needed to retrieve the contents of an external metadata stream referenced by an mdRef element in a Zipped up METS manifest.

Constructor Summary

Constructors

Constructor and Description
AbstractInternalMETSIngester()

Method Summary

Methods

Modifier and Type	Method and Description
protected void	addBitstreams(org.dspace.core.Context context, org.dspace.content.Item item, org.dspace.content.packager.METSManifest manifest, File pkgFile, org.dspace.content.packager.PackageParameters params, AbstractInternalMETSIngester.MdrefManager mdRefCallback) Add Bitstreams to an Item, based on the files listed in the METS Manifest
protected void	addContainerLogo(org.dspace.core.Context context, org.dspace.content.DSpaceObject dso, org.dspace.content.packager.METSManifest manifest, File pkgFile, org.dspace.content.packager.PackageParameters params) Add a Logo to a Community or Collection container object based on a METS Manifest.
abstract void	addLicense(org.dspace.core.Context context, org.dspace.content.Item item, String license, org.dspace.content.Collection collection, org.dspace.content.packager.PackageParameters params) Add license(s) to Item based on contents of METS and other policies.
protected void	addManifestBitstream(org.dspace.core.Context context, org.dspace.content.Item item, org.dspace.content.packager.METSManifest manifest) Save/Preserve the METS Manifest as a Bitstream attached to the given DSpace item.
protected void	addTemplateItem(org.dspace.core.Context context, org.dspace.content.DSpaceObject dso, org.dspace.content.packager.METSManifest manifest, File pkgFile, org.dspace.content.packager.PackageParameters params, AbstractInternalMETSIngester.MdrefManager callback) Add a Template Item to a Collection container object based on a METS Manifest.
abstract void	crosswalkObjectDmd(org.dspace.core.Context context, org.dspace.content.DSpaceObject dso, org.dspace.content.packager.METSManifest manifest, AbstractInternalMETSIngester.MdrefManager callback, org.jdom.Element[] dmds, org.dspace.content.packager.PackageParameters params) Select the dmdSec element(s) to apply to the Item.
protected String	decodeHandleURN(String value) Parse the hdl: URI/URN format into a raw Handle.
abstract void	finishBitstream(org.dspace.core.Context context, org.dspace.content.Bitstream bs, org.jdom.Element mfile, org.dspace.content.packager.METSManifest manifest, org.dspace.content.packager.PackageParameters params) Subclass-dependent final processing on a Bitstream; could include fixing up the name, bundle, other attributes.
abstract void	finishObject(org.dspace.core.Context context, org.dspace.content.DSpaceObject dso, org.dspace.content.packager.PackageParameters params) Hook for final "finishing" operations on the new Object.
abstract String	getConfigurationName() Returns keyword that makes the configuration keys of this subclass unique, e.g.
protected InputStream	getFileInputStream(File pkgFile, org.dspace.content.packager.PackageParameters params, String path, org.dspace.core.Context context) Retrieve the inputStream for a File referenced from a specific path within a METS package.
protected String	getManifestBitstreamFormat()
String	getObjectHandle(org.dspace.content.packager.METSManifest manifest) Determines the handle of the DSpace object represented in this METS doc.
abstract int	getObjectType(org.dspace.content.packager.METSManifest manifest) Determines what type of DSpace object is represented in this METS doc.
String	getParameterHelp() Returns a user help string which should describe the additional valid command-line options that this packager implementation will accept when using the -o or --option flags with the Packager script.
org.dspace.content.DSpaceObject	getParentObject(org.dspace.core.Context context, org.dspace.content.packager.METSManifest manifest) Determines what parent DSpace object is referenced in this METS doc.
org.dspace.content.DSpaceObject	ingest(org.dspace.core.Context context, org.dspace.content.DSpaceObject parent, File pkgFile, org.dspace.content.packager.PackageParameters params, String license) Create a new DSpace object out of a METS content package.
protected org.dspace.content.DSpaceObject	ingestObject(org.dspace.core.Context context, org.dspace.content.DSpaceObject parent, org.dspace.content.packager.METSManifest manifest, File pkgFile, org.dspace.content.packager.PackageParameters params, String license) Ingest/import a single DSpace Object, based on the associated METS Manifest and the parameters passed to the METSIngester
protected org.dspace.content.packager.METSManifest	parsePackage(org.dspace.core.Context context, File pkgFile, org.dspace.content.packager.PackageParameters params) Parse a given input package, ultimately returning the METS manifest out of the package.
protected boolean	preserveManifest()
protected void	removeObject(org.dspace.core.Context context, org.dspace.content.DSpaceObject dso) Remove an existing DSpace Object (called during a replace)
org.dspace.content.DSpaceObject	replace(org.dspace.core.Context context, org.dspace.content.DSpaceObject dsoToReplace, File pkgFile, org.dspace.content.packager.PackageParameters params) Replace an existing DSpace object with the contents of a METS-based package.
protected org.dspace.content.DSpaceObject	replaceObject(org.dspace.core.Context context, org.dspace.content.DSpaceObject dso, org.dspace.content.packager.METSManifest manifest, File pkgFile, org.dspace.content.packager.PackageParameters params, String license) Replace the contents of a single DSpace Object, based on the associated METS Manifest and the parameters passed to the METSIngester.
protected boolean	useCollectionTemplate()

Methods inherited from class org.dspace.content.packager.AbstractPackageIngester

addPackageReference, addToIngestedList, getIngestedList, getPackageReferences, ingestAll, replaceAll

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AbstractInternalMETSIngester

public AbstractInternalMETSIngester()

Method Detail

ingest

public org.dspace.content.DSpaceObject ingest(org.dspace.core.Context context,
                                     org.dspace.content.DSpaceObject parent,
                                     File pkgFile,
                                     org.dspace.content.packager.PackageParameters params,
                                     String license)
                                       throws org.dspace.content.packager.PackageValidationException,
                                              org.dspace.content.crosswalk.CrosswalkException,
                                              org.dspace.authorize.AuthorizeException,
                                              SQLException,
                                              IOException

Create a new DSpace object out of a METS content package. All contents are dictated by the METS manifest. Package is a ZIP archive (or optionally bare manifest XML document). In a Zip, all files relative to top level and the manifest (as per spec) in mets.xml.

Parameters:

context - DSpace context.

parent - parent under which to create new object (may be null -- in which case ingester must determine parent from package or throw an error).

pkgFile - The package file to ingest

params - Properties-style list of options (interpreted by each packager).

license - may be null, which takes default license.

Returns:

DSpaceObject created by ingest.

Throws:

PackageValidationException - if package is unacceptable or there is a fatal error turning it into a DSpaceObject.

org.dspace.content.crosswalk.CrosswalkException

org.dspace.authorize.AuthorizeException

SQLException

IOException

parsePackage

protected org.dspace.content.packager.METSManifest parsePackage(org.dspace.core.Context context,
                                                    File pkgFile,
                                                    org.dspace.content.packager.PackageParameters params)
                                                         throws IOException,
                                                                SQLException,
                                                                org.dspace.authorize.AuthorizeException,
                                                                org.dspace.content.crosswalk.MetadataValidationException

Parse a given input package, ultimately returning the METS manifest out of the package. METS manifest is assumed to be a file named 'mets.xml'

Parameters:

context - DSpace Context

pkgFile - package to parse

params - Ingestion parameters

Returns:

parsed out METSManifest

Throws:

IOException

SQLException

org.dspace.authorize.AuthorizeException

org.dspace.content.crosswalk.MetadataValidationException

ingestObject

protected org.dspace.content.DSpaceObject ingestObject(org.dspace.core.Context context,
                                           org.dspace.content.DSpaceObject parent,
                                           org.dspace.content.packager.METSManifest manifest,
                                           File pkgFile,
                                           org.dspace.content.packager.PackageParameters params,
                                           String license)
                                                throws IOException,
                                                       SQLException,
                                                       org.dspace.authorize.AuthorizeException,
                                                       org.dspace.content.crosswalk.CrosswalkException,
                                                       org.dspace.content.crosswalk.MetadataValidationException,
                                                       org.dspace.content.packager.PackageValidationException

Ingest/import a single DSpace Object, based on the associated METS Manifest and the parameters passed to the METSIngester

Parameters:

context - DSpace Context

parent - Parent DSpace Object

manifest - the parsed METS Manifest

pkgFile - the full package file (which may include content files if a zip)

params - Parameters passed to METSIngester

license - DSpace license agreement

Returns:

completed result as a DSpace object

Throws:

IOException

SQLException

org.dspace.authorize.AuthorizeException

org.dspace.content.crosswalk.CrosswalkException

org.dspace.content.crosswalk.MetadataValidationException

PackageValidationException

replaceObject

protected org.dspace.content.DSpaceObject replaceObject(org.dspace.core.Context context,
                                            org.dspace.content.DSpaceObject dso,
                                            org.dspace.content.packager.METSManifest manifest,
                                            File pkgFile,
                                            org.dspace.content.packager.PackageParameters params,
                                            String license)
                                                 throws IOException,
                                                        SQLException,
                                                        org.dspace.authorize.AuthorizeException,
                                                        org.dspace.content.crosswalk.CrosswalkException,
                                                        org.dspace.content.crosswalk.MetadataValidationException,
                                                        org.dspace.content.packager.PackageValidationException

Replace the contents of a single DSpace Object, based on the associated METS Manifest and the parameters passed to the METSIngester.

Parameters:

context - DSpace Context

dso - DSpace Object to replace

manifest - the parsed METS Manifest

pkgFile - the full package file (which may include content files if a zip)

params - Parameters passed to METSIngester

license - DSpace license agreement

Returns:

completed result as a DSpace object

Throws:

IOException

SQLException

org.dspace.authorize.AuthorizeException

org.dspace.content.crosswalk.CrosswalkException

org.dspace.content.crosswalk.MetadataValidationException

PackageValidationException

addBitstreams

protected void addBitstreams(org.dspace.core.Context context,
                 org.dspace.content.Item item,
                 org.dspace.content.packager.METSManifest manifest,
                 File pkgFile,
                 org.dspace.content.packager.PackageParameters params,
                 AbstractInternalMETSIngester.MdrefManager mdRefCallback)
                      throws SQLException,
                             IOException,
                             org.dspace.authorize.AuthorizeException,
                             org.dspace.content.crosswalk.MetadataValidationException,
                             org.dspace.content.crosswalk.CrosswalkException,
                             org.dspace.content.packager.PackageValidationException

Add Bitstreams to an Item, based on the files listed in the METS Manifest

Parameters:

context - DSpace Context

item - DSpace Item

manifest - METS Manifest

pkgFile - the full package file (which may include content files if a zip)

params - Ingestion Parameters

mdRefCallback - MdrefManager storing info about mdRefs in manifest

Throws:

SQLException

IOException

org.dspace.authorize.AuthorizeException

org.dspace.content.crosswalk.MetadataValidationException

org.dspace.content.crosswalk.CrosswalkException

PackageValidationException

addManifestBitstream

protected void addManifestBitstream(org.dspace.core.Context context,
                        org.dspace.content.Item item,
                        org.dspace.content.packager.METSManifest manifest)
                             throws IOException,
                                    SQLException,
                                    org.dspace.authorize.AuthorizeException,
                                    org.dspace.content.packager.PackageValidationException

Save/Preserve the METS Manifest as a Bitstream attached to the given DSpace item.

Parameters:

context - DSpace Context

item - DSpace Item

manifest - The METS Manifest

Throws:

SQLException

org.dspace.authorize.AuthorizeException

PackageValidationException

IOException

addContainerLogo

protected void addContainerLogo(org.dspace.core.Context context,
                    org.dspace.content.DSpaceObject dso,
                    org.dspace.content.packager.METSManifest manifest,
                    File pkgFile,
                    org.dspace.content.packager.PackageParameters params)
                         throws SQLException,
                                IOException,
                                org.dspace.authorize.AuthorizeException,
                                org.dspace.content.crosswalk.MetadataValidationException,
                                org.dspace.content.packager.PackageValidationException

Add a Logo to a Community or Collection container object based on a METS Manifest.

Parameters:

context - DSpace Context

dso - DSpace Container Object

manifest - METS Manifest

pkgFile - the full package file (which may include content files if a zip)

params - Ingestion Parameters

Throws:

SQLException

IOException

org.dspace.authorize.AuthorizeException

org.dspace.content.crosswalk.MetadataValidationException

PackageValidationException

addTemplateItem

protected void addTemplateItem(org.dspace.core.Context context,
                   org.dspace.content.DSpaceObject dso,
                   org.dspace.content.packager.METSManifest manifest,
                   File pkgFile,
                   org.dspace.content.packager.PackageParameters params,
                   AbstractInternalMETSIngester.MdrefManager callback)
                        throws SQLException,
                               IOException,
                               org.dspace.authorize.AuthorizeException,
                               org.dspace.content.crosswalk.CrosswalkException,
                               org.dspace.content.packager.PackageValidationException

Add a Template Item to a Collection container object based on a METS Manifest.

Parameters:

context - DSpace Context

dso - DSpace Container Object

manifest - METS Manifest

pkgFile - the full package file (which may include content files if a zip)

params - Ingestion Parameters

callback - the MdrefManager (manages all external metadata files referenced by METS mdref elements)

Throws:

SQLException

IOException

org.dspace.authorize.AuthorizeException

org.dspace.content.crosswalk.MetadataValidationException

PackageValidationException

org.dspace.content.crosswalk.CrosswalkException

replace

public org.dspace.content.DSpaceObject replace(org.dspace.core.Context context,
                                      org.dspace.content.DSpaceObject dsoToReplace,
                                      File pkgFile,
                                      org.dspace.content.packager.PackageParameters params)
                                        throws org.dspace.content.packager.PackageValidationException,
                                               org.dspace.content.crosswalk.CrosswalkException,
                                               org.dspace.authorize.AuthorizeException,
                                               SQLException,
                                               IOException

Replace an existing DSpace object with the contents of a METS-based package. All contents are dictated by the METS manifest. Package is a ZIP archive (or optionally bare manifest XML document). In a Zip, all files relative to top level and the manifest (as per spec) in mets.xml.

This method is similar to ingest(), except that if the object already exists in DSpace, it is emptied of files and metadata. The METS-based package is then used to ingest new values for these.

Parameters:

context - DSpace Context

dsoToReplace - DSpace Object to be replaced (may be null if it will be specified in the METS manifest itself)

pkgFile - The package file to ingest

params - Parameters passed from the packager script

Returns:

DSpaceObject created by ingest.

Throws:

PackageValidationException - if package is unacceptable or there is a fatal error turning it into a DSpace Object.

IOException

SQLException

org.dspace.authorize.AuthorizeException

org.dspace.content.crosswalk.CrosswalkException

preserveManifest

protected boolean preserveManifest()

getManifestBitstreamFormat

protected String getManifestBitstreamFormat()

useCollectionTemplate

protected boolean useCollectionTemplate()

decodeHandleURN

protected String decodeHandleURN(String value)

Parse the hdl: URI/URN format into a raw Handle.

Parameters:

value - handle URI string

Returns:

raw handle (with 'hdl:' prefix removed)

removeObject

protected void removeObject(org.dspace.core.Context context,
                org.dspace.content.DSpaceObject dso)
                     throws org.dspace.authorize.AuthorizeException,
                            SQLException,
                            IOException

Remove an existing DSpace Object (called during a replace)

Parameters:

dso - DSpace Object

Throws:

org.dspace.authorize.AuthorizeException

SQLException

IOException

getParentObject

public org.dspace.content.DSpaceObject getParentObject(org.dspace.core.Context context,
                                              org.dspace.content.packager.METSManifest manifest)
                                                throws org.dspace.content.packager.PackageValidationException,
                                                       org.dspace.content.crosswalk.MetadataValidationException,
                                                       SQLException

Determines what parent DSpace object is referenced in this METS doc.

This is a default implementation which assumes the parent will be specified in a <structMap LABEL="Parent">. You should override this method if your METS manifest specifies the parent object in another location.

Parameters:

context - DSpace Context

manifest - METS manifest

Returns:

a DSpace Object which is the parent (or null, if not found)

Throws:

PackageValidationException - if parent reference cannot be found in manifest

org.dspace.content.crosswalk.MetadataValidationException

SQLException

getObjectHandle

public String getObjectHandle(org.dspace.content.packager.METSManifest manifest)
                       throws org.dspace.content.packager.PackageValidationException,
                              org.dspace.content.crosswalk.MetadataValidationException,
                              SQLException

Determines the handle of the DSpace object represented in this METS doc.

This is a default implementation which assumes the handle of the DSpace Object can be found in the <mets> @OBJID attribute. You should override this method if your METS manifest specifies the handle in another location. If no handle was found then null is returned.

Parameters:

manifest - METS manifest

Returns:

handle as a string (or null, if not found)

Throws:

PackageValidationException - if handle cannot be found in manifest

org.dspace.content.crosswalk.MetadataValidationException

SQLException

getFileInputStream

protected InputStream getFileInputStream(File pkgFile,
                             org.dspace.content.packager.PackageParameters params,
                             String path,
                             org.dspace.core.Context context)
                                  throws org.dspace.content.crosswalk.MetadataValidationException,
                                         IOException

Retrieve the inputStream for a File referenced from a specific path within a METS package.

If the packager is set to 'manifest-only' (i.e. pkgFile is just a manifest), we assume the file is available for download via a URL.

Otherwise, the pkgFile is a Zip, so the file should be retrieved from within that Zip package.

Parameters:

pkgFile - the full package file (which may include content files if a zip)

params - Parameters passed to METSIngester

path - the File path (either path in Zip package or a URL)

Returns:

the InputStream for the file

Throws:

org.dspace.content.crosswalk.MetadataValidationException

IOException

getParameterHelp

public String getParameterHelp()

Returns a user help string which should describe the additional valid command-line options that this packager implementation will accept when using the -o or --option flags with the Packager script.

Returns:

a string describing additional command-line options available with this packager

crosswalkObjectDmd

public abstract void crosswalkObjectDmd(org.dspace.core.Context context,
                      org.dspace.content.DSpaceObject dso,
                      org.dspace.content.packager.METSManifest manifest,
                      AbstractInternalMETSIngester.MdrefManager callback,
                      org.jdom.Element[] dmds,
                      org.dspace.content.packager.PackageParameters params)
                                 throws org.dspace.content.crosswalk.CrosswalkException,
                                        org.dspace.content.packager.PackageValidationException,
                                        org.dspace.authorize.AuthorizeException,
                                        SQLException,
                                        IOException

Select the dmdSec element(s) to apply to the Item. The implementation is responsible for choosing which (if any) of the metadata sections to crosswalk to get the descriptive metadata for the item being ingested. It is responsible for calling the crosswalk, using the manifest's helper i.e. manifest.crosswalkItemDmd(context,item,dmdElement,callback); (The callback argument is a reference to itself since the class also implements the METSManifest.MdRef interface to fetch package files referenced by mdRef elements.)

Note that item and manifest are available as protected fields from the superclass.

Parameters:

context - the DSpace context

manifest - the METSManifest

callback - the MdrefManager (manages all external metadata files referenced by METS mdref elements)

dmds - array of Elements, each a METS dmdSec that applies to the Item as a whole.

params -

Throws:

org.dspace.content.crosswalk.CrosswalkException

org.dspace.content.packager.PackageValidationException

org.dspace.authorize.AuthorizeException

SQLException

IOException

addLicense

public abstract void addLicense(org.dspace.core.Context context,
              org.dspace.content.Item item,
              String license,
              org.dspace.content.Collection collection,
              org.dspace.content.packager.PackageParameters params)
                         throws org.dspace.content.packager.PackageValidationException,
                                org.dspace.authorize.AuthorizeException,
                                SQLException,
                                IOException

Add license(s) to Item based on contents of METS and other policies. The implementation of this method controls exactly what licenses are added to the new item, including the DSpace deposit license. It is given the collection (which is the source of a default deposit license), an optional user-supplied deposit license (in the form of a String), and the METS manifest. It should invoke manifest.getItemRightsMD() to get an array of rightsMd elements which might contain other license information of interest, e.g. a Creative Commons license.

This framework does not add any licenses by default.

Note that crosswalking rightsMD sections can also add a deposit or CC license to the object.

Parameters:

context - the DSpace context

collection - DSpace Collection to which the item is being submitted.

license - optional user-supplied Deposit License text (may be null)

Throws:

org.dspace.content.packager.PackageValidationException

org.dspace.authorize.AuthorizeException

SQLException

IOException

finishObject

public abstract void finishObject(org.dspace.core.Context context,
                org.dspace.content.DSpaceObject dso,
                org.dspace.content.packager.PackageParameters params)
                           throws org.dspace.content.packager.PackageValidationException,
                                  org.dspace.content.crosswalk.CrosswalkException,
                                  org.dspace.authorize.AuthorizeException,
                                  SQLException,
                                  IOException

Hook for final "finishing" operations on the new Object. This method is called when the new Object is otherwise complete and ready to be returned. The implementation should use this opportunity to make whatever final checks and modifications are necessary.

Parameters:

context - the DSpace context

dso - the DSpace Object

params - the Packager Parameters

Throws:

org.dspace.content.packager.PackageValidationException

org.dspace.content.crosswalk.CrosswalkException

org.dspace.authorize.AuthorizeException

SQLException

IOException

getObjectType

public abstract int getObjectType(org.dspace.content.packager.METSManifest manifest)
                           throws org.dspace.content.packager.PackageValidationException

Determines what type of DSpace object is represented in this METS doc.

Returns:

one of the object types in Constants.

Throws:

org.dspace.content.packager.PackageValidationException

finishBitstream

public abstract void finishBitstream(org.dspace.core.Context context,
                   org.dspace.content.Bitstream bs,
                   org.jdom.Element mfile,
                   org.dspace.content.packager.METSManifest manifest,
                   org.dspace.content.packager.PackageParameters params)
                              throws org.dspace.content.crosswalk.MetadataValidationException,
                                     SQLException,
                                     org.dspace.authorize.AuthorizeException,
                                     IOException

Subclass-dependent final processing on a Bitstream; could include fixing up the name, bundle, other attributes.

Throws:

org.dspace.content.crosswalk.MetadataValidationException

SQLException

org.dspace.authorize.AuthorizeException

IOException

getConfigurationName

public abstract String getConfigurationName()

Returns keyword that makes the configuration keys of this subclass unique, e.g. if it returns NAME, the key would be: "mets.NAME.ingest.preserveManifest = true"