Package org.dspace.content.packager

Class AbstractMETSIngester

java.lang.Object

org.dspace.content.packager.AbstractPackageIngester

org.dspace.content.packager.AbstractMETSIngester

All Implemented Interfaces:

PackageIngester

Direct Known Subclasses:

DSpaceAIPIngester, DSpaceMETSIngester

public abstract class AbstractMETSIngester
extends AbstractPackageIngester

Base class for package ingester of METS (Metadata Encoding and Transmission Standard) Packages.
See http://www.loc.gov/standards/mets/.

This is a generic packager framework intended to be subclassed to create ingesters for more specific METS "profiles". METS is an abstract and flexible framework that can encompass many different kinds of metadata and inner package structures.

Package Parameters:

• validate -- true/false attempt to schema-validate the METS manifest.
• manifestOnly -- package consists only of a manifest document.
• ignoreHandle -- true/false, ignore AIP's idea of handle when ingesting.
• ignoreParent -- true/false, ignore AIP's idea of parent when ingesting.

Configuration Properties:

• mets.CONFIGNAME.ingest.preserveManifest - if true, the METS manifest itself is preserved in a bitstream named mets.xml in the METADATA bundle. If it is false (the default), the manifest is discarded after ingestion.
• mets.CONFIGNAME.ingest.manifestBitstreamFormat - short name of the bitstream format to apply to the manifest; MUST be specified when preserveManifest is true.
• mets.default.ingest.crosswalk.MD_SEC_NAME = PLUGIN_NAME Establishes a default crosswalk plugin for the given type of metadata in a METS mdSec (e.g. "DC", "MODS"). The plugin may be either a stream or XML-oriented ingestion crosswalk. Subclasses can override the default mapping with their own, substituting their configurationName for "default" in the configuration property key above.
• mets.CONFIGNAME.ingest.useCollectionTemplate - if true, when an item is created, use the collection template. If it is false (the default), any existing collection template is ignored.

Author:

Larry Stone, Tim Donohue

See Also:

• METSManifest
• AbstractPackageIngester
• PackageIngester

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static final class	AbstractMETSIngester.MdrefManager	An instance of ZipMdrefManager holds the state needed to retrieve the contents of an external metadata stream referenced by an mdRef element in a Zipped up METS manifest.

Field Summary

Fields

Modifier and Type	Field	Description
protected final BitstreamFormatService	bitstreamFormatService
protected final BitstreamService	bitstreamService
protected final BundleService	bundleService
protected final CollectionService	collectionService
protected final CommunityService	communityService
protected final ConfigurationService	configurationService
protected final HandleService	handleService
protected final ItemService	itemService
protected final WorkspaceItemService	workspaceItemService

Constructor Summary

Constructors

Constructor	Description
AbstractMETSIngester()

Method Summary

Modifier and Type	Method	Description
protected void	addBitstreams(Context context, Item item, METSManifest manifest, File pkgFile, PackageParameters params, AbstractMETSIngester.MdrefManager mdRefCallback)	Add Bitstreams to an Item, based on the files listed in the METS Manifest
protected void	addContainerLogo(Context context, DSpaceObject dso, METSManifest manifest, File pkgFile, PackageParameters params)	Add a Logo to a Community or Collection container object based on a METS Manifest.
abstract void	addLicense(Context context, Item item, String license, Collection collection, PackageParameters params)	Add license(s) to Item based on contents of METS and other policies.
protected void	addManifestBitstream(Context context, Item item, METSManifest manifest)	Save/Preserve the METS Manifest as a Bitstream attached to the given DSpace item.
protected void	addTemplateItem(Context context, DSpaceObject dso, METSManifest manifest, File pkgFile, PackageParameters params, AbstractMETSIngester.MdrefManager callback)	Add a Template Item to a Collection container object based on a METS Manifest.
abstract void	crosswalkObjectDmd(Context context, DSpaceObject dso, METSManifest manifest, AbstractMETSIngester.MdrefManager callback, org.jdom2.Element[] dmds, PackageParameters params)	Select the dmdSec element(s) to apply to the Item.
protected String	decodeHandleURN(String value)	Parse the hdl: URI/URN format into a raw Handle.
abstract void	finishBitstream(Context context, Bitstream bs, org.jdom2.Element mfile, METSManifest manifest, PackageParameters params)	Subclass-dependent final processing on a Bitstream; could include fixing up the name, bundle, other attributes.
abstract void	finishObject(Context context, DSpaceObject dso, PackageParameters params)	Hook for final "finishing" operations on the new Object.
abstract String	getConfigurationName()	Returns keyword that makes the configuration keys of this subclass unique, e.g. if it returns NAME, the key would be: "mets.NAME.ingest.preserveManifest = true"
protected static InputStream	getFileInputStream(File pkgFile, PackageParameters params, String path)	Retrieve the inputStream for a File referenced from a specific path within a METS package.
protected String	getManifestBitstreamFormat()
String	getObjectHandle(METSManifest manifest)	Determines the handle of the DSpace object represented in this METS doc.
UUID	getObjectID(METSManifest manifest)
abstract int	getObjectType(METSManifest manifest)	Determines what type of DSpace object is represented in this METS doc.
String	getParameterHelp()	Returns a user help string which should describe the additional valid command-line options that this packager implementation will accept when using the -o or --option flags with the Packager script.
DSpaceObject	getParentObject(Context context, METSManifest manifest)	Determines what parent DSpace object is referenced in this METS doc.
DSpaceObject	ingest(Context context, DSpaceObject parent, File pkgFile, PackageParameters params, String license)	Create a new DSpace object out of a METS content package.
protected DSpaceObject	ingestObject(Context context, DSpaceObject parent, METSManifest manifest, File pkgFile, PackageParameters params, String license)	Ingest/import a single DSpace Object, based on the associated METS Manifest and the parameters passed to the METSIngester
protected METSManifest	parsePackage(Context context, File pkgFile, PackageParameters params)	Parse a given input package, ultimately returning the METS manifest out of the package.
protected boolean	preserveManifest()
protected void	removeObject(Context context, DSpaceObject dso)	Remove an existing DSpace Object (called during a replace)
DSpaceObject	replace(Context context, DSpaceObject dsoToReplace, File pkgFile, PackageParameters params)	Replace an existing DSpace object with the contents of a METS-based package.
protected DSpaceObject	replaceObject(Context context, DSpaceObject dso, METSManifest manifest, File pkgFile, PackageParameters params, String license)	Replace the contents of a single DSpace Object, based on the associated METS Manifest and the parameters passed to the METSIngester.
protected boolean	useCollectionTemplate()

Methods inherited from class org.dspace.content.packager.AbstractPackageIngester

addPackageReference, addToIngestedMap, getIngestedList, getIngestedMap, getPackageReferences, ingestAll, replaceAll

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

bitstreamService

protected final BitstreamService bitstreamService

bitstreamFormatService

protected final BitstreamFormatService bitstreamFormatService

bundleService

protected final BundleService bundleService

communityService

protected final CommunityService communityService

collectionService

protected final CollectionService collectionService

itemService

protected final ItemService itemService

handleService

protected final HandleService handleService

workspaceItemService

protected final WorkspaceItemService workspaceItemService

configurationService

protected final ConfigurationService configurationService

Constructor Details

AbstractMETSIngester

public AbstractMETSIngester()

Method Details

ingest

public DSpaceObject ingest(Context context,
 DSpaceObject parent,
 File pkgFile,
 PackageParameters params,
 String license)
                    throws PackageValidationException,
CrosswalkException,
AuthorizeException,
SQLException,
IOException,
WorkflowException

Create a new DSpace object out of a METS content package. All contents are dictated by the METS manifest. Package is a ZIP archive (or optionally bare manifest XML document). In a Zip, all files relative to top level and the manifest (as per spec) in mets.xml.

Parameters:

context - DSpace context.

parent - parent under which to create new object (may be null -- in which case ingester must determine parent from package or throw an error).

pkgFile - The package file to ingest

params - Properties-style list of options (interpreted by each packager).

license - may be null, which takes default license.

Returns:

DSpaceObject created by ingest.

Throws:

PackageValidationException - if package validation error if package is unacceptable or there is a fatal error turning it into a DSpaceObject.

CrosswalkException - if crosswalk error

AuthorizeException - if authorization error

SQLException - if database error

IOException - if IO error

WorkflowException - if workflow error

parsePackage

protected METSManifest parsePackage(Context context,
 File pkgFile,
 PackageParameters params)
                             throws IOException,
SQLException,
AuthorizeException,
MetadataValidationException

Parse a given input package, ultimately returning the METS manifest out of the package. METS manifest is assumed to be a file named 'mets.xml'

Parameters:

context - DSpace Context

pkgFile - package to parse

params - Ingestion parameters

Returns:

parsed out METSManifest

Throws:

IOException - if IO error

SQLException - if database error

AuthorizeException - if authorization error

MetadataValidationException - if metadata validation error

ingestObject

protected DSpaceObject ingestObject(Context context,
 DSpaceObject parent,
 METSManifest manifest,
 File pkgFile,
 PackageParameters params,
 String license)
                             throws IOException,
SQLException,
AuthorizeException,
CrosswalkException,
PackageValidationException,
WorkflowException

Ingest/import a single DSpace Object, based on the associated METS Manifest and the parameters passed to the METSIngester

Parameters:

context - DSpace Context

parent - Parent DSpace Object

manifest - the parsed METS Manifest

pkgFile - the full package file (which may include content files if a zip)

params - Parameters passed to METSIngester

license - DSpace license agreement

Returns:

completed result as a DSpace object

Throws:

IOException - if IO error

SQLException - if database error

AuthorizeException - if authorization error

CrosswalkException - if crosswalk error

MetadataValidationException - if metadata validation error

WorkflowException - if workflow error

PackageValidationException - if package validation error

replaceObject

protected DSpaceObject replaceObject(Context context,
 DSpaceObject dso,
 METSManifest manifest,
 File pkgFile,
 PackageParameters params,
 String license)
                              throws IOException,
SQLException,
AuthorizeException,
CrosswalkException,
MetadataValidationException,
PackageValidationException

Replace the contents of a single DSpace Object, based on the associated METS Manifest and the parameters passed to the METSIngester.

Parameters:

context - DSpace Context

dso - DSpace Object to replace

manifest - the parsed METS Manifest

pkgFile - the full package file (which may include content files if a zip)

params - Parameters passed to METSIngester

license - DSpace license agreement

Returns:

completed result as a DSpace object

Throws:

IOException - if IO error

SQLException - if database error

AuthorizeException - if authorization error

CrosswalkException - if crosswalk error

MetadataValidationException - if metadata validation error

PackageValidationException - if package validation error

addBitstreams

protected void addBitstreams(Context context,
 Item item,
 METSManifest manifest,
 File pkgFile,
 PackageParameters params,
 AbstractMETSIngester.MdrefManager mdRefCallback)
                      throws SQLException,
IOException,
AuthorizeException,
MetadataValidationException,
CrosswalkException,
PackageValidationException

Add Bitstreams to an Item, based on the files listed in the METS Manifest

Parameters:

context - DSpace Context

item - DSpace Item

manifest - METS Manifest

pkgFile - the full package file (which may include content files if a zip)

params - Ingestion Parameters

mdRefCallback - MdrefManager storing info about mdRefs in manifest

Throws:

SQLException - if database error

IOException - if IO error

AuthorizeException - if authorization error

MetadataValidationException - if metadata validation error

CrosswalkException - if crosswalk error

PackageValidationException - if package validation error

addManifestBitstream

protected void addManifestBitstream(Context context,
 Item item,
 METSManifest manifest)
                             throws IOException,
SQLException,
AuthorizeException,
PackageValidationException

Save/Preserve the METS Manifest as a Bitstream attached to the given DSpace item.

Parameters:

context - DSpace Context

item - DSpace Item

manifest - The METS Manifest

Throws:

IOException - if IO error

SQLException - if database error

AuthorizeException - if authorization error

PackageValidationException - if package validation error

addContainerLogo

protected void addContainerLogo(Context context,
 DSpaceObject dso,
 METSManifest manifest,
 File pkgFile,
 PackageParameters params)
                         throws SQLException,
IOException,
AuthorizeException,
MetadataValidationException,
PackageValidationException

Add a Logo to a Community or Collection container object based on a METS Manifest.

Parameters:

context - DSpace Context

dso - DSpace Container Object

manifest - METS Manifest

pkgFile - the full package file (which may include content files if a zip)

params - Ingestion Parameters

Throws:

SQLException - if database error

IOException - if IO error

AuthorizeException - if authorization error

MetadataValidationException - if metadata validation error

PackageValidationException - if package validation error

addTemplateItem

protected void addTemplateItem(Context context,
 DSpaceObject dso,
 METSManifest manifest,
 File pkgFile,
 PackageParameters params,
 AbstractMETSIngester.MdrefManager callback)
                        throws SQLException,
IOException,
AuthorizeException,
CrosswalkException,
PackageValidationException

Add a Template Item to a Collection container object based on a METS Manifest.

Parameters:

context - DSpace Context

dso - DSpace Container Object

manifest - METS Manifest

pkgFile - the full package file (which may include content files if a zip)

params - Ingestion Parameters

callback - the MdrefManager (manages all external metadata files referenced by METS mdref elements)

Throws:

SQLException - if database error

IOException - if IO error

AuthorizeException - if authorization error

MetadataValidationException - if metadata validation error

PackageValidationException - if package validation error

CrosswalkException

replace

public DSpaceObject replace(Context context,
 DSpaceObject dsoToReplace,
 File pkgFile,
 PackageParameters params)
                     throws PackageValidationException,
CrosswalkException,
AuthorizeException,
SQLException,
IOException,
WorkflowException

Replace an existing DSpace object with the contents of a METS-based package. All contents are dictated by the METS manifest. Package is a ZIP archive (or optionally bare manifest XML document). In a Zip, all files relative to top level and the manifest (as per spec) in mets.xml.

This method is similar to ingest(), except that if the object already exists in DSpace, it is emptied of files and metadata. The METS-based package is then used to ingest new values for these.

Parameters:

context - DSpace Context

dsoToReplace - DSpace Object to be replaced (may be null if it will be specified in the METS manifest itself)

pkgFile - The package file to ingest

params - Parameters passed from the packager script

Returns:

DSpaceObject created by ingest.

Throws:

PackageValidationException - if package validation error if package is unacceptable or there is a fatal error turning it into a DSpace Object.

IOException - if IO error

SQLException - if database error

AuthorizeException - if authorization error

CrosswalkException - if crosswalk error

WorkflowException - if workflow error

preserveManifest

protected boolean preserveManifest()

getManifestBitstreamFormat

protected String getManifestBitstreamFormat()

useCollectionTemplate

protected boolean useCollectionTemplate()

decodeHandleURN

protected String decodeHandleURN(String value)

Parse the hdl: URI/URN format into a raw Handle.

Parameters:

value - handle URI string

Returns:

raw handle (with 'hdl:' prefix removed)

removeObject

protected void removeObject(Context context,
 DSpaceObject dso)
                     throws AuthorizeException,
SQLException,
IOException

Remove an existing DSpace Object (called during a replace)

Parameters:

context - context

dso - DSpace Object

Throws:

IOException - if IO error

SQLException - if database error

AuthorizeException - if authorization error

getParentObject

public DSpaceObject getParentObject(Context context,
 METSManifest manifest)
                             throws PackageValidationException,
MetadataValidationException,
SQLException

Determines what parent DSpace object is referenced in this METS doc.

This is a default implementation which assumes the parent will be specified in a <structMap LABEL="Parent">. You should override this method if your METS manifest specifies the parent object in another location.

Parameters:

context - DSpace Context

manifest - METS manifest

Returns:

a DSpace Object which is the parent (or null, if not found)

Throws:

PackageValidationException - if package validation error if parent reference cannot be found in manifest

MetadataValidationException - if metadata validation error

SQLException - if database error

getObjectHandle

public String getObjectHandle(METSManifest manifest)
                       throws PackageValidationException,
MetadataValidationException,
SQLException

Determines the handle of the DSpace object represented in this METS doc.

This is a default implementation which assumes the handle of the DSpace Object can be found in the <mets> @OBJID attribute. You should override this method if your METS manifest specifies the handle in another location. If no handle was found then null is returned.

Parameters:

manifest - METS manifest

Returns:

handle as a string (or null, if not found)

Throws:

PackageValidationException - if package validation error if handle cannot be found in manifest

MetadataValidationException - if validation error

SQLException - if database error

getFileInputStream

protected static InputStream getFileInputStream(File pkgFile,
 PackageParameters params,
 String path)
                                         throws MetadataValidationException,
IOException

Retrieve the inputStream for a File referenced from a specific path within a METS package.

If the packager is set to 'manifest-only' (i.e. pkgFile is just a manifest), we assume the file is available for download via a URL.

Otherwise, the pkgFile is a Zip, so the file should be retrieved from within that Zip package.

Parameters:

pkgFile - the full package file (which may include content files if a zip)

params - Parameters passed to METSIngester

path - the File path (either path in Zip package or a URL)

Returns:

the InputStream for the file

Throws:

MetadataValidationException - if validation error

IOException - if IO error

getParameterHelp

public String getParameterHelp()

Returns a user help string which should describe the additional valid command-line options that this packager implementation will accept when using the -o or --option flags with the Packager script.

Returns:

a string describing additional command-line options available with this packager

crosswalkObjectDmd

public abstract void crosswalkObjectDmd(Context context,
 DSpaceObject dso,
 METSManifest manifest,
 AbstractMETSIngester.MdrefManager callback,
 org.jdom2.Element[] dmds,
 PackageParameters params)
                                 throws CrosswalkException,
PackageValidationException,
AuthorizeException,
SQLException,
IOException

Select the dmdSec element(s) to apply to the Item. The implementation is responsible for choosing which (if any) of the metadata sections to crosswalk to get the descriptive metadata for the item being ingested. It is responsible for calling the crosswalk, using the manifest's helper i.e. manifest.crosswalkItemDmd(context,item,dmdElement,callback); (The callback argument is a reference to itself since the class also implements the METSManifest.MdRef interface to fetch package files referenced by mdRef elements.)

Note that item and manifest are available as protected fields from the superclass.

Parameters:

context - the DSpace context

dso - DSpace Object

manifest - the METSManifest

callback - the MdrefManager (manages all external metadata files referenced by METS mdref elements)

dmds - array of Elements, each a METS dmdSec that applies to the Item as a whole.

params - Packager Parameters

Throws:

CrosswalkException - if crosswalk error

PackageValidationException - if package validation error

IOException - if IO error

SQLException - if database error

AuthorizeException - if authorization error

addLicense

public abstract void addLicense(Context context,
 Item item,
 String license,
 Collection collection,
 PackageParameters params)
                         throws PackageValidationException,
AuthorizeException,
SQLException,
IOException

Add license(s) to Item based on contents of METS and other policies. The implementation of this method controls exactly what licenses are added to the new item, including the DSpace deposit license. It is given the collection (which is the source of a default deposit license), an optional user-supplied deposit license (in the form of a String), and the METS manifest. It should invoke manifest.getItemRightsMD() to get an array of rightsMd elements which might contain other license information of interest, e.g. a Creative Commons license.

This framework does not add any licenses by default.

Note that crosswalking rightsMD sections can also add a deposit or CC license to the object.

Parameters:

context - the DSpace context

item - Item

collection - DSpace Collection to which the item is being submitted.

license - optional user-supplied Deposit License text (may be null)

params - Packager Parameters

Throws:

PackageValidationException - if package validation error

IOException - if IO error

SQLException - if database error

AuthorizeException - if authorization error

finishObject

public abstract void finishObject(Context context,
 DSpaceObject dso,
 PackageParameters params)
                           throws PackageValidationException,
CrosswalkException,
AuthorizeException,
SQLException,
IOException

Hook for final "finishing" operations on the new Object. This method is called when the new Object is otherwise complete and ready to be returned. The implementation should use this opportunity to make whatever final checks and modifications are necessary.

Parameters:

context - the DSpace context

dso - the DSpace Object

params - the Packager Parameters

Throws:

CrosswalkException - if crosswalk error

PackageValidationException - if package validation error

IOException - if IO error

SQLException - if database error

AuthorizeException - if authorization error

getObjectType

public abstract int getObjectType(METSManifest manifest)
                           throws PackageValidationException

Determines what type of DSpace object is represented in this METS doc.

Parameters:

manifest - METS manifest

Returns:

one of the object types in Constants.

Throws:

PackageValidationException - if package validation error

finishBitstream

public abstract void finishBitstream(Context context,
 Bitstream bs,
 org.jdom2.Element mfile,
 METSManifest manifest,
 PackageParameters params)
                              throws MetadataValidationException,
SQLException,
AuthorizeException,
IOException

Subclass-dependent final processing on a Bitstream; could include fixing up the name, bundle, other attributes.

Parameters:

context - context

manifest - METS manifest

bs - bitstream

mfile - element

params - package params

Throws:

MetadataValidationException - if validation error

IOException - if IO error

SQLException - if database error

AuthorizeException - if authorization error

getConfigurationName

public abstract String getConfigurationName()

Returns keyword that makes the configuration keys of this subclass unique, e.g. if it returns NAME, the key would be: "mets.NAME.ingest.preserveManifest = true"

Returns:

name

getObjectID

public UUID getObjectID(METSManifest manifest)
                 throws PackageValidationException

Throws:

PackageValidationException