CUL - Main Content

According to the Columbia University Libraries’ Mission Statement:

The Columbia University Libraries provides Columbia faculty, students, and staff with access to information in all subject areas related to the University’s academic mission and its goals. The Libraries embraces its time-honored obligations of collecting, preserving, and providing access to collections, not only for the Columbia community but also for scholars and students from throughout the world who require access to the Libraries’ unique materials.

Policy

Digital resources are part of the CUL collections and subject to the same criteria for selection and retention decisions as other media. As such, they are included under the central CUL preservation policy: ensuring that the collections remain available over the long term through prevention of damage and deterioration; reversing damage where possible; and, when necessary, changing the format of materials to preserve their intellectual content.

As with other parts of the collections, decisions about preservation are made by selectors, curators, and bibliographers as experts on the value of the content, in consultation with the relevant technical experts, including the Digital Library and Scholarly Technologies, the Library Systems Office, and the Preservation and Digital Conversion Division. Priorities for preservation action are based on this Policy, the CUL Strategic Plan, and available budgetary support. When possible, decisions about the need for long-term retention should be made at the time of creation, acquisition, or licensing of digital resources.

For digital resources that are deemed to be of long-term value, preservation can be defined as the actions needed to assure enduring access to the full content of those resources over time. Content has wider implications than simply assuring that a given file can be accessed. For example, hierarchical and structural relationships among the files (such as the pages of a book) and metadata that make the files usable must be preserved as well as the files themselves.

Digital resources may exist in multiple versions. CUL is committed to preserving the archival version: the fullest, highest-quality available version of the resource, whenever possible; and the descriptive, structural, and administrative metadata associated with it.

Scope of preservation responsibility
CUL recognizes three levels of responsibility for retention and management of digital resources.

1. CUL-held digital resources. CUL has primary responsibility for preservation of:
   1. Digital resources created by CUL and deemed to be of long-term value.
   2. Digital versions of resources reformatted by CUL and deemed to be of long-term value.
   3. Unique digital resources which are acquired by CUL (through donation or purchase) as parts of archival/manuscript collections and which are unlikely to be preserved anywhere else.
   4. Digital records (e.g. bibliographic records, personnel records) essential to CUL’s functioning not preserved through any other arm of the university.
2. Commercially available digital resources. CUL has responsibility for working externally through consortial action, licensing agreements, etc. to assure that someone (possibly but not necessarily CUL) carries out preservation so that CU faculty, staff, and students will have adequate ongoing access to these resources. Particular emphasis should be given to resources which exist in digital form only.
3. Administrative, scholarly, and other digital resources created at the University outside of the Libraries. CUL has responsibility for informing, consulting, and as appropriate coordinating with other units of Columbia University to assure that CU faculty, staff, and students will have adequate ongoing access to these resources. Included here are resources considered to be part of the digital library (EPIC ventures, others) and of long-term value.

Frequency with which preservation/retention policy for digital materials will be updated

This policy will be reviewed by at the beginning of each academic year to assure timely updates as technology and experience mature, or more often if need arises.

CUL is committed to lifecycle management of its digital resources. Guidelines and procedures for each stage have been or are being developed, and are reviewed as technology changes or other need arises. CUL will participate actively where appropriate in research, development, and implementation of new practices for preservation of digital resources.

• Development of preservation strategies, including consideration of:
  • Degree of integration with storage, backup, and preservation for non-digital library resources.
  • Development and use of decision-making tools (e.g., risk analysis, usage monitoring, probability of loss calculations, cost models, etc.).
  • Maintenance strategies (backups - online and/or offline, monitoring, refreshing, redundancy through mirror sites or caching, etc.).
  • Survival strategies (migration, emulation, archeology, etc.).
  • Reliance on outside consulting and archiving services, if any,contract negotiation, etc.
• Selection for long-term retention at time of digitization, acquisition, or licensing; and later re-selection for retention if this decision was not made initially:
  • Primary criteria: based on institutional mission, needs, priorities, and reasons for creating or acquiring the resources (related to long-term institutional mission and linked to conversion guidelines).
  • Secondary criteria: based on regional, national, consortial, and international responsibilities.
• Conversion guidelines:
  • CUL follows relevant standards where they have been established and best practice for digital conversion, as documented by Library of Congress, NARA, the Digital Library Federation (DLF), and other relevant bodies.
• Metadata creation and management (unique IDs and other descriptive, structural and administrative metadata, including ownership/rights management):
  • CUL follows relevant standards where they have been established and best practice for metadata creation and management, as documented by Library of Congress, NARA, DLF, and other relevant bodies.
• Storage (online, offline, redundancy recommendations, etc.):
  • CUL follows relevant standards where they have been established and best practice for storage, as documented by Library of Congress, NARA, DLF, and other relevant bodies.
  • The archival version of the digital resource is preserved in a lossless, non-proprietary format, whenever possible.
  • Storage plan for primary responsibility materials:
    1. Resources currently in use: kept online with regular backup, refreshment, and migration.
    2. Whether online or not: all archival versions (highest resolution, fullest capture, lossless compression) are written to approved storage media and stored off-line in the Digital Library and Scholarly Technologies, with a schedule for regular refreshment, and migration.
    3. For archival versions which are not currently online: a duplicate off-line copy is created for storage at a different site.
    4. All versions, online and offline, are tracked through the CUL local asset management system.
• Access arrangements:
  • CUL follows relevant standards where they have been established and best practice for access arrangements, appropriate rights and permissions requirements, and administrative concerns.

As stated in its Strategic Plan, CUL is committed to preservation of digital content, including:

• Technical infrastructure (equipment purchases, maintenance and upgrades, software/hardware obsolescence monitoring, network connectivity, etc.).
• Financial planning (strategy and methods for financing the digital preservation program, commitment to long-term funding).
• Staffing infrastructure (including hiring and ongoing training).
• Rights management.

In some cases it may be proposed that digital resources created or acquired by CUL and deemed to be of long-term value should be preserved by another institution or organization. CUL will evaluate the proposed capacity to provide long-term access to digital files, metadata, and functionality in order to assure that the resources will be preserved at least as well as they would be by CUL. Issues which must be taken into account include:

• Shared obligations and cooperative preservation.
• Information sharing.
• Technology support (e.g., reuse of same migration paths and strategies by others; following emerging common/recommended practice).
• Development of minimal standards for capture, management, and maintenance (to enable efficient strategies).