Data Rescue
I would like to ...
If you believe that the data you are working with is at risk of being lost, the Checklist for USA federal data backups from MIT is an excellent starting point. You can also reach out to data@library.columbia.edu for a research data consultation related to a specific dataset.
To learn about different data rescue projects for public government data, look to the Data Rescue Project (DRP). It is a resource growing from coordination among data organizations such as IASSIST and RDAP. Their Data Rescue Tracker provides information on specific government datasets, including the responsible federal agency, original URL, rescue status, and the filetype, maintaining organization, and download location of the rescued data. To contribute to an active project or find collaborators for a new project, visit the Data Rescue Project or read their FAQ.
To stay informed about these efforts and to get involved, you can sign up for the Data Rescue Project email list. Additionally, this Columbia Libraries' Data Rescue blog post provides more context and information.
Since 2008, the Internet Archive (IA) has worked with partner organizations to create an End of Term Web Archive by crawling federal government websites–including some datasets–in the .gov, .mil, and other domains. The archives are created at the end of each presidential administration. You can search the 2024 End of Term Web Archive in IA’s Wayback Machine.
For researchers who need to preserve data that they have created or gathered - including for projects that have been grant funded - Research Data at Columbia is an excellent starting point.
Academic Commons is an appropriate platform for public data sets that are 10GB or smaller. Data Dryad, which the Libraries subscribes to, is appropriate for public data sets that are 300GB or smaller. For questions related to Academic Commons, please contact ac@columbia.edu.
We realize that federal data access plays a fundamental role in advancing research at Columbia University.
We encourage researchers to contact data@library.columbia.edu for advice on appropriate data sources or if you are unable to access data needed for research and/or teaching.
Columbia Libraries is focused on the data needs as well as the preservation of research data created by the Columbia community.