FAQ for Website Owners
What is the Ivy Plus Libraries Confederation?
The Ivy Plus Libraries Confederation is a partnership between thirteen leading academic research libraries ― Brown University, the University of Chicago, Columbia University, Cornell University, Dartmouth College, Duke University, Harvard University, Johns Hopkins University, the Massachusetts Institute of Technology, the University of Pennsylvania, Princeton University, Stanford University and Yale University ― that collectively provide access to a rich and unique record of human thought and creativity through resource sharing and collaboration. The Ivy Plus Libraries Confederation Web Collecting Program is a collaborative collection development effort to build curated, thematic collections of freely available, but at-risk, web content in order to support research at participating Libraries and beyond.
What is web archiving?
Web archiving is the process of creating and storing copies (or snapshots) of live websites to ensure that they are preserved and remain publicly accessible for future use. Sites are typically captured using automated web crawlers, which systematically identify and follow hyperlinks on selected websites (referred to as seeds), copying and storing information as they work. The Ivy Plus Libraries Confederation uses the Archive-It service to crawl websites: its crawler is called archive.org_bot.
Will the web crawler interrupt access to my website?
No, it shouldn’t. Ivy Plus Libraries crawls websites at a polite rate so as not to interfere with access to your site. Crawls are typically run semi-annually, operate in the background, and last for at most a few days. Once a crawl is complete, the crawler no longer interacts with your site.
What is your copyright policy?
The Ivy Plus Libraries Confederation attempts to notify all organizations and/or website owners before crawling selected sites. Some websites may contain material that is produced by other parties who may claim copyright ownership of such materials. Third-party copyright holders who believe their rights have been infringed upon may contact us at ivypluslibweb@library.columbia.edu.
What if I do not want my website to be included?
We will honor requests to remove access to archived content. Please contact us at ivypluslibweb@library.columbia.edu.
Where can I access the archived versions of my website?
All websites collected by the Ivy Plus Libraries Confederation can be freely and permanently accessed via Archive-It.
Why do the archived versions of my website look different than my live website?
Though the Ivy Plus Libraries Confederation attempts to create complete copies of each website we collect, web crawlers can struggle to capture portions of the dynamic web. Examples of challenging or impossible-to-capture content include: JavaScript-driven navigation menus, streaming audio and video, and database-driven content. In addition, web crawlers cannot collect password-protected portions of your site: only publicly available content will be crawled. Site owners interested in optimizing their site design to allow full archiving may be interested in these Guidelines for Preservable Websites.