Website Owner FAQ
DO WEBSITE OWNERS HAVE TO CHANGE OR ALTER WEBSITES TO BE INCLUDED IN THE CRAWLS?
No, website owners do not have to change the content, structure, or appearance of their websites to be included in the crawls. Website design can affect the completeness and accuracy of the archival captures, however; following these Guidelines for Preservable Websites helps ensure successful web archiving.
WILL YOUR CRAWLING INTERFERE WITH ACCESS TO OUR WEBSITE?
We crawl websites at a polite rate so as not to interfere with access to your website. Crawls will generally be run quarterly or semi-annually for actively updated websites, and last for a few days. Once a crawl is complete, the crawler no longer interacts with your server. If you encounter any issues or have any additional questions, please contact us at email@example.com.
ARE YOU ABLE TO CAPTURE MEDIA, AUDIO AND VIDEO FILES?
Yes, downloadable media, audio and video files can usually be captured, although videos hosted on third-party services like YouTube or Vimeo can be challenging. Our crawler follows links in order to discover and capture content, so links to content must exist on a website in order for that content to be included in the archive. We can’t capture files that are not linked and have to be retrieved from a database via user query. (For example, a publications database that requires one to execute a search in order to access publications.) Streaming audio and video can’t be captured at all by the current generation of web crawlers.
HOW CAN I VIEW THE WEBSITES THAT HAVE BEEN ARCHIVED? WILL ACCESS ALWAYS BE FREE?
Archived websites will remain freely accessible to the public. Websites can be viewed by date of capture via our Internet Archive partner page. Additional means of viewing archived websites will be explored by program staff.
WHY DO THE ARCHIVED VERSIONS OF SOME WEBSITES APPEAR TO BE INCOMPLETE?
I WOULD LIKE MY ORGANIZATION’S WEBSITE TO BE REMOVED FROM THE WEB RESOURCES COLLECTION. WHO DO I CONTACT?
We will honor requests to remove archived content. Please contact firstname.lastname@example.org.