The WebBack Crawler Script
The WebBack Crawler script is a specialized web crawler written in python. The script is in development, and is recommended for those with programming experience.
The script will later serve as the backbone of the WebBack Machine, a web interface that will allow researchers to run crawls and analysis on their own. The WebBack interface will abstract the implementation of the crawler which will make the crawling process much easier for those without programming experience.
The WebBack Crawler script has only been tested on the Windows platform. It may not work on Linux or Mac OSX.
To get started with the script:
- Download WebBackCrawler.zip and unzip
- At this time, there is no user-level documentation. Please contact dpf8 [at] cornell dot edu. for assistance using the script.
Last updated May 12, 2007