Webarc:Tools Developed
From Adapt
Input Data Preprocessors
- [[Webarc:LRE Monthly Dumper|LRE Monthly Dumper] (Java):
- [[Webarc:MediaWiki-to-TREC Converter|MediaWiki-to-TREC Converter] (Java):
- [[Webarc:Merge DB Constructor|Merge DB Constructor] (Java):
- [[Webarc:Carryover DB Constructor|Carryover DB Constructor] (Java):
For our experiments, we have run the tools listed above sequentially to obtain monthly snapshots of Wikipedia revisions from 2001 to 2007, and also to identify, for each month, the articles that do not have new revisions in the current month, thus need to be carried over and indexed for the current month.
