Araneum Bohemicum Czech Web Corpus

• Crawled in May & June 2013 by SpiderLing 0.72
• TLD restriction set to include the relevant national (.cz, .sk) and generic (.com, .edu, .info, .org, .net) TLDs only
• Language similarity threshold set to 0.65, remaining non-Czech (mostly Slovak) documents removed by subsequent filtration based on character frequencies
• Tagged by MorphoDiTa using the Czech parameter file based on the Jan Hajič tagset
• Native tagset mapped to Araneum Universal Tagset
• Paragraph-level deduplicated by Onion, tokens in duplicate paragraphs marked

Versions available

• Araneum Bohemicum Maius: 1,200,001,911 tokens, 850,194,623 unmarked words
• Araneum Bohemicum Minus: 120,139,611 tokens, 90,809,716 unmarked words

Revision history

• 15.04 Initial publicly released version