Randoom a Michael Friis production

Posts Tagged Scraping

Downloading the EU

… or parts of it anyway. The European Union is generally up to lots of weird and wonderful things, one of the more esoteric being the “Tenders Electronic Daily” (TED) database. Basically, all public procurement above a certain value has to go through this database so that companies from all over the world have a […]

.Net/Firefox Screen Scraping

Need to scrape a website? I have two links for you: Solvent from the MIT SIMILI project. In combination with Piggy Bank it’s a scraper on it’s own, but I only use for its superb XPath generator. Just activate the sprayer and click on an element you want and Solvent will generate an intelligent XPath […]

After →