In association with heise online

03 July 2013, 17:07

Data gathering platform ScraperWiki exits beta

  • Twitter
  • Facebook
  • submit to slashdot
  • StumbleUpon
  • submit to reddit


The ScraperWiki team has announced that the open source data gathering platform has exited beta. The AGPLed software is available as a commercial self-service platform, as a managed service, and as an application that users can download and run themselves. ScraperWiki allows developers to extract data from a large variety of sources and then manage and process it. The results are then presented in a wiki-like interface, giving the software its name.

ScraperWiki's services have been used by the UK government's project to unify a large number of individual department web sites under one domain and a research project at Liverpool John Moores University used it with data from the Office of National Statistics and from emergency services to provide a virtual map of health-related factors like assaults and number of ambulance call-outs across the city. Other users of ScraperWiki include the Guardian, Channel Four and Bridgestone.

The AGPLv3-licensed source code for the different components of the ScraperWiki platform can be downloaded from ScraperWiki's GitHub page.



  • July's Community Calendar

The H Open

The H Security

The H Developer

The H Internet Toolkit