The South Dakota Lobbyists' database provides a simple task that any scraper should be able to cope with. The results are in a table with each cell representing a piece of data.
There are search options, but they aren't necessary to get all of the data.
The only difficulty with the data structure itself is that the table also contains the search form. This may provide a challenge for some tools as the HTML structure doesn't provide clean separation of navigation and content.
DESIRED OUTCOME: Create a database or spreadsheet containing all the information on lobbyists in South Dakota.
Data Toolbar was a quick and almost idiot-proof solution for this test, scraping the names and addresses of 8,117 South Dakota lobbyists from the state's website.READ OUR FULL TEST RESULT »
Needlebase effortlessly pulls down structured data from the South Dakota lobbyist database within minutes.READ OUR FULL TEST RESULT »
Helium Scraper easily recognizes the table-formatted South Dakota lobbyist registry and is capable of automatically navigating and extracting all of the 8,000-plus entries.READ OUR FULL TEST RESULT »
Despite some junk data that requires post-scrape cleaning, Outwit was able to efficiently capture the simple, multi-page South Dakota lobbyist database.READ OUR FULL TEST RESULT »
This is not the right software to help scroll though 406 pages of tables on South Dakota lobbyists. However, it will help to create a nicer looking spreadsheet if you want to copy and paste pages individually.READ OUR FULL TEST RESULT »
While it quickly gathered data from the first page of the South Dakota lobbyists website, Scraper's single-page limitation meant it could not automatically scrape the other pages of lobbyist information.READ OUR FULL TEST RESULT »
Table Capture was able to copy individual tables for importing to Excel, but will not automatically scrape this entire 406-page database of South Dakota lobbyists.READ OUR FULL TEST RESULT »