complaint-scraper

Complaint Scraper

This program scrapes a website for phone number complaint data and provides that data via a RESTful API.

Dependencies

• bottle
• bs4
• cfscrape
• lxml
• os
• socket
• socks
• sqlite3
• sys

References

• https://deshmukhsuraj.wordpress.com/2015/03/08/anonymous-web-scraping-using-python-and-tor/
• http://www.gregreda.com/2013/03/03/web-scraping-101-with-python/
• http://docs.python-guide.org/en/latest/scenarios/scrape/
• https://github.com/Anorov/cloudflare-scrape

Usage

• Run the script with python complaint-scraper.py
• Visit http://localhost:8080 to view the results
• Routes defined:

  /	Returns all results in the database
  /123	Returns the results for this area code only, where 123 is some valid area code