I am looking to have a email harvester/scraper coded (can be either desktop or web-based) that can consistently pull streaming, updated emails from the Google Search Live Updates website. This site searches Twitter exclusively (I believe) for the newest updates. Here is an example link of how it works:
[url removed, login to view];safe=off&tbs=mbl%3A1&q=*%40gmail.com&aq=f&aqi=g10&oq=
This application will need HTTP/SOCKS proxy support (ability to paste a large list of IPs that will rotate) and needs to update streaming directly from Google's Search. These emails need to be stored (.txt with filename of the date for each dump for DESKTOP APP, SQL database with tables set up to allow for storage and searching SQL based on date of harvest for WEB APP) - I need advanced functions like searching based on the email domain that is harvested, searching by date range of harvest, etc.
I'd like streaming emails to be shown in the actual GUI of the interface, so emails are being shown as they are harvested both on the application and being stored in the database. I want it to automatically detect and have prevention from recording duplicate addresses, and I want to be able to enter a list of email domains I'd like to harvest from. (For example, if I enter "gmail.com" and "comcast.net" it would run the query "*@[url removed, login to view] OR *@comcast.net" for the search.)
I've seen this coded for cheap before and I know there are existing softwares out there already that can do this, so my winning bid will be chosen both according to understanding of what I want AND lowest offer.