I need someone to write me two pieces Python scripts. The scripts work with mysql database.
This page queries [login to view URL] (or other similar sites on web ranking) to discover popular websites and store the website URLs in table websites. An example table entry:
ID site_domain global_rank checked_for_sso
2 freelancer.com 449 no
3 [login to view URL] 6476 no
4 [login to view URL] 760 no
Requirement: it is able to discover top 1 million websites
This page get a website not analyzed (checked_for_sso field is “no”) from table websites, then visit this website and explore it to figure out what single sign on services it uses. If any single sign on services discovered, store the information in table sso. Also update checked_for_sso field in table websites to “yes” after the analysis is done. An example entry for table sso:
ID login_url supported_SSO
2 http://www.freelancer.com/#login facebook
3 [login to view URL] google
4 [login to view URL] facebook,google,yahoo
Requirement: false positive rate is 0, and false negative rate is less than 1%
This is small project. My budget is around $300. Please don't bid more than $400.
10 freelancere byder i gennemsnit $630 på dette job
Hi. I'm able to write super fast crawler that will download thoushands of pages per second. Shall it crawl whole domain (~2000 pages per domain) or just one page?