Crawl the web to find websites which use single sign-on

I need someone to write me two pieces Python scripts. The scripts work with mysql database.

Crawler script:

This page queries [login to view URL] (or other similar sites on web ranking) to discover popular websites and store the website URLs in table websites. An example table entry:

ID site_domain global_rank checked_for_sso

2 449 no

3 [login to view URL] 6476 no

4 [login to view URL] 760 no

Requirement: it is able to discover top 1 million websites

Analysis script:

This page get a website not analyzed (checked_for_sso field is “no”) from table websites, then visit this website and explore it to figure out what single sign on services it uses. If any single sign on services discovered, store the information in table sso. Also update checked_for_sso field in table websites to “yes” after the analysis is done. An example entry for table sso:

ID login_url supported_SSO

2 facebook

3 [login to view URL] google

4 [login to view URL] facebook,google,yahoo

Requirement: false positive rate is 0, and false negative rate is less than 1%

This is small project. My budget is around $300. Please don't bid more than $400.

Evner: MySQL, Python

Se mere: www the freelancer com, www freelancer web page, www freelancer login com, www freelancer in login, www freelancer in facebook, www freelancer id, www freelancer com sign, www freelancer com find freelancer, www freelancer com b, www find freelancer com, what to write on freelancer, what other freelancer sites, websites to work from home, web python freelancer, web freelancer com, web app freelancer, web analysis services, top www freelancer com, to find freelancer, to find a freelancer, smartsheet freelancer, smartsheet app, sites similar freelancer com, similar sites to freelancer, sign out freelancer

Om arbejdsgiveren:
( 12 bedømmelser ) Bloomington, United States

Projekt ID: #5992764

10 freelancere byder i gennemsnit $630 på dette job


Hello, sir I'm interested with your project. I have good skill and my place is 8th in freelancer. I'll do it for you. I can do it well more than you could guess. :) If you give me t Flere

$773 USD in 10 dage
(201 bedømmelser)

Hello, I'm Anna - a project manager in a Russian-Canadian web development company - A2 Design. You can check our recent projects on our website [The administrator removed this message for encouraging communicati Flere

$350 USD in 5 dage
(20 bedømmelser)

Dear Sir / Madam, Warm Greetings from Genpex IT Solutions. As soon as I saw your posting for a “Crawl the web to find websites which use single sign-on”. It is the perfect position for us .Our AIM to give the bes Flere

$526 USD in 10 dage
(38 bedømmelser)

Dear Client, I can help in your project. We have already experience of working on similar projects. Please see below to get idea of our experience: Amazon/Ebay Bots: [login to view URL] Flere

$526 USD in 10 dage
(71 bedømmelser)

Hi, Iam interested in your project and I'll be happy to do that for you. I have rich experince in scrapping using curl regular expressions Dom and Selenium RC. I worked for [login to view URL] and [login to view URL] sear Flere

$400 USD in 10 dage
(13 bedømmelser)

I am willing to discuss further about the project specifications and deliver the same to your needs .

$526 USD in 10 dage
(46 bedømmelser)

Hi, Hope you are doing well! We have gone through your requirement and we understand that you are looking for highly skilled, qualified, and experience Python development team for your project. Our developer Flere

$773 USD in 15 dage
(15 bedømmelser)

Hi, You need a crawler which collects the most popular websites, checks whether the sites use a single sign on or not and stores the data in MySQL. If you are interested I will show you a demo of this project on Flere

$400 USD in 10 dage
(2 bedømmelser)

Hi. I'm able to write super fast crawler that will download thoushands of pages per second. Shall it crawl whole domain (~2000 pages per domain) or just one page?

$1666 USD in 14 dage
(0 bedømmelser)

I have 2 years experience specializing in website scraping with [login to view URL] post on website . I do a lot of sites like facebook, twitter, google, ... in the normal way or used in conjunction with multi-threading api do Flere

$355 USD in 9 dage
(0 bedømmelser)