Price Grapper Crawler on 5 sites

We need to crawl few information from 5 sites: (we can pm to you if you request)


We need to crawl the products price from 5 sites.

some of the sites is very simple, as they have product ID in the URL.

e.g. [url removed, login to view]

so, you can do it very easily.

We only need to store

1. list price

2. special price (if any)

3. sku / item code

4. product id (if any)

5. product title

6. product weight (if any)

7. product URL

8. site ID, to indicate which site you store

9. product image, just the small listing image is fine.

10. any other field that you may need to store in order to let your program run, such as last crawl date, flag, hash, etc

All the data is stored in MySQL 5.x db. (the image file is store in local folder)

We use those data to compare the price (you DO NOT need to write this part, just store the data format as we request is enough),so, no need to store other information.

but if you think that storing other information, such as category is helpful, you can store it.

we don't mind.

Pay attention:


All the product price should be in USD.

The script must be able to run in cron job OR manually.

If the product is crawled before, update the exist record.

It should allow us to input a product URL or product ID range, so that we can crawl some product

manually. it is mainly use to fix some products which is timeout during crawl.

It should allow us to set which site to be crawl in the "web submit form", just like a normal web form.

The project must be written in module, so that we can add extra CRAWL site php file, say the 9th site:

[url removed, login to view] in the module/ folder.

then your script will auto load that file and we need to change nothing in the php files.

the auto crawl setting should be store in db, so, please give us a web form so that we can modify it.

such as when to crawl, which site to crawl

don't request us to do it in phpmyadmin. a normal end user don't know how to use phpmyadmin.

After crawl the product, it is needed to update the mysql LOG so that we can check which NEW product is added and which product price is changed.

We allow this project to be done within 10 days. (included holidays)

Our Sever:


Redhat Linux, Apache 2.x, php 5.3.x



We will ecrow 50% of the payment once you are accepted to do this project.

you need to send us the code to test on our server or you can show us in your server.

Once we got your working code (at least 90% work), we will escrow another 50% to you.

The payment will be released to you if any only if after we TESTED and WORKED without problem in our server.

Normally the test takes about 1-2 days, it depends on how fast is your script can crawl.

it don't makes send s for us to test for 1 day if your script need 10 days to crawl, right?

Bonus will be given if your script is fast, clean and well communication.

normally we pay extra 10%-20% depends on your work.

Deadline is super important for us, if we can't finish it in the period you stated.

We have the right to cancel this project and you CANNOT argue on that.

If you DO NOT agree or DO NOT understand what we said, please DO NOT bid this project.

Evner: PHP

Se mere: working of web crawler, will price, submit to web crawler, small job sites, sites that pay you to write, server load test, redhat linux price, price file, php price project, payment module escrow, job web crawler, job sites in us, job listing sites, job flag, fix test request, fix it sites, fine products, escrow web sites, escrow price, escrow com small price

Om arbejdsgiveren:
( 23 bedømmelser ) HK, Hong Kong

Projekt ID: #499371

17 freelancere byder i gennemsnit $274 på dette job


Check your PMB for details!!!

$550 USD in 15 dage
(86 bedømmelser)

Please view PMB

$450 USD in 10 dage
(60 bedømmelser)

waiting for more details

$250 USD in 5 dage
(96 bedømmelser)

Hello sir. I can help you with this project. I've done this kind of projects before. Regards, Sphinks!

$200 USD in 3 dage
(88 bedømmelser)

Hello,Please refer your [login to view URL] you.

$250 USD in 7 dage
(48 bedømmelser)

Hi. I just built such a price comparison application. I will need the 5 URLs you wish to scrape. Please check PM for details. Regards.

$600 USD in 7 dage
(29 bedømmelser)

Check PM please.

$175 USD in 5 dage
(14 bedømmelser)

Hi,please check PM.

$250 USD in 5 dage
(4 bedømmelser)

Please check PM

$190 USD in 5 dage
(10 bedømmelser)

Hi. I can help you with this project. I am sure i am the right person because i have a good experience in data grabbing from website. I will send you a pm with some of my projects.

$150 USD in 3 dage
(1 bedømmelse)

Hi I am basically a perl programmer and like to scrape sites using perl. I can do it in php though but i think perl would be a better choice as it is fast. Please accept my bid Thanks

$150 USD in 25 dage
(5 bedømmelser)

Hi, Please check PM.

$150 USD på 1 dag
(6 bedømmelser)

Lets start...................send the complete project requirement

$250 USD in 24 dage
(2 bedømmelser)

Over 3 years of experience in PHP/MySQL. I believe I can do this. Looking forward to your response.

$500 USD in 10 dage
(0 bedømmelser)

lets do this asap

$150 USD in 3 dage
(0 bedømmelser)

Please check PMB

$180 USD in 4 dage
(0 bedømmelser)

Hi we have years of experience and expertise in PHP. with our experienced and expert people we can assure you accurate,creative and timely results,according to your requirements.

$210 USD in 8 dage
(0 bedømmelser)