Crawler for shopping-engine

Data Base


We are building a web-shopping engine, we need a robot to crawl online web catalogs and excel file; the sellers inventory. Then treat the data in order to add it to a standardised-searchable database.

Every inventory will have different shape and format so we hope to have a clever robot that is going to be able to treat the information without need of writhing a script for each excel file and html catalogs.

• What does this html page and file characterize?

They are wine shops price list, inventory.

• Where are they located?

90 % of them are located on a server where we have access simply with the URL

10 % are located on wine shop’s computer these once are going to be sent to us by email

• What is their format?

Every shop builds their inventory with different software: Html, Excel … But also they use their own style to design the inventory.

Plus, they use different currency, abbreviation (France – FR), (Bordeaux – BDX), (Bottle – 75cl) and some of them do spelling mistake. Some include taxes, some don’t.

• What is our format?

In order to have a searchable data base we need to standardize all this data in our own format.

For example we will change: Bordeaux, BDX, and Bordau to → BRDX

• What we need at the end?

A clean Mysql database with all their data in our format

We need a robot-engine that is going to be able to:

Crawl the webpage inventory of the seller

Download the online excel format inventory of seller from url

Receive excel file by email a treat them automatically

Change the prices to us dollars

Remove sale tax if price include tax

Change the data to our format

Add to the database

The table we want to enrich is as follow:

• Shop identification code

• Date and time we have collected this information

• Product name

• Vintage

• Condition

• Packaging (1x75cl, 12X75cl, [url removed, login to view])

• Quantity

• Price excluding taxes in dollars

• IB, DP

• Url of the product’s page if exist

As the database could probably end up with a few million offer, it should be optimized to be searchable very fast.

Evner: .NET, MySQL, PHP, Python, Ruby on Rails

Se mere: shopping engine software, wine by design, where to design and order packaging, what's dp, what is dp, what is a dp, web design robot, web design product packaging, web design packaging, software shopping, shape up, sale packaging design, robot's name, robot name, python shop, product packaging design software, product packaging design online, product design packaging online, product design for sale, price for packaging design

Om arbejdsgiveren:
( 0 bedømmelser ) phulet patong, France

Projekt ID: #611646

25 freelancere byder i gennemsnit $2768 på dette job


Hello. Please check PMB. I'm experienced Programmer with skills & knowledge. I can write this crawler following Your guidelines in the specified time frames. I'm available 8 hours a day on-line.

$1750 USD in 40 dage
(139 bedømmelser)

Hi, we are a company focused on creating custom-made software solutions. We have great experience in creating Internet crawlers and indexers. We have created one of the UKs biggest media intelligence projects and we h Flere

$3000 USD in 45 dage
(2 bedømmelser)

Hello, We have read the posting and would like develop as per your [login to view URL] check your private messages for more information about this project. I hope to assist you with it.

$1500 USD in 0 dage
(14 bedømmelser)

Please check your PMB. Thanks.

$3000 USD in 30 dage
(3 bedømmelser)

Hi I have experience with such types of robots Unfortunately such robots could not run with out human help. But I i could offer you instrument which will make it easy and fast - to add new stores or tune up spider whe Flere

$15000 USD in 300 dage
(4 bedømmelser)

Hello. This is an interesting project. We will most likely need to do some fuzzy logic for the spelling differences/errors, but that won't be a problem. There is a bit of complexity as you want the software to scrape w Flere

$1500 USD in 15 dage
(1 bedømmelse)

I have similar project experience. Check out my github repo ([login to view URL]) for some of my projects.

$1500 USD in 10 dage
(1 bedømmelse)

Kindly Check Your PMB Thanks Alex Grey

$3000 USD in 30 dage
(1 bedømmelse)

Hi, love making smart robots, name the robot and we'll make it! contact us on PMB Yacov

$1500 USD in 5 dage
(0 bedømmelser)

Hello, United Software Associates Inc is a CA based software services and solutions company, focused on building and maintaining high quality software applications to the global market. We have understood your requirem Flere

$2300 USD in 30 dage
(0 bedømmelser)

have good experience in data processing and standardization with PHP, Mysql.

$3000 USD in 45 dage
(0 bedømmelser)

Easy-peasy. I have written several web screen-scrapers, one used by 70,000 users!

$1800 USD in 14 dage
(0 bedømmelser)

We will provide only binary, not source code

$3000 USD in 20 dage
(0 bedømmelser)

please check PMB.

$2000 USD in 20 dage
(0 bedømmelser)

Plz check your PM.

$2000 USD in 50 dage
(0 bedømmelser)

Dear Sir, We have gone through the details of your crawler. Having similar experience of crawler and scrappers for various types of websites we are confident that we can produce quality work for you. We will deliver Flere

$1500 USD in 30 dage
(0 bedømmelser)

Hi, I've been doing screen scraping since 2000, when I was working at [login to view URL], the first metabrowsing site. So, I've got a lot of experience there. The variable I need to know is how many sites do you wan Flere

$3000 USD in 10 dage
(0 bedømmelser)

DASGROUP GROUP OF FREELANCER software solution 7th genaration it solution Following benefits you will get from us: Flere

$3000 USD in 25 dage
(0 bedømmelser)

Hello !! Alphasoft Technologies is a company providing professional-level customized solution, Web applications development, full-featured online shopping stores, Software development, Client server technology. We d Flere

$2500 USD in 30 dage
(1 bedømmelse)

We have the experts and specialized for the various field of IT which develop Software and Website. And have the expert skill of Different sector of Software Development like a PHP, JSP, Ajax, ASP, VBScript,Flash, J Flere

$2250 USD in 70 dage
(0 bedømmelser)