Price comparison

A web scraping program written in C# to crawl multiple competitors' websites, and record time-stamped product pricing to MSSQL tables with minimal impact on the competitors' website performance.

9/17/10 @ 10:00am US Central: Updated project with change from PHP and MySQL to C# and Microsoft SQL Server 2005 (with 2008 compatibility).

## Deliverables

These are the websites that will need to be parsed for product id and pricing (in order of importance): [url removed, login to view], [url removed, login to view], [url removed, login to view], and hamcity.com. UPDATE: [url removed, login to view], [url removed, login to view] will be excluded from this initial project but possibly added as a separate project later.

Since potentially thousands of individual product IDs are involved and we want to run this on a daily basis, we'd prefer not to hit their site with an individual search for each product ID but rather formulate and optimize generic search terms that will return lots of results.

It may not be necessary to obtain their entire list of products because we are mostly interested in the prices of the products that we sell and that they sell also. If necessary, we could compile a list of manufacturer part numbers that we are interested in comparing to our own.

We want to keep historical records of pricing, so every time we obtain pricing from a competitor's site, additional rows will be added to the table(s), rather than overwriting existing rows.

Update: To further clarify, the scope of this project does not include an interface for retrieving, viewing, or analyzing the database. However, error logging and email notification upon error will likely be necessary. <s>Each scraping job (including separate jobs for sites that require multiple scrapes to minimize server load) should be able to be launched from an url like: http://{server}/{path}/[url removed, login to view]


Evner: C# Programmering, Ingeniørarbejde, Microsoft, MySQL, Projekt Ledelse, Software Arkitektur, Software Testning, SQL, Windows Skrivebord

Se mere: web scraping price, web scraping part time, universal jobs, time management jobs, sql server price list, sql server 2008 prices, sql price, scraping prices from websites, record management jobs, radio jobs, program management jobs, part time sql jobs, mysql search performance, microsoft sql server price, microsoft project jobs, microsoft.com/jobs, manufacturer prices, logging jobs, jobs search sites, importance of com

Om arbejdsgiveren:
( 1 bedømmelse ) United States

Projekt ID: #3719729

Tildelt til:


See private message.

$297.5 USD in 14 dage
(11 bedømmelser)

3 freelancere byder i gennemsnit $247 på dette job


See private message.

$340 USD in 14 dage
(93 bedømmelser)

See private message.

$102 USD in 14 dage
(2 bedømmelser)