We are looking for someone who can spider a specific website for data, capture the data and enter the data into a database. It doesn't matter what technology or programming language you use to do this. The choice is yours. It also doesn't matter what database you choose to use. The end result will need to be SQL Server, but we can always do an import. This is the site in question:
[url removed, login to view]
On this specific page, you will see a list of main categories in the middle of the page. We need to follow each of those categories to the sub categories and keep drilling down until we get to the products. The product pages are all virtually identical besides some minor branding of the pages depending on what the manufacturer is.
Once you are at a specific product, the data that will need to be captured is as follows and each of these items will end up being a separate field in the database:
1. The category path i.e. "Audio and Videoconferencing Equipment > Audio > Conference and Speakerphones"
2. The brand name
3. The model name
4. The model series (if provided)
5. The model short description
6. The model long description
7. The model MSRP
8. The model Specs
9. The model Features
10 The accessory models for the model
11 The related models for the model
12. The image for the model. This can be downloaded and put into a specific directory and referenced with an ID number in the database or whatever method we mutually come up with.
13. Any files for the model i.e. PDF docs, word docs etc. Same as above as far as putting into a directory
This is a very complex project, but should be facilitated by the fact that if you view the source on the product page, you will notice that all the necessary elements in the page are enclosed in their own DIV tags with ID's, which should make the spidering process fairly simple.
Please don't hesitate to ask questions regarding this project.
26 freelancers are bidding on average $767 for this job
I am the creator of the [url removed, login to view] spider technology. It has been used to spider hundreds of websites with excellent results. More info in your PMB
Hi, I am currently in the process of creating a similar kind of thing for a different website. Woudn't mind earning a extra buck doing the same thing for you. Regards, Pritam Dhar
I have 9 years of C / C++ programming and a strong education background. At present I work as a senior programmer in SIIC Microsystems ltd. ([url removed, login to view]) -- internet data mining company. Please, see PM.