I gang

Data collection / web page scraping

We need someone to create a script that will extract data from a few search and shopping engines for us. We need to scrape the top engines that have paid listings on a set of keywords in order to gather certain information. The same process will be used to gather this information from the shopping engines. This information collected will be extracted into a database. Project outline described briefly below.

A brief description of the requirements are as follow:

We need a program that can scrape the following information (title, description, URL, and placement (1st , 2nd, etc) for paid or sponsored listings from 5 different search engines, and 3 shopping engines. The program will grab the keywords that will be searched from a database or excel file. The program needs to multi-tread the queries to be efficient. The program will run (pulling from a list of keywords) until all the keywords have been exhausted. Keywords may be batched; IE 5000 keywords per day or something like that. The program needs to handle at least 5000 keywords per day. When complete with the entire list, it will start over from the beginning.

The results from the data collection / scrapping will be added to a database. The results from the scraping will need to be processed by the database.

Example

I have attached an excell files(named [url removed, login to view]) which will show you what the output will include.

A single URL might have thousands of keywords. Along with the associated titles, descriptions, and placement.. We will need some reporting created to properly analyze the data.

About us.

We a online advertiser with thousands of keywords (keywords change and are added to daily). We want to see what other advertisers in our pace are doing, in order to be competitive.

This project needs to be completed ASAP

Færdigheder: .NET, Java, Perl, Visual Basic

Se mere: we excell, web search engines list, web scraping process, web page scrapping, top data, to create web page, set data, search files on web, scraping data from web database, run java program online, online advertisers, online advertiser, multi program online, java start web, i want to create web page, excell program, data process online, data 0, daily advertiser, competitive set example, advertisers online, java data collector, webpage data collection, perl webpage scraping, scrape webpage perl

Om arbejdsgiveren:
( 7 bedømmelser ) Las Vegas, United States

Projekt-ID: #89459

Tildelt til:

cliver

Hello, Please look at the PMB. Thanks, Sergey

$300 USD in 5 dage
(26 bedømmelser)
6.8

12 freelancere byder i gennemsnit $205 for dette job

dims

I can do with Perl. I can create demo for you

$270 USD in 14 dage
(116 bedømmelser)
6.1
Stylishcoder

Please check PMB

$100 USD in 2 dage
(0 bedømmelser)
0.0
cisinfosys

i do it many time

$100 USD in 30 dage
(0 bedømmelser)
0.0
Venkatachhalam

I have already created a content harvestor kind of a program and is very much succeeded using perl. I think it will not take a lot of time for me and can meet the expected in this schedule.

$100 USD in 7 dage
(0 bedømmelser)
0.0
supremenetpark

i have already worked on this project.. please contact me for further details

$100 USD in 5 dage
(0 bedømmelser)
0.0
mladenb

Please see the pmb.

$190 USD in 0 dage
(0 bedømmelser)
0.0
saitechinc

Lets Start, Please see us in PM so that we can discuss about project and our experience

$200 USD in 15 dage
(0 bedømmelser)
0.0
Gudoc

Ready to start immediately on your project, as we are the experts in such things. Waiting for your selection. Thanks!

$300 USD in 7 dage
(0 bedømmelser)
0.0
sharptech

We are web design ,graphic design and website development company comprises a pool of professionals who deliver quality work on a wide range of communication web site design and web application projects. The web desi Mere

$250 USD in 8 dage
(0 bedømmelser)
0.0
lipa

Dear Sirs, This project seems to be very interesting for us, please see PM.

$300 USD in 15 dage
(0 bedømmelser)
0.0
dengyn

Hi, I have 5 years experience in VB,C#,ASP.NET,especially in VB. Just before several days I had left my original outsourcing IT company in SZ in order to have further time to pursuit personal career target, I can conce Mere

$250 USD in 6 dage
(0 bedømmelser)
1.8