Annulleret

Web scrape of 5-7 online retailers using Ruby Mechanize/Watir/Nokogiri (repost)

I run a data analytics consultancy in Sydney, Australia. I have a requirement to setup weekly scrapes of at least 5 large (50,000+ SKUs) online retail websites. The project will be split to scrape a single website to start with, extending to the others assuming satisfaction with the work completed.

Technical requirements:

- Scrapes must be written in Ruby using Mechanize/Nokogiri or Watir as appropriate

- Where possible, avoiding JS and scrape via direct HTTP requests using Mechanize - i.e requires strong skills in tracing/debugging HTTP requests/responses

- Alternatively, load page using Watir webdriver then pass off cookies to Mechanize

- Last resort would be brute force Watir webdriver approach (scroll, point, click, etc)

- Implement utilities for use of proxy / IP rotation where required (access to pool of anonymous IP addresses will be provided)

- Scheduling via cron job to run at least weekly

- Must use object oriented approach and design in a DRY manner so that code can be leveraged for future scrapes of additional sites and ensure future code manageability

- Must use Activerecord to persist records scraped

- Must implement logging (production logging to file + verbose debug option to STDOUT)

Personal requirements:

- Must be a great communicator both in terms of coding concepts and project management

- Good at clearly laying out the workplan, milestones and progress

- Must be available for 2x daily stand-up calls with Sydney AUS (timing TBC at mutually agreed time)

- Must be able to demonstrate prior experience with ruby web scrape projects similar to above

- Code must be well-structured and well-documented / commented

Opportunity for ongoing Ruby backend and RoR web app build if this project is a success.

Evner: Django, Python, Ruby, Ruby on Rails, Web Skrabning

Se mere: watir headless, watir examples, ruby web scraping, ruby watir, ruby mechanize examples, watir gem, Web development using Ruby on Rails, 8 5 x 7 5 print online brochure, web scrape 5 sites, quick app site web scrape using proxies, simple web application using ruby, web scraping using ruby rails, online sharing using web service, creating portal using ruby, web scrape asp login, online bookstore using php, flash web scrape, example online form using php, create online admission using php javascript html, php web scrape script

Om arbejdsgiveren:
( 0 bedømmelser ) Australia

Projekt ID: #15865779

21 freelancere byder i gennemsnit $37/time for dette job

bestit4u

Hello,sir I am very interested in your project, because I have much experience in such projects. I have good skills with the program language including C/C++, C#, java, php, asp.net, python, VB.NET. So I have expert Flere

$41 AUD / time
(47 bedømmelser)
6.1
yingli1987

Hello, I read your job posting carefully and I'm interesting in your job. My major language for scraping is python with some packages(scrapy, scrapyD, lxml, BS, selenium packages). I can make scraping script as Flere

$41 AUD / time
(16 bedømmelser)
5.4
londonlance

We are a London (Shoreditch) based rails studio. Please get in touch to discuss further. Following are some of our recent projects; [login to view URL] a freelancing marketplace built using Rails 4.2 a Flere

$36 AUD / time
(1 bedømmelse)
5.6
kchg

As the TOP 5th freelancer of this site, I can deliver the good result in time. Developed some kind of data scraping using Ruby on Rails and PHP. With the wonderful experience and enough knowledge, I can make the su Flere

$35 AUD / time
(11 bedømmelser)
6.0
zixy

We have worked on the large scale scrapers which work in the same way as you described in the project requirements. Are you available to discuss the details now? Talk soon, Zoran

$40 AUD / time
(2 bedømmelser)
5.3
arvindkushwah9

Dear, I have 5+ years experienced in Web Development. I'm full stack Ruby On Rails developer, I have experience in developing the application like you want with, i can do this job very well. Specialties: Ruby on Flere

$27 AUD / time
(35 bedømmelser)
5.6
andriphone0

Hello, I am scraping expert, I have did too many similar projects, Please check my feedback then you will know. Can you tell me more details? Then I will provide demo data for you.

$25 AUD / time
(48 bedømmelser)
5.3
TenStar718

I have good experience for web scrap app development. I can setup an app that scrapes at leat 5 large online retail websites. Let's discuss more in PM.

$39 AUD / time
(4 bedømmelser)
4.6
abhilashtv

Hi, ➲ 7+ years of full-time experience in Ruby and Python with 50,000+ Upwork hours billed and 50+ successful Python projects ➲ Upwork Top 10 Certification for Ruby and Pythn ➲ Guaranteed Results Policy: Pay only if Flere

$27 AUD / time
(11 bedømmelser)
4.8
BizVenture

I've special skills in web scraping using Python and Ruby. Generally use direct HTTP requests, but if website asks javascript running on client, no problem by using selenium and webdriver. Have scraped on-demand, soc Flere

$44 AUD / time
(2 bedømmelser)
4.1
DarinX

Web design&development project Relevant Skills and Experience Mobile: Java, Swift, Objective-C Design: graphic design, web design, UI/UX design (Photoshop, Sketch, Illustrator, Axure, Balsamiq) Web: (front end) Angu Flere

$30 AUD / time
(2 bedømmelser)
3.1
leadconcept

Hi there, I have read everything and our team is very experienced in Ruby on Rails, so please share complete requirements or let me know from which websites, we have to scrap the data? Do you want to display the dat Flere

$38 AUD / time
(1 bedømmelse)
3.0
mike199

Hi, I’m a Web Designer/Developer from the UK. My name is Mike. Your project description sounds interesting to me and I do have skills & experience that are required to complete this project. Let's have a quick chat whe Flere

$41 AUD / time
(1 bedømmelse)
3.2
zdesign77

Hey, how is it going today? My name is Michael, I'm a web designer/developer from Boston. I've had a look at your project description and feel that my skills match your requirements perfectly. In fact, I did a very sim Flere

$41 AUD / time
(0 bedømmelser)
0.0
AzaLess

Hi, We are focusing on Data analytics and Rails development as well. So it should be a perfect fit. Here are the case studies about the proects where we worked with Text Processing and scraped a lot of data: Flere

$41 AUD / time
(0 bedømmelser)
0.0
paultitov

Thank you for your job posting. I am a talented python developer with rich experience of data scrapping and web programming(django) for 8+ years. About your task: After reading your job posting, I feel I am good Flere

$41 AUD / time
(0 bedømmelser)
0.0
ftp2012

Hello!, do you need a script or my work? I have scraped Amazon 150000 URL (Title, Price, Bullets, Pictures). look please at my portfolio.

$41 AUD / time
(0 bedømmelser)
0.0
suhaa18

Dear Sir! Your project is very interesting to me. I want to discuss with you about your project and participate in your project. I have 5+ years of Python and Ruby development experience and have a good knowledge of th Flere

$41 AUD / time
(0 bedømmelser)
0.0
becritical

Hey there, I will develop a web scraper for online retailers using Ruby in combination with Watir. I will implement advance proxy rotation in order to achieve maximum performance. I have a very complex scraping Flere

$37 AUD / time
(0 bedømmelser)
0.0
$37 AUD / time
(0 bedømmelser)
0.0