Færdiggjort

Scrapy/Selenium (Python) to extract texts and files(specific webpages) based on keywords.

I need to make a Python script (Scrapy or Selenium, I am up to suggestions) to extract information within some specific(I have around 12) websites - daily(auto) or manually.

The pages are in portuguese, but I can guide you into the key input-fields and key-pages to look for.

1. User input:

- Time period (if the page has this feature)

- Website to scrap.

- Keywords(can be a list of words) to look for.

- User chooses the local path to the files to be downloaded.

2. Back-end:

- Access the page.

- Searches the tabs that can have useful information(I will provide the specific parts for each webpage to make the queries) links inside that domain.

- Download(if the page serves files in doc, html or pdf) and look for the keywords.

- Extract all the related content (files or the text in html).

- Go around Captchas(if the page has captcha)

3. Logging:

- All the extracted content must have the URL which the information/file is available in the webpage - can be done by logs.

- All the extracted content must have the DATE which the scrapping has been made - can be done by logs.

4. Configuration:

- All key-fields (like CSSSelector for a date field) should be configurable for each spider.

- The URL to start scrapping each webpage should be configurable.

- If page contains Authentication(Login/Password), user will fill the configuration for it.

IMPORTANT:

1. My plan is to pay for each 4 mapped websites (so total project is for 3 "packs" of websites)

2. The content in few cases will need to be extracted from images.

3. Start your bid with the word forward, so I can know if you did read all the description.

4. If you can't extract properly the content I can give you another one to replace that one, so you still need to deliver 4 websites per milestone.

5. I WILL RELEASE THE MILESTONES ONLY AFTER YOU SEND ME THE CODE AND I AM TOTALLY SATISFIED (I WILL RUN TESTS TO CHECK FUNCTIONALITY).

I have many projects at hand and would be great to stablish a good relation with you, since I constantly need someone to work with me.

Thank you.

Evner: Data Scraping, Python, Scrapy, Selenium Webdriver, Web Skrabning

Se mere: scrapy examples, python scrapy example, scrapy vs selenium, python web scraping, scrapy python 3, scrapy documentation, scrapy vs beautifulsoup, web scraping, extract dbx files, extract 3gp files, mapguide enterprise extract shp files, extract xml files website, python script text files, extract embedded files doc, extract bkf files systools bkf repair tool, testsuite example selenium python, extract perl files server, extract ole files rich text, extract mht files, test suites selenium python

Om arbejdsgiveren:
( 5 bedømmelser ) FORTALEZA, Brazil

Projekt ID: #19212037

Tildelt til:

etuannv

Hi there, I am interested in your project. I would approach your project by using Python with Scrapy. The website will be written in Python with Django. Here is a demo project: Price tracking system: https://etuannv.c Flere

$250 USD in 10 dage
(63 bedømmelser)
6.2

36 freelancere byder i gennemsnit $579 på dette job

Vlzinch

Hi! I’m experienced Python developer, and web-scraping is one of my main fields of knowledge, so I’m 100% confident that I can complete your project and extract data from the sites you need. Please contact me to d Flere

$748 USD in 7 dage
(61 bedømmelser)
7.7
mhmhz

Hi Can you provide the sites so i can analysis them? Thanks

$800 USD in 5 dage
(103 bedømmelser)
7.4
zhangyingtai

forward Hello sir I have 9 years of experience about web scraping and have made 200+ crawlers with python. I have fully understood the project and I am confident. I can start the work right now. Best Regards, Flere

$588 USD in 10 dage
(111 bedømmelser)
7.5
$1000 USD in 7 dage
(93 bedømmelser)
7.2
zekovicm

Forward Hi there,I am Python Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and fi Flere

$705 USD in 10 dage
(91 bedømmelser)
7.2
polarjin2017

Here is my selenium with python working result. [login to view URL] python selenium web driver app to scrap live data from the web site and export to excel file. This is just what I've done. I can do pytho Flere

$250 USD in 3 dage
(49 bedømmelser)
6.4
dreammate0621

Hello! Let's just rest a moment. <Actions speak louder than words!> Nice to meet You! I am a WEB expert! I am interested in Your project. I wanna work with You. If you hire me, I am gonna do my best for Your proj Flere

$555 USD in 10 dage
(5 bedømmelser)
6.3
C3guru

forward I've read your requirements about User Input,Back-end,Logging and Configuration. I have a good experience with selenium and python. Recently,I've developed B*T for Telegram. That acts like human 100% exactl Flere

$1000 USD in 10 dage
(15 bedømmelser)
5.8
lightingdavid

Hello. I have good skills in "Data Scraping, Python, Scrapy, Selenium Webdriver, Web Scraping". I have working for 7+ years in this field. I 'm very interest to your project. I have checked your project description Flere

$250 USD in 3 dage
(31 bedømmelser)
5.1
kunitsynartem

Hello! I have 2 years of experience in web scraping using Python and I'm interested in your project. I can use both Selenium and Scrapy depending on what is better for certain website. Also I can handle logins, file do Flere

$600 USD in 10 dage
(27 bedømmelser)
5.1
smsaurabhv

‌Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHO Flere

$444 USD in 10 dage
(49 bedømmelser)
4.9
drishinfotech

forward HI, I read your job description and would like to assist you in website scraping task. I understand your conditions and will surely provide you the code after completion of the each task. Please share Flere

$750 USD in 10 dage
(9 bedømmelser)
4.7
albertpopov46

Dear, sir @I am fulltime freelancer@ I read your description in carefully. I am python expert and I have rich experience with scrapping. Also i have selenium experience . So I think that i can do your project in Flere

$500 USD in 10 dage
(10 bedømmelser)
4.2
yongbeauty1996

hello how are you? I am very interested in your project. I have read your description very carefully. I can do your job in time. kind regards

$555 USD in 10 dage
(4 bedømmelser)
4.2
NIKE9

Hi, I am a senior selenium/python expert and I can build the script as requirements in the description. I have 7+ years of professional experiences in web development. I can start immediately, also finish your proje Flere

$750 USD in 7 dage
(3 bedømmelser)
3.6
KGeorgy

Hi, Thanks for your job posting. I've read your project description carefully. You are going to build scrapy that gets data based on keywords. As a senior scraping developer, I have rich experience in scrapy and pyt Flere

$500 USD in 10 dage
(6 bedømmelser)
3.6
chirag9700

I have more than 6+ years of experience into IT field. Since last 6 years, I am dealing with different kind of field such like : - Laravel, CI, YII - Angular.js - Node.js - Ionic Framework - PHP - HTML - Python - Djan Flere

$666 USD in 10 dage
(2 bedømmelser)
4.2
BoyVit85

How are you. Credit is my motto. I am expert web scraping. I can do your job with BS4 and Seleinum framework of python. I can do any project in your demand completely by my good experiences of last ago. I think thi Flere

$555 USD in 10 dage
(3 bedømmelser)
3.1
edison4mobile

HI, how are you? I have checked your description carefully. I can say I understood fully what you want. As I have rich experienced in python(2, 3) so that your project is not problem for me. I am really confident an Flere

$777 USD in 10 dage
(1 bedømmelse)
2.8
vorasiddh4it

We have 11+ years of experience in software development. We have developed 400+ projects and the research paper in the field of Machine Learning, Artificial Intelligence and Image processing (GIS), Network, SEO based W Flere

$1000 USD in 10 dage
(4 bedømmelser)
3.4