Simple web scraper using Python + Scrapy (+ XPath) + PostgreSQL (current versions of products).
Data will be scraped from one particular site, it contains three types of pages (based on page is different scrap logic).
It will works as follows:
- Reading input data for page from DB input table (based on selection criteria in script), sending to server
- Reading the response, parsing (based on page type) and saving to DB output table
From every page will be created 1-N rows in database – exact rows count is dependent on one from scraped values.
Database structure of input/output table + data and detailed decrciption including tracing of scraped data to table fields will be provided.
Functions for avoiding to be blocked also requested (using proxies, changing headers, etc.)
37 freelancere byder i gennemsnit €138 på dette job
I am an scraping and automation [login to view URL] free to take a look at my profile and portfolio. send me the database structure and lets discuss more in inbox
Dear, sir. I am so happy to see your scraping project. I am so familiar with Scrapping and Entry. I will use Python and PostgreSQL. I can send good result you want. Hoping reply from you. Best Regards.
Dear client. I am an expert of Python scrapy and web scraping. I have rich experiences in postgresql. I can help you perfectly. Please contact me. Thank you.