We're an insurance company from to Santiago of Chile. We need to scrape a website and pull prices of our competitors using dummy data as an input. We are mainly interested in the code algorithm of a scraper but under certain conditions. This should be done with Python+Selenium with random proxies and random agents, this is to avoid banning our bots.
We have a code Jupyter-Notebook to scrape the website but without random proxies-agents, we are attaching the code as a file. The main goal is to create a Python-Selenium algorithm (3.8 version or upper, and we're working with jupyter-notebook) under Chrome or Mozilla browser, furthermore is very important to use random proxies (public proxies desirably), random agents and this should be working for every requests on the website without bans or failures (bans for ip, bans for requesting many times the same customer id, etc...). The process starts in "[login to view URL]", you should click the button "Cotiza Aquí" and will load a page of customer information (this could be a dummy input), after providing this information you would get to prices of all competitors, you should save these prices like an object in Python.
We offer $$$$ US for who is interested in this project and meet the profile, the needed skills are:
- Python + Selenium for scraper websites
- Knowledge about the parameters proxies and agent in Selenium
• If accepted, the above project shall be undertaken under the utmost confidentiality, both as regards its results and the dummy data provided therein.
• Without prejudice to above, if selected, you shall be required to enter into a binding NDA and a Data Processing Agreement under terms satisfactory to the Company.
• The project is to be carried out within the limits set forth under applicable regulation and your commitment to comply with all applicable regulation is expected.
Hi there, I have good experience in python with selenium and also have good experience in proxy handling. Message me and lets take this forward. Thanks