Installing , Setting up and Running BuBING Web Crawler on Digital Ocean Droplet

As a company we have some droplets hosted on Digital Ocean and we have already a very long list of websites and domains (and subdomains) that need to be crawled: from every url, the crawler need to fetch webpage title and metatags (if any).

In order to do so, there is a crawler that is already there, [login to view URL] but it is quite hard for our team to deploy it, install it and run it.

This job requires the installation of BuBing crawler into the droplet, the correct setup in order to not get blocked by webpages and run it. After that, for each page the title and metatags need to be saved into a file that will be then parsed by one of our scripts and given to the databases.

Long story short: install, setup and run BuBing given a list of urls.

Evner: Shell Script, Web Crawling, Web Skrabning

Se mere: scrapy tutorial pdf, scrapy python 3, python web crawler from scratch, web crawler in python pdf, web crawler python 3, python web crawler source code, how to crawl a website, python web crawler github, java web crawler image capture, web crawler cpu usage, write web crawler, web crawler names, web crawler system requirement, web crawler demo gathering information function, java spring web crawler, setting upan adult web cam paige, web crawler traffic stats, auto daily web crawler, web crawler sale, web crawler asp spider

Om arbejdsgiveren:
( 0 bedømmelser ) Uppsala, Sweden

Projekt ID: #18537468

7 freelancere byder i gennemsnit €184 på dette job


I am very interested in your project, because I have much experience in web Scraping I have good skills with the program language including python, C# , JAVA ,PHP . So I am a expert with web scraping (1000 over pro Flere

€333 EUR in 3 dage
(13 bedømmelser)

can do your job. These are my skills related to web scraping and crawling Have done scraping in CasperJS Phantomjs, python. Have done testing and automation with selenium also. Know to deal with database like mon Flere

€277 EUR in 2 dage
(26 bedømmelser)

Hello? How are you? I have good experiences in "Installing , Setting up and Running BuBING Web Crawler on Digital Ocean Droplet" as you can see my profile for these (Shell Script, Web Crawling, Web Scraping). I have Flere

€155 EUR in 3 dage
(9 bedømmelser)

Hello, I can install, configure and setup the BuBING Web Crawler in the digital ocean server. Please let me know if you are interested so we can discuss the project further. Waiting for your reply. Thank you

€154 EUR in 3 dage
(2 bedømmelser)

As a software and scrap expert, I am very interested in your project. I own a variety of scrapping experiences and technologies. I hope to discuss this project with you. I think we can get good results if we coopera Flere

€80 EUR in 3 dage
(2 bedømmelser)

Hi. BuBing crawler is a little hard to install because it need do a lot of adjust on system variables and install extra dependencies. I can help you installing it and running it. Better is you're using ubunt Flere

€133 EUR in 2 dage
(3 bedømmelser)

Hi, We would like to Design and Build your online store / personal website / simple website / Management CRM application, We would like to provide you Website Design and then you select it and then if changes need we c Flere

€155 EUR in 3 dage
(0 bedømmelser)