Lukket

I need a web crawler

I need a webcrawler that can find websites that are using or have Flash elements on their website.

Furthermore it needs to be able to differentiate the language of the website. I am ONLY interested in finding German and English language websites!

The project literally requires that I find ONLY websites in German and English language. That could be a whole number of different domain endings such as .de .at .ch for German language and .com .[login to view URL] .[login to view URL] [login to view URL] and so forth for English. But I definitely don't want websites that are in any other language (french, spanish, italian, polish, russian, japanese, chinese etc).

The crawler needs to generate a list of domains and their e-mail addresses for THAT language.

The crawler needs to have a GUI where the above parameters most be adjustable. It also must have the possibility to check against a blacklist, so if he hits a known domain or email, it gets skipped from the beginning.

There is a second stage where the crawler is supposed to find all websites that do not have an SSL certificate.

And a third stage to find those websites that use cookies, but do not have a cookie banner.

If the crawler can do all 3 of the above things, that much the better. But the first stage is to find websites with Flash elements.

I need a minimum of 2 million addresses per language

I am looking forward to your offers!

Evner: Web Skrabning, PHP, Python

Om arbejdsgiveren:
( 1 bedømmelse ) Kensington, South Africa

Projekt ID: #20352419

53 freelancere byder i gennemsnit $1234 på dette job

arhamsoftltd

Hi, I can design and develop a web crawler for you. I am familiar with related libraries and frameworks such as Tensorflow, Keras, Theano, Torch Scipy, Scikit-learn. and I am skilled in Python, R, Lua, Go etc. So thi Flere

$3000 USD in 35 dage
(166 bedømmelser)
9.0
aqualeafitsolpl

I have checked your requirements where you mentioned that you need a webcrawler that can find websites that are using or have Flash elements on their website. Please invite me in chat so we can discuss in detail.

$1500 USD in 20 dage
(143 bedømmelser)
8.3
wpoppo

Hello, As per need, you need a web crawler, we have shared our company portfolio please go through it: We have a team of professional Full Stack developers with an experience of more than 8+ years in creating and cus Flere

$1500 USD in 20 dage
(186 bedømmelser)
8.3
mingxiao2008

Hello, Dear How are you? I have checked your project description and am ready for discussing with you about project as Web scraping developer. I have experienced in PHP , Python, Web Scraping. I will work very hard and Flere

$1125 USD in 20 dage
(104 bedømmelser)
8.1
seekdeveloper

Hi, I have read your post and understood your requirement. Looking for the freelancer to work on your next project? Or just need some issues/bugs/fixes ASAP? I have 11+ years of experience and I'm here for you! My sk Flere

$750 USD in 7 dage
(104 bedømmelser)
8.7
mingah429

Hello, I read your description and got what to do. I am looking forward to discussing more for your project. I am ready now, please message me to start your project. I have 7+ years of experience in this field I am Flere

$1125 USD in 7 dage
(20 bedømmelser)
7.9
OutsourceMan

Greetings! My name is Neha. We would love to have the opportunity to discuss your project with you. We can read through the job details extremely carefully and absolutely sure that we can do the project very well. Flere

$1000 USD in 12 dage
(335 bedømmelser)
8.1
infowider11

Hello, How are you? Yes i am expert in web scraping . i have more than 6 year experienced in php. i can do this scraping in php, You can check my work:- [login to view URL] Thanks Shweta

$1500 USD in 7 dage
(114 bedømmelser)
7.6
latatestTech

With Freelancer Preferred badge bound to give 100% Quality (Let's Chat) Hello, A great team is here for your service!! I read the project description properly, and I agreed to fulfill 100% of all your requirements. Flere

$1125 USD in 7 dage
(151 bedømmelser)
7.8
ZhenExpert

Hi, I'm experienced web scraping developer with 5 years of hands-on experience. I have created several scraping projects and even developed automatic scraping bots. I'm good at python programming language and used scra Flere

$1500 USD in 20 dage
(39 bedømmelser)
7.0
Hariomtechnology

Hello BrandBuilder63, I have checked your project(I need a web crawler) requirements. We are one of the best programming team on freelancer. Awarded PREFERRED FREELANCER BADGE by Freelancer.com We have more than 127 Flere

$1625 USD in 7 dage
(83 bedømmelser)
7.1
darshring

Greetings! Seeing is believing - please check my recent reviews and portfolios first. I am proficient in Web scrapping and Data mining, and I have already finished several smiliar projects before. I am confident in m Flere

$1125 USD in 7 dage
(37 bedømmelser)
7.2
bestit4u

Hi. I am very interested in your project, because I have much experience in such projects. I have good skills with the program language including C/C++, C#, java, php, asp.net, python, VB.NET. So I have expert and smar Flere

$1125 USD in 7 dage
(112 bedømmelser)
7.1
C3guru

Hello I am a talented Web developer. To detect Flash elements, I can scrape Web pages from each website using selenium. In this way, we can also detect languages. The second and third stage is a bit difficult but I ca Flere

$1000 USD in 7 dage
(28 bedømmelser)
6.7
WebnAppExperts

Hello there, As you have posted jobs "I need a web crawler". We are a team of 60+ members in PHP, CodeIgniter, WordPress, Magento, Big commerce, Interspire, , Node Js, AngularJs, and Mobile Apps having experience ove Flere

$1500 USD in 70 dage
(22 bedømmelser)
6.3
pvsysgroupinc

Dear Sir, Requirements : - CMS AS per your need or we can make custom CMS too - Design : HTML5, CSS 3, Bootstrap - Jquery - CMS will Custom CMS - Validation Java Script - Google Map API - API ( FACEBOOK ) https:/ Flere

$1500 USD in 40 dage
(24 bedømmelser)
6.3
sayno2bugs

Highly experienced web scraper developer. My web scrapers scrape more than 10 Million pages each day without getting blocked. Please check my reviews here to know more about the quality you can expect : [login to view URL] Flere

$1500 USD in 10 dage
(30 bedømmelser)
6.4
Max01

Hello, I love to create Web crawler that can find out German and English language websites! using flash on their websites. I see web crawler should generate a list of domains and their e-mail addresses for THAT langua Flere

$1000 USD in 7 dage
(251 bedømmelser)
6.6
Bluesky122

Hi. I am Guicai Y. I have read your post content and I am able to help you. I am web expert using python. I have made more project using it for client. And otherwise I am web scraping expert too. I have more knowledge Flere

$1125 USD in 7 dage
(45 bedømmelser)
6.3
tekzee

Hello, Greeting for the day! I recently came across your job post and feel that I can assist you perfectly. I am a senior web developer with an experience of 8+ years, working proficiently on various domains which in Flere

$900 USD in 7 dage
(37 bedømmelser)
6.3