This job post is about specific open source web crawler: [login to view URL]
You can find overview setup guide: [login to view URL]
This web crawler looks hard to setup for me because I lack Java ecosystem knowlege and documentation on crawler setup is not very detailed.
The Idea would be to setup working docker image of bubing crawler. Requirements for image:
1) It should be parametererized - it must be possible to somehow pass initial seed of URL's
2) Image should do all Bubing configurations listed in [login to view URL]
3) Configurations should be tailored for 16 vCPU core 64 GB RAM 10 Gbit Network VPS machine.
4) One should be able to run container from this image in AWS EC2 Spot Instances.
5) Once started, container should work immediately - crawl must start on container.
6) Container should store list of crawler page files in some folder. The file of crawler page should contain page URL and all HTML page content.
Additional requirements for job:
1) You have to provide Dockerfile that was used to setup docker image
2) You have to provide short Readme description how docker image behave
2 freelancere byder i gennemsnit $570 timen for dette job
Hi, Hope you are doing well. I have full experience about Java/JavaFX so that I have confident to complete your project perfectly. I will be very happy to discuss about your project via chatting. Thank you.