Hi there,
I am a fulltime freelance web development professional and I specialize in web scraping.
I have built a web scraping framework in python and can provide progress reports, on demand task activation, an api and a multitude of output formats via a web based front end.
To accomplish your task would require only a minimum configuration and setup within my framework as it already handles scraping scenarios quite similar to yours. However, I do write bespoke custom scrapers for clients on a regular basis should that be required.
I feel I would be a good match for your project as I can, going forward, offer you a degree of control as to how and when you run your scrapes as well as being able to monitor the progress. At the same time I would still be able to maintain the scraper code and add to and amend it as required.
Looking forward to hearing from you.
All the best,
Ulfur