Additional work on Dekalb county crawler as discussed.

To deal with the issue of one website acting up, can you change the crawler to do two things? First, can it detect an issue retrieving that page and, if it detects an issue with pulling that page it would do one retry before moving on? Second, if it does move on after two fails could it increment an error counter and display that error counter as part of the "Crawling page 99 with parcel id XXXXXXXX" message. This particular crawler requires a few weeks to complete, and if I see the error counter jump up to an unacceptable level and continue to climb, I can stop it and restart it when things are working again. Other than that, the PIN number I mentioned in the previous e-mail is all I am looking for. Thanks in advance.

Færdigheder: Python

Se mere: i parcel, message crawler, additional work discussed, website change detect, restart page, net additional work, additional work polish native, moving message display, mail crawling, web crawler work explain main features, pin number, detect page change, companies work crawler, work crawler php, detect website working, work counter, bot crawler work, project work library system management college level, page crawler, county, crawler page, mail crawler, work crawler, additional work

Om arbejdsgiveren:
( 6 bedømmelser ) Johns Creek, United States

Projekt-ID: #5996777

1 freelancer is bidding on average $25 for this job


Hired by the Employer

$25 USD in 4 dage
(2 bedømmelser)