I need a spider built with following features:
Must spider all URLs we enter in database.
It must spider to endless level : it must keep on spidering all pages it finds on that website and it must enter all outside URLs into its database and start spidering those websites also and do the same with those URLs as well. So it will be endless process.
This spider will crawl Title Tag/Meta Description Tag/Meta Keyword Tag of each web page.
Rest of the features we can discuss in PMB. We dont need any website built - we need just the spider which can be installed on Windows OS and can run as stand alone application.
Please show some of your similar work samples.