Contract PHP Developer for Long-Term Contract to Develop and Support Web Crawlers

We are interested in hiring a PHP developer/team on a long-term contract basis who is knowledgeable about the design and development of web crawlers using PHP. We currently have crawlers developed for two retail sites—one with local store information and the other with online store information—and are interested in significantly expanding the number of sites crawled (see listings below). Database used is MySQL database.

The current code is written in PHP and are of two varieties:

1. “On-Demand” Scripts – utilized to crawl sites with specific information and populating a table with that information

2. Nightly Batch Scripts – executed nightly to send and receive data from a website and populating a table with that information

We also have the need for the following scripts/functions:

1. Keyword Matching Script – Script to ensure that duplicate records are not being created for the same information. Code must gauge the proximity of the match, based on an algorithm which matches key words. This script will be utilized to match stores, products, and listings and thus be fairly flexible.

2. Deletion Script – Script which deletes repetitive or expired listings from tables and inserts them into the table of purged records ([purged_record]).

3. Record Matching Script – Code which attempts to match nearly identical records found during the execution of the crawlers to existing records in various tables based on matching criteria. Algorithm will be explained.

4. Local Retailers Store Number Script – Given stores with a brick and mortar presence in the database, will search the brick and mortar stores’ websites to retrieve the store information, specifically the store number (e.g., “BestBuy #48” matching an actual physical location) and any other missing store information not already recorded in the store table.

5. Product Picture Script – Given a product, be able to capture and store the picture of the product in a picture table in the database. Avoid duplication of pictures.

6. UPC Code Matching Algorithm – Match a product based on various attributes (namely Brand and Model and/or Key Words) to a product in the UPC database to retrieve the product’s UPC Code, which will be stored in the product table. Use the Keyword Matching Script to achieve this.

Addendum for Future PHP Scripts:

SITES: We will be expanding the web crawls to search for all items. This listing might include, and are not limited to, Books, Cellular products, Clothing & Shoes, Computers, Food, Gas Prices, Entertainment, Housewares, Jewelry, Music & DVDs, Office Supply, Pets, Services, Travel, Toys, and Wholesale.

Local Shopping Sites:

1. [url removed, login to view]

2. [url removed, login to view]

3. [url removed, login to view]

4. [url removed, login to view]

Online Shopping Sites:

1. [Trademark masked].com

2. [url removed, login to view]

3. [url removed, login to view]

4. [url removed, login to view]

5. [url removed, login to view]

6. [url removed, login to view]

7. [url removed, login to view]

8. [url removed, login to view]

9. [url removed, login to view]

10. [url removed, login to view]

** Note: Furthermore, the Scripts should be flexible enough (extensible) to extend this search to other price comparison websites in the future.

Brick-and-Mortar Companies which have Online Presence:

1. [url removed, login to view]

2. [url removed, login to view]

3. CompUSA

4. [url removed, login to view]

5. [url removed, login to view]

6. [url removed, login to view]

7. B&[url removed, login to view]

8. J&R

9. PC Richards

** Note: For local stores and pricing of products above, you will need to specify a zip code. The script should be able to take a text file of zip codes and run the search accordingly. The script can run in a batch manner with zip-code text file as a feed.


1. For the local online sites, the Script should read zip codes in from a text file and search within a certain radius from that zip code. The text file will contain zip codes separated by carriage returns.

2. The Script should crawl all subcategories under the primary categories on each website. This hierarchy should be retained in the database. The script should be intelligent enough to search down each hierarchy of items categorized under the parent category as specified by each individual site. Hierarchical format will be specified in the tables.

3. Data will need to be captured in the respective product tables, include the category, product, and brand tables without duplication of records. See each table for more information.

4. The script should enable another PHP page to call it On-Demand (i.e., initiated when required) and the data parsed into the appropriate tables, without duplication of records.

5. Contractor will agree to and sign the Service Agreement contract. Finalization of the contract will be dependent on progress during the first week, to ensure a mutual fit.

6. Contractual rate is negotiated for each script/website or a quote provided on a bundled basis. Each payment will be contingent upon meeting milestones throughout the project (i.e., delivering working code). The final payment will be made before the end of the entire project dependent on meeting all requirements and conclusion of testing.

7. Contractor must participate in bi-weekly meetings and provide daily status updates. Once a project starts, you must provide daily updates and must be available on a ad-hoc basis for conference calls via Skype or a toll-number to the United States.

8. Scripts must be periodically backed up, uploaded and tested on our servers. Final acceptance of a script is contingent on testing and approval by management.

9. Contractor will provide project timeline, noting milestones of when various scripts and web crawlers will be delivered. If the contractor anticipates that it will miss the deadline, management must be notified or risk breach of contract. Any delays in the project timeline must be agreed by both parties.

10. Contractor will work closely with PeerShopper™ team members in testing and integrating scripts into existing code.

11. Contractor will provided evidence of previous work using PHP, preferably for developing web crawlers.

12. You must include 4 months' of assistance and upgrades on a weekly basis.

Evner: Databehandling, PHP, Script Installering

Se mere: written service agreement, work php online, work php developer, work online web developer, work online php developer, work online for companies, work online as web developer, work of a developer, working online web developer, working online developer, working online as web developer, working for first data, working for companies online, work for online companies, work for ebay online, who is website developer, who is php developer, who is develop php, who is a php developer, we need a web development companies

Om arbejdsgiveren:
( 5 bedømmelser ) Wilmington, United States

Projekt ID: #41239

6 freelancere byder i gennemsnit $290 på dette job


a placeholder bid, thanx for the invitation, we are interested but this would cost you more, pls revise you budget range,thanx

$300 USD in 30 dage
(34 bedømmelser)

Our bid is 299 $ for the whole project. Bid includes development , deployment and testing. we offer 30 days free support. We are ready start and send daily reports. we communicate through mail,messenger,phone and sky Flere

$240 USD in 15 dage
(22 bedømmelser)

Hello, Please look at the PMB. Thanks, Sergey

$300 USD in 3 dage
(25 bedømmelser)

Hello Sir, we have good experience in website development. Please view [login to view URL]

$300 USD in 10 dage
(68 bedømmelser)

Thanks for the invitation, are ready to cooperate.

$300 USD in 30 dage
(3 bedømmelser)

We are a Toronto, Canada based web development firm and have the team you need for ongoing development of your crawlers. We have exteneive experience working with variety of crawlers in PHP/MySQL and have developed cus Flere

$300 USD in 7 dage
(1 bedømmelse)