Generic Crawler (PHP & Code Igniter)


I need a crawler that knows to identify products from any site/page (the site owner knows about it!):

1. You get the URL of the homepage, parse it with PHP ganon, find all the images of the homepage and their links that should be the SRC of the closest A parent backwards

2. You group the found images by one of these options IN THIS ORDER (the first that is applicable):

a) Grouped by [url removed, login to view] (or DOM OBJECT POSITION)

b) Grouped by [url removed, login to view] + [url removed, login to view] if there are into the IMG TAG (don't try to reach width and height in PHP at this stage)

c) Grouped by number of directory levels (/) into [url removed, login to view] && number of directory levels (/) into [url removed, login to view]

d) Grouped by margin of similar_text of [url removed, login to view] && margin of similar_text of [url removed, login to view]

e) Grouped by [url removed, login to view] + [url removed, login to view] after reaching width and height of the images via PHP

3. You start with the group of images that have the higher number of images, and you look backwards for the price and the name of the item after checking that the name looks like [url removed, login to view] (in case of doubt open the item page and take the TITLE but it should be rare!)

4. When you find 3 images of the same image's group that have url, name and price, that's all! You will stop the headaches and start to search in the homepage and then in all the sites ONLY the CSS classes that you have just found now

5. For every product, you find the tags from CONCAT([url removed, login to view], '', [url removed, login to view], '', NAME) - a word that appears at least twice is a tag...

You release first homepage results ASAP and IN PARALLEL you run a script that takes other links from the sitemap or from this analyze where you already have a ganon object that will give you all the links of the page easily and quickly

The crawler must be multi thread thanks to Multi Thread CURL of PHP and it will work of course for the most of the site = millions sites, so it should be 1000% generic!

This task is very complex and relevant only for PHP engineers that wants to progress in their career...

Good luck!

Evner: Codeigniter, MySQL, PHP, Software Arkitektur, Web Skrabning

Se mere: mp3 crawler source code php, php crawler product, php code web crawler database, crawler code php, ebay product list code php, php code website crawler, php crawler code, website crawler code php, simple web crawler script code php, product rating reviews open source code php, code php add product, code igniter interface php, php crawler html tags, code web crawler php, php crawler tags contents, add product code php, security code php, mail code php post check, video url embedd code php, convertir code php php, php crawler image title link script, random alphanumeric code php online

Om arbejdsgiveren:
( 5 bedømmelser ) Petach Tikva, Israel

Projekt ID: #12550540

54 freelancere byder i gennemsnit $586 på dette job


Hello, and thanks for the opportunity to bid on your project. I am an expert in many different area’s of web and mobile applications based on the following languages: W Flere

$526 USD in 10 dage
(155 bedømmelser)

Hi I have gone through the details of your project and we find it well within our capabilities. I offer a wide range of services, including Web design, PHP/MySQL web application development, Open sources like Joo Flere

$567 USD in 14 dage
(425 bedømmelser)

Hello, Hope you are doing well !! php codeigniter Experts!! I have gone through your requirements. Lets discuss more over Chat box, So i can define the best suitable time frame to get it finished at the earliest Flere

$750 USD in 10 dage
(156 bedømmelser)

Dear sir, I am scraping expert, I have did too many scraping projects, please check my reviews then you will know. Can you tell me more details? then I will provide example data/script for you. Thanks, Flere

$557 USD in 6 dage
(253 bedømmelser)

Dear Client, Greetings from Flowgica technologies, I have experience with these skills. We do have similar experience doing something similar to yours therefore I am looking forward to discuss more about more details a Flere

$525 USD in 12 dage
(40 bedømmelser)

Dear Sir/Madam, We are in top 25 company in , we made lots of system and as well also earned more then 1000000+ USD and give us one chance then you will know our level of expertise and quality wor Flere

$722 USD in 20 dage
(164 bedømmelser)

Hey Friend, Myself having has 10 + years experience of web site development. Expert in Custom programming in php / Mysql / Jquery / Wordpress/ Joomla / OSC / Magento /OpenCart / Android / Mobile development / R Flere

$580 USD in 10 dage
(198 bedømmelser)

======EXPERTS HERE READY TO START===== Hi, we have a creative and logical web development Professional Team with great skills specialized in WORDPRESS, CSS, HTML, PHP, ONLINE STORES, ECOMMERCE , GRAPHIC DESIGNING , et Flere

$250 USD in 5 dage
(131 bedømmelser)

Hello, my name is Olya. I represent Ukrainian IT Company «Webbook». We provide website design and web and app development services for organizations, public and government institutions, company or private web-pages. Flere

$600 USD in 20 dage
(60 bedømmelser)

Hello, I understood the initial scope of this project. Although i want to discuss further this job in order to prepare the final concept for this project. After Complete discussion over the call or in chat, i wi Flere

$773 USD in 10 dage
(50 bedømmelser)

Hello, Thanks for giving me the opportunity to discuss your project. I am a professional PHP developer with an experience of more than 8 years in creating and customizing PHP based websites, themes and plugins/m Flere

$750 USD in 10 dage
(61 bedømmelser)

Dear client, Thank you for allowing me to post bid on your project. We have an expert team of professional blog, web application, responsive and mobile website developers in [login to view URL] have many satisfied clients Flere

$700 USD in 10 dage
(89 bedømmelser)

Hi there, My name is Momin. I’ve read your brief and can see that I’d like to do this task. My team has 7 years experience designing and developing. I would approach your project by starting with wireframes Flere

$555 USD in 10 dage
(133 bedømmelser)

Dear Client , We gone through project description We are Really interested to do work with you , We are Web & Software development company having more than 8 year experience in Software development we totally co Flere

$555 USD in 10 dage
(154 bedømmelser)

Hello ! We want to discuss about your project as we have experience in it. What Differentiates us from the other freelancers : # Experience of more than 5 years in Unity 3D, Xcode, CoCoa 2d, Phonegap and major Flere

$1000 USD in 10 dage
(56 bedømmelser)

I have gone through the requirement and still need some more brief of the project. I have more than 6 years of experience in skills like Ecommerce, Enable Shopping cart, Payment and API Implement/Integration, Javas Flere

$526 USD in 15 dage
(33 bedømmelser)

Hello, I have gone through your outsourced requirements "Generic Crawler (PHP & Code Igniter)" and have analyzed that I have right skills (Codeigniter, MySQL, PHP, Software Architecture, Web Scraping) to execu Flere

$586 USD in 365 dage
(102 bedømmelser)

Hi There, Ready To Start now. I have 4+ years experience in PHP as well as Wordpress, Magento,OpenCart, Prestashop, OsCommerce….and also in responsive CSS3, bootstrap, html5 . etc You can check our work http://www Flere

$333 USD in 2 dage
(140 bedømmelser)

Hi! I am a experienced developer myself and leads a team of 3 designers and 5 developers. We are specializing in designing and building websites. If you want your website to be more professional, then we are the righ Flere

$650 USD in 10 dage
(95 bedømmelser)

Hi, We are go through your brief and can see that you would like to build a WEBSITE. We make a detail review of your given description and understand it very well. We will build with all features as you required. Flere

$1000 USD in 18 dage
(52 bedømmelser)