Scrape API

Program written in whatever language you want that will run on a linux server, communicate with a mysql database, support multithreading, and make use of multiple ips.

1. Write a class to communicate with an HTTP XML api

1. Use this class to scrape listings off the website

1. Support configurable scraping intervals and randomize those intervals to avoid detection

2. Make use of multiple ips on a linux server for the web requests. Website has a minimum polling rate of 2 minutes per ip. Use multiple ips to poll every ~5 seconds.

3. Use pagination.

4. Use regex to extract a value

5. Insert listings into a mysql database

6. Create algorithm to link duplicate listings by the same user together in database

7. Create algorithm to link users with multiple accounts together in database

8. Create algorithm to compare listing prices to local database

9. Compare listings against a configurable table of criteria

2. Create methods to list/edit/post/delete listings on website

3. Create methods to send/receive/list messages on website

1. Insert all conversations into local mysql database

2. Automatically send message with preconfigured text. Attempt to exact answers from any replies

4. Once daily check listings in local database against website for any changes. Insert any changes in a table

5. Integrate Plivo to send/receive text messages

1. Send message with preconfigured text. Attempt to exact answers from any replies.

This program needs to have clear and concise code. Please do not bid unless you are very experienced. I also need a website front end designed and possibly an iPhone app. If you are able to do this as well let me know.

Evner: HTML, Linux, PHP, Python, Software Arkitektur

Se mere: create sharepoint front end access database, dreamweaver create dreamweaver html front end access database, web front end excel database, front end mssql database, create web front end sql database, front end view database, create front end excel database, flags delete post, bulk delete post wordpress, forums pay per post payout daily, front end excel database, vbnet front end access database, html front end access database, phpbb3 users automatically join group post, scrape data website submitting post data php, wordpress automatically hyperlink url post text, web html front end access database, php front end mysql database, web front end sql database, html front end excel database, building front end mysql database

Om arbejdsgiveren:
( 3 bedømmelser ) Regina, Canada

Projekt ID: #12549271

52 freelancere byder i gennemsnit $1198 på dette job


Hello, my name is Olya. I represent Ukrainian IT Company «Webbook». We provide website design and web and app development services for organizations, public and government institutions, company or private web-pages. Flere

$1200 CAD in 20 dage
(109 bedømmelser)

Hi I work towards providing reliable, relevant and robust IT solutions at most competitive prices to my customers. I ensure 100% customer satisfaction so lets start Thanks

$927 CAD in 28 dage
(377 bedømmelser)

We are your perfect outsourcing partner. We have many years of experience in writing web scrapers. Last project we worked on was a movies database, where we had to scrap existing information of more than 13000 movies, Flere

$1200 CAD in 20 dage
(32 bedømmelser)

Hi I am Mustafa from Appxone. I am interested to accomplish your project of hiring people like I will take care of your project from start to end and source code will be provided. My profile:   Flere

$1263 CAD in 20 dage
(57 bedømmelser)

We already work on a product listing and price comparison website, kindly have a look on our similar kind of job and let me know your views: [login to view URL] Here, I am sharing my main tasks what I ne Flere

$1250 CAD in 25 dage
(55 bedømmelser)

Dear Sir/Madam, We are in top 25 company in , we made lots of system and as well also earned more then 1000000+ USD and give us one chance then you will know our level of expertise and quality wor Flere

$1111 CAD in 20 dage
(226 bedømmelser)

I have gone through the requirement and still need some more brief of the project. I have more than 6 years of experience in skills like Ecommerce, Enable Shopping cart, Payment and API Implement/Integration, Javas Flere

$842 CAD in 20 dage
(48 bedømmelser)

My name is Mike and I’m from UK. I work with individual clients and also provide outsourcing services for a number of UK and USA based agencies. Your project description sounds interesting to me and I do have skills & Flere

$1250 CAD in 20 dage
(50 bedømmelser)

Dear Client , We gone through project description We are Really interested to do work with you , We are Web & Software development company having more than 8 year experience in Software development we totally co Flere

$1250 CAD in 20 dage
(204 bedømmelser)

Hi I read your project description carefully It is perfect for me I'm a web site builder and developer of web scrap crawler, spider, bot with python Web site was developed by django which is python framework so tha Flere

$1250 CAD in 20 dage
(56 bedømmelser)

Hi, Im web developer from Poland, I have 5+ years of experience in webdesign/ development, WordPress, OperCart, E-commerce, obile Development, SEO and I have done number of projects. I can handle project indep Flere

$888 CAD in 10 dage
(37 bedømmelser)

Dear Client, Greetings from Flowgica technologies, I have experience with these skills. We do have similar experience doing something similar to yours therefore I am looking forward to discuss more about more details a Flere

$1050 CAD in 24 dage
(41 bedømmelser)

Hello, I understood the initial scope of this project. Although i want to discuss further this job in order to prepare the final concept for this project. After Complete discussion over the call or in chat, i wi Flere

$1546 CAD in 20 dage
(64 bedømmelser)

Hello! We are a professional team of web developers with huge experience in using python for custom webapps based on django and odoo. We are available and will be happy to help you with the project. Looking forward Flere

$1350 CAD in 25 dage
(56 bedømmelser)

Hello, Hope you are doing Good. I am available to start asap. Please lemme know more details about the project. I am 8+ year experienced in Web Designing as well as Development and have linux experience too. Have Flere

$1250 CAD in 20 dage
(197 bedømmelser)

Hello, I have gone through your outsourced requirements "Scrape API" and have analyzed that I have right skills (HTML, Linux, PHP, Python, Software Architecture) to execute your esteemed project. I will complete Flere

$1229 CAD in 365 dage
(123 bedømmelser)

Hi, I can start work on your project immediately once you confirm me and i am also open for further discussion if you want. I have 7+ years in web designing & development with good experience in following skills Flere

$1250 CAD in 20 dage
(81 bedømmelser)

Hi! I am professional C/C++/C#/Java/Python programmer. I can do this project with highest quality! Best Regards, Szymszteinsl

$1500 CAD in 15 dage
(59 bedømmelser)

Hi, We are go through your brief and can see that you would like to build an application. We make a detail review of your given description and understand it very well. We will build with all features as you requ Flere

$1250 CAD in 20 dage
(72 bedømmelser)

Hello ! We want to discuss about your project as we have experience in it. What Differentiates us from the other freelancers : # Experience of more than 5 years in Unity 3D, Xcode, CoCoa 2d, Phonegap and major Flere

$1500 CAD in 20 dage
(64 bedømmelser)