
Closed
Posted
Paid on delivery
I need a developer to collect data from multiple public websites and deliver it in a clean, structured format. This is for legitimate data extraction from publicly available pages. I will share the target URLs and exact data fields with shortlisted candidates. Scope of work Scrape data from multiple public websites (details shared after shortlisting) Extract specific fields consistently and handle pagination/filtering where needed Normalize/clean the data (remove duplicates, consistent formatting) Export results to CSV/Excel/JSON (format to be confirmed) Provide a repeatable solution (script or small app) that I can run on demand Basic documentation: how to run it, how to adjust settings, where outputs go Quality requirements Reliable scraping with error handling and retries Respectful request rate / throttling to avoid overloading sites Clear logging (success/fail, pages processed) Ability to adapt if page structure changes Experience with Python (Scrapy/BeautifulSoup/Selenium/Playwright) or Node.js Proxy / rotating user-agents experience (only if needed) Scheduling/automation (cron, Docker, or cloud run) Deliverables Working scraper + instructions Sample output file(s) Final dataset from agreed sources (initial run) To apply, please include Examples of similar scraping work you’ve done
Project ID: 40209152
172 proposals
Remote project
Active 11 days ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
172 freelancers are bidding on average $143 USD for this job

Hello, With vast experience in web services, including web scraping and data extraction, my team and I at Our Software are highly skilled in the relevant technologies required for this project - in your case, JavaScript. We have extensively worked with data normalization, filtering, and exporting as per clients' specific requirements. Our priority isn't just delivering results but providing well-documented, repeatable solutions that you can use any time you need them. Quality is also key to us. We ensure successful scraping through error handling and retries, respectful request rates to avoid overloading sites, and clear logging for easy tracking of progress. If required, we can adapt our system to any changes in page structures promptly. Finally, through my Website Development background, I'm familiar with the use of cron jobs, Dockerization, as well as other scheduling/automation techniques which will help streamline the entire scraping process. Entrusting me with this task will not only guarantee a working scraper but also a satisfied you! Together, we can turn your vision of clean structured data from multiple public sites into a tangible reality that propels your business to new heights again. Thanks!
$130 USD in 2 days
7.7
7.7

Hi I have expertise in Web Automation using Python and can assist you with developing customized scripts/programs for extracting data from your targeted websites, into structured format such as CSV, Excel or JSON. I'm available to discuss details in chat and can start right away. Abdul H.
$100 USD in 2 days
7.8
7.8

I have extensive experience in JavaScript, Python, Data Processing, Web Scraping, and Puppet, making me a perfect fit for the project "Build Separate Web Scraping Tools for Multiple Public Sites -- 2". The budget can be adjusted after discussing the full scope, and I am committed to working within your budget. Please review my 15-year-old profile to see my past work. Your satisfaction is my top priority, and I am ready to start working on the project immediately. Let's discuss the details and get started right away. Looking forward to your response.
$175 USD in 7 days
7.5
7.5

With extensive experience in web automation and data scraping using languages like Python and JavaScript, I am confident that I can provide you with the robust, clean, and consistent data you need for your project. I have previously worked on similar projects utilizing tools such as Scrapy, BeautifulSoup, and Selenium to extract specific fields from multiple sources while accommodating pagination, filtering, and potentially changing page structures. Furthermore, I understand the significance of data normalization and cleaning for reliable analyses. My expertise in backend technologies like PHP (Laravel, CI, CakePHP, Yii), Node.js and Python/Django would be instrumental in delivering your desired clean, structured datasets for every agreed-upon source. Moreover, my proficiency in various CMS platforms like WordPress will aid me in generating good documentation with clear instructions on how to run and adjust the solution I provide. In addition to this adherence software quality measures, you can also expect clear logging and meticulous error-handling from me. Ultimately, my goal is to provide a repeatable solution allowing you to efficiently run the scrapers on-demand while catering to any necessary adjustments along the way. Let's make your data journey more effortless together! With Regards!
$250 USD in 7 days
7.5
7.5

Hello, Hope you are doing great, i am expert in web scraping , I can easily scrape all the target data from the website using Python or any other script so you don't have to spend any time or effort doing it manually. Plus, I provide quality results quickly and efficiently within your budget. Lets connect through chat for further detailed discussion, i can start the work right after the discussion., thank you Gaurav D.
$200 USD in 7 days
7.1
7.1

Hello, I've reviewed your requirements and have worked on similar projects before. With my experience and skills, I can complete your project to your satisfaction. Please contact me via chat to discuss the details. Thank you.
$140 USD in 12 days
7.3
7.3

As a seasoned developer with more than 12 years of experience, I have honed my skills in web scraping using various tools including Scrapy, BeautifulSoup, Selenium and Playwright in Python and NodeJS. This proficiency has not only allowed me to extract data from numerous public websites, but also enabled me to handle pagination and filtering seamlessly. My acute attention to detail means I can ensure the extracted data is consistently structured and formatted, efficiently removing duplicates. Reliability is a cornerstone of my work ethic. I create scrapers that throttle request rates and employ faulty page structure contingencies while logging successes and failures diligently. With an aim to provide a repeatable solution, I will build you a script or small app that aligns with your needs and is complemented by thorough documentation - affording you full control over the process. Additionally, if needed, my expertise with proxies and rotating user-agents ensures respectful scraping without overloading target sites. An added advantage of working with me is my familiarity with technologies like cron jobs, Docker, cloud runs etc which allows for seamless scheduling and automation. Moreover, having developed applications in diverse domains allows me to be adaptable to any changes in your target sites structure. Thanks....
$250 USD in 7 days
7.3
7.3

As a highly skilled and experienced web developer with a speciality in JavaScript and web scraping, I am confident I can deliver a top-quality, structured solution for your data extraction needs. My extensive experience in building websites from scratch, utilizing various languages and technologies like Scrapy, BeautifulSoup, Selenium, etc., equips me with the profound understanding of extracting information from public sites in a clean-cut manner. In particular, my dedication to error handling and retries ensures that the scraping process remains reliable even in the face of unexpected hurdles or page structure changes. Additionally, my knowledge of proxy/rotating user-agents if needed assures respectful request rate which would save you from overloading sites. Another area I excel at is providing clear logging which keeps you informed about the success/failure of pages processed during scraping. To prove my competence hands-on, I have some sample output files ready from similar past projects that I am willing to share. Additionally, my ability to schedule and automate tasks using cron, Docker or cloud run provides extra convenience for maintaining repeatable solutions that can be run on demand. So let's embark on this project together; with my skills and your unique data requirements, we can build an efficient web scraping tool that makes data collection easier than ever before!
$120 USD in 2 days
6.7
6.7

As an experienced professional with over 13 years specializing in developing customized python web automation, scraping, and AI solutions, I’m confident I have the skills and expertise to successfully execute your web scraping project. With a deep understanding of crucial tools in Python such as BeautifulSoup, Scrapy, Selenium, and Playwright - and the ability to handle clean structures for all extracted data - I can deliver quality results promptly. Throughout my career, I've executed similar scraping projects to extract specific fields consistently while handling pagination/filtering as well. Data normalization and cleaning are also matters I prioritize to ensure your final dataset is free from duplicates and uniformly formatted. Exporting the results to your preferred format - CSV/Excel/JSON - is also within my expertise. Overall, choosing me means choosing a reliable developer who understands the importance of respecting sites' request rates to prevent overloading along with having clear logging procedures for every stage (success/failures). With adaptability being a key trait considering evolving site structures, I assure you of my capacity to pivot accordingly.
$100 USD in 1 day
7.1
7.1

With years of experience in data extraction and processing, I can confidently state that I am the best fit for this project. I have an extensive understanding of Python, which includes proficiency in Scrapy, BeautifulSoup, Selenium, and Playwright - all of which are beneficial for scraping the web. My earlier work attests to my ability to adapt quickly to changing page structures, ensuring that the functionality of the scraping tools remains unhindered. One vital quality that sets me apart from others is my attention to detail. For your project, this means I will ensure consistent formatting and clean data by removing duplicates and normalizing it before delivering it to you in a structured format (CSV/Excel/JSON). With a focus on creating a repeatable solution, I will provide a script or small app that you can effortlessly run on demand. Lastly, my understanding of scheduling/automation (cron, Docker, or cloud run) will be valuable as it aligns with the need for a dependable scraping process with clear logging and respectful request rate to avoid overloading the sites. Please consider me for this project; I guarantee meticulous delivery of quality results well within your timeframe.
$140 USD in 7 days
6.4
6.4

We can build a reliable, repeatable web scraping solution to extract structured data from multiple public websites with proper pagination handling, normalization, and clean exports (CSV/Excel/JSON). We work with Python (Scrapy, BeautifulSoup, Playwright/Selenium) and Node.js, implement throttling, retries, logging, and error handling, and design scrapers that are easy to maintain if page structures change. The final delivery will include a runnable script/app, clear documentation, and the initial dataset from agreed sources. Can you share the target URLs, required data fields, and preferred output format so we can confirm the best tech stack and approach?
$200 USD in 7 days
6.9
6.9

Leveraging my extensive experience in data processing and web scraping, I am confident in my ability to provide you with a tailored solution for your specific needs. Having worked on numerous projects similar to yours, I've honed my skills with Python libraries like Scrapy, BeautifulSoup, and Selenium - ensuring efficient and reliable data extraction, handling pagination/filtering impeccably. Additionally, my knowledge in Automation and Web scheduling (such as cron, Docker, or cloud run) will allow me to deliver an adaptable and repeatable solution that can be effortlessly used on demand. Moreover, my dedication to maintaining data integrity through normalization and cleaning will ensure you receive a precise and clean dataset. My proficiency in exporting results to CSV/Excel/JSON will guarantee compatibility with your desired format of data delivery. In conclusion, my technical expertise combined with my ability to adapt quickly to structural changes on websites makes me an ideal candidate for this project. I'm confident that I can meet all your quality requirements by implementing respectful request rate/throttling and employing clear logging protocols. With me on board, you can expect not only efficient deliveries but also an enriching collaboration where I communicate regularly, provide updates on progress and incorporate your feedback throughout the process.
$150 USD in 6 days
6.0
6.0

Hello there, I have solid experience scraping public websites and delivering clean, well-structured data. I can handle pagination, data cleaning, exports, and build a repeatable Python solution with proper logging and error handling. I follow best scraping practices and provide clear documentation. Thank you
$250 USD in 2 days
6.2
6.2

I'm Abhishek, a Senior Web & Mobile Application Developer with over a decade of experience and your go-to person for your web scraping needs. My expertise spans across several fields that are directly relevant to your project such as Web Development, AI/ML and Python, ERP/CRM systems, CMS and E-commerce, Mobile App Development, and most importantly Cloud Integration. I have a commendable command over the listed tools - Scrapy, Selenium, BeautifulSoup - and their use in conjunction with Python makes me highly effective in the task of data extraction from diverse websites. My advanced understanding of scheduling/automation tools like Cron Jobs, Docker, and cloud run could be an added advantage if you ever decide to take automation to the next level. My commitment to delivering scalable yet secure projects is integral to any data scraping endeavour. I prioritize the client's contentment and realize it through clear communication, fast responses, on-time delivery while ensuring long-term support. With my meticulous work approach and ability to adapt, rest assured any minor or major changes from select sites won't disrupt your data flow. Overall, I believe my skill sets are a perfect fit for your project and I look forward to working together towards its successful completion. With Regards!
$140 USD in 7 days
6.3
6.3

Hello there, I can build a reliable, repeatable scraping solution to collect data from multiple public websites and deliver it in clean, structured formats (CSV/Excel/JSON). I’ll extract the exact fields you specify, handle pagination and filters, normalize and de-duplicate data, and include solid error handling, throttling, logging, and retries. The scraper will be easy to run on demand, with clear documentation on configuration and outputs, and can be scheduled via cron/Docker if needed. I have strong experience with Python (Scrapy/BeautifulSoup/Selenium/Playwright) and Node.js and can adapt quickly to structure changes. Regards, Rahul
$180 USD in 7 days
6.0
6.0

Greetings! I am an experienced web scraping developer well-versed in Python (Scrapy/BeautifulSoup/Selenium) and Node.js. I will extract, clean, and deliver data from multiple public websites in a structured format. With reliable scraping, respectful request rates, and clear logging, I will create a repeatable solution for you to run on demand. Let's work together to build a scraper that meets your quality standards and delivers accurate results. Looking forward to your feedback on the proposal.
$155 USD in 1 day
5.7
5.7

Hi, I’ll build you a reliable, repeatable scraper that pulls data from your target sites, cleans it up, and exports it in your preferred format, whether it's CSV, Excel, or JSON. I use Python + Playwright (handles dynamic content, pagination, and structure changes gracefully) with smart retries, rate limiting, and clear logging so you always know what’s working. The script will be easy to run on demand, and you can tweak selectors or fields via config. So no code changes needed. You’ll get the full script, setup instructions, sample output, and the initial dataset. I’ve done many similar jobs and examples are already on my portfolio. And yes, I offer FREE post-delivery support if a site changes or you need to adjust the output later. Just send the URLs and fields, and I’ll check them and get started right away.
$150 USD in 3 days
5.8
5.8

https://www.freelancer.com/projects/data-scraping/Automated-Counterfeit-Detection/reviews Dear. Nice to meet you. I am very pleasure to submit my proposal on your scrapping and automation project. I have many experiences in these field using python. Recently, I developed Automated Counterfeit Detection and Reporting System on Amazon. You can check this in my portfolio. I am sure and I can start immediately. I will wait for your good news. Thank you.
$140 USD in 3 days
5.6
5.6

Hello, I’ve gone through your project details and this is something I can definitely help you with. I have 10+ years of experience in mobile and web app development, and extensive expertise in web scraping technologies such as Python with Scrapy, BeautifulSoup, and Selenium. I focus on reliable data extraction, ensuring a clean, structured format while respecting request rates to avoid overloading your target sites. I will create a tailored solution that handles pagination, normalizes the data, and exports it in the desired format. Additionally, I'll provide clear documentation on how to run and adjust the script, ensuring you have a repeatable solution. Here is my portfolio: https://www.freelancer.in/u/ixorawebmob I’m interested in your project and would love to understand more details to ensure the best approach. Could you clarify: 1. What specific data fields are you looking to extract? What specific data fields are you looking to extract? Let’s discuss over chat! Regards, Arpit Jaiswal
$155 USD in 25 days
5.4
5.4

Hi there, I’ve read your project on building separate, repeatable web-scraping tools for multiple public sites and delivering clean, structured data. I’m confident I can deliver a robust, maintainable solution using Python (Scrapy/BeautifulSoup/Selenium/Playwright) or Node.js, with solid error handling, respectful throttling, clear logging, and easy run instructions. What I’ll deliver: a modular scraper architecture with per-site extractors, a normalization pipeline to remove duplicates and standardize formats, and flexible output options (CSV/Excel/JSON). You’ll receive a small app or CLI you can run on demand, plus basic docs on running, adjusting settings, and output destinations. I’ll include optional Docker/cron scheduling so you can automate runs in your environment. The pilot will cover the initial set of URLs and fields you share, followed by a scalable plan to add more sites. Proposed approach: define a reusable data model, implement site-specific parsers, incorporate retry logic and polite throttling, implement proxy handling if needed, and generate reproducible outputs with sample files from the first run. I’ll also provide a changelog and logging for success/failure and pages processed. Timeline: initial setup and pilot run within 7-9 days, then iterative refinements. Final dataset delivered after validation and your confirmation of fields/formats.
$100 USD in 2 days
5.4
5.4

montreuil, France
Payment method verified
Member since Apr 28, 2016
$250-750 USD
$30-250 USD
$30-250 USD
$30-250 USD
€8-30 EUR
$10000-20000 USD
$30-250 USD
€250-750 EUR
$30-250 USD
$750-1500 USD
$30-250 AUD
₹12500-37500 INR
₹1500-12500 INR
$15 USD
$10-30 USD
$250-750 USD
₹1500-12500 INR
$25-50 USD / hour
$30-250 USD
₹12500-37500 INR
$30-250 USD
₹600-1500 INR
₹600-1500 INR
$10-30 USD
$250-450 USD