Research off the shelf solutions for a back end batch system (scraping application)

  • Status: Closed
  • Præmier: $170
  • Modtagne indlæg: 2
  • Vinder: TheGuyver2040

Konkurrence Instruktioner

The task is to research and suggest possible off the shelf solutions for a back end batch system (scraping application) that fulfill the following requirements:

Business Objectives:
Get competitors product information for branded shoes on a daily basis. Information like Product name, Product colorname, other product attributes example materials, list of competitors, Competitor prices (Black and discounted prices)).

Proposed Work Flow:
The system fetches a product list from our system with EAN codes (for identification). The system then uses the EAN in google shopping to fetch product information and prices per competitor.

Required Features:
Scraping engine to handle over 15.000 requests per day. The normal request per day will be 50-500. The information should be structured on model level and SKU / EAN level (ie we send over/fetch one size/EAN to scrape).


Inputs:
There will be a product feed into the system from our product system. Alternativly we want to be able to fetch the data through a feed / API.


Dependencies:
The system should have an easy way to export / import information preferably using some API / feed.

Integrations:
There should be an API available. To fetch the scraped products. And possibly what products of interest to scrape.

Server / Hosting:
AWS on a Kubernetes server, if hosted on our servers. Otherwise software as a service is fine.

User Count:
Performance is hard to estimate but roughly 50-500 daily requests but up to 15.000 when going through our total range of products.

Deliverable:
A summary of the system you have found with a short explanation of its functionalities and how it can be used to fulfil our requirements

Anbefalede færdigheder

Arbejdsgiverfeedback

“It was great! ”

Profilbillede Footway, Sweden.

Bedste indlæg fra denne konkurrence

Se flere indlæg

Offentlig Præciserings Opslagstavle

Ingen beskeder endnu.

Sådan kommer du i gang med konkurrencer

  • Opret din konkurrence

    Opret din konkurrence Hurtigt og nemt

  • Få tonsvis af indlæg

    Få tonsvis af indlæg Fra hele verden

  • Tildel det bedste indlæg

    Tildel det bedste indlæg Download filerne - Nemt!

Opret en Konkurrence Nu eller slut dig til os i dag!