Amazgo:

REST APIs for Amazon scraping using colly.
Contains POST (REST) API in Golang Which scrapes an Amazon web page given its URL The code should scrape following data from the page

• Product Name/Title
• Product image url
• Product description
• Product price
• Product total number of reviews.
  API handler takes the JSON structure from the request payload,
  Handler writes the obtained payload to a new/updated Document,
  Also write timestamp of the create/update within the document.\

To run:-

git clone https://github.com/pratik-anurag/Amazgo.git
cd Amazego
docker-compose up

Sample cURL to insert a single product in mongoDB:-

curl -X POST \
  http://localhost:8000/products \
  -H 'cache-control: no-cache' \
  -H 'content-type: application/json' \
  -H 'postman-token: c6a68fb0-2b20-db8a-d3bc-1bf189dbd567' \
  -d '{
        "Name": "PlayStation 2 Slim Console PS2 Renewed",
        "ImageUrl": "https://m.media-amazon.com/images/I/4116HYcMH0L._AC_UY218_.jpg",
        "TotalReview": "273",
        "Description": "test",
        "Price": "209.99"
    }'

/

Sample cURL to fetch document from given url and insert it to DB:-

curl -X POST \
  'http://localhost:8000/scrapes?url=https%3A%2F%2Fwww.amazon.com%2Fs%3Fk%3Dps2%26ref%3Dnb_sb_noss_2' \
  -H 'cache-control: no-cache' \
  -H 'postman-token: 7a628957-c693-03c0-58cc-0578b677e87e'