Skip to content

Sync Scraper Docs

  1. Create a request to the Sync scraper endpoint with Public API

  2. Endpoint: POST /job/async

  3. Authentication: API KEY required.
  4. Request Body: JSON object describing the job (see example below)
  5. Response: A job object with a unique identifier
Example Request:

Imagine you want to scrape Amazon product title and you need to do it fast and effortlessly and synchronously response.

{
  "website": [
    "https://www.amazon.in/Daikin-Fixed-Copper-Filter-FTL28U/dp/B09R4SF5SP/ref=sr_1_2?_encoding=UTF8&content-id=amzn1.sym.58c90a12-100b-4a2f-8e15-7c06f1abe2be&dib=eyJ2IjoiMSJ9.LpujZ4uISPUK8sa_6yNGVY-3zoi-I7NYK-eHPsE7wGCkuXTtpSz8w4eclgDyER6vnaTNizv9kGMGiKdwparoq8lJaW0Vd17KpYqWHu_4_2GsTsYqHLUj7RlihL7PYGaAeQF7xGwW1TcRhicMypOlur1X1fGG3rEbE8SpaHXxsLRbwd8EOhSc7BNzWvc1RV0pFSB-Mub0Fai7s2kcdQk3ZHYQ33C5a-M944LcIQlaOyAap-kBzWcJD97e-RotIJI87t8jEE7mTeZTI_7RxBca6bwezt24iUGmJ5_wc2h63eE.TtxFyZ-e3P0yqkJWWnvLfgJlMTH3hl6CFmqBb2kB5bA&dib_tag=se&pd_rd_r=03eee08c-1962-45b1-9fb9-7b6b121ad312&pd_rd_w=eZPmI&pd_rd_wg=FeJad&pf_rd_p=58c90a12-100b-4a2f-8e15-7c06f1abe2be&pf_rd_r=B8KXZK2D0MT4ZVR2DHSS&qid=1731507695&refinements=p_85%3A10440599031&rps=1&s=kitchen&sr=1-2"
  ],
  "data_to_scrape": [
    {
      "field": "product_name",
      "selector": {
        "type": "css-selector",
        "selector": "#productTitle"
      }
    }
  ],
}

Here are the key components:

  1. website: The target URL to scrape. (in sync endpoint isn't that possible to schedule multiple URLs, it receives an array but just one element array)
  2. data_to_scrape: Defines what data to extract. Each item specifies a field (the name you want to give the data) and a selector. The selector is an object compound of two components: The type, which can take either css-selector or xpath and selector which is the selector.

Example response:

The API response with the sync job request.

{
  "website": "https://www.amazon.in/Daikin-Fixed-Copper-Filter-FTL28U/dp/B09R4SF5SP/ref=sr_1_2?_encoding=UTF8&content-id=amzn1.sym.58c90a12-100b-4a2f-8e15-7c06f1abe2be&dib=eyJ2IjoiMSJ9.LpujZ4uISPUK8sa_6yNGVY-3zoi-I7NYK-eHPsE7wGCkuXTtpSz8w4eclgDyER6vnaTNizv9kGMGiKdwparoq8lJaW0Vd17KpYqWHu_4_2GsTsYqHLUj7RlihL7PYGaAeQF7xGwW1TcRhicMypOlur1X1fGG3rEbE8SpaHXxsLRbwd8EOhSc7BNzWvc1RV0pFSB-Mub0Fai7s2kcdQk3ZHYQ33C5a-M944LcIQlaOyAap-kBzWcJD97e-RotIJI87t8jEE7mTeZTI_7RxBca6bwezt24iUGmJ5_wc2h63eE.TtxFyZ-e3P0yqkJWWnvLfgJlMTH3hl6CFmqBb2kB5bA&dib_tag=se&pd_rd_r=03eee08c-1962-45b1-9fb9-7b6b121ad312&pd_rd_w=eZPmI&pd_rd_wg=FeJad&pf_rd_p=58c90a12-100b-4a2f-8e15-7c06f1abe2be&pf_rd_r=B8KXZK2D0MT4ZVR2DHSS&qid=1731507695&refinements=p_85%3A10440599031&rps=1&s=kitchen&sr=1-2",
  "html": "raw html here",
  "json_data": {
    "title": "Daikin 0.8 Ton 3 Star, Fixed Speed Split AC (Copper, PM 2.5 Filter, 2022 Model, FTL28U, White)"
  }
}