API Documentation

APIFull Documentation

Overview of the API parameters


GET https://app.apifull.com/api/v1/


Parameter Type # Description
api_key string Required The API Key is the only way to authenticate your application with APIFull. A unique API key will be provided after you signup to one of our plans.

Ex : fgehkwgDgNRbMbIuCvSi4UyDCPP97
url string Required The url of the page you want to scrape.

Ex :
https://app.apifull.com/api/v1/?api_key=fgehkwgDgNRbMbIuCvSi4UyDCPP97&url=https://example.com
js_render boolean Optional Should the JavaScript be executed or not. Set to false by default.
js_code base64 encoded string Optional JavaScript code that you want to execute on the page to scrape (click on a button, scroll down, etc...).
proxy boolean Optional Set proxy=true to bypass difficult to scrape website. With this parameter the API request will hide behind one of our own rotating proxies.
cookies string Optional Set custom cookies on the webpage you want to scrape. Pass a JSON array of cookies.

Ex : [{ "name": "cookie1", "value": "val1" },{ "name": "cookie2", "value": "val2" },{ "name": "cookie3", "value": "val3" }].

WARNING : Do not forget to encode the URI to manage special characters. (Ex "encodeURI" function in Javascript). Wikipedia Documentation about URL encoding




Example : API call to scrape an HTML webpage

The two required parameters to scrape a webpage are your API_KEY and the URL of the webpage you want to scrape.


Each request with this parameter will count as 1 API credit.

Run the following command in a terminal (Replace with your own API Key and the URL of your choice) :
curl -L "https://app.apifull.com/api/v1/?api_key=YOUR_API_KEY&url=https://example.com"

You will directly receive the remote HTML code in your terminal.
<html>
  <head>
     ...
  </head>
  <body>
     ...
  </body>
</html>




Example : API call to scrape an HTML webpage with Javascript rendering enabled

If you are crawling a webpage that requires you to render the javascript on the remote page, we can fetch these pages using a headless browser. This feature is only available on the Business and Enterprise plans. To render javascript, simply set js_render=true and we will use a headless Google Chrome instance to fetch the page.

Each request with this parameter will count as 5 API credits.

Run the following command in a terminal (Replace with your own API Key and the URL of your choice) :
curl -L "https://app.apifull.com/api/v1/?api_key=YOUR_API_KEY&url=https://example.com&js_render=true"

You will directly receive the remote HTML code in your terminal.
<html>
  <head>
     ...
  </head>
  <body>
  <div>Javascript generated or fetched content</div>
     ...
  </body>
</html>




Example : API call to execute a Javascript snippet remotely before scraping the webpage

You can ask the API to execute a Javascript code snippet inside our Headless Chrome instance. Executing your own javascript code can be useful if you want to scroll down the page to get more data from an infinite scroll web page.

Or if you simply want to click a button to access specific data from a popup.

Keep in mind that with this feature, Javascript rendering will be set to true : set js_render=true

To use this feature, you need to add the js_code parameter with your code encoded in base64.

Let's suppose you want to click on a button with the following Javascript code :

document.getElementsByTagName("BODY")[0].style.backgroundColor = 'red';

This line of Javascript code encoded in base64 will become :
ZG9jdW1lbnQuZ2V0RWxlbWVudHNCeVRhZ05hbWUoIkJPRFkiKVswXS5zdHlsZS5iYWNrZ3JvdW5kQ29sb3IgPSAncmVkJzsK

Run the following command in a terminal (Replace with your own API Key and the URL of your choice) :
curl -L "https://app.apifull.com/api/v1/?api_key=YOUR_API_KEY&url=https://example.com&js_render=true&js_code=ZG9jdW1lbnQuZ2V0RWxlbWVudHNCeVRhZ05hbWUoIkJPRFkiKVswXS5zdHlsZS5iYWNrZ3JvdW5kQ29sb3IgPSAncmVkJzsK"

You will directly receive the remote HTML code in your terminal.
<html>
  <head>
     ...
  </head>
  <body>
  <button id="myButton" >My button</button> <!-- Clicked button -->
     ...
  </body>
</html>




Example : API call to scrape an HTML webpage setting your own cookies

You can ask the API to set up custom cookies to the webpages you want to crawl.
Set the cookies parameter with a JSon Array of your cookies's values and keys.

Here is an example showcasing how to structure your cookies in a simple JSON Array :
 [{
  "name": "cookie1",
  "value": "val1"
},{
  "name": "cookie2",
  "value": "val2"
},{
  "name": "cookie3",
  "value": "val3"
}]
We currently only handle name and value of custom cookies.

Warning : Don't forget to encode the URI to manage special characters. (Ex : "encodeURI()" function in Javascript). Wikipedia Documentation about URL encoding

After URL encoding, the above JSON array should look like this :
%5B%7B%22name%22%3A%22cookie1%22%2C%22value%22%3A%22val1%22%7D%2C%7B%22name%22%3A%20%22cookie2%22%2C%22value%22%3A%22val2%22%7D%2C%7B%22name%22%3A%20%22cookie3%22%2C%20%22value%22%3A%20%22val3%22%7D%5D

Keep in mind that with this feature, Javascript rendering will be set to true : set js_render=true

Run the following command in a terminal (Replace with your own API Key and the URL of your choice) :
curl -L "https://app.apifull.com/api/v1/?api_key=YOUR_API_KEY&url=https://example.com&cookies=%5B%7B%22name%22%3A%22cookie1%22%2C%22value%22%3A%22val1%22%7D%2C%7B%22name%22%3A%20%22cookie2%22%2C%22value%22%3A%22val2%22%7D%2C%7B%22name%22%3A%20%22cookie3%22%2C%20%22value%22%3A%20%22val3%22%7D%5D"

You will directly receive the remote HTML code in your terminal.
<html>
  <head>
     ...
  </head>
  <body>
     ...
  </body>
</html>




Example : API call to scrape an HTML webpage using rotating proxies

One of the most frustrating parts of automated web scraping is constantly dealing with IP blocks and CAPTCHAs. APIFull can rotate IP addresses with each request using premium proxies (or Residential proxies). Those proxies almost never get blocked.

To enable Residential rotating proxies, set the proxy parameter to true.

Each request with this parameter will count as 100 API credits (Javascript rendering will be set to true : set js_render=true).

Run the following command in a terminal (Replace with your own API Key and the URL of your choice) :
curl -L "https://app.apifull.com/api/v1/?api_key=YOUR_API_KEY&url=https://example.com&proxy=true"

You will directly receive the remote HTML code in your terminal.
<html>
  <head>
     ...
  </head>
  <body>
     ...
  </body>
</html>




Credit cost by request type

Request type API credit cost
- -
Without JavaScript rendering 1
With JavaScript rendering (default) 5
Rotating Residential with JavaScript rendering 100