Http request is fetching only 100 items

Help Forum
, ,
1

Hi, I am using http activity and storing the output in a string variable then i am using deserialize json array activity. Then i am checking the count of the json variable. It is always giving me the count as 100. Though the data in the server is more than 100. It is only getting the 100 items. Please help

1 Like
2

Hi @Konda_Sai_Charan_Goud,

Which HTTP activity are you using?

Kind regards,
Kenneth

3

Http request

4

Hi @Konda_Sai_Charan_Goud ,

I’ve checked the HTTP Request activity, it does not have any parameter for setting the result limit.

Can you check if the server only gives a max of 100? Is it possible to share the link as well?

Happy automation!

Kind regards,
Kenneth

2 Likes
5

Hi @Konda_Sai_Charan_Goud,

The reason why you’re getting only 100 items after deserializing the JSON array is most likely due to the default page size of the API endpoint you’re calling.

Many APIs limit the number of records returned in each page of the response to a certain number, such as 100. This means that even if there are more than 100 records in the response, only the first 100 records will be returned in the first page.

To retrieve all the records from the API, you’ll need to make multiple requests to fetch all the pages of data. You can usually achieve this by specifying a page number or offset in your API request to retrieve the next set of records.

1 Like
6

@Konda_Sai_Charan_Goud can you show us your body/header which you are passing in http request.

1 Like
7

@ABHIMANYU_THITE1 Can you please elaborate how to achieve this? setting the page number or offset?

8

Sure! When making an API request that returns paginated data, you can typically include parameters in your request that specify the page number or offset of the data that you want to retrieve. Here’s a brief overview of how this works:

  1. Determine the size of the page: The API documentation should indicate how many records are returned per page, and you’ll need to use this information to calculate the page number or offset for the data you want to retrieve.
  2. Include the page parameter in your API request: Depending on the API, you may need to include a parameter in your request that specifies the page number or offset of the data you want to retrieve. For example, some APIs use a page parameter to specify the page number, while others use an offset parameter to specify the starting record for the page.
  3. Retrieve the data for the current page: Once you’ve included the appropriate page parameter in your API request, you’ll receive a response containing the data for the current page.
  4. Determine if there are more pages of data: Check the API documentation to determine if there are more pages of data available. If there are, you can repeat steps 2-3 for each subsequent page of data until you have retrieved all the records.
  5. Combine the data: Once you have retrieved all the pages of data, you can combine the records to get the complete set of data.

Here’s an example of what an API request with a page parameter might look like:

https://api.example.com/records?page=2

In this example, we’re requesting the second page of records from the API. The API would return a response containing the records for page 2, along with metadata indicating if there are additional pages available.

@Konda_Sai_Charan_Goud Note that the specific details of how to retrieve paginated data will vary depending on the API you’re using, so be sure to consult the API documentation for instructions on how to do this properly.