Not sure what’s your business case which is demanding for returning all the assets.
But anyway using sub folder is possible through the request header of assets endpoint which returns you with the assets only belong to that particular folder.
I want in fact to only return all assets that belong to a certain category (i.e. assets that store the results of the transactions). I was thinking of grouping those assets using a sub folder in order to achieve that. If there is a better option I am open to suggestion .
You can send the data scraped by bot as an out argument from the main workflow and the same can be received at the API call side instead of saving it in assets.