Ingest a file using Python 3
This article describes how to ingest a file using Python 3.
The main steps in the code involve using your Tenant API Key (for security measures, this should be a variable, and not hardcoded!) along with the name of the file you want to upload (this may be a variable, in case you want to use the same script to periodically upload data incrementally) to generate a Signed URL.
You then use the generated Signed URL to upload the file to the Peak Data lake.
Prerequisites: Tenant API Key.
Sample Code
#####
# Sending Peak a file via SignedURL using python3
# Please note this is only a quick and simple code example
# Other file types/formats etc. may vary
# No responsibility is taken by the author for the running of the code
####
import requests
import os
import sys
# make sure your tenant api key is set as a variable
# please do not hard code sensitive information in code!
peak_tenant_api_key = os.environ["peak_tenant_api_key"]
file_name = "sample_20181112.csv"
######
## First call - Generating the SignedURL
######
# filename in the payload may be a variable
# use the same script to send incremental data updates
payload = '{"fileName": "' + file_name + '"}'
headers = {'Content-Type': 'application/json', 'Authorization': peak_tenant_api_key}
url = 'https://api.peak.ai/file/signedurl/client'
# make a post request
print('Generating URL...')
first_response = requests.post(url, data=payload, headers=headers)
if first_response.status_code == 200:
print('Status code: ' + str(first_response.status_code) + ' - Success!')
generated_url=first_response.text
else:
print('Status code: ' + str(first_response.status_code) + ' - Something went wrong!')
sys.exit(1)
######
## Second call - Uploading the file
######
# This time the url will be the signedURL we generated
# so we use the result of the first call
# remove the leading and training double quotes
url = generated_url.strip('"')
headers = {'Content-Type': 'application/json'}
# read in the file for upload
payload=open(file_name, 'rb')
print('Uploading file...')
# This time we use a put request to upload the file
put_response = requests.put(url, data=payload, headers=headers)
if put_response.status_code == 200:
print('Status code: ' + str(put_response.status_code) + ' - Success!')
else:
print('Status code: ' + str(put_response.status_code) + ' - Something went wrong!')
sys.exit(1)
Python
Next Steps:
Once the data has been ingested into the Peak data lake, a feed can be created and scheduled.