Integration With S3
Pulling a dataset from S3
You may want to pull a dataset directly from S3. This may be used either to upload a baseline dataset, or to publish production traffic to Fiddler.
You can use the following code snippet to do so. Just fill out each of the string variables (S3_BUCKET
, S3_FILENAME
, etc.) with the correct information.
Uploading the data to Fiddler
If your goal is to use this data as a baseline dataset within Fiddler, you can then proceed to upload your dataset (see Uploading a Baseline Dataset).
If your goal is to use this data as a batch of production traffic, you can then proceed to publish the batch to Fiddler (see Publishing Batches of Events ).
What if I donβt want to hardcode my AWS credentials?
If you donβt want to hardcode your credentials, you can use an AWS profile instead. For more information on how to create an AWS profile, click here.
You can use the following code snippet to point your boto3
session to the profile of your choosing.
What if I don't want to load the data into memory?
If you would rather save the data to a disk instead of loading it in as a pandas DataFrame, you can use the following code snippet instead.
Last updated