S3 - Testing

For tests that run against the AWS S3 service you need to provide the relevant credentials to S3. The most typical way to provide these credentials is with the usage of environment variables, e.g.

export PEKKO_CONNECTORS_S3_AWS_CREDENTIALS_PROVIDER=static
export PEKKO_CONNECTORS_S3_AWS_CREDENTIALS_ACCESS_KEY_ID="my key"
export PEKKO_CONNECTORS_S3_AWS_CREDENTIALS_SECRET_ACCESS_KEY="my secret"
export PEKKO_CONNECTORS_S3_REGION_PROVIDER=static
export PEKKO_CONNECTORS_S3_REGION_DEFAULT_REGION=eu-central-1

Utilities

Guardian provides a utility to help deal with running S3 related tests. Due to the possibility of this tool making unintentional consequences to your S3 account, it needs to be manually run in sbt. To run the tool without any parameters do this

sbt "coreS3/test:runMain io.aiven.guardian.kafka.s3.Main"

Current commands

• cleanup-buckets: Helps in cleaning up S3 buckets that have been inadvertently left over by tests.

Tagging S3 Tests

Due to a current limitation where there is no way to expose Github secrets to PR’s made from external forks, tests which run against S3 need to be Tagged using RealS3Available.