Set up Datastore for S3 Archive, with Role Based credentials

I have an AWS S3 Archive and would like to access data. However, the permissions to access the S3 archive are Role Based, not individual accounts. In that case, how do I set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY?? Thanks.

[Piyush Kumar]

Hi Kevin,

By "S3 Archive", are you referring to using S3 for storing data that is not accessed frequently, or to the data stored in S3 Glacier or S3 Glacier Deep Archive tiers, which are intended for archival purposes?

[Kevin]

S3 for storing data. NOT Glacier.

[Piyush Kumar]

Hi Kevin,

If the permissions to access the S3 archive are Role-Based, you would not need to set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for each user. Instead, you may create an IAM role that has the necessary permissions and attach it to your EC2 instance. This way, any application running on the instance will have the necessary permissions to access the S3 archive.

If you want to access S3 from EC2, these documentations should help -

• https://repost.aws/knowledge-center/ec2-instance-access-s3-bucket
• https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html

[Kevin]

We have an IAM role that is used for the instance and I can access the S3 bucket using AWS CLI. I successfully copied an file from the S3 bucket to my local drive on my instance. When I try to copy the same file from MATLAB using copyfile(s3file,localDir) I keep getting the following error: A low level error has occurred. The s3file is the bucketpath. Any advice on how to copy data from an S3 and access h5 file content when we have the IAM role attached to the instance?