How to load or display content of the file from AWS S3

31 visualizaciones (últimos 30 días)
Mukesh Dangi
Mukesh Dangi el 29 de Jun. de 2018
Respondida: Mukesh Dangi el 3 de Jul. de 2018
I have following code which load files from AWS s3 however i'm not sure how to display os access the these files one by one: -
%Set S3 Credentials
setenv('AWS_ACCESS_KEY_ID', '{ID}');
setenv('AWS_SECRET_ACCESS_KEY','{KEY}');
setenv('AWS_REGION', 'us-east-1');
%Load Data
imds = fileDatastore('s3://{bucket}/',...
'FileExtensions',{'.txt', '.pdf'},...
'ReadFcn',@load,...
'IncludeSubfolders',true);
file=imds;
disp(file);
output
FileDatastore with properties:
Files: {
's3://{bucket}/2017%20EBA%20-%20Relets%20(part%201)%20-%20signed.pdf'
's3://{bucket}/AQS94%20-%20Official%20Website.pdf'
's3://{bucket}/summary.txt'
}
UniformRead: 0
ReadFcn: @load
AlternateFileSystemRoots: {}
I tried
file=imds.Files;
disp(file);
However it's not displaying the content of the file.

Respuesta aceptada

Mukesh Dangi
Mukesh Dangi el 3 de Jul. de 2018
Finally. Well following worked :
fileName = strrep(fileName,"s3://","https://s3.amazonaws.com/");
%fileName ='https://s3.amazonaws.com/{bucket}/summary.txt';
options = weboptions('ContentType','text');
data = webread(fileName, options);

Más respuestas (1)

Hatem Helal
Hatem Helal el 2 de Jul. de 2018
You need to call read on the datastore. Its hard to make a specific recommendation, but if your goal is to open the txt and pdf files in your S3 bucket locally: you could try using the open command instead of load as the ReadFcn of your fileDatastore. Something like the following
fds = fileDatastore('s3://{bucket}/',...
'FileExtensions',{'.txt', '.pdf'},...
'ReadFcn',@open,...
'IncludeSubfolders',true);
while hasdata(fds)
read(fds)
end
  1 comentario
Mukesh Dangi
Mukesh Dangi el 2 de Jul. de 2018
Editada: Mukesh Dangi el 2 de Jul. de 2018
Thank You for the suggestion. Unfortunately due to some reasons hasdata() is giving an error stating the provide valid input. I'm not sure whether i should use fileDataStore or not. My in objective is just to get the files from S3 and play{/r/w} with those files. The problem with my code is that it's giving me following output and i'm unable to get the file individually from it. I'm not sure whether it's list
Files: {
's3://{bucket}/2017%20EBA%20-%20Relets%20(part%201)%20-%20signed.pdf'
's3://{bucket}/AQS94%20-%20Official%20Website.pdf'
's3://{bucket}/summary.txt'
}
UniformRead: 0
ReadFcn: @load
AlternateFileSystemRoots: {}
Moreover My code is able to fetch the files however unable to display desired results which is content of the file.
function handle()
%Set S3 Credentials
setenv('AWS_ACCESS_KEY_ID', '{ID}');
setenv('AWS_SECRET_ACCESS_KEY','{KEY}');
setenv('AWS_REGION', 'us-east-1');
%Load Data
imds = fileDatastore('s3://{buket}/',...
'FileExtensions',{'.txt'},...
'ReadFcn',@load,...
'IncludeSubfolders',true);
data=imds.Files{1};
AWSRead(data);
end
function AWSRead(fileName)
fileID = fopen(fileName,'r');
txt= textscan(fileID, '%s');
fclose(fileID);
whos txt
end
Error : Error using textscan
Invalid file identifier. Use fopen to generate a valid file identifier.

Iniciar sesión para comentar.

Categorías

Más información sobre Data Import and Analysis en Help Center y File Exchange.

Etiquetas

Aún no se han introducido etiquetas.

Productos


Versión

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by