Loop through files of remote server
1 visualización (últimos 30 días)
Mostrar comentarios más antiguos
I'm using matlab to perform simulations on a cluster, where the jobs are submitted from a local machine through matlab to the cluster.
A sample submission script might look like this:
%%Get handle to the job scheduler
sched = findResource();
%%Create a job
job = createJob(sched, 'FileDependencies', {'Analysis.m'});
%%Create the tasks
filelist=dir('/dir1/dir2/')
for tidx = 1:length(filelist)
tasks(tidx) = createTask(job,@Analysis, 1, {tidx});
end
%%Submit the job
submit(job)
I'm now trying to obtain and loop through some files on the cluster and run a script on the files, say, Analysis.m
How would I do this to get a file list on the cluster and not on the local machine from which the job-scheduling, and then pass each file to the Analysis.m one at a time?
0 comentarios
Respuesta aceptada
Edric Ellis
el 10 de Oct. de 2011
Perhaps a PARFOR loop on the cluster would be simplest. Something like this:
function x = doStuff
% list the files on the cluster
d = dir( '/path/on/cluster/*.dat' );
% loop over the files - this will spread the work
% among the workers
parfor ii = 1:numel( d )
x(ii) = someFcn( d(ii).name );
end
Then, you need to submit 'doStuff' as a matlabpool job to the cluster, something like this:
job = createMatlabpoolJob( sched, 'MaximumNumberOfWorkers', 4 );
createTask( job, @doStuff, 1, ... );
job = batch( @doStuff, 1, {}, 'Matlabpool', 4 );
4 comentarios
Más respuestas (1)
Walter Roberson
el 10 de Oct. de 2011
You have dirlist.name(idxd) but that should be dirlist(idxd).name
Ver también
Categorías
Más información sobre Startup and Shutdown en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!