Is it possible to set a remote folder on HPC as a current folder?
2 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Hi everyone!
I would like to know if there is any way to set the current folder on a remote location. I am working using matlab parallel server and since my data is huge I put data and functions inside a folder DATA locatd in the cluster and send job by batch as follow:
c = parcluster('short');
job = batch(c, 'forHPC', 'Pool', 27, 'CurrentFolder', '~/DATA', 'AutoAddClientPath', false, 'AutoAttachFiles', false);
It perfectly works without transferring data between my computer and workers and finally it saves outputs in the folder DATA in the cluster.
I want to do the same thing in the interactive mode inside Matlab. Put differently, I would like to work with matlab interactively on my computer while my workers and current folder are remote to avoid trasferring data. Is it possible?
Any idea is highly appriciated.
0 comentarios
Respuestas (1)
Raymond Norris
el 15 de Dic. de 2020
Can you provide a bit more info on how you'd like this to work. Given that you're using parcluster, batch, etc., I suspect you're also familiar with parpool -- is that what you mean by running MATLAb in interactive mode? Would the following work?
c = parcluster('short');
p = parpool(c,28);
pctRunOnAll cd ~/DATA
forHpc
forHpc runs locally, except when you hit parfor, spmd, etc. That code (e.g. parfor) will be run on the cluster and the workers will run in ~/DATA. Any code in forHpc that is not embedded within a parfor or spmd block, distributed array, will be run on your local machine.
Does this address what you're asking? You didn't mention if your local machine and the cluster share a mount disk. Is ~/DATA reachable from your local machine?
7 comentarios
Raymond Norris
el 19 de Dic. de 2020
Can you post forHpc? I have a thought or two, but would need to see what the code looks like.
I won't be checking this forum till the new year. Others may be able to provide some guidance as well.
Ver también
Categorías
Más información sobre Parallel Computing Fundamentals en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!