Aim
On a Linux High Performance Computing (HPC) system, I am using multiple text files to set variables and array variables in a bash script. These are variables are then the input arguments to a MATLAB function that is run through bash. This MATLAB function runs iteratively creates url download strings (based on the variables mentioned above) and then download corresponding temperature data using MATLAB's websave function. Notably, the MATLAB script uses parfor to parallelise a for loop and spread it over 44 cores on a Linux high performance computing cluster. The MATLAB functions runs fine on the laptop, but extremely slowly, that's why I need to run it from the High Performance Cluster.
Issue
If I open the matlab app via the linux commandline then I can get the function to run. However, I want to send this as a job to a HPC scheduler - meaning I cannot work interactively. Thus, I need to run the function from a bash script.
I am having trouble passing the bash script variables to the matlab function. Note I have tried removing the single quotes from around the variables in the matlab function call, however I get the same error.
Function Example
function f_Download_CRW_Temp(Lat, Long, StartYear, EndYear, SiteNumber, NumCores, FileType)
parpool(NumCores)
parfor d = 1:length(Years)
options = weboptions('Timeout',Inf);
url =
filename =
try
websave(filename,url,options);
catch
end
end
end
Bash Script (not in MATLAB syntax)
cd CRW_Temp_Profiles
# Set Variables
IFS=$'\r\n' GLOBIGNORE='*' command eval 'LatArray=($(cat Lat.txt))'
IFS=$'\r\n' GLOBIGNORE='*' command eval 'LongArray=($(cat Long.txt))'
IFS=$'\r\n' GLOBIGNORE='*' command eval 'SitesArray=($(cat Sites.txt))'
Site=$(echo ${SitesArray[${SLURM_ARRAY_TASK_ID}-1]})
FileType=".csv"
# Check Variables were read properly
echo 'Site is Number '${Site}
echo 'Row-specific LatLons are:'
echo ' deg N = '${LatArray[${SLURM_ARRAY_TASK_ID}-1]}
echo ' deg S = '${LongArray[${SLURM_ARRAY_TASK_ID}-1]}
echo ' All Lats are'
echo ${LatArray[*]}
# run MATLAB function
module load MATLAB
matlab -nodisplay -nojvm -nodesktop -nosplash -r "try; f_Download_CRW_Temp('$LatArray', '$LongArray', "1985", "2019", '$Site', "44", ".csv"); end; quit;"
module purge
Bash Output and Error Message
Site is Number 1
Row-specific LatLons are:
deg N = -16.5
deg S = 149.8
All Lats are
-16.5 -20 -19.5 -11.5 -23.5 -21.5 -31.5 ...
< M A T L A B (R) >
Copyright 1984-2018 The MathWorks, Inc.
R2018b (9.5.0.944444) 64-bit (glnxa64)
August 28, 2018
For online documentation, see https://www.mathworks.com/support
For product information, visit www.mathworks.com.
try; f_Download_CRW_Temp('-16.5', '149.8', 1985, 2019, '1', 44, .csv); end; quit;
Error: Invalid use of operator.