Borrar filtros
Borrar filtros

How to open a file in the remote server, read the contents, and save it to work space

72 visualizaciones (últimos 30 días)
Hi,
I have some large size NetCDF files store on a remote server. Those files are at least 100GB, and that is just the data for 1 day. I would need data for multiple days for some analysis. I certainly do not want to download each file to the local.
Is there any way that I can connect to the remote server from local Matlab, open these NetCDF files, grab the variables I need, and load them to the local workspace?
I have tried using
unix('ssh xxxxx@xxxx.gov')
It does connect me to the server. However, I do not know how to open the NetCDF files and grab the variables to my local workspace. Is this even doable?
Thank you very much for your help!
  5 comentarios
Ching An Yang
Ching An Yang el 10 de Jul. de 2023
Hi Walter, thank you! I learn a lot from your comments. I am not sure which case is the server, is there a way to tell? I asked some collegues and one told me that server that requires Duo when log in would not work.
Walter Roberson
Walter Roberson el 11 de Jul. de 2023
I am not sure what Duo is: is that the Single Sign-On security software?
Your reference to xxxx.gov suggests a US government site. US government sites tend to support OpenDap for science data. If, however, you happen to be working with data that is restricted access (possibly classified) then I do not have any information about what various US government sites support.

Iniciar sesión para comentar.

Respuestas (1)

Aditya
Aditya el 17 de Nov. de 2023
Hi Ching,
I understand that you want to retrieve some data from the NetCDF files present on the remote server.
For this you can follow these steps:
  1. Connect to the remote server using ‘ssh’ command via the ‘system’ function.
  2. Navigate to the directory where the NetCDF files are located using the ‘cd’ command via the ‘system’ function.
  3. Launch MATLAB command line interface on the remote server using the ‘matlab -nodesktop’ command via the ‘system’ function.
  4. Use NetCDF related commands like ‘ncread’ and ‘ncinfoto read and retrieve information from the files.
  5. Save the extracted variables to a file using MATLAB’s ‘save’ function.
  6. Exit the remote MATLAB session using ‘exit’ command.
  7. Download the extracted data file to your local maching using the ‘scp’ command via the ‘system’ function.
You can refer to the below links for the above-mentioned functions:
Hope this will resolve the issue!

Productos


Versión

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by