Collect cmorph data through matlab command window

I'm trying to extract precipitation data for every day of the month for specific latitudes and longitudes through the matlab command window. Just below I will insert the script I am using along with the matlab error response.

I would be very grateful for the help.

% Download the CPC data used in the script below
% Directory
% https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/2000/01/
% Directory with all years
% https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/

year_start = 20010101; % first year to download
year_stop = 20010131; % last year to download
nb_of_years = year_stop - year_start +1;

httpsUrl = "https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/2001/01/";

filename_tmp="temp.nc"; % temporary filename (for loop); data will be always stored under this temp name

LG=[203.875	204.25	204.375];  % longitude (NB range is 0 : 360°)
LT=[3.875 3.875 3.875];     % latitude

% export results to excel 
filename_export='PRP_CMORPH.xlsx';

%% main loop 
for ci = 1:nb_of_years
    
    clear Cmorph
    
    filename = strcat("CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_", num2str(year_start+ci-1), ".nc"); % create filename according to list above
    disp(strcat(filename, "  is processed"));
    
    dataUrl = strcat(httpsUrl, filename);
    data = webread(dataUrl);
    
    filename_out = websave(filename_tmp,dataUrl); % save downloaded data
%     ncdisp(filename_tmp,'/','min');      % commented, just to avoid filling the command window
    cmorph=ncread(filename_tmp,'cmorph');  % Dimensions: lon,lat,time
    long=ncread(filename_tmp,'lon');
    lat=ncread(filename_tmp,'lat');
    time=ncread(filename_tmp,'time');
    for i=1:3 
        LGG=find(long==LG(i));
        LTT=find(lat==LT(i));
         if isempty(LGG)||isempty(LTT)
            disp('LOCATION NOT FOUND IN THIS NetCDF FILE')
            break
         end
        % Step 5 :Convert 3 dimensional array to column vector array for one station
        Cmorph(:,i)=cmorph(LGG,LTT,:);  % Unrecognized function or variable 'precip'.
        X(i)={'Precipitation(mm)'};% label for variable
    end
    latt2=[LG ; LT];
    time=double(time);
    AA=time/24+datenum('1900-01-01 00:00:0.0');
    [yyyy,mm,dd,~,~,~] = datevec(AA);
    date1=[yyyy mm dd];
    DD={'Year','Month','Day'};
    sheet=ci;  % NB : every year on a new sheet - same file
    
    C={'LOCATION: NOT','','','Longitude';'DATE FROM: NOT','','','Latitude'};
    writecell(C,filename_export,"Sheet",sheet,"Range",'A1');
    writecell(DD,filename_export,"Sheet",sheet,"Range",'A3');
    writematrix(latt2,filename_export,"Sheet",sheet,"Range",'E1');
    writecell(X,filename_export,"Sheet",sheet,"Range",'E3');
    writematrix(date1,filename_export,"Sheet",sheet,"Range",'A4');
    writematrix(Cmorph,filename_export,"Sheet",sheet,"Range",'E4');
    
end

Below are the errors:

Warning: Added specified worksheet. 
> In matlab.io.internal.writing.writeXLSFile>getSheetFromBook (line 373)
In matlab.io.internal.writing.writeXLSFile (line 27)
In writetable (line 404)
In writecell (line 190) 
CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010103.nc  is processed
LOCATION NOT FOUND IN THIS NetCDF FILE
Warning: Added specified worksheet. 
> In matlab.io.internal.writing.writeXLSFile>getSheetFromBook (line 373)
In matlab.io.internal.writing.writeXLSFile (line 27)
In writetable (line 404)
In writecell (line 190) 
CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010104.nc  is processed
LOCATION NOT FOUND IN THIS NetCDF FILE
Warning: Added specified worksheet. 
> In matlab.io.internal.writing.writeXLSFile>getSheetFromBook (line 373)
In matlab.io.internal.writing.writeXLSFile (line 27)
In writetable (line 404)
In writecell (line 190) 
CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010105.nc  is processed
LOCATION NOT FOUND IN THIS NetCDF FILE
Warning: Added specified worksheet. 
> In matlab.io.internal.writing.writeXLSFile>getSheetFromBook (line 373)
In matlab.io.internal.writing.writeXLSFile (line 27)
In writetable (line 404)
In writecell (line 190) 
>> 

Best regards,

Augusto.

[Dave Correa]

Hi;

I did a test modifying the second longitude value, with a file on my PC and it works fine.

Change 204.25 x 204.125

LG=[203.875 204.125 204.375]; % de longitud (rango NB es 0 : 360°)

I attach a sample.

David

[Mathieu NOE]

hello @Augusto Gabriel da Costa Pereira

as proved by @Dave Correa your code works but simply one long value did not exist in the nc file so

I also changed

LG=[203.875 204.25 204.375]; % longitude (NB range is 0 : 360°)

to

LG=[203.875 204.125 204.375]; % longitude (NB range is 0 : 360°)

then OK.

No error mesage :

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010101.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010102.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010103.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010104.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010105.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010106.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010107.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010108.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010109.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010110.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010111.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010112.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010113.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010114.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010115.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010116.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010117.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010118.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010119.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010120.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010121.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010122.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010123.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010124.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010125.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010126.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010127.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010128.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010129.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010130.nc is processed

CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_20010131.nc is processed

[Augusto Gabriel da Costa Pereira]

But I want to collect precipitation data from each file, the result in .xlsx is just one file. How do I solve this problem?

[Mathieu NOE]

well your code has created 31 sheets with one day data per sheet

[Augusto Gabriel da Costa Pereira]

Ok, I just saw that 31 sheet was created. I don't understand this becomes very difficult. Can't the data be generated one below the others in just 1 sheet, just to make it easier at the time of analysis?

if there is a way to gather all the data in only 1 sheet, how do I do it?

Best Regards,

AP

[Mathieu NOE]

if you wanted one month of data in one sheet , I modified your code accordingly

now we still have a problem with "time" to be converted in yy/mm/dd . we could pick it directly from the nc file name but if yu prefer to process it as it is now the relationship must be updated.

attached is the excel file I got so far

% Download the CPC data used in the script below
% Directory
% https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/2000/01/
% Directory with all years
% https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/
date_start = 20010101; % first year to download
date_stop = 20010131; % last year to download
nb_of_days = date_stop - date_start +1;
httpsUrl = "https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/2001/01/";
filename_tmp="temp.nc"; % temporary filename (for loop); data will be always stored under this temp name
LG=[203.875 204.125 204.375];  % longitude (NB range is 0 : 360°)
LT=[3.875 3.875 3.875];     % latitude
% export results to excel 
filename_export='PRP_CMORPH.xlsx';

%% main loop 
    
sheet=1;  % NB : all data on 1st sheet (use append writte mode)
C={'LOCATION: NOT','','','Longitude';'DATE FROM: NOT','','','Latitude'};
DD={'Year','Month','Day'};
latt2=[LG ; LT];
X={'Precipitation(mm)','Precipitation(mm)','Precipitation(mm)'};
writecell(C,filename_export,"Sheet",sheet,"Range",'A1');
writecell(DD,filename_export,"Sheet",sheet,"Range",'A3');
writematrix(latt2,filename_export,"Sheet",sheet,"Range",'E1');
writecell(X,filename_export,"Sheet",sheet,"Range",'E3');


for ci = 1:nb_of_days
    
    clear Cmorph
    
    filename = strcat("CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_", num2str(date_start+ci-1), ".nc"); % create filename according to list above
    disp(strcat(filename, "  is processed"));
    
    dataUrl = strcat(httpsUrl, filename);
    data = webread(dataUrl);
    
    filename_out = websave(filename_tmp,dataUrl); % save downloaded data
%     ncdisp(filename_tmp,'/','min');      % commented, just to avoid filling the command window
    cmorph=ncread(filename_tmp,'cmorph');  % Dimensions: lon,lat,time
    long=ncread(filename_tmp,'lon');
    lat=ncread(filename_tmp,'lat');
    time=ncread(filename_tmp,'time');
    for i=1:3 
        LGG=find(long==LG(i));
        LTT=find(lat==LT(i));
         if isempty(LGG)||isempty(LTT)
            disp('LOCATION NOT FOUND IN THIS NetCDF FILE')
            break
         end
        % Step 5 :Convert 3 dimensional array to column vector array for one station
        Cmorph(:,i)=cmorph(LGG,LTT,:);  % Unrecognized function or variable 'precip'.
        X(i)={'Precipitation(mm)'};% label for variable
    end
    
    time=double(time);
    AA=time/24+datenum('1900-01-01 00:00:0.0');
    [yyyy,mm,dd,~,~,~] = datevec(AA);
    date_and_precip_data(ci,:)=[yyyy mm dd NaN Cmorph];
        
end

writematrix(date_and_precip_data,filename_export,"Sheet",sheet,"Range",'A4');

Answer 1

[Mathieu NOE]

Quick and dirty fix for the date , as we have it in the nc file name

% Download the CPC data used in the script below
% Directory
% https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/2000/01/
% Directory with all years
% https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/
date_start = 20010101; % first year to download
date_stop = 20010131; % last year to download
nb_of_days = date_stop - date_start +1;
httpsUrl = "https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/2001/01/";
filename_tmp="temp.nc"; % temporary filename (for loop); data will be always stored under this temp name
LG=[203.875 204.125 204.375];  % longitude (NB range is 0 : 360°)
LT=[3.875 3.875 3.875];     % latitude
% export results to excel 
filename_export='PRP_CMORPH.xlsx';

%% main loop 
    
sheet=1;  % NB : all data on 1st sheet (use append writte mode)
C={'LOCATION: NOT','','','Longitude';'DATE FROM: NOT','','','Latitude'};
DD={'Year','Month','Day'};
latt2=[LG ; LT];
X={'Precipitation(mm)','Precipitation(mm)','Precipitation(mm)'};
writecell(C,filename_export,"Sheet",sheet,"Range",'A1');
writecell(DD,filename_export,"Sheet",sheet,"Range",'A3');
writematrix(latt2,filename_export,"Sheet",sheet,"Range",'E1');
writecell(X,filename_export,"Sheet",sheet,"Range",'E3');


for ci = 1:nb_of_days
    
    clear Cmorph
    current_date = date_start+ci-1;
    filename = strcat("CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_", num2str(current_date), ".nc"); % create filename according to list above
    disp(strcat(filename, "  is processed"));
    
    dataUrl = strcat(httpsUrl, filename);
    data = webread(dataUrl);
    
    filename_out = websave(filename_tmp,dataUrl); % save downloaded data
%     ncdisp(filename_tmp,'/','min');      % commented, just to avoid filling the command window
    cmorph=ncread(filename_tmp,'cmorph');  % Dimensions: lon,lat,time
    long=ncread(filename_tmp,'lon');
    lat=ncread(filename_tmp,'lat');
    time=ncread(filename_tmp,'time');
    for i=1:3 
        LGG=find(long==LG(i));
        LTT=find(lat==LT(i));
         if isempty(LGG)||isempty(LTT)
            disp('LOCATION NOT FOUND IN THIS NetCDF FILE')
            break
         end
        % Step 5 :Convert 3 dimensional array to column vector array for one station
        Cmorph(:,i)=cmorph(LGG,LTT,:);  % Unrecognized function or variable 'precip'.
        X(i)={'Precipitation(mm)'};% label for variable
    end
    
%     time=double(time);
%     AA=time/24+datenum('1900-01-01 00:00:0.0');
%     [yyyy,mm,dd,~,~,~] = datevec(AA);
    current_date_str = num2str(current_date);
    yyyy = str2num(current_date_str(1:4));
    mm = str2num(current_date_str(5:6));
    dd = str2num(current_date_str(7:8));
    
    date_and_precip_data(ci,:)=[yyyy mm dd NaN Cmorph];
        
end

writematrix(date_and_precip_data,filename_export,"Sheet",sheet,"Range",'A4');

[Augusto Gabriel da Costa Pereira]

Awesome, Mathieu!!!

Thank you so much for your help, you are amazing.

If I have another question, can I contact you via mathworks and email?

Best Regards,

AP

[Rik]

Given the flag by the OP ("Question solved here"), I moved the comment to the answer section.

@Augusto Gabriel da Costa Pereira, now you will have see the 'accept this answer' button.

Regarding the latter part of your comment: I suspect Mathieu will be like most other frequent contributors: if you want to attract their attention, simply post a question. If you want to specifically attract someones attention you can @-mention them. That will work for all users who look at their activity feed. If you click on his contact card, you can also see the message icon, so you can send him an email via Mathworks as well, without him having to post his email address on the public internet.

[Mathieu NOE]

@Augusto Gabriel da Costa Pereira

yes you can send me an email via MathWorks

all the best