In hydrological and climatological studies, handling large datasets such as those stored in NetCDF (Network Common Data Form) format is common. This blog post guides you through the process of loading, extracting, and analyzing specific data from a NetCDF file using MATLAB. Our focus will be on handling the spei06.nc
file, which contains monthly Standardized Precipitation-Evapotranspiration Index (SPEI) data from January 1900 to December 2018.
Step-by-Step Analysis
Loading Geographic Data
Before diving into the SPEI data, it’s crucial to understand the geographical context of the data points. We start by loading coastline data, which will later help us determine which data points fall on land.
The load coastlines
command in MATLAB fetches the latitude (coastlat
) and longitude (coastlon
) of global coastlines. These variables are vectors that outline the boundaries between land and water, essential for geographic analyses where differentiation between land and ocean grid points is needed.
Loading and Displaying NetCDF File Data
Here, we specify the path to our NetCDF file and use ncdisp
to display its contents. This function provides a detailed overview of the variables within the file, including dimensions and attributes, which is crucial for understanding how to access and manipulate the data.
Creating Grid Coordinates
meshgrid
creates matrices for latitude and longitude, which are then reshaped into a two-column matrix where each row represents a geographic coordinate. This matrix (all_latlon
) contains coordinates for every grid point in the dataset, crucial for spatial analysis.
Extracting Land Grids
inpolygon
determines which points from all_latlon
fall within the polygon defined by coastlat
and coastlon
(the coastlines). The resulting logical array (index
) indicates whether each point is on land. This method is highly effective for isolating land-specific data from global datasets.
Verifying Land Grid Coordinates
Once we have identified which grid points fall on land using the inpolygon
function, it’s important to visually confirm that these points are correct. This verification step helps ensure that our geographical analysis aligns with real-world data.
Extracting and Flattening SPEI Data
First, let’s break down the MATLAB code used to process the SPEI data:
- Flattening the Data: The SPEI data is stored in a 3D array where each slice along the third dimension corresponds to a different time step. We flatten this 2D slice into a 1D array so that it’s easier to work with and extract specific values.
- Extracting Data for Land Grids: Using the
index
array, which identifies positions that correspond to land areas, we pull out the relevant SPEI values from our flattened data. This is done for each time step within our loop, effectively filtering out ocean grid points.
Data Structuring and Export
Once we’ve processed the SPEI data, we need to structure it for export:
- final_spei_ts has a shape of (1416, 78018), where 1416 represents the total number of monthly time series entries from January 1900 to December 2018.
- 78018 represents the number of columns: 78016 for each land grid and the first 2 columns are the year and month.
- Understanding the Output Array: The
final_spei_ts
array combines temporal data (year and month) with the SPEI values for each land grid point. Each row in this array represents a specific month, and each column after the first two represents SPEI data for a specific grid point on land. - Exporting the Data: To facilitate further analysis, we export this structured array to a text file using MATLAB’s
writematrix
function. This allows the data to be easily accessed in other software for detailed analysis or visualization.
Saving the SPEI data and the geographic latitude and longitude coordinates
Finally, we save the latitude and longitude data for the land grids:
The entire code for this blog post are available here in my github. The spei data, used in this blog, is more than 300 mb. Thats why I can’t upload it to Github; but you can download it from SPEI database. Please let me know about your thoughts and if you are helpful after reading this.