Data from Models & Grids¶
Introduction¶
This section describes the process of submitting model-generated or gridded data products to a data portal, including:
3-D or 4D formatted spatial time-series data
forecasts
nowcasts
hindcasts
model results
satellite data
unstructured grids
Terminology¶
We’ve struggled with what to call this data. “Models” is confusing to most other people outside of Axiom because it doesn’t technically include satellite data. “Grids” is better, but doesn’t capture everything. It also only tangentially includes unstructured grids, which are not really grids at all.
The dimensions of these data will be the following:
Time-series: one of the dimensions is time. For every time period, the dataset should contain a 2D spatial or 3D depth-enabled spatial block of data.
Spatial: Each time slice contains X and Y dimensions that corresponds to real locations on the Earth (e.g., longitude and latitude). The spatial component may be in a projection or not.
Elevation (optional): the last optional dimension is elevation, depth, or some proxy (e.g., sigma (pressure values).
Use community standards. Follow CF Conventions for naming variables and structuring data. Structured, self-describing formats like NetCDF are highly preferred.
Data formats¶
Please provide your dataset in one of the following formats, in order of preference:
NetCDF: NetCDF files are the preferred format and the backbone of archival data storage at both Axiom and NCEI. They are machine-independent, flexible, can store data and dimensions in any combination, store metadata as global and/or variable attributes within the file itself (i.e., self-describing). They are a community and industry standard for oceanographic and atmospheric data, and a suite of tools (e.g., java toolkits, web services such as THREDDS, etc.) are readily available. Axiom follows the format of the most recent NCEI templates.
HDF: HDF files are very similar to NetCDF files in format. In fact, since 2008, NetCDF files are built on the HDF5 standard and NetCDF tools can generally read HDF files. However, HDF tools cannot generally do the reverse, making them slightly less flexible.
GRIB2: GRIB2 files are a very common way to store weather models are forecasts. However, it has a rigid format (i.e., named fields) that do not readily accept other types of data such as biological fields. They also have limited (pre-named) metadata attributes.
GEOTIFF: a common format for regularly-gridded (i.e., raster) data. This format is simple to read and interpret and commonly includes metadata such as projection information, spatial resolution, pixel-size, etc., but other metadata attributes such as source, owner, publisher, history, parameter names, etc. are often not included. They also can accommodate only a limited number of “bands” (typically RGBA), so are not often used for time-series data.
NetCDF guidelines¶
In nearly all cases, Axiom will convert non-NetCDF data into NetCDF. Please consider the following:
follow conventions: ACDD 1.3
include quality control if you have it, or metadata speaking to it.
use a compliance checker (available online) and follow its feedback
use the value -9999 for null values
Implementation¶
Gridded and model output data shall be made available via OPeNDAP and WMS. THREDDS, ERDDAP, and Hyrax all support OpenDAP. Data providers that deploy THREDDS should also enable the WCS service.
Submitting Data¶
If your gridded data are not available through a machine-readable endpoint (e.g., OPeNDAP), you may upload up to 1 TB of files using this form, or upload your files to the Research Workspace if you already have an account: