HDF Dimension Scales in HDF5 HDF-EOS Workshop IX San Francisco, CA November 30 - December 2, 2005 Pedro Vicente Nunes THG/NCSA Champaign-Urbana, IL HDF

HDF Overview and definition of Dimension Scales Dimension Scales Metadata Dimension Scales API functions and code example

HDF C code Example: I have an array of 12 elements My array data of 12 elements: int dset_data[12] = {1,2,3,4,5,6,7,8,9,10,11,12};

HDF My spatial domain But the spatial phenomenon I am trying to model has different width and height definitions X Y

HDF My array data of 12 elements: int dset_data[12] = {1,2,3,4,5,6,7,8,9,10,11,12}; My X axis spatial domain: float X[4] = {10,20,50,100}; My Y axis spatial domain: double Y[3] = {10,20,30}; C code

HDF Dimension Scales in HDF4 SD - Multifile Scientific Data Interface API functions –SDgetdimscale –SDsetdimscale –SDdiminfo

HDF Save my arrays as HDF5 datasets with the High Level HDF5 API function H5LTmake_dataset_int (fid,“My data",rank,dims, dset_data ); HDF5 API C code

HDF H5LTmake_dataset_int (fid,“X axis",rankx,dimsx, X ); H5LTmake_dataset_int (fid,“Y axis",ranky,dimsy, Y ); HDF5 API C code for the X and Y arrays

HDF /* get the DS dataset id and dataset*/ did = H5Dopen(fid,” My data”); dsid = H5Dopen(fid,” X axis”); /* attach the “ X axis” dimension scale to “My data" at dimension 0 */ H5DSattach_scale(did,dsid,DIM0); HDF5 API Call H5DSattach_scale function

HDF /* get the DS dataset id and dataset*/ did = H5Dopen(fid,” My data”); dsid = H5Dopen(fid,” X axis”); /* attach the “ X axis” dimension scale to “My data" at dimension 0 */ H5DSattach_scale(did,dsid,DIM0); Dimension Scales : HDF5 API Call H5DSattach_scale function

HDF HDF Explorer: Tree View

HDF HDF Explorer: Grid View

HDF HDF Explorer: Map View

HDF What is a Dimension Scale (in HDF5) ? A HDF5 dataset. With additional metadata that identifies the dataset as a Dimension Scale Typically Dimension Scales are logically associated with the dimensions of HDF5 Datasets The meaning of the association is left to applications.

HDF Example: 3D dataset Dataset: 3D Array with 5 x 7 x 10 dimensions 3 Dimension Scales Datasets

HDF Relationships between dataset dimensions and their corresponding dimension scales are not to be directly maintained or enforced by the HDF5 library Few restrictions as possible on the use of dimension scale dataset A dimension scale is not required to be a 1-D array, or to have a specific datatype Overview

HDF A dataset dimension can have more than 1 associated dimension scale A Dimension Scale can be shared by two or more Dataset dimensions

HDF Example: 3D dataset Dataset: 3D Array with 5 x 7 x 10 dimensions Several Dimension Scales Datasets

HDF Metadata The following dataset attributes are used to describe dimension scale datasets: –Attribute named “CLASS” with the value “DIMENSION_SCALE” –Optional attribute named “NAME” –Attribute references to any associated Dataset

HDF When the Dimension Scale is associated with a dimension of a Dataset, the association is represented by attributes of the two datasets.

HDF

HDF In the Dataset, the DIMENSION_LIST attribute is an array of object references to scales (Dimension Scale Datasets), and in the Dimension Scale Dataset the REFERENCE_LIST is an array of object references to Datasets.

HDF

HDF

HDF

HDF APIs Functions for creating and using Dimension Scales are implemented as high level functions Other High Level APIs include –Image –Table –Lite See

HDF API Functions

HDF Name: H5DSset_scale Signature: herr_t H5DSset_scale(hid_t dsid, char *dimname) Purpose: –Convert dataset dsid to a dimension scale, with optional name, dimname. Description: –The dataset dsid is converted to a Dimension Scale dataset. Creates the CLASS attribute, set to the value "DIMENSION_SCALE" and an empty REFERENCE_LIST attribute. If dimname is specified, then an attribute called NAME is created, with the value dimname.

HDF Name: H5DSattach_scale Signature: herr_t H5DSattach_scale(hid_t did, hid_t dsid, unsigned int idx) Purpose: –Attach dimension scale dsid to dimension idx of dataset dsid. Description: –Define Dimension Scale dsid to be associated with dimension idx of Dataset did. Entries are created in the DIMENSION_LIST and REFERENCE_LIST attributes.

HDF –Name: H5DSdetach_scale –Signature: –herr_t H5DSdetach_scale(hid_t did, hid_t dsid, unsigned int idx) –Purpose: Detach dimension scale dsid from the dimension idx of Dataset did. –Description: If possible, deletes association of Dimension Scale dsid with dimension idx of Dataset did. This deletes the entries in the DIMENSION_LIST and REFERENCE_LIST attributes

HDF Name: H5DSis_attached Signature: htri_t H5DSis_attached(hid_t did, hid_t dsid, unsigned int idx) Purpose: –Verifies if a dimension scale is attached to a dataset Description: –Report if dimension scale dsid is currently attached to dimension idx of dataset did.

HDF Name: H5DSiterate_scales Signature: herr_t H5DSiterate_scales(hid_t did, unsigned dim, int *idx, H5DS_iterate_t visitor, void *visitor_data) Purpose: –Iterates the operation visitor through the scales attached to dimension dim. Description: –H5DSiterate_scales iterates over the scales attached to dimension dim of dataset dset. For each scale in the list, the visitor_data and some additional information, specified below, are passed to the visitor function.

HDF Name: H5DSset_label Signature: herr_t H5DSset_label(hid_t did, unsigned int idx, const char *label) Purpose: –Set label for the dimension idx of did to the value label. Description: –Sets the DIMENSION_LABEL_LIST for dimension idx of dataset did.

HDF Name: H5DSget_label Signature: ssize_t H5DSget_label(hid_t did, unsigned int idx, char *label, size_t *size) Purpose: –Read the label for dimension idx of did into buffer label. Description: –Returns the value of the DIMENSION_LABEL_LIST for dimension idx of dataset did, if set.

HDF Name: H5DSget_scale_name Signature: herr_t H5DSget_scale_name(hid_t did, char *name, size_t *size) Purpose: –Read the name of scale did into buffer name. Description: –Read the value of the NAME attribute for scale did. Reads up to size characters into name.

HDF Name: H5DSis_scale Signature: htri_t H5DSis_scale(hid_t did) Purpose: –Determines whether dset is a Dimension Scale. Description: –H5DSis_scale determines if did is a Dimension Scale, i.e., has CLASS="DIMENSION_SCALE").

HDF Name: H5DSget_num_scales Signature: int H5DSget_num_scales(hid_t did, unsigned int idx) Purpose: –Determines how many Dimension Scales are attached to dimension idx of did. Description: –H5DSget_num_scales determines how many Dimension Scales are attached to dimension did of dataset dset.

HDF Code example #include "H5DS.h" #include "H5LT.h" #define RANK 2 #define DIM_DATA 12 #define DIM1_SIZE 3 #define DIM2_SIZE 4 #define DIM3_SIZE 2 #define DIM0 0 #define DIM1 1 #define DIM2 2 #define DS_1_NAME "ds_a_1" #define DS_2_NAME "ds_a_2"

HDF int main(void) { hid_t fid; /* file ID */ hid_t did; /* dataset ID */ hid_t dsid; /* DS dataset ID */ int rank = RANK; /* rank of data dataset */ int rankds = 1; /* rank of DS dataset */ hsize_t dims[RANK] = {DIM1_SIZE,DIM2_SIZE}; /* size of data dataset */ int buf[DIM_DATA] = {1,2,3,4,5,6,7,8,9,10,11,12}; /* data of data dataset */ hsize_t s1_dim[1] = {DIM1_SIZE}; /* size of DS 1 dataset */ hsize_t s2_dim[1] = {DIM2_SIZE}; /* size of DS 2 dataset */ int s1_wbuf[DIM1_SIZE] = {10,20,30}; /* data of DS 1 dataset */ int s2_wbuf[DIM2_SIZE] = {100,200,300,400}; /* data of DS 2 dataset */

HDF /* create a file using default properties */ if ((fid=H5Fcreate("myfile.h5",H5F_ACC_TRUNC,H5P_DEFAULT,H5P_ DEFAULT))<0) goto out; /* make a dataset */ if (H5LTmake_dataset_int(fid,"dset_a",rank,dims,buf)<0) goto out; /* make a DS dataset for the first dimension */ if (H5LTmake_dataset_int(fid,DS_1_NAME,rankds,s1_dim,s1_wbuf)<0) goto out; /* make a DS dataset for the second dimension */ if (H5LTmake_dataset_int(fid,DS_2_NAME,rankds,s2_dim,s2_wbuf)<0) goto out;

HDF /* * attach the DS_1_NAME dimension scale to "dset_a" * */ /* get the dataset id for "dset_a" */ if ((did = H5Dopen(fid,"dset_a"))<0) goto out; /* get the DS dataset id */ if ((dsid = H5Dopen(fid,DS_1_NAME))<0) goto out; /* attach the DS_1_NAME dimension scale to "dset_a" at dimension 0 */ if (H5DSattach_scale(did,dsid,DIM0)<0) goto out; /* close DS id */ if (H5Dclose(dsid)<0) goto out;

HDF /* * attach the DS_2_NAME dimension scale to "dset_a" * */ /* get the DS dataset id */ if ((dsid = H5Dopen(fid,DS_2_NAME))<0) goto out; /* attach the "ds2" dimension scale to "dset_a" as the 2nd dimension */ if (H5DSattach_scale(did,dsid,DIM1)<0) goto out; /* close DS id */ if (H5Dclose(dsid)<0) goto out; /* close */ H5Fclose(fid);

HDF

HDF

HDF

HDF RFC document /H5DimScales

HDF Acknowledgements Special thanks to Bob McGrath (NCSA) This presentation is based upon work supported in part by a Cooperative Agreement with the National Aeronautics and Space Administration (NASA) under NASA grant NNG05GC60A. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NASA. Other support provided by NCSA and other sponsors and agencies: