1. HDF Dimension Scales in HDF5 HDF-EOS Workshop IX San Francisco, CA November 30 - December 2, 2005 Pedro Vicente Nunes THG/NCSA Champaign-Urbana, IL HDF

2. HDF - 2 - Overview and definition of Dimension Scales Dimension Scales Metadata Dimension Scales API functions and code example

3. HDF - 3 - C code Example: I have an array of 12 elements My array data of 12 elements: int dset_data[12] = {1,2,3,4,5,6,7,8,9,10,11,12};

4. HDF - 4 - My spatial domain But the spatial phenomenon I am trying to model has different width and height definitions X Y

5. HDF - 5 - My array data of 12 elements: int dset_data[12] = {1,2,3,4,5,6,7,8,9,10,11,12}; My X axis spatial domain: float X[4] = {10,20,50,100}; My Y axis spatial domain: double Y[3] = {10,20,30}; C code

6. HDF - 6 - Dimension Scales in HDF4 SD - Multifile Scientific Data Interface API functions –SDgetdimscale –SDsetdimscale –SDdiminfo

7. HDF - 7 - Save my arrays as HDF5 datasets with the High Level HDF5 API function H5LTmake_dataset_int (fid,“My data",rank,dims, dset_data ); HDF5 API C code

8. HDF - 8 - H5LTmake_dataset_int (fid,“X axis",rankx,dimsx, X ); H5LTmake_dataset_int (fid,“Y axis",ranky,dimsy, Y ); HDF5 API C code for the X and Y arrays

9. HDF - 9 - /* get the DS dataset id and dataset*/ did = H5Dopen(fid,” My data”); dsid = H5Dopen(fid,” X axis”); /* attach the “ X axis” dimension scale to “My data" at dimension 0 */ H5DSattach_scale(did,dsid,DIM0); HDF5 API Call H5DSattach_scale function

10. HDF - 10 - /* get the DS dataset id and dataset*/ did = H5Dopen(fid,” My data”); dsid = H5Dopen(fid,” X axis”); /* attach the “ X axis” dimension scale to “My data" at dimension 0 */ H5DSattach_scale(did,dsid,DIM0); Dimension Scales : HDF5 API Call H5DSattach_scale function

11. HDF - 11 - HDF Explorer: Tree View

12. HDF - 12 - HDF Explorer: Grid View

13. HDF - 13 - HDF Explorer: Map View

14. HDF - 14 - What is a Dimension Scale (in HDF5) ? A HDF5 dataset. With additional metadata that identifies the dataset as a Dimension Scale Typically Dimension Scales are logically associated with the dimensions of HDF5 Datasets The meaning of the association is left to applications.

15. HDF - 15 - Example: 3D dataset Dataset: 3D Array with 5 x 7 x 10 dimensions 3 Dimension Scales Datasets 5 7 10

16. HDF - 16 - Relationships between dataset dimensions and their corresponding dimension scales are not to be directly maintained or enforced by the HDF5 library Few restrictions as possible on the use of dimension scale dataset A dimension scale is not required to be a 1-D array, or to have a specific datatype Overview

17. HDF - 17 - A dataset dimension can have more than 1 associated dimension scale A Dimension Scale can be shared by two or more Dataset dimensions

18. HDF - 18 - Example: 3D dataset Dataset: 3D Array with 5 x 7 x 10 dimensions Several Dimension Scales Datasets

19. HDF - 19 -Metadata The following dataset attributes are used to describe dimension scale datasets: –Attribute named “CLASS” with the value “DIMENSION_SCALE” –Optional attribute named “NAME” –Attribute references to any associated Dataset

20. HDF - 20 - When the Dimension Scale is associated with a dimension of a Dataset, the association is represented by attributes of the two datasets.

21. HDF - 21 -

22. HDF - 22 - In the Dataset, the DIMENSION_LIST attribute is an array of object references to scales (Dimension Scale Datasets), and in the Dimension Scale Dataset the REFERENCE_LIST is an array of object references to Datasets.

23. HDF - 23 -

24. HDF - 24 -

25. HDF - 25 -

26. HDF - 26 - APIs Functions for creating and using Dimension Scales are implemented as high level functions Other High Level APIs include –Image –Table –Lite See http://hdf.ncsa.uiuc.edu/HDF5/hdf5_hl/

27. HDF - 27 - API Functions

28. HDF - 28 - Name: H5DSset_scale Signature: herr_t H5DSset_scale(hid_t dsid, char *dimname) Purpose: –Convert dataset dsid to a dimension scale, with optional name, dimname. Description: –The dataset dsid is converted to a Dimension Scale dataset. Creates the CLASS attribute, set to the value "DIMENSION_SCALE" and an empty REFERENCE_LIST attribute. If dimname is specified, then an attribute called NAME is created, with the value dimname.

29. HDF - 29 - Name: H5DSattach_scale Signature: herr_t H5DSattach_scale(hid_t did, hid_t dsid, unsigned int idx) Purpose: –Attach dimension scale dsid to dimension idx of dataset dsid. Description: –Define Dimension Scale dsid to be associated with dimension idx of Dataset did. Entries are created in the DIMENSION_LIST and REFERENCE_LIST attributes.

30. HDF - 30 - –Name: H5DSdetach_scale –Signature: –herr_t H5DSdetach_scale(hid_t did, hid_t dsid, unsigned int idx) –Purpose: Detach dimension scale dsid from the dimension idx of Dataset did. –Description: If possible, deletes association of Dimension Scale dsid with dimension idx of Dataset did. This deletes the entries in the DIMENSION_LIST and REFERENCE_LIST attributes

31. HDF - 31 - Name: H5DSis_attached Signature: htri_t H5DSis_attached(hid_t did, hid_t dsid, unsigned int idx) Purpose: –Verifies if a dimension scale is attached to a dataset Description: –Report if dimension scale dsid is currently attached to dimension idx of dataset did.

32. HDF - 32 - Name: H5DSiterate_scales Signature: herr_t H5DSiterate_scales(hid_t did, unsigned dim, int *idx, H5DS_iterate_t visitor, void *visitor_data) Purpose: –Iterates the operation visitor through the scales attached to dimension dim. Description: –H5DSiterate_scales iterates over the scales attached to dimension dim of dataset dset. For each scale in the list, the visitor_data and some additional information, specified below, are passed to the visitor function.

33. HDF - 33 - Name: H5DSset_label Signature: herr_t H5DSset_label(hid_t did, unsigned int idx, const char *label) Purpose: –Set label for the dimension idx of did to the value label. Description: –Sets the DIMENSION_LABEL_LIST for dimension idx of dataset did.

34. HDF - 34 - Name: H5DSget_label Signature: ssize_t H5DSget_label(hid_t did, unsigned int idx, char *label, size_t *size) Purpose: –Read the label for dimension idx of did into buffer label. Description: –Returns the value of the DIMENSION_LABEL_LIST for dimension idx of dataset did, if set.

35. HDF - 35 - Name: H5DSget_scale_name Signature: herr_t H5DSget_scale_name(hid_t did, char *name, size_t *size) Purpose: –Read the name of scale did into buffer name. Description: –Read the value of the NAME attribute for scale did. Reads up to size characters into name.

36. HDF - 36 - Name: H5DSis_scale Signature: htri_t H5DSis_scale(hid_t did) Purpose: –Determines whether dset is a Dimension Scale. Description: –H5DSis_scale determines if did is a Dimension Scale, i.e., has CLASS="DIMENSION_SCALE").

37. HDF - 37 - Name: H5DSget_num_scales Signature: int H5DSget_num_scales(hid_t did, unsigned int idx) Purpose: –Determines how many Dimension Scales are attached to dimension idx of did. Description: –H5DSget_num_scales determines how many Dimension Scales are attached to dimension did of dataset dset.

38. HDF - 38 - Code example #include "H5DS.h" #include "H5LT.h" #define RANK 2 #define DIM_DATA 12 #define DIM1_SIZE 3 #define DIM2_SIZE 4 #define DIM3_SIZE 2 #define DIM0 0 #define DIM1 1 #define DIM2 2 #define DS_1_NAME "ds_a_1" #define DS_2_NAME "ds_a_2"

39. HDF - 39 - int main(void) { hid_t fid; /* file ID */ hid_t did; /* dataset ID */ hid_t dsid; /* DS dataset ID */ int rank = RANK; /* rank of data dataset */ int rankds = 1; /* rank of DS dataset */ hsize_t dims[RANK] = {DIM1_SIZE,DIM2_SIZE}; /* size of data dataset */ int buf[DIM_DATA] = {1,2,3,4,5,6,7,8,9,10,11,12}; /* data of data dataset */ hsize_t s1_dim[1] = {DIM1_SIZE}; /* size of DS 1 dataset */ hsize_t s2_dim[1] = {DIM2_SIZE}; /* size of DS 2 dataset */ int s1_wbuf[DIM1_SIZE] = {10,20,30}; /* data of DS 1 dataset */ int s2_wbuf[DIM2_SIZE] = {100,200,300,400}; /* data of DS 2 dataset */

40. HDF - 40 - /* create a file using default properties */ if ((fid=H5Fcreate("myfile.h5",H5F_ACC_TRUNC,H5P_DEFAULT,H5P_ DEFAULT))<0) goto out; /* make a dataset */ if (H5LTmake_dataset_int(fid,"dset_a",rank,dims,buf)<0) goto out; /* make a DS dataset for the first dimension */ if (H5LTmake_dataset_int(fid,DS_1_NAME,rankds,s1_dim,s1_wbuf)<0) goto out; /* make a DS dataset for the second dimension */ if (H5LTmake_dataset_int(fid,DS_2_NAME,rankds,s2_dim,s2_wbuf)<0) goto out;

41. HDF - 41 - /*------------------------------------------------------------------------- * attach the DS_1_NAME dimension scale to "dset_a" *------------------------------------------------------------------------- */ /* get the dataset id for "dset_a" */ if ((did = H5Dopen(fid,"dset_a"))<0) goto out; /* get the DS dataset id */ if ((dsid = H5Dopen(fid,DS_1_NAME))<0) goto out; /* attach the DS_1_NAME dimension scale to "dset_a" at dimension 0 */ if (H5DSattach_scale(did,dsid,DIM0)<0) goto out; /* close DS id */ if (H5Dclose(dsid)<0) goto out;

42. HDF - 42 - /*------------------------------------------------------------------------- * attach the DS_2_NAME dimension scale to "dset_a" *------------------------------------------------------------------------- */ /* get the DS dataset id */ if ((dsid = H5Dopen(fid,DS_2_NAME))<0) goto out; /* attach the "ds2" dimension scale to "dset_a" as the 2nd dimension */ if (H5DSattach_scale(did,dsid,DIM1)<0) goto out; /* close DS id */ if (H5Dclose(dsid)<0) goto out; /* close */ H5Fclose(fid);

43. HDF - 43 -

44. HDF - 44 -

45. HDF - 45 -

46. HDF - 46 - RFC document http://hdf.ncsa.uiuc.edu/RFC/ /H5DimScales

47. HDF - 47 -Acknowledgements Special thanks to Bob McGrath (NCSA) This presentation is based upon work supported in part by a Cooperative Agreement with the National Aeronautics and Space Administration (NASA) under NASA grant NNG05GC60A. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NASA. Other support provided by NCSA and other sponsors and agencies: http://hdf.ncsa.uiuc.edu/acknowledge.html