1. NCL
Introduction
Dennis Shea
National Center for
Atmospheric Research

2. Workshop Overview Objective Schedule
comfortable with NCL; minimize learning curve
workshop will not make you an expert
access, process and visualize data
Schedule
day 1:
Intro/Language Basics, File I/O, Graphics: Lab
day 2:
Data Processing [two sessions]: Lab
day 3 / optional day 4 [8-12]
Lab
Labs important: you must invest the time
“osmosis” method of learning does not work

3. Size = 3 1D array time (3) scalar (vector) time(time) x(8), y(8)
Q(8,8)
Q(y,x)
3D array
Z(4)
x(8), y(8), z(4)
R(4,8,8)
R(z,y,x)
Time
x(8), y(8), z(4), t(3)
S(3,4,8,8)
(time,z,y,x)
4D array

4. NCL variable based on a netCDF variable model
Introduction to
netCDF
NCL variable based on a netCDF variable model

5. netCDF files self describing portable [machine independent]
(ideally) all info contained within file
no external information needed to determine file contents
portable [machine independent]
Supported by many software tools / languages
NCL, IDL, Matlab, ferret, GrADS,... / F, C, C++, Java,…
NCO: netCDF Operators
perform various tasks very efficiently
ncview
quick visualization: COARDS
Many modelling groups use netCDF

6. Examining a netCDF file
ncdump file_name | less
dumps the entire contents of a file
ncdump -h file_name | less
dumps the header info [most commonly used]
NCL equivalent: ncl_filedump file_name | less
ncdump -v U file_name | less
NCL equivalent: ncl_filedump –v U file_name | less
Note: ncdump is a Unidata utility
not a netCDF Operator (NCO)
not associated with NCL
ncview to visualize file contents [COARDS]
ncl_filedump file_name [ more general ]
netCDF, GRIB, HDF, HDF-EOS

7. Parts of netCDF file ncdump -h 1999.nc VARIABLES:
Names , Attributes, Coordinates
variables:
float lat(lat) ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
float lon(lon) ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
int time(time) ;
time:long_name = "time" ;
time:units = "Month of Year" ;
double T(time, lat, lon) ;
T:long_name = “Temperature”;
T:units = “C" ;
T:missing_value = 1.e+20f ;
T:_FillValue = 1.e+20f
DIMENSIONS:
dimensions:
lat = 64 ;
lon = 128 ;
time = 12 ;
FILE ATTRIBUTES:
global attributes:
:title = “Temp: 1999" ;
:source = “NCAR” ;
:conventions = "None" ;
exercise: ncl_filedump –h UV300.nc | less

8. Simple Array [fortran]
array [regular shape; could be of length 1 (scalar)]
parameter (nrow=7, ncol=5)
real x(nrow,ncol) x
name: x
type: real (float)
shape: 2-dimensions [ 2D ]
size: (rows) x 5 (columns)
values: x(3,4) = [col major]

9. netCDF/NCL variable array [could be of length 1 (scalar)]
(may) have additional information x
name: x
type: float [real]
shape: 2-dimensions
size: (rows) x 5 (columns)
values: x(2,3) = [row major]
long_name: “Temperature”
units: “C”
named dimensions: x(time,lat)
lat: (/ -60, -30 ,0, 30, 60 /)
time: (/2000,2001,2002,2003,2004,2005,2006 /)

10. visual: simple 2D netCDF Variable
coordinate variables
long_name
units
Latitude coordinate variable (1D, &)
Longitude coordinate variable (1D, &)
NCL is NOT LIMITED to netCDF conforming variables
eg: 2D coordinate arrays (curvilinear coords)

11. netCDF [NCL] Variable modelX
Scalar or Array
f = addfile(“foo.nc”,”r”) ; grb/hdf
x = f->X
NCL reads the scalar/array, attributes, and coordinate variables as an object
attributes
long_name
_FillValue
units
add_offset
scale_factor
etc. X
accessed
accessed via &
values
Scalar or Array
attributes
long_name
_FillValue
units
add_offset
scale_factor
etc.
time
lev
lat
lon
etc.
coord var
time
lev
lat
lon
etc.
coordinates

12. Detailed Look netCDF Variable (NCL)
ncl <return> ; interactive mode
ncl 0 > f = addfile ("UV300.nc", "r") ; open file
ncl 1 > u = f->U ; import STRUCTURE
ncl 2 > printVarSummary (u) ; overview of variable
Variable: u
Type: float
Total Size: bytes
16384 values
Number of Dimensions: 3
Dimensions and Sizes: [time | 2] x [lat | 64] x [lon | 128]
Coordinates:
time: [ ]
lat: [ ]
lon: [ ]
Number of Attributes: 5
_FillValue : e36
units : m/s
long_name : Zonal Wind Component
short_name : U
missing_value : 1e36
Classic netCDF
Variable Model
NCL
syntax/funcs
query
use
modify
add
any aspect of
data object

13. when creating output file in NCL (optional)
UNLIMITED dimension
special dimension
essentially a “record” dimension
time dimension is most frequently “unlimited”
used by the NCO to concatenate files
no special meaning to NCL
when creating output file in NCL (optional)
filedimdef (outputfile, “time”, -1, True )
example: ncdump -h T2m_ud.nc
netcdf T2m_ud {
dimensions:
time = UNLIMITED ; // (204 currently)
lat = 94 ;
lon = 192 ;
lev = 18}

14. Integrated processing environment
NCL Overview
Integrated processing environment
pdf
Vis5D
freeware: supported, public domain
portable: Linux/Unix, Windows, MacOSX
general purpose: unique capabilities
excellent graphics

15. NCL Support Documentation and Examples http://www.ncl.ucar.edu/
numerous downloadable examples to get you going
downloadable reference manuals [pdf], FAQ
users must subscribe ,

16. Complete Programming Language
NCAR Command Language
Complete Programming Language
data types
variables
operators
expressions
conditional statements
loops
functions/procedures/graphics
Features
query / manipulate meta data
import data in a variety of formats
array syntax / operations
can use user fortran/C codes and commercial libraries
most functions/procedures ignore missing data

17. Running NCL Batch Mode [ .ncl suffix is optional]
Interactive Mode (Command line)
ncl [options][command-line-arguments] <return>
ncl> enter commands
ncl> quit <return>
can save interactive commands
ncl> record (“file_name”)
ncl> stop record
Batch Mode [ .ncl suffix is optional]
ncl [options][arguments] script.ncl
ncl < script.ncl [also acceptable]
ncl [options][arguments] script.ncl >&! out
ncl [options][arguments] script.ncl >&! out &
appending "&" means put in background
note: the >&! & are csh and tcsh syntax

18. NCL predefined options
ncl –hnxV [predfined options are preceded by dash]
may be used for interactive or batch mode
informational
ncl –h [display predefined options and usage and exit]
ncl –V [print the NCL version and exit]
action
ncl –x [echo statements as encountered (debug)]
ncl –n [don't enumerate dimensions of values in print() ]
multiple predefined options
ncl –nx [ not ncl –n –x ]

19. Sample Batch Script: sample.ncl
load “$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl“
load “$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl“
begin ; optional
f1 = addfile("TMP_58-97.nc", "r") ; open netCDF file
T = f1->Tmp ; T(time,lev,lat,lon) =>(480,17,73,144)
f2 = addfile(“P_ grb", "r") ; open GRIB file
P = f2->Pres ; P(time,lev,lat,lon)
f3 = addfile(“Q_ hdfeos", "r") ; open hdf –eos file
Q = f3->Specific_Humidity ; Q(time,lev,lat,lon)
pot = T* ( *Q) *(1000/P)^ ; potential temperature (array)
print("pot: min="+ min(pot) + " max="+ max(pot) )
wks = gsn_open_wks("ps", “sample") ; open a graphic workstation
gsn_define_colormap (wks,"gui_default") ; change from default
plot = gsn_csm_contour_map_polar (wks,T(0,5,:,:),False)
res = True ; change default plot
= True ; use colors
= True ; use entire color map
plot = gsn_csm_pres_hgt (wks,T(0,:,{50},:), res )
end ; only if begin is present

21. Language
Basics

22. Outline: Language Basics
special syntax characters
data types
Variables  netCDF/NCL variable model
attributes
_FillValue
named dimensions
coordinate variables
print and printVarSummary
shaping
subscripting

23. NCL Syntax Characters ; - comment [can appear anywhere]
@ - reference/create attributes
! - reference/create named dimension
& - reference/create coordinate variable
{…} - coordinate subscripting
$ - enclose strings when (im/ex)port variables via addfile
(/../) - array construct characters
: - array syntax
| - separator for named dimensions
\ - continue character [statement to span multiple lines]
:: - syntax for external shared objects (eg, fortran/C)
-> - use to (im/ex)port variables via addfile function

24. Data Types Numeric double (64 bit) float (32 bit) long (32 or 64 bit)
integer (32 bit)
short (16 bit)
byte ( 8 bit, 0-255)
complex NOT supported
non-Numeric
string
character
graphic
file
logical
list

25. netCDF/NCL variable array [could be of length 1 (scalar)]
(may) have additional information x
name: x
type: float [real]
shape: 2-dimensions
size: (rows) x 5 (columns)
values: x(2,3) = [row major]
long_name: “Temperature”
units: “C”
named dimensions: x(time,lat)
lat: (/ -60, -30 ,0, 30, 60 /)
time: (/2000,2001,2002,2003,2004,2005,2006 /)
Meta data

26. NCL variable model (2 of 2)
(possibly) has much more info: meta data
information about the variable
descriptive variable name, units, etc
coordinate information
dimensions can be 1D variables
x(lat,lon) lat/lon are 1D vectors (coord variables)
NCL variable => “variable data object”
[C: struct , f90: derived type]
values
(possibly) descriptive info (textual/numeric)
(possibly) coordinate information (numeric)

27. Why Know NCL Variable Model
variables imported (read) from nc, grib, hdf, hdf-eos
will have the same [consistent] structure
powerful programming concept
facilitates writing netCDF and HDF files
facilitates writing robust functions
functions can query, add, use meta data
some functions access/use the meta data
eg: gsn_csm graphic functions
aware of common netCDF conventions
will query and use for labels and mapping

28. Simple Variable Assignment
a_int = 1 , a_float = [2e-5] a_double = 3.2d
a_string = "a” , a_logical = True [False]
array constructor characters (/…/)
a_integer = (/1,2,3/)
a_float = (/1.0, 2.0, 3.0/)
a_double = (/1., 2 , 3.2 /) *1d5
a_string = (/"abcd","e", "Hello, World”/)
a_logical = (/True, False,True/)
a_2darray = (/ (/1,2,3/), (/4,5,6/), (/7,8,9/) /)

29. Conversion between data types
coercion
implicit conversion of one type to another
automatic coercion when no info is lost
fortran: x=i and i=x
NCL: x=i and i=floattointeger(x)
many functions to perform conversions

30. Attributes assign/access with @ character
info about a variable or file [meta data]
attributes can be any data type but file or list
scalar, multi dimensional array (string, numeric)
assign/access character
= “deg C”
= (/ 0.25, 0.5, 0.25 /)
= (/ (/1,2,3/), (/4,5,6/), (/7,8,9/) /)
= -999
title =
attribute functions [isatt, getfilevaratts]
if (isatt(T,"units")) then end if
atts = getfilevaratts (fin, "T")
delete can eliminate an attribute

31. netCDF Operators [NCO] look for “missing_value” attribute
_FillValue
Unidata and NCL reserved attribute
most NCL functions ignore _FillValue
Note: “missing_value” attribute: no special status to NCL
if “T” has “missing_value” attribute but no “_FillValue ” =
use built-in function “ismissing” to check for _FillValue
if (any(ismissing(T))) then … end if
NOTE: if will not work
netCDF Operators [NCO]
look for “missing_value” attribute
best to create netCDF files with both
NCL: best to not use zero as a _FillValue
OK except when contouring [random bug]

32. Interactive: Variable_1
- demonstrate interactive mode
illustrate simple calculations and function use
first view of meta data [attributes]

33. Dimensions may be “named” assigned with ! Character {let T(:,:,:)}
provides alternative way to reference subscripts
recommendation: always name dimensions
use NCL syntax
assigned with ! Character {let T(:,:,:)}
T!0 = "time" ; leftmost [slowest varying] dim
T!1 = "lat“
T!2 = "lon" ; rightmost [fastest varying] dim
Dim names may be renamed, retrieved
T!1 = "LAT" … dName = T!2
delete can eliminate: delete (T!2)
Named dimensions used to reshape

34. Coordinate Variables used in graphics, coordinate subscripting
data coordinates [ eg: time, level, lat, lon ]
strict netCDF definition
1D array
monotonically in[de]creasing numeric values
can only be assigned to a named dimension
1D array must be the same size as dimension
should not have _FillValue or missing_value attributes
values assigned via & character
T&lat = latitude
used in graphics, coordinate subscripting
coordinate functions
iscoord , isfilevarcoord
delete can eliminate coordinate array
delete(T&time)

35. Create and Assign Coordinate Variables
create 1D array
time = (/ 1980, 1981, 1982 /) ; integer
lon = ispan(0, 355, 5)* ; integer->float
assign dimension name(s)
time!0 = “time”
lon!0 = “lon”
assign values to named dimension
time&time = time
lon&lon = lon
let x be 2D: name dimensions
x!0 = “time” … x!1 = “lon”
assign coordinate variables to x
x&time = time … x&lon = lon

36. Access/Change/Create/Delete Meta Data
@ attributes
= "U"
lonName =
! named dimensions
u!0 = "TIME"
tName = u!0
& coordinate variable
u&lat = (/ -90., -85, .... , 85., 90. /)
latitude = u&lat
$ substitute string
x = fin->$variable(n)$ … x = fin->$”T: p”$

37. netCDF [NCL] Variable modelX
Scalar or Array
f = addfile(“foo.nc”, “r”) ; grb/hdf
x = f->X
NCL reads the scalar/array, attributes, and coordinate variables as an object
attributes
long_name
_FillValue
units
add_offset
scale_factor
etc. X
accessed
accessed via &
values
Scalar or Array
attributes
long_name
_FillValue
units
add_offset
scale_factor
etc.
time
lev
lat
lon
etc.
coord var
time
lev
lat
lon
etc.
coordinates

38. Variable Assignment Variable-to-Variable assignment
consider y = x where x is previously defined
if y not defined:
y has same type/shape/values/meta data as x
if y predefined:
y must have same shape and type
or “x” must be coerceible to the type of y
y attributes, dimension names and coordinate variables, if they exist, will be over written
Value-only assignment (no meta copy)
U multi-dimensional array with meta data
Uval = (/ U /) or Uval = (/ f->U/)
the (/ ... /) operator pair strips meta data

39. Interactive: Variable_2
named dimensions
create a variable data-object [structure] from scratch
standard and coordinate subscripting
reordering via named dimensions

40. Arrays row major ….. like C
left dimension varies slowest; right varies fastest
dimension numbering left to right [0,1,..]
indicies [subscripts] are zero based [0,N-1]
Consider T(time,lat,lon)
left dimension is 0 [ T!0 ]
middle dimension is [ T!1 ]
right dimension is [ T!2 ]

41. Variable Shaping transpose, thin, reorder dimensions
functions may require data in specific order
map plot functions want array order T(...,lat,lon)
can and should be done without loops
use NCL syntax or functions
very fast for variables in memory
how? … two approaches: let T(time,lat,lon)
named dimensions: t = T(lat|:, lon|:, time|:)
NCL functions:
ndtooned: t1D = ndtooned(T)
onedtond: t2D = onedtond(t1D, (/N,M/) )

42. Variable Subscripting (1 of 3)
Standard Array Subscripting
ranges: start/end and [optional] stride
indices separated by :
omitting start/end index implies default begin/end
Consider T(time,lat,lon)
T  entire array [ don't use T(:,:,:) ]
T(0,:,::5)  1st time index, all lat, every 5th lon T(0,::-1,:50)  1st time index, reverse lat order, 1st lon
T(:1,45,10:20)  1st 2 time indices, 46th index of lat, indicies of lon

43. Variable Subscripting 2 of 3)
Coordinate Variable Subscripting
only applies to netCDF conforming variables
same rules apply for ranges, strides, defaults
use curly brackets {…}
standard and coordinate subs can be mixed
[if no reorder]
T(:,{-30:30},:)  all times/lon, lat -30° to +30° (inclusive)
T(0,{-20},{-180:35:3})  1st time, lat nearest - 20°, every 3rd lon between -180° and 35°

44. Variable Subscripting (3 of 3)
Named Dimensions
only used for dimension reordering
indicated by |
dimension names must be used for each subscript
named/coordinate subscripting can be mixed
Consider T(time,lat,lon)
t = T(lat|:, lon|:, time|:)  makes t(lat,lon,time)
t = T(time|:,{lon|90:120},{lat|-20:20})  all times,
90-120° lon, ° lat

45. Standard and Coordinate Subscripting
Latitude coordinate variable (1D)
Combined:
T({-10:20}, 1:8)
Longitude coordinate variable (1D)

46. Interactive: Variable_3
standard and coordinate subscripting
reorder via named dimensions
shape remains the same [same number of dimensions]
reshape via ndtooned and onedtond
number of dimensions changes

47. “printing” printVarSummary print write_matrix
gives gross overview of a variable
print
same info as printVarSummary
prints values
write_matrix
print to standard out or a file
format control of numerical output
can write to file also

48. printVarSummary Print out variable (data object) information
type
dimension information
coordinate information (if present)
attributes (if present)
printVarSummary (u)
Variable: u
Type: double
Total Size: bytes
values
Number of Dimensions: 4
Dimensions / Sizes: [time | 1] x [lev | 18] x [lat | 64] x [lon | 128]
Coordinates:
time: [ ]
lev: [ ]
lat: [ ]
lon: [ ]
Number of Attributes: 2
long_name: zonal wind component
units: m/s

49. print (1 of 3) Prints out all variable information including
meta data, values
T(lat,lon): print (T)
Variable: T
Type: float
Total Size: bytes
values
Number of Dimensions: 2
Dimensions / Sizes: [lat | 64] x [lon | 128]
Coordinates:
lat: [ ]
lon: [ ]
Number of Attributes: 2
long_name: Temperature
units: C
(0,0)
(0,1)
(0,2)
(0,3)
(0,4) etc. [entire T array will be printed]

50. print (2 of 3) can be used to print a subset of array
meta data, values
T(lat,lon): print( T(:,103) ) or print( T(:,{110}) )
Variable: T (subsection)
Type: float
Total Size: 256 bytes
64 values
Number of Dimensions: 1
Dimensions / Sizes: [lat | 64]
Coordinates:
lat: [ ]
Number of Attributes: 3
long_name: Temperature
units: C
lon: [ added ]
(0)
(1)
-25.1
-20.0
(4) etc.

51. print (3 of 3) print with embedded strings
no meta data
print ( "min(T)="+min(T)+" max(T)="+max(T) )
(0) min(T)= max(T)=
sprintf and sprinti provide formatting
often used in graphics
print ( "min(T) = "+ sprintf("%5.2f ", min(T)) )
(0) min(T) =
sprinti can left fill with zeros (ex: let n=3)
fnam = "h" + sprinti ("%0.5i", n) + ".nc"
print("file name = "+fnam)
(0) file name = h00003.nc

52. write_matrix(x[*][*], fmt, opt)
pretty-print 2D array (table) to standard out
integer, float, double
user format control (fmt)
T(N,M), N=7, M=5: write_matrix (T, “5f7.2”, False)
can also create an ASCII file
opt = True
= “foo.ascii” ; file name
write_matrix (T, “5f7.2”, opt)