CDAT uses the Python language as a glue to link together the various tools. Python has one of the fastest growing user bases among programming languages today and is extensively documented online and in publications. It must be stressed that it is not essential to have any knowledge of Python before being able to use CDAT tools. The Graphical User Interface (VCDAT) can be used with no knowledge of Python and is the first topic introduced in this chapter. This is highly recommended for beginners. For the more adventurous beginner who wishes to learn the use of scripting capabilities, a brief background of the Python language and syntax is provided in the next section.
CDAT comes with a suite of tutorials in the form of online video tutorials and scripts to help you learn how to use it. These tutorials are described in brief to enable the user to find the appropriate tutorials and to give a flavor of the power of CDAT. Finally, the various documentation and help avenues available to the user are described in this chapter.
The beginning user is encouraged to try the VCDAT browser first. This is accomplished by simply typing the command "vcdat" at your shell prompt, i.e:
Online video tutorials are available through the CDAT home page at http://cdat.sf.net. These short streaming video tutorials are a convenient way to see VCDAT in action. VCDAT was designed to be used from left-to-right and top-to-bottom and has on-line help balloons to assist the user in navigating through the interface. That is, if the user becomes unsure of what to do at any give time, then by moving the mouse over the region of question and letting it rest will result in a help balloon popup with information to assist the user. VCDAT allows the user to enter command line instructions and logs most button click instructions in a script file for later reference. Although it is not required to learn CDAT scripting in order to use VCDAT, the interface can be used as a CDAT script-learning tool by translating every button press and keystroke into a script file. All scripts include comment lines to assist the user. The script file can be modified, saved, and executed by the user. Thus, helping the user learn how to read and write CDAT scripts. This facility also allows the non-interactive repetition of common tasks.
To demonstrate quickly, the use of VCDAT try the following example sequence of button clicks:
Example 1: To quickly browse and plot data:
Example 2: To select data ranges and plot type:
Example 3: To average over a dimension:
Example 4: To define a variable in memory:
There are six sections of the VCDAT interface (figure 1) that are listed here from top-to-bottom: Main Menu (i.e., File, Options, Tools, PCMDI Tools, Help); Select Variable (Directory, File, and Variable); Graphics (i.e., Plot, Boxfill, VCS Canvas 1, Options, Define); Dimensions (Z-time, Y-latitude, X-longitude); Defined Variables (Operational Icons, Defined Variable List, Function Icons); and Variable Information.
There are five static pull down options in VCDAT's main menu. In addition, the user can add their own customized pull down menus to VCDAT's main menu.
The main purpose of this section is to select a variable for analysis and/or display. There are five main components here: the "Directory"; the "Database", this component take the place of the "Directory" when it is invoked; the "File"; the "Datasets", this component takes the place of the "File" when the "Database" is invoked, and the Variable.
There are only four components to "Graphics", but because of screen real estate and design considerations, the "Define" button was added to this row. Therefore, "Graphics" consist of five components: the "Plot" button, the "Graphics Method" (e.g., "Boxfill", "Isofill", etc.) choice button, the "VCS Canvas" choice button, the graphics "Options" menu, and the "Define" button.
The "Dimension Panel" is blank if no variable or defined variable is selected. It is located just under the "Graphics" section. It can currently display up to six dimensions. But more display dimensions can be easily added if necessary. The displayed dimensions are separated by lines and contain four components: the "Dimension Menu"; the "Dimension Selection"; the "Dimension Sliders"; and the "Dimension Functions".
The "Defined Variables" section operates on variables that are stored in memory. Variables can be defined in memory by selecting the "Define" button located in the "Graphics" section. Variables can also be defined by operations located in the "Main Menu" section. For example: selecting from "Main Menu" -> "PCMDITools" -> "Statistics" -> "Mean" on a variable would produce "_Statistics_mean_variablename ()" in the "Defined Variables List". Other possible ways to produce a defined variable are: by the use of "Function Icons" and command line operations via VCDAT's "Command Line Window". For example: typing "import MV; a = MV.array((1,2,3))" in the VCDAT "Command Line Window" would produce "a (3)" in the "Defined Variables List".
Note that everything associated with defined variables has the same background color as the "Define" button located in the "Graphics" section.
The "Variable Information" section immediately displays variable or defined variable information to the user. To quickly browser through many variables and examine their content, resize the variable information panel by selecting the small button located to the upper right of the "Variable Information" text. While keeping the button depressed, move the panel to the desired location.
Python is documented down to the last detail at the Python language website http://python.org and the SourceForge website http://python.sourceforge.net. There are now many books discussing the use of Python and how to use it with other open source software. We can recommend the free tutorial available at www.python.org in the documentation section. The book "Learning Python" from O'Reilly Press is another excellent resource. This section provides a very brief introduction to some basic concepts aimed at "getting started". The interested reader can refer to any of the other sources referred to above for details.
We begin with the traditional "Hello World" program. To start the python interpreter type:
Python 2.2.1c2 (#1, Apr 11 2002, 12:36:10)
[GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Programs can be placed in files with a .py extension. The helloworld.py program would contain the following lines:
The program can then be executed by providing the name of the program file to the interpreter as follows:
A simple program can best illustrate the use of arithmetic expressions.
print degs, " degrees = ", rads, " radians"
The output of this program is the following table:
10.0 degrees = 0.174527777778 radians
20.0 degrees = 0.349055555556 radians
30.0 degrees = 0.523583333333 radians
40.0 degrees = 0.698111111111 radians
50.0 degrees = 0.872638888889 radians
60.0 degrees = 1.04716666667 radians
70.0 degrees = 1.22169444444 radians
80.0 degrees = 1.39622222222 radians
90.0 degrees = 1.57075 radians
Python is a dynamically typed language (i.e you do not need to declare the variable types apriori) and the names can represent different types depending on the arithmetic operations performed. In the above example, "degs" was initially set to the integer 0, and is seen in the first line printed. Subsequently a real number (10.) is added at each stage of the loop and this changed the type of "degs" as it can be seen that "degs" values printed are real. Note that the above example is contrived to illustrate this point and there is no real reason to add a real value 10.0 in the loop.
The output of the program looks less than ideally formatted. To make it look better, we can make use of format strings. For example:
>>> print "%3d degrees = %0.4f radians" %(degs, rads)
would produce output that looks like this:
The format strings %d, %s denote integers and strings respectively, and %g and %f denote floats.
The if and else statements provide an easy way to perform tests. For instance:
>>> print 'x is not equal to y'
The indentation is required to isolate the if and else clauses, but the else clause is optional. Do nothing clauses can be created by using the pass statement.
>>> print 'x and y are equal!'
Multiple test cases can be implemented using the elif clause.
Boolean expressions can be formed by using or, and, and not keywords.
>>> print 'z is the max value'
>>> if not (x==z or y==z or x==y):
To open a text file and read its contents you would
# The above line returns a file object
# The readline() method is invoked on file
# and one line is read from the file.
# The following section keeps printing and reading
# subsequent lines of data while there are new lines to
# The while loop is exited when there are no more lines
# to be read. To close the open file:
>>> fout = open("out.txt", `w')
# The `w' indicates file should be opened for writing
Lists and tuples are sequences of arbitrary objects. Lists can be created by:
>>> mylist = ["a", 1.0, "c", 4]
Note that you can mix items of any type in a list. You can also have lists nested inside lists.
>>> my_other_list = ["a", "b", [1,2,3], "d"]
The lists are indexed by integers starting with zero. To see the first item in mylist, you would:
# Changes the item in the index position 2.
To append new members to the list, the append() method is used as follows:
["a", 1.0, "x", 4, "nextitem"]
Similarly, individual items can be removed from the list using the remove() method:
You can also insert items into specific positions:
>>> mylist.insert(1, "inserted_item")
["a", "inserted_item", "x", 4, "nextitem"]
Lists can be concatenated by using the "+" operator:
>>> newlist = mylist + [9, 10, 11]
["a", "inserted_item", "x", 4, "nextitem", 9, 10, 11]
Tuples are very similar to lists and are constructed by using parentheses instead of brackets or just a comma-separated list.
>>> myvector = (magnitude, direction)
>>> myvector = magnitude, direction
Tuples support the same operations as are supported by lists except the appending or modification of elements after they are created. That is, you cannot modify elements or append elements to a tuple.
We saw the simple loop using the while statement in the previous examples. Other looping constructs such as the for statement are available to the user. It is important to note that statements within the loop are indented. An example of its usage is:
>>> print "10 raised to ", i, " is ", 10**i
Note that a=range(3) is equivalent to a=[0,1,2]. Similarly the range(1, 6) is equivalent to [1,2,3,4,5] and range(10, 8, -1) is equivalent to [10, 9]. To generalize, range(i, j, k) produces a list of integers from i to j-1 using a stride k. If i is omitted, it is taken to be zero and k defaults to 1 if omitted. A more efficient (in terms of memory and runtime) is the xrange() function used exactly like range().
A dictionary allows you to associate values index by keys. Dictionaries can be created using values in curly braces like this:
To access members of the dictionary, we use the key-indexing facility
>>> modelling_center = a["center"]
The keys associated with a dictionary can be obtained as a list by:
and you can check the dictionary membership by using the has_key() method:
A function can be created using the def statement as shown below.
Note once again that the statements inside the function are indented after the "def" statement. To invoke the function we do the following:
It is possible to return multiple values using comma separated names in the return statement inside the function.
In this case the function is invoked as follows:
>>> addvalue, subvalue = myaddsub(2, 5)
The function definition can also be done in a way such that default values for input parameters can be set.
Then, when you need base to take a different value than the default 10, you can
To keep your programs manageable as they grow in size, you may want to break them up into several files. Python allows you to put multiple function definitions into a file and use them as a module that can be imported into other scripts and programs. These files must have a .py extension. For example
To use the above module in other programs, you would use the import statement.
The tutorials are designed to introduce you to the most common operations for climate data analysis. They are available as a Python script from our download area under "Tutorials". If you want to try executing these examples, follow these steps:
There are five tutorial files provided with data required to run them. The tutorials are CDAT scripts that give the user a flavor of how scripts work and to give an idea of how to accomplish specific tasks. A brief description of the tutorials follows:
This tutorial is the first one to study. The tutorial consists of three examples:
This example runs through some basic operations such as opening, closing, writing to files, and reading data by location or at specified time intervals. The tutorial is presented with a series of questions and answers in the style of a FAQ.
This example shows (in excruciating detail) how to read in Fortran formatted data, use it within CDAT, and writing to a netCDF file. Some simple averaging operations are carried out on the data. This example also shows how to change/insert metadata for the variable.
This tutorial demonstrates the uses of the time averaging functions. Basic examples cover topics such as computing the December-February seasonal means, computing the climatology, anomalies (departures) from climatology, construction of seasons not already defined, customization, and use of powerful criteria to specify minimum temporal coverage and data distribution.
This tutorial covers some of the basic statistical functions such as rms, correlation, mean absolute difference and their usage with climate data.
This tutorial guides you through some basic plotting functions and features to visualize the data and produce presentation quality output. This is by no means an exhaustive demonstration of features - just a very basic set of capabilities are addressed. Specific examples cover animations, creating and altering graphics methods, creating and altering display templates, producing output files for printing and displaying and changing colormaps etc.
In this tutorial, the interface to the XmGrace utility is demonstrated by showing simple plot generation and customization. XmGrace is a plotting tool developed independent of CDAT and has a wide user base. To download and install XmGrace, see the Grace home page at http://plasma-gate.weizmann.ac.il/Grace
The CDAT home page http://cdat.sf.net is your source for documentation- both online and printable and support. Documentation is available to help users at various levels of expertise starting with the beginning user's guide (this document) to the advanced user who needs a quick reference to commands or details of usage in the programming API reference guides. CDAT makes heavy use of Numerical Python, a fast array facility for Python. The Numerical Python website at http://numpy.sourceforge.net contains extensive documentation.
The VCDAT graphical user interface has a "help" button that provides access to all the documentation that a user may require at the click of a button. The interface also has helpful information in pop-up balloons that instruct the user when the cursor is moved over the buttons.
Python also has a powerful feature: most objects in Python, including the modules, classes, and functions, have documentation strings ("docstrings") attached to them. The special attribute name "__doc__" (which has two underscores on each end) is used to access this string supplied by the module author. If you have an object x, try the command
You can also use the pydoc utility. This is a standard facility for extracting documentation from Python installations.
You can generate Python documentation in HTML or text for interactive use.
In the Python interpreter, do "from pydoc import help" to provide online help. Calling help(thing) on a Python object documents the object. Also, typing help() will put the user in a help environment indicated by the prompt changing to
At the shell command line outside of Python: Run "pydoc <name>" to show documentation on something. <name> may be the name of a function, module, package, or a dotted reference to a class or function within a module or module in a package. If the argument contains a path segment delimiter (e.g. slash on Unix, backslash on Windows) it is treated as the path to a Python source file. Run "pydoc -k <keyword>" to search for a keyword in the synopsis lines of all available modules.
pydoc -g starts an HTTP server and also pops up a little window for controlling it.
The documents cdms_quick_start.pdf and vcs_quick_start.pdf provide a quick look at useful commands in the core CDAT modules; cdms and vcs respectively on a single page that is useful even to the experienced user.
The core CDAT modules cdms and vcs are extensively documented in separate documents that can be downloaded from http://cdat.sf.net. The most up-to-date versions of the following guides can be found there.
The CDAT Home Page is http://cdat.sf.net. CDAT is hosted at SourceForge, a free service provided by VA Linux, Inc. to the Open Source Community. SourceForge enables us to provide the following services for users:
You can use these facilities without registering at SourceForge, but registration, which is quick, easy, and free, will enable you to participate in the fullest possible way. In particular, it is very helpful to us if you are registered when you submit a bug report.