Gri: `open' command

Chapters: 1: Introduction 2: Simple example 3: Invocation 4: Finer Control 5: X-Y Plots 6: Contour Plots 7: Image Plots 8: Examples 9: Gri Commands 10: Programming 11: Environment 12: Emacs Mode 13: History 14: Installation 15: Gri Bugs 16: Test Suite 17: Gri in Press 18: Acknowledgments 19: License Indices: Concepts Commands Variables | ### 9.3.28: ``open`' There are two styles of ``open`' command. In the first style, a simple file is to be opened. In the second style a unix-like "pipe" is opened, i.e. Gri will read the output of a system command instead of a file. Opening Simple Files: Opening Pipes: Opening URLs: #### 9.3.28.1: Opening simple files Ascii Files: Binary Files: NetCDF Files: Ascii Files Most applications involve ascii files, and these are very easy to handle in Gri. For example given a data file named ``foo.dat`', just use the command
    open foo.dat
and then you can read the data using various commands. Thus a complete program might be
    open foo.dat
    read columns x y
    draw curve
If a filename contains blanks or punctuation symbols, you must put it in double quotes (``"`'), e.g.
    open "foo bar.dat"
Indeed, Gri accepts double-quotes on any ``open`' command and some folks use it on all commands, as a matter of habit. Gri can handle compressed files appropriately, e.g.
    open foo.data.gz
so that there is no need to uncompress data for use with Gri. Gri is quite persistant in looking for your file, and if a given file is not found, it will then check to see if a compressed version is available, and use that instead. Thus
    open foo.dat
will look for a file named ``foo.dat.gz`' if ``foo.dat`' is not available. (Only files compressed with the GNU ``gzip`' utility are handled.) If the ``open`' command was successful in opening the file, it will set the value of the synonym ``\.return_value.`' to the full pathname of the file. Thus, if ``open a.dat`' is done in directory ``/home/gri`', then ``\.return_value.`' will equal the string ``/home/gri/a.dat`'. Binary Files Like most computer programs, Gri has some trouble with binary files. One big issue is the so-called "endian" character of the computer. Some computers store multi-byte values with the most significant bytes first, while others store them with the most significant bytes last. The problem is that nothing is stored in data files to indicate which convention was employed. For this reason, a version of Gri compiled on a so-called "big-endian" computer will misinterpret multi-byte values that were created on a so-called "little-endian" computer. Many folks in the scientific community have converted to using the NetCDF format (see next section) for precisely this reason, since this format is independent of the endian character of the computer. Presuming an appropriate endian character, however, reading is straightforward. A command of the form
    open foo.dat binary
tells Gri that the data are stored in a binary format. With the above syntax, Gri expects images to be in ``unsigned char`' (8 bits), while other data, such as columns and grids, are expected to be in 32-bit format (suitable for reading into a so-called "float" variable in the C programming language). You may also specify the format directly, as in the following examples; Gri then interprets all data as being in the indicated format and then converts to the internal format before using the data.
    open \filename binary uchar
    open \filename binary 8bit
    open \filename binary int
    open \filename binary float
    open \filename binary double
    open \filename binary 16bit
As with ascii files, Gri will automatically uncompress any files that are compressed, and if it fails to find a given filename, it will try to open a compressed version of it (i.e. one with a ``.gz`' suffix). NetCDF Files The NetCDF format provides the best of both worlds. It is binary, so that data are relatively compact, and may be read very quickly. (Reading ascii data is time-consuming in C++, the language in which Gri is written.) But it does not suffer the endian problem problem of normal binary files (see previous section), since information about the endian character is stored in the file itself, and Gri uses this information to decode the data without difficulty, regardless of the endian characteristics of the computer on which Gri is running and of the computer that created the data. For more information on netCDF format, see ``http://www.unidata.ucar.edu/packages/netcdf/index.html`' here . The syntax of opening NetCDF files is as below
    open foo.nc netCDF
and the syntax for reading such files is described in sections on the various ``read`' commands (see e.g. see Read Columns). #### 9.3.28.2: Opening pipes Sometimes it makes sense to get Gri to work with the results of another command in the OS. Gri handles this by creating a so-called "pipe", thus reading the output from the other command. (Readers familiar with the unix OS will know what pipes are all about, and especially why they are a good thing. Other readers might wish to skip this section.) Suppose we wish to plot an x-y plot using just the first few lines of a datafile named ``foo.dat`'. Unix users will know that a good way to see the first few lines of such a file would be to type the command ``head foo.dat`'. They also know that these lines could be provided to a second unix command, named ``do_foo`' say, by the command ``head foo.dat | do_foo`'. This uses a so-called "pipe", designated by the vertical line (called a pipe symbol below). Gri can read the output from system commands by using a syntax in which the (quoted) system command ends in a pipe symbol, e.g.
    open "head foo.dat |"
1. as in the example above. Aside: When pipe-open commands are used, Gri creates a temporary file (often located in ``/usr/tmp`', but that varies with machine). This is automatically cleaned up when Gri completes executation, but if Gri dies (or is interrupted) before it finishes, you'll be left with an extra file in this temporary-storage directory. It's up to you to clean that directory up from time to time. Some common examples of pipe-open commands are given below. Comma-separated values are common in files created by, or intended for, spreadsheets. Since Gri expects data elements to be separated by blanks (or tabs), you'll have to convert the commas into blanks. There are many ways to do that using pipes, e.g. ``sed`' system utility, e.g.
       open "sed -e 's/,/ /g' foo.dat |"
   Other unix facilities, such as ``tr`' will also work, of course. If the file has headers, you'll want to remove them also. This can be done with the ``skip`' command (see Skip) but you could also do it at the open stage, e.g. to remove the first two lines, use
       open "sed -e 's/,/ /g' foo.dat | tail +2 |"
2. Manipulating column data is done by e.g.
       open "cat foo.dat | awk '{$1, $2 * 22}' |"
3. where ``awk`' has been used to multiply the second column in the file named ``foo.dat`' by 22. Time-based and geographical data are sometimes encountered. For an example, suppose that longitude/latitude (i.e. x/y) data are stored in Hour.minutesecond format, e.g. 12.2133 means hour 12, minute 21, second 33. Gri doesn't read HMS format, but gawk can be told to:
       open "cat datafile.HMS |        \
           awk '{                      \
           split($1, hms, \".\");      \
           h = hms[1];                 \
           m = int(hms[2] / 100);      \
           s = hms[2] - 100 * m;       \
           x = h + m / 60 + s / 3600;  \
           split($2, hms, \".\");      \
           h = hms[1];                 \
           m = int(hms[2] / 100);      \
           s = hms[2] - 100 * m;       \
           y = h + m / 60 + s / 3600;  \
           print(x,y)                  \
           }' | "
           read columns x y
4. Timeseries data are often stored in formats that blend letters and numbers. For one thing, using letters (e.g. ``aug`') removes an ambiguity in numerically-based data. (Example: 02/03/2000 means one thing to an American and another thing in the rest of the world. However, everybody agrees on what 2000-Feb-03 means.) Suppose, for example, that we have data in a format such as
       Tue_Jul_25_11:07:51 0.62
       Tue_Jul_25_11:22:51 0.59
       Tue_Jul_25_11:37:51 0.56
   (stored in a file called ``foo.dat`' say) and we want a graph of the y-variable (0.62, 0.59, 0.56) versus x-variable, time expressed say as seconds in the day. Then here is how that could be done:
       open "cat foo.dat |\
           sed -e 's/_/ /g' -e 's/:/ /g' |\
           awk '{print ($4*3600+$5*60+$6, $7)}' |"
       read columns x y
       draw curve
Note that the actual day information is skipped in this example; seasoned ``awk`' users could easily fill in the code to handle datasets spanning several days. #### 9.3.28.3: Opening URLs Gri can open a URL, if you have the ``wget`' program on your machine. (``wget`' is available from the GNU website ``http://www.gnu.org/software/wget/`'.) The URL must be enclosed in quotes (since otherwise, Gri will interpret the ``//`' sequence as indicating an old way of denoting comments). For example,
    open "http://gri.sourceforge.net/gridoc/examples/example1.dat"
    read columns x y
    show columns
If you don't have ``wget`' installed on your machine, the above won't work, but you can always use another fetching program, with a system call, as in the following:
    \url = "http://gri.sourceforge.net/gridoc/html/examples/example1.dat"
    open "lynx -dump \url |"
    read columns x y
    draw curve