User's Guide - Appendix B. Importing Data: File Formats

## Appendix B. Importing Data: File Formats

Partial Table-of-Contents

- B.1 General Array Importer: Keyword Information from Data Files
- B.2 Data Explorer Native Files
- Overview of the Native File Format
- Examples
- Syntax of the Native File Format
- Objects
- Group Objects
- Series Objects
- Multigrid Objects
- Composite Field Objects
- Field Objects
- Array Objects
- Constant Array Objects
- gridpositions Keyword
- Regular Array Objects
- Product Array Objects
- gridconnections Keyword
- Path Array Objects
- Mesh Array Objects
- Xform Objects
- String Objects
- Light Objects
- Camera Objects
- Clipped Objects
- Screen Objects
- Data Mode Clause
- Default Clause
- End Clause
- B.3 CDF Files
- B.4 netCDF Files
- Regular Grids
- B.5 netCDF Files: Complex Fields
- Irregular Arrays
- Series Data
- Examples
- B.6 HDF Files

  Importing your data into Data Explorer will be the first step in creating a visualization of that data. In order to take this step you should have some understanding of the Data Explorer data model and a working knowledge of a Field. An informal description of a Field is provided in Chapter 2. "Introduction to Visualization". A formal description is given in Chapter 3. "Understanding the Data Model".

  A number of methods for importing data are available for use with Data Explorer: the General Array Importer, Data Explorer native file format, netCDF, CDF, and HDF.

  ## B.1 General Array Importer: Keyword Information from Data Files

  In addition to the syntaxes for the grid, points, and positions keywords described in IBM Visualization Data Explorer QuickStart Guide, it is also possible to derive information for these keywords directly from the data file. This allows you to write "filters" for specific applications that output their data in a set format which includes the grid size within the file.

  The syntax for the grid keyword is (in addition to the syntax given in 5.3 , "Header File Syntax: Keyword Statements" in IBM Visualization Data Explorer QuickStart Guide):

                              bytes n                                      bytes n
      grid =  [format],[type]{lines n   , [skip, width] [skip, width],..  {lines n, ...
                              marker string                                marker string

  where:

  `format` | is the format in which the grid values will be found, and must be one of the following: `binary`, `ieee`, `text`, or `ascii`. The first two parameters are synonymous, as are the second two.
  `type` | is the type of the values, and should be one of the following:
      byte               int               short
      unsigned byte      signed int        signed short
      signed byte        unsigned int      unsigned short
  Note that in each of the three groupings shown here, the first and second (reading down) are equivalent to each other.
  `bytes`, `lines`, and `marker` | specify where to begin reading the grid values.
  `skip`, and `width` | are optional and should be used when two pieces of information are on the same line with other information separating them (see Example 2). If necessary, different portions of the grid specification can be read separately by repeating the bytes, lines, or marker specification (see Example 3).

  #### Example 1

  Suppose that the data file contains the following first line:

         dimensions 100 300

  You can specify that this information is to be derived from the data file by the following statement:

         grid = lines 0, 11, 3, 1, 3

  This specifies that 0 lines are to be skipped. Then 11 characters are skipped, and the first grid dimension is read from 3 characters. Then 1 character is skipped, and the second grid dimension is read from 3 characters.

  You could also have used the statement

         grid = bytes 11

  which simply specifies that the grid information will be found after skipping 11 bytes in the file.

  #### Example 2

  Suppose the that data file contains the following line (not at the top of the file)

        xdim = 5 ydim = 20

  You could use the statement

        grid = marker "xdim =", 0, 2, 8, 2

  This specifies that one should start reading after "xdim =," read the first dimension from 2 characters, skip 8 characters, then read the second dimension from 2 characters.

  #### Example 3

  suppose that the data file contains the following lines

        xsize = 20
        ysize = 30

  You could use the statements

        grid = marker "xsize =" X marker "ysize ="

  or

        grid = marker "xsize =",0,3 X lines 1,8,2

  The first specifies that the first dimension should be read following the marker "xsize =," and the second dimension should be read following the marker "ysize =." The second statement specifies that the first dimension should be read from 3 characters, after skipping 0 characters following "xsize =," and that the second dimension should be read from 2 characters after skipping 1 line and 8 characters.

  The syntax for the `points` keyword is (in addition to the syntax given in 5.3 , "Header File Syntax: Keyword Statements" in IBM Visualization Data Explorer QuickStart Guide):

                                 bytes n
      points =  [format],[type]{ lines n   , [skip, width]
                                 marker string

  where:

  `format` | is the format in which the grid values will be found, and must be one of the following: `binary`, `ieee`, `text`, or `ascii`. The first two parameters are synonymous, as are the second two. For `type` and the other parameters, see the preceding description of `grid`.

  The syntax for the `positions` keyword is (in addition to the syntax given in 5.3 , "Header File Syntax: Keyword Statements" in IBM Visualization Data Explorer QuickStart Guide):

                                  bytes n
      positions =[format],[type]{ lines n   , [skip, width]  | ?, ...
                                  marker string

  (This syntax may be used only if you are specifying regular positions: origin and delta pairs for each dimension. As described in IBM Visualization Data Explorer QuickStart Guide, the origins and deltas are specified as origin1, delta1, origin2, delta2, etc.)

  `format` | is the format in which the grid values will be found, and must be one of the following: `binary`, `ieee`, `text`, or `ascii`. The first two parameters are synonymous, as are the second two. For `type` and the other parameters, see the description of `grid` above. A question mark (?) signifies that the default should be used (origin=0 or delta=1) for a particular origin or delta value. This would be used if only the origins or only the deltas are to be found in the file.

  [ OpenDX Home at IBM | OpenDX.org ]