________________________________________________________________________
PARSEC CONFIG FILE PARSER

A framework for constructing nested-INI-style config file formats with
automatic parsing, validation, default values, on-the-fly item
obsoletion, deprecation and upgrade, and site/user-style file override
(e.g. User's global.rc overrides site's overrides spec default values).
_____________________________
  USED IN CYLC FOR:
   * suite definition files
   * gcylc config files
   * site/user config files

________________________________________________________________________
DESIGN AND IMPLEMENTATION
______________
  PARSING
  lib/parsec/fileparse.py

Parses any nested-INI format file into a corresponding nested dict
structure, after "%include"-inlining, Jinja2 processing, and line
continuation joining. Also: trailing comments are stripped; single-,
double-, or un-quoted values; and triple-quoted multiline string values.
string-internal comments are retained.

______________
  VALIDATION
  lib/parsec/validate.py

Validates a config structure (from the parser) against a file spec that
defines the "file format" (next).

______________
  FILE SPEC
  e.g. lib/parsec/test_spec.py for lib/parsec/test.rc
  and cylc file specs in lib/cylc/cfgspec/

A nested dict that contains all legal items in their proper place, and
"values" that are *validator* objects prescribing the type of the value
(string, integer, etc.), other requirements, and default values. There
are pre-defined validators for string, integer, float; list (of strings,
integers, floats, and lists with multipliers as for cylc "retry
delays"); and cylc times. Other validators can be defined.

______________
  DEPRECATION AND AUTOMATIC UPGRADE
  lib/parsec/upgrade.py

Allows the file spec module to specify mapping between deprecated items
and their new counterparts, along with an upgrader object to do any
value transformation required.  If affected items are found in the
config file, a deprecation warning will be emitted and the item upgraded
on-the-fly for compatibility with the current file spec.

______________
  HIGH LEVEL "INTERFACE"(?)
  lib/parsec/loadcfg.py

Defines functions that combine parsing and validation for (presumably)
common use cases. Includes combine two config files with precedence
order.

________________________________________________________________________
TESTING

  * lib/parsec/test.rc
  An example config file that does (or should/will) contain all types of
  config item that parsec is supposed to support.

  * lib/parsec/test_spec.py
  Defines a spec for test.rc, and a unit test to parse, validate, and
  print the config file. Does not test deprecation/upgrade at this stage.

  * lib/cylc/parsec/upgrade.py
  Contains a unit test to test on-the-fly upgrade of deprecated items.

________________________________________________________________________
TODO
 * Some suite-definition-specific special behaviour (namely combining
   identical graph sections, and use of OrderedDicts for environment and
   directives sections) is encoded in parsec - we need a way to specify
   this sort of thing in file-specific spec modules.

 * Figure out how to use OrderedDicts *only* for environment (and
   directives?) sections from the outset. Currently all sections are
   added as OrderedDicts by the fileparser.

________________________________________________________________________
NOTE ON USE OF ORDERED DICTS

Files are parsed into ordered dicts, in case order is important. In cylc
it is important in at least two places: variables defined in [runtime]
[environment] sections may depend on other varialbes previously defined
in the same section; and in site/user config files, parsed host sections
need to remain ordered in case of an ordered hierarchy (specific to
general) of hostname match patterns.

This generally only matters for items parsed from config files, not for
the default values defined in file spec modules - so there is no need to
use ordered dicts in file specs unless the order of defaults matters.
