\section{Suite.rc Reference}
\label{SuiteRCReference}

\lstset{language=bash}

This appendix defines all legal suite definition config items.
Embedded Jinja2 code (see~\ref{Jinja2}) must process to a valid
raw suite.rc file. See also~\ref{SuiteRCFile} for a descriptive
overview of suite.rc files, including syntax (\ref{Syntax}).

\subsection{Top Level Items}

The only top level configuration items at present are the suite title
and description.

\subsubsection{title}

A single line description of the suite. It is displayed in the db viewer
window and can be retrieved at run time with the
\lstinline=cylc show= command.

\begin{myitemize}
\item {\em type:} single line string
\item {\em default:} (none)
\end{myitemize}

\subsubsection{description}

A multi-line description of the suite. It can be retrieved by the db viewer
right-click menu, or at run time with the \lstinline=cylc show= command.

\begin{myitemize}
\item {\em type:} multi-line string
\item {\em default:} (none)
\end{myitemize}

\subsubsection{group}

A single line group name for a suite. It is displayed in the gscan
window and can be used as a way of grouping related suites together.
\lstinline=cylc show= command.

\begin{myitemize}
\item {\em type:} single line string
\item {\em default:} (none)
\end{myitemize}

\subsubsection{URL} \label{SuiteURL}

A web URL to suite documentation.  If present it can be browsed with the
\lstinline=cylc doc= command, or from the gcylc Suite menu.  The variable
\lstinline=${CYLC_SUITE_NAME}= will be replaced with the actual suite name
(note this is not a shell environment variable in this context). See also
task URLs (\ref{TaskURL}).

\begin{myitemize}
\item {\em type:} string (URL)
\item {\em default:} (none)
\item {\em example:} \lstinline=http://suites/${CYLC_SUITE_NAME}/index.html=
\end{myitemize}

\subsection{[cylc]}

This section is for configuration that is not specifically task-related.

\subsubsection[required run mode]{ [cylc] \textrightarrow required run mode}

If this item is set cylc will abort if the suite is not started in the
specified mode. This can be used for demo suites that have to be
run in simulation mode, for example, because they have been taken out of
their normal operational context; or to prevent accidental submission of
expensive real tasks during suite development.
\begin{myitemize}
    \item {\em type:} string
    \item {\em legal values:} live, dummy, dummy-local, simulation
    \item {\em default:} None
\end{myitemize}

\subsubsection[UTC mode]{ [cylc] \textrightarrow UTC mode}
\label{UTC-mode}

Cylc runs off the suite host's system clock by default. This item allows
you to run the suite in UTC even if the system clock is set to local time.
Clock-trigger tasks will trigger when the current UTC time is equal to
their cycle point date-time plus offset; other time values used, reported,
or logged by the suite daemon will usually also be in UTC. The default for
this can be set at the site level (see ~\ref{SiteUTCMode}).

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False, unless overridden at site level.
\end{myitemize}

\subsubsection[cycle point format]{ [cylc] \textrightarrow cycle point format}
\label{cycle-point-format}

To just alter the timezone used in the date/time cycle point format, see
\ref{cycle-point-time-zone}. To just alter the number of expanded year digits
(for years below 0 or above 9999), see
\ref{cycle-point-num-expanded-year-digits}.

Cylc usually uses a \lstinline=CCYYMMDDThhmmZ= (\lstinline=Z= in the special
case of UTC) or \lstinline=CCYYMMDDThhmm+hhmm= format (\lstinline=+= standing
for \lstinline=+= or \lstinline=-= here) for writing down date/time cycle
points, which follows one of the basic formats outlined in the ISO 8601
standard. For example, a cycle point on the 3rd of February 2001 at 4:50 in
the morning, UTC (+0000 timezone), would be written
\lstinline=20010203T0450Z=. Similarly, for the the 3rd of February 2001 at
4:50 in the morning, +1300 timezone, cylc would write
\lstinline=20010203T0450+1300=.

You may use the isodatetime library's syntax to write dates and times in ISO
8601 formats - \lstinline=CC= for century, \lstinline=YY= for decade and
decadal year, \lstinline=+X= for expanded year digits and their positive or
negative sign, thereafter following the ISO 8601 standard example notation
except for fractional digits, which are represented as \lstinline=,ii= for
\lstinline=hh=, \lstinline=,nn= for \lstinline=mm=, etc. For example, to write
date/times as week dates with fractional hours, set cycle point format to
\lstinline=CCYYWwwDThh,iiZ= e.g. \lstinline=1987W041T08,5Z= for 08:30 UTC on
Monday on the fourth ISO week of 1987.

You can also use a subset of the strptime/strftime POSIX standard - supported
tokens are \lstinline=\%F=, \lstinline=\%H=, \lstinline=\%M=, \lstinline=\%S=,
\lstinline=\%Y=, \lstinline=\%d=, \lstinline=\%j=, \lstinline=\%m=,
\lstinline=\%s=, \lstinline=\%z=.

To use the old cylc date-time format (e.g. \lstinline=2014020106= for 06:00
on the 1st of February 2014), set cycle point format to
\lstinline=\%Y\%m\%d\%H=.

Please note that using characters like "/" is not allowed, as it will break
task output files ("/" is a reserved character in POSIX file and directory
naming). Using ":" is also not allowed, as it is likely to interfere with
usage of commands like "rsync" when applied to task output files.

\subsubsection[cycle point num expanded year digits]{ [cylc] \textrightarrow
cycle point num expanded year digits}
\label{cycle-point-num-expanded-year-digits}

For years below 0 or above 9999, the ISO 8601 standard specifies that an
extra number of year digits and a sign should be used. This extra number needs
to be written down somewhere (here).

For example, if this extra number is set to 2, 00Z on the 1st of January in
the year 10040 will be represented as \lstinline=+0100400101T0000Z= (2 extra
year digits used). With this number set to 3, 06Z on the 4th of May 1985 would
be written as \lstinline=+00019850504T0600Z=.

This number defaults to 0 (no sign or extra digits used).

\subsubsection[cycle point time zone]{ [cylc] \textrightarrow
cycle point time zone}
\label{cycle-point-time-zone}

If you set UTC mode to True (\ref{UTC-mode}) then this will default to
\lstinline=Z=. If you use a custom cycle point format
(\ref{cycle-point-format}), you should specify the timezone choice (or null
timezone choice) here as well.

You may set your own time zone choice here, which will be used for all
date/time cycle point dumping. Time zones should be expressed as ISO 8601 time
zone offsets from UTC, such as \lstinline=+13=, \lstinline=+1300=,
\lstinline=-0500= or \lstinline=+0645=, with \lstinline=Z= representing the
special \lstinline=+0000= case. Cycle points will be converted to the time
zone you give and will be represented with this string at the end.

Cycle points that are input without time zones (e.g. as an initial cycle point
setting) will use this time zone if set. If this isn't set (and UTC mode is
also not set), then they will default to the current local time zone.

Note that the ISO standard also allows writing the hour and minute separated
by a ":" (e.g. \lstinline=+13:00=) - however, this is not recommended, given
that the time zone is used as part of task output filenames.

\subsubsection[abort if any task fails]{[cylc] \textrightarrow abort if any task fails}

Cylc does not normally abort if tasks fail, but if this item is turned
on it will abort with exit status 1 if any task fails.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False
\end{myitemize}

\subsubsection[health check interval]{[cylc] \textrightarrow health check interval}
\label{health-check-interval}

Specify the time interval on which a running cylc suite will check that its run
directory exists and that its contact file contains the expected information.
If not, the suite will shut itself down automatically.

\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation (e.g.
 \lstinline=PT5M=, 5 minutes (note: by contrast, \lstinline=P5M= means 5
 months, so remember the \lstinline=T=!)).
    \item {\em default:} PT10M
\end{myitemize}

\subsubsection[task event mail interval]{[cylc] \textrightarrow task event mail interval}
\label{task-event-mail-interval}

Group together all the task event mail notifications into a single email within
a given interval. This is useful to prevent flooding users' mail boxes when
many task events occur within a short period of time.

\begin{myitemize}
  \item {\em type:} ISO 8601 duration/interval representation (e.g. \lstinline=PT10S=, 10 seconds, or \lstinline=PT1M=, 1 minute).
  \item {\em default: PT5M}
\end{myitemize}

\subsubsection[disable automatic shutdown]{[cylc] \textrightarrow disable automatic shutdown}

This has the same effect as the \lstinline{--no-auto-shutdown} flag for
the suite run commands: it prevents the suite daemon from shutting down
normally when all tasks have finished (a suite timeout can still be used to
stop the daemon after a period of inactivity, however).  This option can
make it easier to re-trigger tasks manually near the end of a suite run,
during suite development and debugging.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False
\end{myitemize}

\subsubsection[log resolved dependencies]{[cylc] \textrightarrow log resolved dependencies}

If this is turned on cylc will write the resolved dependencies of each
task to the suite log as it becomes ready to run (a list of the IDs of
the tasks that actually satisfied its prerequisites at run time). Mainly
used for cylc testing and development.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False
\end{myitemize}

\subsubsection[{[[}parameters{]]}]{[cylc] \textrightarrow [[parameters]]}

Define parameter values here for use in expanding {\em parameterized tasks} -
see Section~\ref{Parameterized Tasks}.
\begin{myitemize}
    \item {\em type:} list of strings, or an integer range
        \lstinline=LOWER..UPPER..STEP= (two dots, inclusive bounds, STEP optional)
    \item {\em default:} (none)
    \item {\em examples:}
        \begin{myitemize}
            \item \lstinline@run = control, test1, test2@
            \item \lstinline@mem = 1..5@  (equivalent to \lstinline@1, 2, 3, 4, 5@).
        \end{myitemize}
\end{myitemize}

\subsubsection[{[[}parameter templates{]]}]{[cylc] \textrightarrow [[parameter templates]]}
\label{RefParameterTemplates}

Parameterized task names (see previous item, and Section~\ref{Parameterized
Tasks}) are expanded, for each parameter value, using string templates.  You
can assign templates to parameter names here, to override the default templates.

\begin{myitemize}
    \item {\em type:} a Python-style string template
    \item {\em default} for an integer-valued parameter \lstinline=p=:
        \lstinline=_p%(p)s= \\
        e.g.\ \lstinline=foo<run>= becomes \lstinline=foo_run3= for
            \lstinline@run@ value \lstinline@3@.
    \item {\em default} for a non integer-valued parameter \lstinline=p=:
        \lstinline=_%(p)s= \\
        e.g.\ \lstinline=foo<run>= becomes \lstinline=foo_top= for
            \lstinline@run@ value \lstinline@top@.
    \item {\em example:} \lstinline@run = -R%(run)s@ \\
            e.g.\ \lstinline=foo<run>= becomes \lstinline=foo-R3= for
            \lstinline@run@ value \lstinline@3@.
\end{myitemize}

Note that the values of a parameter named \lstinline=p= are substituted for
\lstinline=%(p)s=.  The \lstinline=s= indicates a string value and should
be used even for integer-valued parameters, because cylc converts integer
parameter values to strings, with (depending on size) zero-padding. In
\lstinline=_run%(run)s= the first ``run'' is a string literal, and the second
gets substituted with each value of the parameter.

\subsubsection[{[[}events{]]}]{[cylc] \textrightarrow [[events]]}
\label{SuiteEventHandling}

Cylc has internal ``hooks'' to which you can attach handlers that are
called by the suite daemon whenever certain events occur. This section
configures suite event hooks; see~\ref{TaskEventHandling} for
task event hooks.

Event handler commands can send an email or an SMS, call a pager, intervene in
the operation of their own suite, or whatever.
They can be held in the suite bin directory, otherwise it is up to you
to ensure their location is in \lstinline=$PATH= (in the shell in which
cylc runs, on the suite host). The commands should require
very little resource to run and should return quickly.

Each event handler can be specified as a list of command lines or command
line templates.

A command line template may have any or all of these patterns which will be
substituted with actual values:
\begin{myitemize}
    \item \%(event)s: event name (see below)
    \item \%(suite)s: suite name
    \item \%(suite\_url)s: suite URL
    \item \%(message)s: event message, if any
\end{myitemize}

Otherwise, the command line will be called with the following command line
arguments:
\begin{lstlisting}
<suite-event-handler> %(event)s %(suite)s '%(message)s'
\end{lstlisting}

Additional information can be passed to event handlers via
[cylc] \textrightarrow [[environment]].

\paragraph[EVENT handler]{[cylc] \textrightarrow [[events]] \textrightarrow EVENT handler}

A comma-separated list of one or more event handlers to call when one of the
following EVENTs occurs:
\begin{myitemize}
    \item {\bf startup}  - the suite has started running
    \item {\bf shutdown} - the suite is shutting down
    \item {\bf timeout}  - the suite has timed out
    \item {\bf stalled} - the suite has stalled
    \item {\bf inactivity} - the suite is inactive
\end{myitemize}

Default values for these can be set at the site level via the siterc file
(see ~\ref{SiteCylcHooks}).

Item details:
\begin{myitemize}
    \item {\em type:} string (event handler script name)
    \item {\em default:} None, unless defined at the site level.
    \item {\em example:} \lstinline@startup handler = my-handler.sh@
\end{myitemize}

\paragraph[handlers]{[cylc] \textrightarrow [[[events]]] \textrightarrow handlers}

Specify the general event handlers as a list of command lines or command line
templates.

\begin{myitemize}
    \item {\em type:} Comma-separated list of strings (event handler command line or command line templates).
    \item {\em default:} (none)
    \item {\em example:} \lstinline@handlers = my-handler.sh@
\end{myitemize}

\paragraph[handler events]{[cylc] \textrightarrow [[events]] \textrightarrow handler events}

Specify the events for which the general event handlers should be invoked.

\begin{myitemize}
    \item {\em type:} Comma-separated list of events
    \item {\em default:} (none)
    \item {\em example:} \lstinline@handler events = timeout, shutdown@
\end{myitemize}

\paragraph[mail events]{[cylc] \textrightarrow [[events]] \textrightarrow mail events}

Specify the suite events for which notification emails should be sent.

\begin{myitemize}
    \item {\em type:} Comma-separated list of events
    \item {\em default:} (none)
    \item {\em example:} \lstinline@handler = startup, shutdown, timeout@
\end{myitemize}

\paragraph[mail footer]{[cylc] \textrightarrow [[events]] \textrightarrow mail footer}

Specify a string or string template to insert to footers of notification emails
for both suite events and task events.

A template string may have any or all of these patterns which will be
substituted with actual values:
\begin{myitemize}
    \item \%(host)s: suite host name
    \item \%(port)s: suite port number
    \item \%(owner)s: suite owner name
    \item \%(suite)s: suite name
\end{myitemize}

\begin{myitemize}
    \item {\em type:} 
    \item {\em default:} (none)
    \item {\em example:} \lstinline@mail footer = see: http://localhost/%(owner)s/notes-on/%(suite)s/@
\end{myitemize}

\paragraph[mail from]{[cylc] \textrightarrow [[events]] \textrightarrow mail from}

Specify an alternate \lstinline=from:= email address for suite event notifications.

\begin{myitemize}
    \item {\em type:} string
    \item {\em default:} None, (notifications@HOSTNAME)
    \item {\em example:} \lstinline|mail from = no-reply@your-org|
\end{myitemize}

\paragraph[mail smtp]{[cylc] \textrightarrow [[events]] \textrightarrow mail smtp}

Specify the SMTP server for sending suite event email notifications.

\begin{myitemize}
    \item {\em type:} string
    \item {\em default:} None, (localhost:25)
    \item {\em example:} \lstinline@mail smtp = smtp.yourorg@
\end{myitemize}

\paragraph[mail to]{[cylc] \textrightarrow [[events]] \textrightarrow mail to}

A list of email addresses to send suite event notifications. The list can be
anything accepted by the \lstinline=mail= command.

\begin{myitemize}
    \item {\em type:} string
    \item {\em default:} None, (USER@HOSTNAME)
    \item {\em example:} \lstinline@mail to = your.colleague@
\end{myitemize}

\paragraph[timeout]{[cylc] \textrightarrow [[events]] \textrightarrow timeout}

If a timeout is set and the timeout event is handled, the timeout event
handler(s) will be called if the suite stays in a stalled state for some period
of time. The timer is set initially at suite start up. It is possible to set a
default for this at the site level (see ~\ref{SiteCylcHooks}).

\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation (e.g.
 \lstinline=PT5S=, 5 seconds, \lstinline=PT1S=, 1 second) - minimum 0 seconds.
    \item {\em default:} (none), unless set at the site level.
\end{myitemize}

\paragraph[inactivity]{[cylc] \textrightarrow [[events]] \textrightarrow inactivity}

If inactivity is set and the inactivity event is handled, the inactivity event
handler(s) will be called if there is no activity in the suite for some period
of time. The timer is set initially at suite start up. It is possible to set a
default for this at the site level (see ~\ref{SiteCylcHooks}).

\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation (e.g.
 \lstinline=PT5S=, 5 seconds, \lstinline=PT1S=, 1 second) - minimum 0 seconds.
    \item {\em default:} (none), unless set at the site level.
\end{myitemize}

\paragraph[reset timer]{[cylc] \textrightarrow [[events]] \textrightarrow reset timer}

If \lstinline=True= (the default) the suite timer will continually reset
after any task changes state, so you can time out after some interval
since the last activity occured rather than on absolute suite execution
time.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} True
\end{myitemize}

\paragraph[abort on stalled]{[cylc] \textrightarrow [[events]] \textrightarrow abort on stalled}

If this is set to True it will cause the suite to abort with error status
if it stalls. A suite is considered "stalled" if there are no active,
queued or submitting tasks or tasks waiting for clock triggers to be met. It is
possible to set a default for this at the site level
(see ~\ref{SiteCylcHooks}).

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False, unless set at the site level.
\end{myitemize}

\paragraph[abort on timeout]{[cylc] \textrightarrow [[events]] \textrightarrow abort on timeout}

If a suite timer is set (above) this will cause the suite to abort with
error status if the suite times out while still running. It is possible to set
a default for this at the site level (see ~\ref{SiteCylcHooks}).

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False, unless set at the site level.
\end{myitemize}

\paragraph[abort on inactivity]{[cylc] \textrightarrow [[events]] \textrightarrow abort on inactivity}

If a suite inactivity timer is set (above) this will cause the suite to abort
with error status if the suite is inactive for some period while still running.
It is possible to set a default for this at the site level
(see ~\ref{SiteCylcHooks}).

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False, unless set at the site level.
\end{myitemize}

\paragraph[abort if startup handler fails]{[cylc] \textrightarrow [[events]] \textrightarrow abort if EVENT handler fails}

Cylc does not normally care whether an event handler succeeds or fails,
but if this is turned on the EVENT handler will be executed in the
foreground (which will block the suite while it is running) and the
suite will abort if the handler fails.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False
\end{myitemize}

\subsubsection[{[[}environment{]]} ]{[cylc] \textrightarrow [[environment]]}

Environment variables defined in this section are passed to suite and
task event handlers.

\begin{myitemize}
    \item These variables are not passed to tasks - use task runtime
        variables for that. Similarly, task runtime variables are not
        available to event handlers - which are executed by the suite daemon,
        (not by running tasks) in response to task events.

    \item Cylc-defined environment variables such as
        \lstinline=$CYLC_SUITE_RUN_DIR= are not passed to task event
        handlers by default, but you can make them available by
        extracting them to the cylc environment like this:
\begin{lstlisting}
[cylc]
    [[environment]]
        CYLC_SUITE_RUN_DIR = $CYLC_SUITE_RUN_DIR
\end{lstlisting}

    \item These variables - unlike task execution environment variables
        which are written to job scripts and interpreted by the shell at
        task run time - are not interpreted by the shell prior to use
        so shell variable expansion expressions cannot be used here.
\end{myitemize}

\paragraph[\_\_VARIABLE\_\_ ]{[cylc] \textrightarrow [[environment]] \textrightarrow \_\_VARIABLE\_\_}

Replace \_\_VARIABLE\_\_ with any number of environment variable
assignment expressions.
Values may refer to other local environment variables (order of
definition is preserved) and are not evaluated or manipulated by
cylc, so any variable assignment expression that is legal in the
shell in which cylc is running can be used (but see the warning
above on variable expansions, which will not be evaluated).
White space around the `$=$' is allowed (as far as cylc's file
parser is concerned these are just suite configuration items).

\begin{myitemize}
    \item {\em type:} string
    \item {\em default:} (none)
    \item {\em examples:}
        \begin{myitemize}
            \item \lstinline@FOO = $HOME/foo@
        \end{myitemize}
\end{myitemize}


\subsubsection[{[[}reference test{]]}]{[cylc] \textrightarrow [[reference test]] }
\label{ReferenceTestConfig}

Reference tests are finite-duration suite runs that abort with non-zero
exit status if cylc fails, if any task fails, if the suite times
out, or if a shutdown event handler that (by default) compares the test
run with a reference run reports failure. See~\ref{AutoRefTests}.

\paragraph[suite shutdown event handler]{[cylc] \textrightarrow [[reference test]] \textrightarrow suite shutdown event handler}

A shutdown event handler that should compare the test run with the
reference run, exiting with zero exit status only if the test run
verifies.

\begin{myitemize}
    \item {\em type:} string (event handler command name or path)
    \item {\em default:} \lstinline=cylc hook check-triggering=
\end{myitemize}
As for any event handler, the full path can be ommited if the script is
located somewhere in \lstinline=$PATH= or in the suite bin directory.

\paragraph[required run mode]{[cylc] \textrightarrow [[reference test]] \textrightarrow required run mode}

If your reference test is only valid for a particular run mode, this
setting will cause cylc to abort if a reference test is attempted
in another run mode.

\begin{myitemize}
    \item {\em type:} string
    \item {\em legal values:} live, dummy, dummy-local, simulation
    \item {\em default:} None
\end{myitemize}

\paragraph[allow task failures]{[cylc] \textrightarrow [[reference test]] \textrightarrow allow task failures}

A reference test run will abort immediately if any task fails, unless
this item is set, or a list of {\em expected task failures} is provided
(below).

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False
\end{myitemize}

\paragraph[expected task failures]{[cylc] \textrightarrow [[reference test]] \textrightarrow expected task failures}

A reference test run will abort immediately if any task fails, unless
{\em allow task failures} is set (above) or the failed task is found
in a list IDs of tasks that are expected to fail.

\begin{myitemize}
    \item {\em type:} Comma-separated list of strings (task IDs: \lstinline=name.cycle_point=).
    \item {\em default:} (none)
    \item {\em example:} \lstinline=foo.20120808, bar.20120908=
\end{myitemize}

\paragraph[live mode suite timeout]{[cylc] \textrightarrow [[reference test]] \textrightarrow live mode suite timeout}

The timeout value, expressed as an ISO 8601 duration/interval, after which the
test run should be aborted if it has not finished, in live mode. Test runs
cannot be done in live mode unless you define a value for this item, because
it is not possible to arrive at a sensible default for all suites.

\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation, e.g.
        \lstinline=PT5M= is 5 minutes (note: by contrast \lstinline=P5M= means 5
         months, so remember the \lstinline=T=!).
    \item {\em default:} PT1M (1 minute)
\end{myitemize}

\paragraph[simulation mode suite timeout]{[cylc] \textrightarrow [[reference test]] \textrightarrow simulation mode suite timeout}

The timeout value in minutes after which the test run should be aborted
if it has not finished, in simulation mode. Test runs cannot be done in
simulation mode unless you define a value for this item, because it is
not possible to arrive at a sensible default for all suites.

\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation (e.g.
 \lstinline=PT5M=, 5 minutes (note: by contrast, \lstinline=P5M= means 5
 months, so remember the \lstinline=T=!)).
    \item {\em default:} PT1M (1 minute)
\end{myitemize}

\paragraph[dummy mode suite timeout]{[cylc] \textrightarrow [[reference test]] \textrightarrow dummy mode suite timeout}

The timeout value, expressed as an ISO 8601 duration/interval, after which the
test run should be aborted if it has not finished, in dummy mode.  Test runs
cannot be done in dummy mode unless you define a value for this item, because
it is not possible to arrive at a sensible default for all suites.

\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation (e.g.
 \lstinline=PT5M=, 5 minutes (note: by contrast, \lstinline=P5M= means 5
 months, so remember the \lstinline=T=!)).
    \item {\em default:} PT1M (1 minute)
\end{myitemize}

\subsubsection[{[[}authentication{]]}]{[cylc] \textrightarrow [[authentication]] }
\label{SuiteAuth}

Authentication of client programs with suite daemons can be set in the
global site/user config files and overridden here if necessary.
See~\ref{GlobalAuth} for more information.

\paragraph[public]{[cylc] \textrightarrow [[authentication]] \textrightarrow public}

The client privilege level granted for public access - i.e.\ no suite passphrase
required.  See~\ref{GlobalAuth} for legal values.

\subsubsection[{[[}simulation{]]} ]{[cylc] \textrightarrow [[simulation]]}

Suite-level configuration for the {\em simulation} and {\em dummy} run modes
described in Section~\ref{SimulationMode}.

\paragraph[disable suite event handlers]{[cylc] \textrightarrow [[simulation]] \textrightarrow disable suite event handlers}

If this is set to \lstinline=True= configured suite event handlers will not be
called in simulation or dummy modes.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} \lstinline=True=
\end{myitemize}

\subsection{[scheduling]}

This section allows cylc to determine when tasks are ready to run.

\subsubsection[cycling]{ [scheduling] \textrightarrow cycling mode }
\label{cycling-mode}

Cylc runs using the proleptic Gregorian calendar by default. This item allows
you to either run the suite using the 360 day calendar (12 months of 30 days
in a year) or using integer cycling.

\begin{myitemize}
    \item {\em type:} string
    \item {\em legal values:} gregorian, 360day, integer
    \item {\em default:} gregorian

\end{myitemize}

\subsubsection[initial cycle point]{[scheduling] \textrightarrow initial cycle point}
\label{initial cycle point}

In a cold start each cycling task (unless specifically excluded under
[special tasks]) will be loaded into the suite with this cycle point,
or with the closest subsequent valid cycle point for the task.  This item can
be overridden on the command line or in the gcylc suite start panel.

In date-time cycling, if you do not provide time zone information for this,
it will be assumed to be local time, or in UTC if~\ref{UTC-mode} is set, or in
the time zone determined by \ref{cycle-point-time-zone} if that is set.

\begin{myitemize}
    \item {\em type:} ISO 8601 date/time point representation (e.g.
 \lstinline=CCYYMMDDThhmm=, 19951231T0630) or ``now''.
    \item {\em default:} (none)
\end{myitemize}

The string ``now'' converts to the current date-time on the suite host (adjusted
to UTC if the suite is in UTC mode but the host is not) to minute resolution.
Minutes (or hours, etc.) may be ignored depending on your cycle point format
(\ref{cycle-point-format}).

\subsubsection[final cycle point]{[scheduling] \textrightarrow final cycle point}

Cycling tasks are held once they pass the final cycle point, if one is
specified. Once all tasks have achieved this state the suite will shut
down. If this item is provided you can override it on the command line
or in the gcylc suite start panel.

In date-time cycling, if you do not provide time zone information for this,
it will be assumed to be local time, or in UTC if \ref{UTC-mode} is set, or in
the \ref{cycle-point-time-zone} if that is set.

\begin{myitemize}
    \item {\em type:} ISO 8601 date/time point representation (e.g.
 \lstinline=CCYYMMDDThhmm=, 19951231T1230) or ISO 8601 date/time offset
    (e.g. +P1D+PT6H)
    \item {\em default:} (none)
\end{myitemize}

\subsubsection[initial cycle point constraints]{[scheduling] \textrightarrow initial cycle point constraints}
\label{initial cycle point constraints}

In a cycling suite it is possible to restrict the initial cycle point by
defining a list of truncated time points under the initial cycle point
constraints.

\begin{myitemize}
    \item {\em type:} Comma-separated list of ISO 8601 truncated time point
        representations (e.g. T00, T06, T-30).
    \item {\em default:} (none)
\end{myitemize}

\subsubsection[final cycle point constraints]{[scheduling] \textrightarrow final cycle point constraints}
\label{final cycle point constraints}

In a cycling suite it is possible to restrict the final cycle point by
defining a list of truncated time points under the final cycle point
constraints.

\begin{myitemize}
    \item {\em type:} Comma-separated list of ISO 8601 truncated time point
        representations (e.g. T00, T06, T-30).
    \item {\em default:} (none)
\end{myitemize}

\subsubsection[hold after point]{[scheduling] \textrightarrow hold after point}

Cycling tasks are held once they pass the hold after cycle point, if one is
specified. Unlike the final cycle point suite will not shut down once all tasks
have passed this point. If this item is provided you can override it on the
command line or in the gcylc suite start panel.

\subsubsection[runahead limit]{[scheduling] \textrightarrow runahead limit}
\label{runahead limit}

Runahead limiting prevents the fastest tasks in a suite from getting too far
ahead of the slowest ones, as documented in~\ref{RunaheadLimit}.

This config item specifies a hard limit as a cycle interval between the
slowest and fastest tasks. It is deprecated in favour of the newer default
limiting by \lstinline=max active cycle points= (\ref{max active cycle points}).

\begin{myitemize}
    \item {\em type:} Cycle interval string e.g. \lstinline=PT12H=
    for a 12 hour limit under ISO 8601 cycling.
    \item {\em default:} (none)
\end{myitemize}

\subsubsection[max active cycle points]{[scheduling] \textrightarrow
 max active cycle points}
\label{max active cycle points}

Runahead limiting prevents the fastest tasks in a suite from getting too far
ahead of the slowest ones, as documented in~\ref{RunaheadLimit}.

This config item supersedes the deprecated hard \lstinline=runahead limit=
(\ref{runahead limit}). It allows up to \lstinline=N= (default 3) consecutive
cycle points to be active at any time, adjusted up if necessary for
any future triggering.

\begin{myitemize}
    \item {\em type:} integer
    \item {\em default:} 3
\end{myitemize}

\subsubsection[spawn to max active cycle points]{[scheduling] \textrightarrow
 spawn to max active cycle points}
\label{spawn to max active cycle points}

Allows tasks to spawn out to \lstinline=max active cycle points=
(\ref{max active cycle points}), removing restriction that a task has to have
submitted before its successor can be spawned.

{\em Important:} This should be used with care given the potential impact of
additional task proxies both in terms of memory and cpu for the cylc daemon as
well as overheads in rendering all the additional tasks in gcylc. Also, use
of the setting may highlight any issues with suite design relying on the
default behaviour where downstream tasks would otherwise be waiting on ones
upstream submitting and the suite would have stalled e.g. a housekeeping task
at a later cycle deleting an earlier cycle's data before that cycle has had
chance to run where previously the task would not have been spawned until its
predecessor had been submitted.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False
\end{myitemize}

\subsubsection[{[[}queues{]]}]{[scheduling] \textrightarrow [[queues]]}

Configuration of internal queues, by which the number of simultaneously
active tasks (submitted or running) can be limited, per queue. By
default a single queue called {\em default} is defined, with all tasks
assigned to it and no limit. To use a single queue for the whole suite
just set the limit on the {\em default} queue as required.
See also~\ref{InternalQueues}.

\paragraph[{[[[}\_\_QUEUE\_\_{]]]}]{[scheduling] \textrightarrow [[queues]] \textrightarrow [[[\_\_QUEUE\_\_]]]}

Section heading for configuration of a single queue. Replace
\_\_QUEUE\_\_ with a queue name, and repeat the section as required.

\begin{myitemize}
\item {\em type:} string
\item {\em default:} ``default''
\end{myitemize}

\paragraph[limit]{[scheduling] \textrightarrow [[queues]] \textrightarrow [[[\_\_QUEUE\_\_]]] \textrightarrow limit}

The maximum number of active tasks allowed at any one time, for this queue.
\begin{myitemize}
\item {\em type:} integer
\item {\em default:} 0 (i.e.\ no limit)
\end{myitemize}

\paragraph[members]{[scheduling] \textrightarrow [[queues]] \textrightarrow [[[\_\_QUEUE\_\_]]] \textrightarrow members}

A list of member tasks, or task family names, to assign to this queue
(assigned tasks will automatically be removed from the default queue).
\begin{myitemize}
\item {\em type:} Comma-separated list of strings (task or family names).
\item {\em default:} none for user-defined queues; all tasks for the ``default'' queue
\end{myitemize}

\subsubsection[{[[}special tasks{]]}]{[scheduling] \textrightarrow [[special tasks]]}

This section is used to identify tasks with special behaviour. Family names can
be used in special task lists as shorthand for listing all member tasks.

\paragraph[clock-trigger]{[scheduling] \textrightarrow [[special tasks]] \textrightarrow clock-trigger}

Clock-trigger tasks (see~\ref{ClockTriggerTasks}) wait on a wall clock time
specified as an offset from their own cycle point.

\begin{myitemize}
    \item {\em type:} Comma-separated list of task or family names with
        associated date-time offsets expressed as ISO8601 interval strings,
        positive or negative, e.g. \lstinline=PT1H= for 1 hour.  The offset
        specification may be omitted to trigger right on the cycle point.
    \item {\em default:} (none)
    \item {\em example:}
\begin{lstlisting}
    clock-trigger = foo(PT1H30M), bar(PT1.5H), baz
\end{lstlisting}
\end{myitemize}

\paragraph[clock-expire]{[scheduling] \textrightarrow [[special tasks]] \textrightarrow clock-expire}
\label{ClockExpireRef}

Clock-expire tasks enter the {\em expired} state and skip job submission if too
far behind the wall clock when they become ready to run.  The expiry time is
specified as an offset from wall-clock time; typically it should be negative -
see~\ref{ClockExpireTasks}.

\begin{myitemize}
    \item {\em type:} Comma-separated list of task or family names with
        associated date-time offsets expressed as ISO8601 interval strings,
        positive or negative, e.g. \lstinline=PT1H= for 1 hour.  The offset
        may be omitted if it is zero.
    \item {\em default:} (none)
    \item {\em example:}
\begin{lstlisting}
    clock-expire = foo(-P1D)
\end{lstlisting}
\end{myitemize}

\paragraph[external-trigger]{[scheduling] \textrightarrow [[special tasks]] \textrightarrow external-trigger}

Externally triggered tasks (see~\ref{ExternalTriggers}) wait on external events
reported via the \lstinline=cylc ext-trigger= command.  To constrain triggers
to a specific cycle point, include \lstinline=$CYLC_TASK_CYCLE_POINT= in the
trigger message string and pass the cycle point to the \lstinline=cylc ext-trigger= command.

\begin{myitemize}
    \item {\em type:} Comma-separated list of task names with associated
        external trigger message strings.
    \item {\em default:} (none)
    \item {\em example:} (note the comma and line-continuation character)
\begin{lstlisting}
    external-trigger = get-satx("new sat-X data ready"), \
                         get-saty("new sat-Y data ready for $CYLC_TASK_CYCLE_POINT")
\end{lstlisting}
\end{myitemize}


\paragraph[sequential]{[scheduling] \textrightarrow [[special tasks]] \textrightarrow sequential}

Sequential tasks are automatically given dependence on their own
predecessor. This is equivalent to use of explicit inter-cycle triggers
in the graph, except that the automatic version does not show in suite
graph visualization. For more on sequential tasks see~\ref{SequentialTasks}.

\begin{myitemize}
    \item {\em type:} Comma-separated list of task or family names.
    \item {\em default:} (none)
    \item {\em example:} \lstinline@sequential = foo, bar@
\end{myitemize}

\paragraph[exclude at start-up]{[scheduling] \textrightarrow [[special tasks]] \textrightarrow exclude at start-up}
\label{EASU}

Any task listed here will be excluded from the initial task pool (this
goes for suite restarts too). If an {\em inclusion} list is also
specified, the initial pool will contain only included tasks that have
not been excluded. Excluded tasks can still be inserted at run time.
Other tasks may still depend on excluded tasks if they have not been
removed from the suite dependency graph, in which case some manual
triggering, or insertion of excluded tasks, may be required.

\begin{myitemize}
    \item {\em type:} Comma-separated list of task or family names.
    \item {\em default:} (none)
\end{myitemize}

\paragraph[include at start-up]{[scheduling] \textrightarrow [[special tasks]] \textrightarrow include at start-up}
\label{IASU}

If this list is not empty, any task {\em not} listed in it will be
excluded from the initial task pool (this goes for suite restarts too).
If an {\em exclusion} list is also specified, the initial pool will
contain only included tasks that have not been excluded. Excluded tasks
can still be inserted at run time. Other tasks may still depend on
excluded tasks if they have not been removed from the suite dependency
graph, in which case some manual triggering, or insertion of excluded
tasks, may be required.

\begin{myitemize}
    \item {\em type:} Comma-separated list of task or family names.
    \item {\em default:} (none)
\end{myitemize}

\subsubsection[{[[}dependencies{]]}]{[scheduling] \textrightarrow [[dependencies]]}

The suite dependency graph is defined under this section.  You can plot
the dependency graph as you work on it, with \lstinline=cylc graph= or
by right clicking on the suite in the db viewer.  See
also~\ref{ConfiguringScheduling}.

\paragraph[graph]{ [scheduling] \textrightarrow [[dependencies]] \textrightarrow graph }

The dependency graph for a completely non-cycling suites can go here.
See also~\ref{GraphDescrip} below and~\ref{ConfiguringScheduling}, for graph
string syntax.
\begin{myitemize}
    \item {\em type:} string
    \item {\em example:} (see~\ref{GraphDescrip} below)
\end{myitemize}

\paragraph[{[[[}\_\_RECURRENCE\_\_{]]]}]{[scheduling] \textrightarrow [[dependencies]] \textrightarrow [[[\_\_RECURRENCE\_\_]]]}

\_\_RECURRENCE\_\_ section headings define the sequence of cycle points for
which the subsequent graph section is valid. These should be specified in
our ISO 8601 derived sequence syntax, or similar for integer cycling:
\begin{myitemize}
    \item {\em examples:}
        \begin{myitemize}
            \item date-time cycling:
                \lstinline@[[[T00,T06,T12,T18]]]@ or \lstinline@[[[PT6H]]]@
            \item integer cycling (stepped by 2):
                \lstinline@[[[P2]]]@
        \end{myitemize}
    \item {\em default:} (none)
\end{myitemize}

See~\ref{GraphTypes} for more on recurrence expressions, and how multiple graph
sections combine.

\subparagraph[graph]{[scheduling] \textrightarrow [[dependencies]] \textrightarrow [[[\_\_RECURRENCE\_\_]]] \textrightarrow graph }
\label{GraphDescrip}

The dependency graph for a given recurrence section goes here. Syntax examples
follow; see also~\ref{ConfiguringScheduling} and~\ref{TriggerTypes}.

\begin{myitemize}
\item {\em type:} string
\item {\em examples:}
  \begin{lstlisting}
graph = """
   foo => bar => baz & waz     # baz and waz both trigger off bar
   foo[-P1D-PT6H] => bar       # bar triggers off foo[-P1D-PT6H]
   baz:out1 => faz             # faz triggers off a message output of baz
   X:start => Y                # Y triggers if X starts executing
   X:fail => Y                 # Y triggers if X fails
   foo[-PT6H]:fail => bar      # bar triggers if foo[-PT6H] fails
   X => !Y                     # Y suicides if X succeeds
   X | X:fail => Z             # Z triggers if X succeeds or fails
   X:finish => Z               # Z triggers if X succeeds or fails
   (A | B & C ) | D => foo     # general conditional triggers
   foo:submit => bar           # bar triggers if foo is successfully submitted
   foo:submit-fail => bar      # bar triggers if submission of foo fails
   # comment
   """
  \end{lstlisting}
\item {\em default:} (none)
\end{myitemize}

\subsection{[runtime]}

This section is used to specify how, where, and what to execute when
tasks are ready to run. Common
configuration can be factored out in a multiple-inheritance hierarchy of
runtime namespaces that culminates in the tasks of the suite. Order of
precedence is determined by the C3 linearization algorithm as used to
find the {\em method resolution order} in Python language class
hiearchies. For details and examples see~\ref{NIORP}.

\subsubsection[{[[}\_\_NAME\_\_{]]}]{[runtime] \textrightarrow [[\_\_NAME\_\_]]}

Replace \_\_NAME\_\_ with a namespace name, or a comma-separated list of
names, and repeat as needed to define all tasks in the suite. Names may
contain letters, digits, underscores, and hyphens. A namespace
represents a group or family of tasks if other namespaces inherit from
it, or a task if no others inherit from it.

%Names may not contain colons (which would preclude use of directory paths
%involving the registration name in \lstinline=$PATH= variables). They
%may not contain the `.' character (it will be interpreted as the
%namespace hierarchy delimiter, separating groups and names -huh?).

\begin{myitemize}
\item {\em legal values:}
    \begin{myitemize}
        \item \lstinline=[[foo]]=
        \item \lstinline=[[foo, bar, baz]]=
    \end{myitemize}
\end{myitemize}

If multiple names are listed the subsequent settings apply to each.

All namespaces inherit initially from {\em root}, which can be
explicitly configured to provide or override default settings
for all tasks in the suite.

\paragraph[inherit]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow inherit}

A list of the immediate parent(s) this namespace inherits from. If no
parents are listed \lstinline=root= is assumed.

\begin{myitemize}
\item {\em type:} Comma-separated list of strings (parent namespace names).
\item {\em default:} \lstinline=root=
\end{myitemize}

\paragraph[title]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow title}

A single line description of this namespace. It is displayed by the
\lstinline=cylc list= command and can be retrieved from running tasks
with the \lstinline=cylc show= command.

\begin{myitemize}
\item {\em type:} single line string
\item {\em root default:} (none)
\end{myitemize}

\paragraph[description]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow description}

A multi-line description of this namespace, retrievable from running tasks with the
\lstinline=cylc show= command.

\begin{myitemize}
\item {\em type:} multi-line string
\item {\em root default:} (none)
\end{myitemize}

\paragraph[URL]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow URL}
\label{TaskURL}

A web URL to task documentation for this suite.  If present it can be browsed
with the \lstinline=cylc doc= command, or by right-clicking on the task in
gcylc.  The variables \lstinline=${CYLC_SUITE_NAME}= and
\lstinline=${CYLC_TASK_NAME}= will be replaced with the actual suite and task
names (note that these are not environment variables in this context). See also
suite URLs (\ref{SuiteURL}).

\begin{myitemize}
\item {\em type:} string (URL)
\item {\em default:} (none)
\item {\em example:} you can set URLs to all tasks in a suite by putting
    something like the following in the root namespace:
    \begin{lstlisting}
[runtime]
    [[root]]
        URL = http://suites/${CYLC_SUITE_NAME}/${CYLC_TASK_NAME}.html
    \end{lstlisting}
\end{myitemize}

(Note that URLs containing the suite comment delimiter
\lstinline=#= must be protected by quotes).

\paragraph[init-script]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow init-script}

This is invoked by the task job script before the task execution
environment is configured, so it does not have access to any suite or task
environment variables. It can be a single command or multiple lines of
scripting. The original intention was to allow remote tasks to
source login scripts to configure their access to cylc, but this should no
longer be necessary (see~\ref{HowTasksGetAccessToCylc}). See also
\lstinline=env-script=, \lstinline=err-script=, \lstinline=pre-script=,
\lstinline=script=, and \lstinline=post-script=.

\begin{myitemize}
\item {\em type:} string
\item {\em default:} (none)
\item {\em example:} \lstinline@init-script = "echo Hello World"@
\end{myitemize}

\paragraph[env-script]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow env-script}

This is invoked by the task job script between the cylc-defined environment
(suite and task identity, etc.) and the user-defined task runtime environment -
i.e.\ it has access to the cylc environment, and the task environment has
access to variables defined by this scripting. It can be a single command or
multiple lines of scripting.  See also \lstinline=init-script=,
\lstinline=err-script=, \lstinline=pre-script=, \lstinline=script=, and
\lstinline=post-script=.

\begin{myitemize}
\item {\em type:} string
\item {\em default:} (none)
\item {\em example:} \lstinline@env-script = "echo Hello World"@
\end{myitemize}

\paragraph[err-script]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow err-script}

This is any custom script to be invoked at the end of the error trap, (if the
error trap is triggered due to failure of a command in the task job). The
output of this will always be sent to STDERR and \lstinline=$1= is set to the
name of the signal caught by the error trap. The script should be fast and use
very little system resource to ensure that the error trap can return quickly.
It can be a single command or multiple lines of scripting. See also
\lstinline=init-script=, \lstinline=env-script=, \lstinline=pre-script=,
\lstinline=script=, and \lstinline=post-script=.

\begin{myitemize}
\item {\em type:} string
\item {\em default:} (none)
\item {\em example:} \lstinline@err-script = "printenv FOO"@
\end{myitemize}

\paragraph[pre-script]{ [runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow pre-script}

This is invoked by the task job script immediately before the \lstinline=script=
item (just below). It can be a single command or multiple lines of scripting.
See also \lstinline=init-script=, \lstinline=env-script=, \lstinline=err-script=,
\lstinline=script=, and \lstinline=post-script=.

\begin{myitemize}
\item {\em type:} string
\item {\em default:} (none)
\item {\em example:}
 \begin{lstlisting}
    pre-script = """
      . $HOME/.profile
      echo Hello from suite ${CYLC_SUITE_NAME}!"""
 \end{lstlisting}
\end{myitemize}

\paragraph[script]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow script}
\label{ScriptItem}

The is the main user-defined scripting to run when the task is ready. It can be a
single command or multiple lines of scripting. See also \lstinline=init-script=,
\lstinline=env-script=, \lstinline=err-script=, \lstinline=pre-script=, and
\lstinline=post-script=.

\begin{myitemize}
\item {\em type:} string
\item {\em root default:} (none)
\end{myitemize}

\paragraph[post-script]{ [runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow post-script}

This is invoked by the task job script immediately after the \lstinline=script=
item (just above).  It can be a single command or multiple lines of scripting.
See also
\lstinline=init-script=, \lstinline=env-script=, \lstinline=err-script=,
\lstinline=pre-script=, and \lstinline=script=.

\begin{myitemize}
\item {\em type:} string
\item {\em default:} (none)
\end{myitemize}

\paragraph[work sub-directory]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow work sub-directory}
\label{worksubdirectory}

Task job scripts are executed from within {\em work directories} created
automatically under the suite run directory. A task can get its own work
directory from \lstinline=$CYLC_TASK_WORK_DIR= (or simply \lstinline=$PWD= if
it does not \lstinline=cd= elsewhere at runtime). The default directory
path contains task name and cycle point, to provide a unique workspace for
every instance of every task. If several tasks need to exchange files and
simply read and write from their from current working directory, this item
can be used to override the default to make them all use the same workspace.

The top level share and work directory location can be changed (e.g.\ to a
large data area) by a global config setting (see~\ref{workdirectory}).

\begin{myitemize}
\item {\em type:} string (directory path, can contain environment variables)
\item {\em default:} \lstinline=$CYLC_TASK_CYCLE_POINT/$CYLC_TASK_NAME=
\item {\em example:} \lstinline=$CYLC_TASK_CYCLE_POINT/shared/=
\end{myitemize}

Note that if you omit cycle point from the work sub-directory path successive
instances of the task will share the same workspace.  Consider the effect on
cycle point offset housekeeping of work directories before doing this.

\paragraph[enable resurrection]{ [runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow enable resurrection}

If a message is received from a failed task cylc will normally treat
this as an error condition, issue a warning, and leave the task in the
``failed'' state.  But if ``enable resurrection'' is switched on failed
tasks can come back from the dead: if the same task job script is
executed again cylc will put the task back into the running state and
continue as normal when the started message is received. This can be
used to handle HPC-style job preemption wherein a resource manager may
kill a running task and reschedule it to run again later, to make way
for a job with higher immediate priority. See also~\ref{PreemptionHPC}
\begin{myitemize}
\item {\em type:} boolean
\item {\em default:} False
\end{myitemize}

\paragraph[{[[[}job{]]]}]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[job]]]}

This section configures the means by which cylc submits task job scripts to run.

\subparagraph[batch system]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[job]]] \textrightarrow batch system}
\label{RuntimeJobSubMethods}

See~\ref{TaskJobSubmission} for how job submission works, and how to define
new handlers for different batch systems. Cylc has a number of built in batch system handlers:
\begin{myitemize}
\item {\em type:} string
\item {\em legal values:}
   \begin{myitemize}
       \item \lstinline=background= - invoke a child process
       \item \lstinline=at= - the rudimentary Unix \lstinline=at= scheduler
       \item \lstinline=loadleveler= - IBM LoadLeveler \lstinline=llsubmit=, with directives defined in the suite.rc file
       \item \lstinline=lsf= - IBM Platform LSF \lstinline=bsub=, with directives defined in the suite.rc file
       \item \lstinline=pbs= - PBS \lstinline=qsub=, with directives defined in the suite.rc file
       \item \lstinline=sge= - Sun Grid Engine \lstinline=qsub=, with directives defined in the suite.rc file
       \item \lstinline=slurm= - Simple Linux Utility for Resource Management \lstinline=sbatch=, with directives defined in the suite.rc file
       \item \lstinline=moab= - Moab workload manager \lstinline=msub=, with directives defined in the suite.rc file
   \end{myitemize}
\item {\em default:} \lstinline=background=
\end{myitemize}

\subparagraph[execution time limit]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[job]]] \textrightarrow execution time limit}

Specify the execution wall clock limit for a job of the task.
For \lstinline=background= and \lstinline=at=, the job script will be invoked using the \lstinline=timeout= command.
For other batch systems, the specified time will be automatically translated into the equivalent directive for wall clock limit.

\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation
    \item {\em example:} \lstinline=PT5M=, 5 minutes, \lstinline=PT1H=, 1 hour
    \item {\em default:} (none)
\end{myitemize}

\subparagraph[batch submit command template]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[job]]] \textrightarrow batch submit command template}

This allows you to override the actual command used by the chosen batch
system. The template's \%(job)s will be substituted by the
job file path.

\begin{myitemize}
\item {\em type:} string
\item {\em legal values:} a string template
\item {\em example:} \lstinline@llsubmit \%(job)s@
\end{myitemize}

\subparagraph[shell]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[job]]] \textrightarrow shell}
\label{JobSubShell}

Location of the command used to interpret the job script submitted by the
suite daemon when a task is ready to run. This can be set to the location of
\lstinline=bash= in the job host if the shell is not installed in the standard
location.
{\em Note: It has no bearing on any sub-shells that may be called by the job script.}

Setting this to the path of a ksh93 interpreter is deprecated. Support of which
will be withdrawn in a future cylc release. Setting this to any other shell is
not supported.

\begin{myitemize}
\item {\em type:} string
\item {\em root default:} \lstinline=/bin/bash=
\end{myitemize}

\subparagraph[submission retry delays]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[job]]] \textrightarrow submission retry delays}
\label{JobSubRefRetries}

A list of duration (in ISO 8601 syntax), after which to resubmit if job
submission fails.
\begin{myitemize}
    \item {\em type:} Comma-separated list of ISO 8601 duration/interval
        representations, optionally {\em preceded} by multipliers.
    \item {\em example:} \lstinline=PT1M,3*PT1H, P1D= is equivalent to
    \lstinline=PT1M, PT1H, PT1H, PT1H, P1D= - 1 minute, 1 hour, 1 hour, 1
    hour, 1 day.
    \item {\em default:} (none)
\end{myitemize}

\subparagraph[execution retry delays]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[job]]] \textrightarrow execution retry delays}
\label{RefRetries}

A list of ISO 8601 time duration/intervals after which to resubmit the task
if it fails. The variable \lstinline=$CYLC_TASK_TRY_NUMBER= in the task
execution environment is incremented each time, starting from 1 for the
first try - this can be used to vary task behaviour by try number.

\begin{myitemize}
    \item {\em type:} Comma-separated list of ISO 8601 duration/interval representations,
    optionally {\em preceded} by multipliers.
    \item {\em example:} \lstinline=PT1.5M,3*PT10M= is equivalent to
    \lstinline=PT1.5M, PT10M, PT10M, PT10M= - 1.5 minutes, 10 minutes,
    10 minutes, 10 minutes.
    \item {\em default:} (none)
\end{myitemize}

\subparagraph[submission polling intervals]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[job]]] \textrightarrow submission polling intervals}
\label{SubmissionPollingIntervals}

A list of intervals, expressed as ISO 8601 duration/intervals, with optional
multipliers, after which cylc will poll for status while the task is in the
submitted state.

For the polling task communication method this overrides the default
submission polling interval in the site/user config files
(\ref{SiteAndUserConfiguration}). For default and ssh task communications,
polling is not done by default but it can still be configured here as a
regular check on the health of submitted tasks.

Each list value is used in turn until the last, which is used repeatedly
until finished.

\begin{myitemize}
    \item {\em type:} Comma-separated list of ISO 8601 duration/interval
        representations, optionally {\em preceded} by multipliers.
    \item {\em example:} \lstinline=PT1M,3*PT1H, PT1M= is equivalent to
    \lstinline=PT1M, PT1H, PT1H, PT1H, PT1M= - 1 minute, 1 hour, 1 hour, 1
    hour, 1 minute.
    \item {\em default:} (none)
\end{myitemize}
A single interval value is probably appropriate for submission polling.

\subparagraph[execution polling intervals]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[job]]] \textrightarrow execution polling intervals}
\label{ExecutionPollingIntervals}

A list of intervals, expressed as ISO 8601 duration/intervals, with optional
multipliers, after which cylc will poll for status while the task is in the
running state.

For the polling task communication method this overrides the default
execution polling interval in the site/user config files
(\ref{SiteAndUserConfiguration}). For default and ssh task communications,
polling is not done by default but it can still be configured here as a
regular check on the health of submitted tasks.

Each list value is used in turn until the last, which is used repeatedly
until finished.

\begin{myitemize}
    \item {\em type:} Comma-separated list of ISO 8601 duration/interval
        representations, optionally {\em preceded} by multipliers.
    \item {\em example:} \lstinline=PT1M,3*PT1H, PT1M= is equivalent to
    \lstinline=PT1M, PT1H, PT1H, PT1H, PT1M= - 1 minute, 1 hour, 1 hour, 1
    hour, 1 minute.
    \item {\em default:} (none)
\end{myitemize}

\paragraph[{[[[}remote{]]]}]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[remote]]]}

Configure host and username, for tasks that do not run on the suite host
account. Non-interactive ssh is used to submit the task by the configured
batch system, so you must distribute your ssh key to allow
this. Cylc must be installed on remote task hosts, but no external
software dependencies are required there.

\subparagraph[host]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[remote]]] \textrightarrow host}
\label{DynamicHostSelection}

The remote host for this namespace. This can be a static hostname, an
environment variable that holds a hostname, or a command that prints a
hostname to stdout. Host selection commands are executed just prior to
job submission. The host (static or dynamic) may have an entry in the
cylc site or user config file to specify parameters such as the location
of cylc on the remote machine; if not, the corresponding local settings
(on the suite host) will be assumed to apply on the remote host.

\begin{myitemize}
\item {\em type:} string (a valid hostname on the network)
\item {\em default:} (none)
\item {\em examples:}
    \begin{myitemize}
        \item static host name: \lstinline@host = foo@
        \item fully qualified: \lstinline@host = foo.bar.baz@
        \item dynamic host selection:
        \begin{myitemize}
            \item shell command (1): \lstinline@host = $(host-selector.sh)@
            \item shell command (2): \lstinline@host = `host-selector.sh)`@
            \item environment variable: \lstinline@host = $MY_HOST@
        \end{myitemize}
    \end{myitemize}
\end{myitemize}


\subparagraph[owner]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[remote]]] \textrightarrow owner}

The username of the task host account. This is (only) used in the
non-interactive ssh command invoked by the suite daemon to submit the remote task
(consequently it may be defined using local environment variables
(i.e.\ the shell in which cylc runs, and [cylc] \textrightarrow [[environment]]).

If you use dynamic host selection and have different usernames on
the different selectable hosts, you can configure your
\lstinline=$HOME/.ssh/config= to handle username translation.

\begin{myitemize}
\item {\em type:} string (a valid username on the remote host)
\item {\em default:} (none)
\end{myitemize}

\subparagraph[retrieve job logs]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[remote]]] \textrightarrow retrieve job logs}
\label{runtime-remote-retrieve-job-logs}

Remote task job logs are saved to the suite run directory on the task host, not
on the suite host. If you want the job logs pulled back to the suite host
automatically, you can set this item to \lstinline=True=. The suite will
then attempt to \lstinline=rsync= the job logs once from the remote host each
time a task job completes. E.g. if the job file is
\lstinline=~/cylc-run/tut.oneoff.remote/log/job/1/hello/01/job=, anything under
\lstinline=~/cylc-run/tut.oneoff.remote/log/job/1/hello/01/= will be retrieved.

\begin{myitemize}
\item {\em type:} boolean
\item {\em default:} False
\end{myitemize}

\subparagraph[retrieve job logs max size]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[remote]]] \textrightarrow retrieve job logs max size}
\label{runtime-remote-retrieve-job-logs-max-size}

If the disk space of the suite host is limited, you may want to set the maximum
sizes of the job log files to retrieve. The value can be anything that is
accepted by the \lstinline@--max-size=SIZE@ option of the \lstinline=rsync=
command.

\begin{myitemize}
\item {\em type:} string
\item {\em default:} None
\end{myitemize}

\subparagraph[retrieve job logs retry delays]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[remote]]] \textrightarrow retrieve job logs retry delays}
\label{runtime-remote-retrieve-job-logs-retry-delays}

Some batch systems have considerable delays between the time when the job
completes and when it writes the job logs in its normal location. If this is
the case, you can configure an initial delay and some retry delays between
subsequent attempts. The default behaviour is to attempt once without any
delay.

\begin{myitemize}
    \item {\em type:} Comma-separated list of ISO 8601 duration/interval representations, optionally {\em preceded} by multipliers.
    \item {\em default:} (none)
    \item {\em example:} \lstinline@handler = PT10S, PT1M, PT5M@
\end{myitemize}

\subparagraph[suite definition directory]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[remote]]] \textrightarrow  suite definition directory}

The path to the suite definition directory on the remote host, needed if
remote tasks require access to files stored there (via
\lstinline=$CYLC_SUITE_DEF_PATH=) or in the suite bin directory (via
\lstinline=$PATH=).  If this item is not defined, the local suite
definition directory path will be assumed, with the suite owner's home
directory, if present, replaced by \lstinline='$HOME'= for
interpretation on the remote host.

\begin{myitemize}
\item {\em type:} string (a valid directory path on the remote host)
\item {\em default:} (local suite definition path with \lstinline=$HOME=
    replaced)
\end{myitemize}


\paragraph[{[[[}events{]]]}]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]]}
\label{TaskEventHandling}

Cylc can call nominated event handlers when certain task events occur. This
section configures specific task event handlers; see~\ref{SuiteEventHandling} for
suite event hooks.

Event handler commands can be located in the suite \lstinline=bin/= directory,
otherwise it is up to you to ensure their location is in \lstinline=$PATH= (in
the shell in which cylc runs, on the suite host). The commands should require
very little resource to run and should return quickly.

Each task event handler can be specified as a list of command lines or command
line templates.

A command line template may have any or all of these patterns which will be
substituted with actual values:
\begin{myitemize}
    \item \%(event)s: event name
    \item \%(suite)s: suite name
    \item \%(point)s: cycle point
    \item \%(name)s: task name
    \item \%(suite\_url)s: suite URL
    \item \%(task\_url)s: task URL
    \item \%(submit\_num)s: submit number
    \item \%(id)s: task ID (i.e. \%(name)s.\%(point)s)
    \item \%(message)s: event message, if any
\end{myitemize}

Otherwise, the command line will be called with the following arguments:
\begin{lstlisting}
<task-event-handler> %(event)s %(suite)s %(id)s %(message)s
\end{lstlisting}

For an explanation of the substitution syntax, see String Formatting Operations
in the Python documentation:
\url{https://docs.python.org/2/library/stdtypes.html#string-formatting}.

Additional information can be passed to event handlers via the
[cylc] \textrightarrow [[environment]] (but not via task
runtime environments - event handlers are not called by tasks).

\subparagraph[EVENT handler]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow EVENT handler}

A list of one or more event handlers to call when one of the following EVENTs occurs:
\begin{myitemize}
    \item {\bf submitted}      - the job submit command was successful
    \item {\bf submission failed}  - the job submit command failed, or the
                                   submitted job was killed before it started executing
    \item {\bf submission retry}   - job submit failed, but cylc will resubmit it
                                   after a configured delay
    \item {\bf submission timeout} - the submitted job timed out without commencing execution

    \item {\bf started}        - the task reported commencement of execution
    \item {\bf succeeded}      - the task reported that it completed successfully
    \item {\bf failed}         - the task reported that if tailed to complete successfully
    \item {\bf retry}          - the task failed, but cylc will resubmit it
                                  after a configured delay
    \item {\bf execution timeout}        - the task timed out after execution commenced
    \item {\bf warning}        - the task reported a warning priority message
\end{myitemize}

Item details:
\begin{myitemize}
    \item {\em type:} Comma-separated list of strings (event handler scripts).
    \item {\em default:} None
    \item {\em example:} \lstinline@failed handler = my-failed-handler.sh@
\end{myitemize}

\subparagraph[submission timeout]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow submission timeout}
\label{runtime-event-hooks-submission-timeout}

If a task has not started after the specified ISO 8601 duration/interval, the
{\em submission timeout} event handler(s) will be called.
\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation (e.g.
 \lstinline=PT30M=, 30 minutes or \lstinline=P1D=, 1 day).
    \item {\em default:} (none)
\end{myitemize}

\subparagraph[execution timeout]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow execution timeout}
\label{runtime-event-hooks-execution-timeout}

If a task has not finished after the specified ISO 8601 duration/interval, the
{\em execution timeout} event handler(s) will be called.
\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation (e.g.
 \lstinline=PT4H=, 4 hours or \lstinline=P1D=, 1 day).
    \item {\em default:} (none)
\end{myitemize}

\subparagraph[reset timer]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow reset timer}
\label{runtime-event-hooks-reset-timer}

If you set an execution timeout the timer can be reset to zero every
time a message is received from the running task (which indicates the
task is still alive).  Otherwise, the task will timeout if it does not
finish in the alotted time regardless of incoming messages.

\begin{myitemize}
\item {\em type:} boolean
\item {\em default:} False
\end{myitemize}

\subparagraph[handlers]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow handlers}

Specify a list of command lines or command line templates as task event handlers.

\begin{myitemize}
    \item {\em type:} Comma-separated list of strings (event handler command line or command line templates).
    \item {\em default:} (none)
    \item {\em example:} \lstinline@handlers = my-handler.sh@
\end{myitemize}

\subparagraph[handler events]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow handler events}

Specify the events for which the general task event handlers should be invoked.

\begin{myitemize}
    \item {\em type:} Comma-separated list of events
    \item {\em default:} (none)
    \item {\em example:} \lstinline@handler events = submission failed, failed@
\end{myitemize}

\subparagraph[handler retry delays]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow handler retry delays}
\label{runtime-events-handler-retry-delays}

Specify an initial delay before running an event handler command and any retry
delays in case the command returns a non-zero code. The default behaviour is to
run an event handler command once without any delay.

\begin{myitemize}
    \item {\em type:} Comma-separated list of ISO 8601 duration/interval representations, optionally {\em preceded} by multipliers.
    \item {\em default:} (none)
    \item {\em example:} \lstinline@handler = PT10S, PT1M, PT5M@
\end{myitemize}

\subparagraph[mail events]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow mail events}

Specify the events for which notification emails should be sent.

\begin{myitemize}
    \item {\em type:} Comma-separated list of events
    \item {\em default:} (none)
    \item {\em example:} \lstinline@handler = submission failed, failed@
\end{myitemize}

\subparagraph[mail from]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow mail from}

Specify an alternate \lstinline=from:= email address for event notifications.

\begin{myitemize}
    \item {\em type:} string
    \item {\em default:} None, (notifications@HOSTNAME)
    \item {\em example:} \lstinline|mail from = no-reply@your-org|
\end{myitemize}

\subparagraph[mail retry delays]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow mail retry delays}

Specify an initial delay before running the mail notification command and any
retry delays in case the command returns a non-zero code. The default behaviour
is to run the mail notification command once without any delay.

\begin{myitemize}
    \item {\em type:} Comma-separated list of ISO 8601 duration/interval representations, optionally {\em preceded} by multipliers.
    \item {\em default:} (none)
    \item {\em example:} \lstinline@handler = PT10S, PT1M, PT5M@
\end{myitemize}

\subparagraph[mail smtp]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow mail smtp}

Specify the SMTP server for sending email notifications.

\begin{myitemize}
    \item {\em type:} string
    \item {\em default:} None, (localhost:25)
    \item {\em example:} \lstinline@mail smtp = smtp.yourorg@
\end{myitemize}

\subparagraph[mail to]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[events]]] \textrightarrow mail to}

A list of email addresses to send task event notifications. The list can be
anything accepted by the \lstinline=mail= command.

\begin{myitemize}
    \item {\em type:} string
    \item {\em default:} None, (USER@HOSTNAME)
    \item {\em example:} \lstinline@mail to = your.colleague@
\end{myitemize}

\paragraph[{[[[}environment{]]]}]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[environment]]]}

The user defined task execution environment. Variables defined here can
refer to cylc suite and task identity variables, which are exported
earlier in the task job script, and variable assignment expressions can
use cylc utility commands because access to cylc is also configured
earlier in the script.  See also~\ref{TaskExecutionEnvironment}.

\subparagraph[\_\_VARIABLE\_\_ ]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[environment]]] \textrightarrow \_\_VARIABLE\_\_}
\label{AppendixTaskExecutionEnvironment}

Replace \_\_VARIABLE\_\_ with any number of environment variable
assignment expressions.
Order of definition is preserved so values can refer to previously
defined variables. Values are passed through to the task job script
without evaluation or manipulation by cylc, so any variable assignment
expression that is legal in the job submission shell can be used.
White space around the `$=$' is allowed (as far as cylc's suite.rc
parser is concerned these are just normal configuration items).

\begin{myitemize}
\item {\em type:} string
\item {\em default:} (none)
\item {\em legal values:} depends to some extent on the task job
    submission shell (\ref{JobSubShell}).
\item {\em examples}, for the bash shell:
   \begin{myitemize}
       \item \lstinline@FOO = $HOME/bar/baz@
       \item \lstinline@BAR = ${FOO}$GLOBALVAR@
       \item \lstinline@BAZ = $( echo "hello world" )@
       \item \lstinline@WAZ = ${FOO%.jpg}.png@
       \item \lstinline@NEXT_CYCLE = $( cylc cycle-point --offset=PT6H )@
       \item \lstinline@PREV_CYCLE = `cylc cycle-point --offset=-PT6H`@
       \item \lstinline@ZAZ = "${FOO#bar}" # <-- QUOTED to escape the suite.rc comment character@
   \end{myitemize}
\end{myitemize}

\paragraph[{[[[}environment filter{]]]}]{ [runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[environment filter]]]}
\label{EnvironmentFilter}

This section contains environment variable inclusion and exclusion
lists that can be used to filter the inherited environment. {\em This is
not intended as an alternative to a well-designed inheritance hierarchy
that provides each task with just the variables it needs.} Filters can,
however, improve suites with tasks that inherit a lot of environment
they don't need, by making it clear which tasks use which variables.
They can optionally be used routinely as explicit ``task environment
interfaces'' too, at some cost to brevity, because they guarantee that
variables filtered out of the inherited task environment are not used.

Note that environment filtering is done after inheritance is completely
worked out, not at each level on the way, so filter lists in higher-level
namespaces only have an effect if they are not overridden by descendants.

\subparagraph[include]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[environment filter]]] \textrightarrow include}

If given, only variables named in this list will be included from the
inherited environment, others will be filtered out. Variables may also
be explicitly excluded by an \lstinline=exclude= list.

\begin{myitemize}
\item {\em type:} Comma-separated list of strings (variable names).
\item {\em default:} (none)
\end{myitemize}

\subparagraph[exclude]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[environment filter]]] \textrightarrow exclude}

Variables named in this list will be filtered out of the inherited
environment.  Variables may also be implicitly excluded by
omission from an \lstinline=include= list.

\begin{myitemize}
\item {\em type:} Comma-separated list of strings (variable names).
\item {\em default:} (none)
\end{myitemize}

\paragraph[{[[[}directives{]]]}]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[directives]]]}

Batch queue scheduler directives.  Whether or not these are used depends
on the batch system. For the built-in methods that support directives
(\lstinline=loadleveler=, \lstinline=lsf=, \lstinline=pbs=, \lstinline=sge=,
\lstinline=slurm=, \lstinline=moab=), directives are written to the top of the
task job script in the correct format for the method. Specifying directives
individually like this allows use of default directives that can be
individually overridden at lower levels of the runtime namespace hierarchy.

\subparagraph[\_\_DIRECTIVE\_\_ ]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[directives]]] \textrightarrow \_\_DIRECTIVE\_\_}

Replace \_\_DIRECTIVE\_\_ with each directive assignment, e.g.
\lstinline@class = parallel@

\begin{myitemize}
\item {\em type:} string
\item {\em default:} (none)
\end{myitemize}

Example directives for the built-in batch system handlers are shown
in~\ref{AvailableMethods}.

\paragraph[{[[[}outputs{]]]}]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[outputs]]]}

Register custom task outputs for use in message triggering in this section
(\ref{MessageTriggers})

\subparagraph[\_\_OUTPUT\_\_ ]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[outputs]]] \textrightarrow \_\_OUTPUT\_\_}

Replace \_\_OUTPUT\_\_ with one or more custom task output messages
(\ref{MessageTriggers}).  The item name is used to select the custom output
message in graph trigger notation.
\begin{myitemize}
    \item {\em type:} string
    \item {\em default:} (none)
    \item{ \em examples:}
\end{myitemize}
\begin{lstlisting}
out1 = "sea state products ready"
out2 = "NWP restart files completed"
\end{lstlisting}

\paragraph[{[[[}suite state polling{]]]}]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[suite state polling]]]}

\lstset{language=transcript}
Configure automatic suite polling tasks as described
in~\ref{SuiteStatePolling}. The
items in this section reflect the options and defaults of the
\lstinline=cylc suite-state= command, except that the target suite name and the
\lstinline=--task=, \lstinline=--cycle=, and \lstinline=--status= options are
taken from the graph notation.

\subparagraph[run-dir]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[suite state polling]]] \textrightarrow run-dir}

For your own suites the run database location is determined by your
site/user config. For other suites, e.g. those owned by others, or
mirrored suite databases, use this item to specify the location
of the top level cylc run directory (the database should be a
suite-name sub-directory of this location).

\begin{myitemize}
    \item {\em type:} string (a directory path on the target suite host)
    \item {\em default:} as configured by site/user config (for your own suites)
\end{myitemize}

\subparagraph[template]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[suite state polling]]] \textrightarrow template}

Cycle point template of the target suite, if different from that of the polling suite.

\begin{myitemize}
    \item {\em type:} string
    \item {\em default:} cycle point format of the polling suite
    \item {\em example:} \lstinline=\%Y-\%m-\%dT\%H=
\end{myitemize}

\subparagraph[interval]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[suite state polling]]] \textrightarrow interval}

Polling interval expressed as an ISO 8601 duration/interval.
\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation (e.g.
 \lstinline=PT10S=, 10 seconds, or \lstinline=PT1M=, 1 minute).
    \item {\em default:} PT1M
\end{myitemize}

\subparagraph[max-polls]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[suite state polling]]] \textrightarrow max-polls}

The maximum number of polls before timing out and entering the `failed' state.

\begin{myitemize}
    \item {\em type:} integer
    \item {\em default:} 10
\end{myitemize}

\subparagraph[user]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[suite state polling]]] \textrightarrow user}

Username of an account on the suite host to which you have access. The
polling \lstinline=cylc suite-state= command will be invoked
on the remote account.

\begin{myitemize}
    \item {\em type:} string (username)
    \item {\em default:} (none)
\end{myitemize}

\subparagraph[host]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[suite state polling]]] \textrightarrow host}

The hostname of the target suite. The polling \lstinline=cylc suite-state= command
will be invoked on the remote account.

\begin{myitemize}
    \item {\em type:} string (hostname)
    \item {\em default:} (none)
\end{myitemize}

\subparagraph[verbose]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[suite state polling]]] \textrightarrow verbose}

Run the polling \lstinline=cylc suite-state= command in verbose output mode.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False
\end{myitemize}

\paragraph[{[[[}simulation{]]]}]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[simulation]]]}
\label{suiterc-sim-config}

\lstset{language=transcript}

Task configuration for the suite {\em simulation} and {\em dummy} run modes
described in Section~\ref{SimulationMode}.

\subparagraph[default run length]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[simulation]]] \textrightarrow default run length}

The default simulated job run length, if \lstinline=[job]execution time limit=
and \lstinline=[simulation]speedup factor= are not set.

\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation (e.g.
 \lstinline=PT10S=, 10 seconds, or \lstinline=PT1M=, 1 minute).
    \item {\em default:} \lstinline=PT10S=
\end{myitemize}

\subparagraph[speedup factor]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[simulation]]] \textrightarrow speedup factor}

If \lstinline=[job]execution time limit= is set, the task simulated run length
is computed by dividing it by this factor.

\begin{myitemize}
    \item {\em type:} float
    \item {\em default:} (none) - i.e.\ do not use proportional run length
    \item {\em example:} \lstinline=10.0=
\end{myitemize}

\subparagraph[time limit buffer]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[simulation]]] \textrightarrow time limit buffer}

For dummy jobs, a new \lstinline=[job]execution time limit= is set to the
simulated task run length plus this buffer interval, to avoid job kill due to
exceeding the time limit.

\begin{myitemize}
    \item {\em type:} ISO 8601 duration/interval representation (e.g.
 \lstinline=PT10S=, 10 seconds, or \lstinline=PT1M=, 1 minute).
    \item {\em default:} PT10S
\end{myitemize}

\subparagraph[fail cycle points]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[simulation]]] \textrightarrow fail cycle points}

Configure simulated or dummy jobs to fail at certain cycle points.

\begin{myitemize}
    \item {\em type:} list of strings (cycle points), or {\em all}
    \item {\em default:} (none) - no instances of the task will fail
    \item {\em examples:}
    \begin{myitemize}
        \item \lstinline=all= - all instance of the task will fail
        \item \lstinline=2017-08-12T06, 2017-08-12T18= - these instances of the
        task will fail
    \end{myitemize}
\end{myitemize}

\subparagraph[fail try 1 only]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[simulation]]] \textrightarrow fail try 1 only}

If this is set to \lstinline=True= only the first run of the task instance will
fail, otherwise retries will fail too.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} \lstinline=True=
\end{myitemize}

\subparagraph[disable task event handlers]{[runtime] \textrightarrow [[\_\_NAME\_\_]] \textrightarrow [[[simulation]]] \textrightarrow disable task event handlers}

If this is set to \lstinline=True= configured task event handlers will not be called
in simulation or dummy modes.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} \lstinline=True=
\end{myitemize}

\subsection{[visualization]}

Configuration of suite graphing for the \lstinline=cylc graph= command (graph
extent, styling, and initial family-collapsed state) and the gcylc graph view
(initial family-collapsed state). Graphviz documentation of node shapes
and so on can be found at http://www.graphviz.org/Documentation.php.

\subsubsection[initial cycle point]{[visualization] \textrightarrow initial cycle point}

The initial cycle point for graph plotting.
\begin{myitemize}
    \item {\em type:} ISO 8601 date/time representation (e.g. CCYYMMDDThhmm)
    \item {\em default:} the suite initial cycle point
\end{myitemize}
The visualization initial cycle point gets adjusted up if necessary to the
suite initial cycling point.

\subsubsection[final cycle point]{[visualization] \textrightarrow final cycle point}

An explicit final cycle point for graph plotting. If used, this overrides the
preferred {\em number of cycle points} (below).
\begin{myitemize}
    \item {\em type:} ISO 8601 date/time representation (e.g. CCYYMMDDThhmm)
    \item {\em default:} (none)
\end{myitemize}
The visualization final cycle point gets adjusted down if necessary to the
suite final cycle point.

\subsubsection[number of cycle points]{[visualization] \textrightarrow number of cycle points}

The number of cycle points to graph starting from the visualization initial
cycle point. This is the preferred way of defining the graph end point, but
it can be overridden by an explicit {\em final cycle point} (above).
\begin{myitemize}
    \item {\em type:} integer
    \item {\em default:} 3
\end{myitemize}

\subsubsection[collapsed families]{[visualization] \textrightarrow collapsed families}

A list of family (namespace) names to be shown in the collapsed state
(i.e.\ the family members will be replaced by a single family node) when
the suite is first plotted in the graph viewer or the gcylc graph view.
If this item is not set, the default is to collapse all families at first.
Interactive GUI controls can then be used to group and ungroup family
nodes at will.

\begin{myitemize}
    \item {\em type:} Comma-separated list of family names.
    \item {\em default:} (none)
\end{myitemize}

\subsubsection[use node color for edges]{[visualization] \textrightarrow use node color for edges}

Plot graph edges (dependency arrows) with the same color as the upstream
node, otherwise default to black.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False
\end{myitemize}

\subsubsection[use node fillcolor for edges]{[visualization] \textrightarrow use node fillcolor for edges}

Plot graph edges (i.e. dependency arrows) with the same fillcolor as the
upstream node, if it is filled, otherwise default to black.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False
\end{myitemize}

\subsubsection[node penwidth]{[visualization] \textrightarrow node penwidth}

Line width of node shape borders.

\begin{myitemize}
    \item {\em type:} integer
    \item {\em default:} 2
\end{myitemize}

\subsubsection[edge penwidth]{[visualization] \textrightarrow edge penwidth}

Line width of graph edges (dependency arrows).

\begin{myitemize}
    \item {\em type:} integer
    \item {\em default:} 2
\end{myitemize}

\subsubsection[use node color for labels]{[visualization] \textrightarrow use node color for labels}

Graph node labels can be printed in the same color as the node outline.

\begin{myitemize}
    \item {\em type:} boolean
    \item {\em default:} False
\end{myitemize}


\subsubsection[default node attributes]{[visualization] \textrightarrow default node attributes}

Set the default attributes (color and style etc.) of graph nodes (tasks and families).
Attribute pairs must be quoted to hide the internal \lstinline@=@ character.

\begin{myitemize}
    \item {\em type:} Comma-separated list of quoted \lstinline@'attribute=value'@ pairs.
    \item {\em legal values:} see graphviz or pygraphviz documentation
    \item {\em default:} \lstinline@'style=filled', 'fillcolor=yellow', 'shape=box'@
\end{myitemize}

\subsubsection[default edge attributes]{[visualization] \textrightarrow default edge attributes}

Set the default attributes (color and style etc.) of graph edges
(dependency arrows).  Attribute pairs must be quoted to hide the
internal \lstinline@=@ character.
\begin{myitemize}
    \item {\em type:} Comma-separated list of quoted \lstinline@'attribute=value'@ pairs.
    \item {\em legal values:} see graphviz or pygraphviz documentation
    \item {\em default:} \lstinline@'color=black'@
\end{myitemize}

\subsubsection[{[[}node groups{]]}]{[visualization] \textrightarrow [[node groups]]}

Define named groups of graph nodes (tasks and families) which can styled
en masse, by name, in [visualization] \textrightarrow [[node attributes]].
Node groups are automatically defined for all task families, including
root, so you can style family and member nodes at once by family name.

\paragraph[\_\_GROUP\_\_]{[visualization] \textrightarrow [[node groups]] \textrightarrow \_\_GROUP\_\_}

Replace \_\_GROUP\_\_ with each named group of tasks or families.

\begin{myitemize}
    \item {\em type:} Comma-separated list of task or family names.
    \item {\em default:} (none)
    \item {\em example:}
\begin{lstlisting}
   PreProc = foo, bar
   PostProc = baz, waz
\end{lstlisting}
\end{myitemize}

\subsubsection[{[[}node attributes{]]}]{[visualization] \textrightarrow [[node attributes]]}

Here you can assign graph node attributes to specific nodes, or to all
members of named groups defined in [visualization] \textrightarrow [[node
groups]]. Task families are automatically node groups. Styling of a
family node applies to all member nodes (tasks and sub-families), but
precedence is determined by ordering in the suite definition.  For
example, if you style a family red and then one of its members green,
cylc will plot a red family with one green member; but if you style one
member green and then the family red, the red family styling will
override the earlier green styling of the member.

\paragraph[\_\_NAME\_\_]{[visualization] \textrightarrow [[node attributes]] \textrightarrow \_\_NAME\_\_}

Replace \_\_NAME\_\_ with each node or node group for style attribute
assignment.

\begin{myitemize}
    \item {\em type:} Comma-separated list of quoted \lstinline@'attribute=value'@ pairs.
    \item {\em legal values:} see graphviz or pygraphviz documentation
    \item {\em default:} (none)
    \item {\em example:} (with reference to the node groups defined above)
\begin{lstlisting}
   PreProc = 'style=filled', 'fillcolor=orange'
   PostProc = 'color=red'
   foo = 'style=filled'
\end{lstlisting}
\end{myitemize}
