execline: variable substitution
Keywords: execline variable substitution el_substitute

execline
Software
skarnet.org

# Variable substitution

In a shell, when you write

     $ A='foobar' ; echo $A

the `echo` command is given the argument `foobar`. The `foobar` value has been substituted for the `A` variable.

Although `execline` maintains no state, and thus has no real variables, it provides such a substitution facility via substitution commands, namely:

- define
- importas
- elglob
- elgetpositionals
- multidefine
- multisubstitute

A substitution command takes a key, i.e. a string (which can contain any character but `$`, `{` and `}`, although it is recommended to use only alphanumerical characters), and a way to compute a value.

## Basics

- If the substitution key is foo, then the substitution command will look for every occurrence of `${foo}` or `$foo` in the rest of its argv. Note that `${foo}bar` matches, but `$foobar` does not. To be safe, always use the syntax with braces, unless `$foo` is a word on its own.
- Every match is then replaced with the value.

The simplest example is the following:

    #!/command/execlineb
    define FOO blah
    echo $FOO

which will replace the `FOO` key with the `blah` value, then execute the `echo` command. So that script will print `blah` on stdout.

## Quoting

execline allows you to write literal `${foo}` constructs even when the foo variable is being substituted, thanks to a quoting mechanism. Brace (pun intended) yourself: the following is the most complex part of the whole language.

### Rationale

If we want to be able to have a literal `${foo}`, then:

- The `${foo}` sequence will mean one of two things: be substituted, or don't be substituted.
- The default (unquoted) action should be: substitute.
- A sequence that means "do not substitute" should be able to appear literally. The quote character should also be able to appear literally before a sequence that means "substitute". (Tricky, eh?)
- There should be as few quote characters as possible, to avoid shell-like quoting nightmares.

### Syntax

Rule:

- The backslash (`\`) is a quote character for substitution commands.
- The following rule applies only if the foo key is explicitly used in a substitution command. If no command tries to substitute anything for foo, sequences like `${foo}` and preceding backslashes are left untouched.
- (Substitute.) If `${foo}` is preceded by `2*n` backslashes (an even number), the whole sequence will be replaced with `n` backslashes, followed by the substituted value.
- (Do not substitute.) If `${foo}` is preceded by `2*n+1` backslashes (an odd number), the whole sequence will be replaced with `n` backslashes, followed by the literal `${foo}`.

And now, the catch: the execlineb launcher, as well as the shell, interprets backslashes as escape characters. To make a word that contains a backlash, you need to write two backslashes in your execline script or shell command line. That means that the whole number of backslashes you must write before your `${foo}` sequence must be doubled for the substitution command to read the proper number of backslashes and perform its work correctly.
Once you keep that in mind, the quoting rule is logical.

### Example

The quoting rule is best illustrated with the following example, where the `A` key is substituted, and the `$B` sequences mean nothing special.

    #!/command/execlineb
    define A val
    foreground { echo $A \\$A \\\\$A \\\\\\$A \\\\\\\\$A \\\\\\\\\\$A }
                 echo $B \\$B \\\\$B \\\\\\$B \\\\\\\\$B \\\\\\\\\\$B

prints

    val $A \val \$A \\val \\$A
    $B \$B \\$B \\\$B \\\\$B \\\\\$B

Phew.

## Value transformations

A value can go through several transformations before it is substituted. It can be crunched, chomped, and/or split.

## Substitution of split values

A split value for `FOO` means that a word containing `${FOO}` will be replaced by zero, one, or (usually) more than one word. The value actually means a list of values.

The rule is: substituting a list of values (v1, v2, ...) for a key A is the same as listing the substitutions of every value v`i` for A.
For instance,

    #!/command/execlineb
    define -s FOO "v1 v2 v3" echo prefix-${FOO}-postfix

will substitute three values for `$FOO`: `v1`, `v2` and `v3`. So the `echo` command will be called with three arguments: `prefix-v1-postfix`, `prefix-v2-postfix`, and `prefix-v3-postfix`.

(Implementation note: the fact that word prefixes are kept is what makes execline's subtitutions secure. Blocks are implemented via prefix space characters; a substitution occurring inside a block will always produce words beginning with the right amount of spaces, thus substituted values cannot prematurely terminate a block.)

### Recursive substitutions

A direct consequence of that rule is that substitutions will be performed recursively if more than one key appears in one word and the values for those keys are split. Parallel substitutions are performed from left to right. For instance, in

    #!/command/execlineb
    define -s B "1 2 3" echo ${B}x${B}

the `${B}x${B}` word will be replaced with nine words: `1x1`, `1x2`, `1x3`, `2x1`, `2x2`, `2x3`, `3x1`, `3x2`, and `3x3`, in that order.
Here is an example with two distinct substitutions in parallel:

    #!/command/execlineb
    multisubstitute
    {
      define -s A "a b c d"
      define -s B "1 2 3"
    }
    echo ${A}x${B}

The `${A}x${B}` word will be replaced with twelve words: `ax1`, `ax2`, `ax3`, `bx1`, `bx2`, `bx3`, `cx1`, `cx2`, `cx3`, `dx1`, `dx2`, and `dx3`, in that order. You can check that the order of the `define` directives in multisubstitute does not matter.

If the left-to-right order does not suit you, then you should perform serial substitutions. For instance, the previous script can be replaced with

    #!/command/execlineb
    define -s B "1 2 3"
    define -s A "a b c d"
    echo ${A}x${B}

and will substitute `${B}` first, then `${A}`. So it will print

    ax1 bx1 cx1 dx1 ax2 bx2 cx2 dx2 ax3 bx3 cx3 dx3

in that order.

## Not for the faint of heart

If you think you have mastered the art of execline substitution, then you can try to do better than these people:

- Joël Riou wrote the first execlineb quine, using only `echo` as non-execline external command.
- Shortly after, Paul Jarc wrote a much shorter quine, using `echo` and `env` as non-execline external commands. He also wrote a revised version, using only `echo`, and a shorter definitive version. The last one is probably very close to the shortest possible execline quine.
- David Madore wrote another quine, using `printf`. His quine is longer than the other ones, but is well-commented and can be used as a tutorial on how to write quines. :)