Author: FCM team

# FCM: User Guide: Annex: The FCM 1 Build System

## Introduction

The FCM 1 build system is deprecated. The documentation for the current build system can be found at FCM Make.

The build system analyses the directory tree containing a set of source code, processes the configuration, and invokes `make` to compile/build the source code into the project executables. In this chapter, we shall use many examples to explain how to use the build system. At the end of this chapter, you should be able to use the build system, either by defining the build configuration file directly or by using the extract system to generate a suitable build configuration file.

## The Build Command

To invoke the build system, simply issue the command:

    fcm build

By default, the build system searches for a build configuration file `bld.cfg` in `$PWD` and then `$PWD/cfg`. If a build configuration file is not found in these directories, the command fails with an error. If a build configuration file is found, the system will use the configuration specified in the file to perform the build. If you use the extract system to extract your source tree, a build configuration should be written for you automatically at the `cfg/` sub-directory of the destination root directory.

If the root directory of the build does not exist, the system performs a new full build at this directory. If a previous build already exists at this directory, the system performs an incremental build. If a full (fresh) build is required for whatever reason, you can invoke the build system using the `-f` option, (i.e. the command becomes `fcm build -f`). If you simply want to remove all the items generated by a previous build in the destination, you can invoke the build system using the `--clean` option.

The build system uses GNU `make` to perform the majority of the build. GNU `make` has a `-j jobs` option to specify the number of jobs to run simultaneously. Invoking the build system with the same option triggers this option when the build system invokes the `make` command. The argument to the option jobs must be an integer. The default is `1`. For example, the command `fcm build -j 4` will allow `make` to perform 4 jobs simultaneously.

For further information on the build command, please see FCM Command Reference > fcm build.

## Basic Features

The build configuration file is the user interface of the build system. It is a line based text file. You can create your own build configuration file or you can use the extract system to create one for you. For a complete set of build configuration file declarations, please refer to the Annex: Declarations in FCM build configuration file.

### Basic build configuration

Suppose we have a directory at `$HOME/example`. Its sub-directory at `$HOME/example/src` contains a source tree to be built. You may want to have a build configuration file `$HOME/example/cfg/bld.cfg`, which may contain:

    # Example 1
    # ----------------------------------------------------------------------
    cfg::type     bld                           # line 1
    cfg::version  1.0                           # line 2

    dest          $HOME/example                 # line 4

    target        foo.exe bar.exe               # line 6

    tool::fc      ifort                         # line 8
    tool::fflags  -O3                           # line 9
    tool::cc      gcc                           # line 10
    tool::cflags  -O3                           # line 11

    tool::ldflags -O3 -L$(HOME)/lib -legg -lham # line 13

Here is an explanation of what each line does:

- line 1: the label `CFG::TYPE` declares the type of the configuration file. The value `bld` tells the system that it is a build configuration file.
- line 2: the label `CFG::VERSION` declares the version of the build configuration file. The current default is `1.0`. Although it is not currently used, if we have to change the format of the configuration file at a later stage, we shall be able to use this number to determine whether we are reading a file with an older format or one with a newer format.
- line 4: the label `DEST` declares the root directory of the current build.
- line 6: the label `TARGET` declares a list of default targets. The default targets of the current build will be `foo.exe` and `bar.exe`.
- line 8: the label `TOOL::FC` declares the Fortran compiler command.
- line 9: the label `TOOL::FFLAGS` declares the options to be used when invoking the Fortran compiler command.
- line 10: the label `TOOL::CC` declares the C compiler command.
- line 11: the label `TOOL::CFLAGS` declares the options to be used when invoking the C compiler command.
- line 13: the label `TOOL::LDFLAGS` declares the options to be used when invoking the linker command.

When we invoke the build system, it reads the above configuration file. It will go through various internal processes, such as dependency generations, to obtain the required information to prepare the `Makefile` of the build. (All of which will be described in later sections.) The `Makefile` of the build will be placed at `$HOME/example/bld`. The system will then invoke `make` to build the targets specified in line 6, i.e. `foo.exe` and `bar.exe` using the build tools specified between line 8 to line 13. On a successful build, the target executables will be sent to `$HOME/example/bin/`. The build system also creates a shell script called `fcm_env.sh` in `$HOME/example/`. If you source the shell script, it will export your PATH environment variable to search the `$HOME/example/bin/` directory for executables.

N.B. You may have noticed that the `-c` (compile to object file only) option is missing from the compiler flags declarations. This is because the option is inserted automatically by the build system, unless it is already declared.

N.B. You can declare the linker using `TOOL::LD`. If it is not specified, the default is to use the compiler command for the source file containing the main program.

Note - declaration of source files for build

  Source files do not have to reside in the `src/` sub-directory of the build root directory. They can be anywhere, but you will have to declare them using the label `SRC::<pcks>`, where `<pcks>` is the sub-package name in which the source belongs. If a directory is specified then the build system automatically searches for all source files in this directory. E.g.

      # Declare a source in the sub-package "foo/bar"
      src::foo/bar  $HOME/foo/bar

  By default, the build system searches the `src/` sub-directory of the build root directory for source files. If all source files are already declared explicitly, you can switch off the automatic directory search by setting the `SEARCH_SRC` flag to false. E.g.

      search_src  false

  As mentioned in the previous chapter, the name of a sub-package <pcks> provides a unique namespace for a file. The name of a sub-package is a list of words delimited by a slash `/`. (The system uses the double colons `::` and the double underscores `__` internally. Please avoid using `::` and `__` for naming your files and directories.)

  Currently, the build system only supports non-space characters in the package name, as the space character is used as a delimiter between the declaration label and its value. If there are spaces in the path name to a file or directory, you should explicity re-define the package name of that path to a package name with no space using the above method. However, we recommend that only non-space characters are used for naming directories and files to make life simple.

  In the build system, the sub-package name also provides an inheritance relationship for sub-packages. For instance, we may have a sub-package called `foo/bar/egg`, which belongs to the sub-package `foo/bar`, which belongs to the package `foo`.

  - If we declare a global build tool, it applies to all packages.
  - If we declare a build tool for `foo`, it applies also to the sub-package `foo/bar` and `foo/bar/egg`.
  - If we declare a build tool for `foo/bar`, it applies also to `foo/bar/egg`, but not to other sub-packages in `foo`.

### Build configuration via the extract system

As mentioned earlier, you can obtain a build configuration file through the extract system. The following example is what you may have in your extract configuration in order to obtain a similar configuration as example 1:

    # Example 2
    # ----------------------------------------------------------------------
    cfg::type          ext                           # line 1
    cfg::version       1.0                           # line 2

    dest               $HOME/example                 # line 4

    bld::target        foo.exe bar.exe               # line 6

    bld::tool::fc      ifort                         # line 8
    bld::tool::fflags  -O3                           # line 9
    bld::tool::cc      gcc                           # line 10
    bld::tool::cflags  -O3                           # line 11

    bld::tool::ldflags -O3 -L$(HOME)/lib -legg -lham # line 13

    # ... and other declarations for source locations ...

It is easy to note the similarities and differences between example 1 and example 2. Example 2 is an extract configuration file. It extracts to a destination root directory that will become the root directory of the build. Line 6 to line 13 are the same declarations, except that they are now prefixed with `BLD::`. In an extract configuration file, any lines prefixed with `BLD::` means that they are build configuration setting. These lines are ignored by the extract system but are parsed down to the output build configuration file, with the `BLD::` prefix removed. (Note: the `BLD::` prefix is optional for declarations in a build configuration file.)

N.B. If you use the extract system to mirror an extract to an alternate location, the extract system will assume that the root directory of the alternate destination is the root directory of the build, and that the build will be carried out in that destination.

### Naming of executables

If a source file called `foo.f90` contains a main program, the default behaviour of the system is to name its executable `foo.exe`. The root name of the executable is the same as the original file name, but its file extension is replaced with `.exe`. The output extension can be altered by re-registering the extension for output EXE files. How this can be done will be discussed later in the sub-section File Type.

If you need to alter the full name of the executable, you can use the `EXE_NAME::` declaration. For example, the declaration:

    bld::exe_name::foo  bar

will rename the executable of `foo.f90` from `foo.exe` to `bar`.

Note: the declaration label is `bld::exe_name::foo` (not `bld::exe_name::foo.exe`) and the executable will be named `bar` (not `bar.exe`).

### Setting the compiler flags

As discussed in the first example, the compiler commands and their flags can be set via the `TOOL::` declarations. A simple `TOOL::FFLAGS` declaration, for example, alters the compiler options for compiling all Fortran source files in the build. If you need to alter the compiler options only for the source files in a particular sub-package, it is possible to do so by adding the sub-package name to the declaration label. For example, the declaration label `TOOL::FFLAGS::foo/bar` will ensure that the declaration only applies to the code in the sub-package `foo/bar`. You can even make declarations down to the individual source file level. For example, the declaration label `TOOL::FFLAGS::foo/bar/egg.f90` will ensure that the declaration applies only for the file `foo/bar/egg.f90`.

N.B. Although the prefix `TOOL::` and the tool names are case-insensitive, sub-package names are case sensitive in the declarations. Internally, tool names are turned into uppercase, and the sub-package delimiters are changed from the slash `/` (or double colons `::`) to the double underscores `__`. When the system generates the `Makefile` for the build, each `TOOL` declaration will be exported as an environment variable. For example, the declaration `tool::fflags/foo/bar` will be exported as `FFLAGS__foo__bar`.

N.B. `TOOL` declarations for sub-packages are only accepted by the system when it is sensible to do so. For example, it allows you to declare different compiler flags, linker commands and linker flags for different sub-packages, but it does not accept different compilers for different sub-packages. If you attempt to make a `TOOL` declaration for a sub-package that does not exist, the build system will exit with an error.

The following is an example setting in an extract configuration file based on example 2:

    # Example 3
    # ----------------------------------------------------------------------
    cfg::type              ext
    cfg::version           1.0

    dest                   $HOME/example

    bld::target            foo.exe bar.exe

    bld::tool::fc          ifort
    bld::tool::fflags      -O3    # line 9
    bld::tool::cc          gcc
    bld::tool::cflags      -O3

    bld::tool::ldflags     -L$(HOME)/lib -legg -lham

    bld::tool::fflags::ops -O1 -C # line 15
    bld::tool::fflags::gen -O2    # line 16

    # ... and other declarations for repositories and source directories ...

In the example above, line 15 alters the Fortran compiler flags for `ops`, so that all source files in `ops` will be compiled with optimisation level 1 and will have runtime error checking switched on. Line 16, alters the Fortran compiler flags for `gen`, so that all source files in `gen` will be compiled with optimisation level 2. All other Fortran source files will use the global setting declared at line 9, so they they will all be compiled with optimisation level 3.

Note - changing compiler flags in incremental builds

  Suppose you have performed a successful build using the configuration in example 3, and you have decided to change some of the compiler flags, you can do so by altering the appropriate flags in the build configuration file. When you trigger an incremental build, the system will detect changes in compiler flags automatically, and update only the required targets. The following hierarchy is followed:

  - If the compiler flags for a particular source file change, only that source file and any targets depending on that source file are re-built.
  - If the compiler flags for a container package change, only source files within that container package and any targets depending on those source files are re-built.
  - If the global compiler flags change, all source files are re-built.
  - If the compiler command changes, all source files are re-built.

N.B. For a full list of build tools declarations, please see Annex: Declarations in FCM build configuration file > list of tools.

### Automatic Fortran 9X interface block

For each Fortran 9X source file containing standalone subroutines and/or functions, the system generates an interface file and sends it to the `inc/` sub-directory of the build root. An interface file contains the interface blocks for the subroutines and functions in the original source file. In an incremental build, if you have modified a Fortran 9X source file, its interface file will only be re-generated if the content of the interface has changed.

Consider a source file `foo.f90` containing a subroutine called `foo`. In a normal operation, the system writes the interface file to `foo.interface` in the `inc/` sub-directory of the build root. By default, the root name of the interface file is the same as that of the source file, and is case sensitive. You can change this behaviour using a `TOOL::INTERFACE` declaration. E.g.:

    bld::tool::interface  program # The default is "file"

In such case, the root name of the interface file will be named in lower case after the first program unit in the file.

The default extension for an interface file is `.interface`. This can be modified through the input and output file type register, which will be discussed in a later section on File Type.

In most cases, we modify procedures without altering their calling interfaces. Consider another source file `bar.f90` containing a subroutine `bar`. If `bar` calls `foo`, it is good practice for `bar` to have an explicit interface for `foo`. This can be achieved if the subroutine `bar` has the following within its declaration section:

    INCLUDE 'foo.interface'

The source file `bar.f90` is now dependent on the interface file `foo.interface`. This can make incremental build very efficient, as changes in the `foo.f90` file will not normally trigger the re-compilation of `bar.f90`, provided that the interface of the `subroutine foo` remains unchanged. (However, the system is clever enough to know that it needs to re-link any executables that are dependent on the object file for the `subroutine bar`.)

By default, the system uses its own internal logic to extract the calling interfaces of top level subroutines and functions in a Fortran source file to generate an interface block. However, the system can also work with the interface generator `f90aib`, which is a freeware obtained from Fortran 90 texts and programs, assembled by Michel Olagnon at the French Research Institute for Exploitation of the Sea. To do so, you need to make a declaration in the build configuration file using the label `TOOL::GENINTERFACE`. As for any other `TOOL` declarations, you can attach a sub-package name to the label. The change will then apply only to source files within that sub-package. If `TOOL::GENINTERFACE` is declared to have the value `NONE`, interface generation will be switched off. The following are some examples:

    # Example 4
    # ----------------------------------------------------------------------
    # This is an EXTRACT configuration file ...

    # ... some other declarations ...

    bld::tool::geninterface       f90aib # line 5
    bld::tool::geninterface::bar  none   # line 6

    # ... some other declarations ...

In line 5, the global interface generator is now set to `f90aib`. In line 6, by setting the interface generator for the package `bar` to the `none` keyword, no interface file will be generated for source files under the package `bar`.

Switching off the interface block generator can be useful in many circumstances. For example, if the interface block is already provided manually within the source tree, or if the interface block is never used by other program units, it is worth switching off the interface generator for the source file to speed up the build process.

### Automatic dependency

The build system has a built-in dependency scanner, which works out the dependency relationship between source files, so that they can be built in the correct order. The system scans all source files of known types for all supported dependency patterns. Dependencies of source files in a sub-package are written in a cache, which can be retrieved for incremental builds. (In an incremental build, only changed source files need to be re-scanned for dependency information. Dependency information for other files are retrieved from the cache.) The dependency information is passed to the `make` rule generator, which writes the `Makefile`.

The `make` rule generator generates different `make` rules for different dependency types. The following dependency patterns are automatically detected by the current system:

- The `USE <module>` statement in a Fortran source file is the first pattern. The statement has two implications: 1) The current file compiles only if the module has been successfully compiled, and needs to be re-compiled if the module has changed. 2) The executable depending on the current file can only resolve all its externals by linking with the object file of the compiled module. The executable needs to be re-linked if the module and its dependencies has changed.
- The `INCLUDE '<name>.interface'` statement in a Fortran source file is the second pattern. (The default extension for an interface file is `.interface`. This can be modified through the input and output file type register, which will be discussed in a later section on File Type.) It has two implications: 1) The current file compiles only if the included interface file is in the INCLUDE search path, and needs to be re-compiled if the interface file changes. 2) The executable depending on the current file can only resolve all its externals by linking with the object file of the source file that generates the interface file. The executable needs to be re-linked if the source file (and its dependencies) associated with the interface file has changed. It is worth noting that for this dependency to work, the root <name> of the interface file should match with that of the source file associated with the interface file. (Please note that you can use pre-processor [#include "<name>.interface] instead of Fortran INCLUDE, but it will not work if you switch on the pre-processing stage, which will be discussed in a later section.)
- The `INCLUDE '<file>'` statement (excluding the INCLUDE interface file statement) in a Fortran source file is the third pattern. It has two implications: 1) The current file compiles only if the included file is in the INCLUDE search path, and needs to be re-compiled if the include file changes. 2) The executable needs to be linked with any objects the include file is dependent on. It needs to be re-linked if these objects have changed.
- The `#include '<file>'` statement in a Fortran/C source or header file is the fourth pattern. It has similar implications as the Fortran INCLUDE statement. However, they have to be handled differently because `#include` statements are processed by the pre-processor, which may be performed in a separate stage of the FCM build process. This will be further discussed in a later sub-section on Pre-processing.

If you want your code to be built automatically by the FCM build system, you should also design your code to conform to the following rules:

1. Single compilable program unit, (i.e. program, subroutine, function or module), per file.
2. Unique name for each compilable program unit.
3. Always supply an interface for subroutines and functions, i.e.:
   - Put them in modules.
   - Put them in the CONTAINS section within the main program unit.
   - Use interface files.
4. If interface files are used, it is good practice to name each source file after the program unit it contains. It will make life a lot simpler when using the Automatic Fortran 9X interface block feature, which has already been discussed in the previous section.
   - The problem is that, by default, the root name of the interface file is the same as that of the source file rather than the program unit. If they differ then the build system will create a dependency on the wrong object file (since the object files are named according to the program unit).
   - This problem can be avoided by changing the behaviour of the interface file generator to use the name of the program unit instead (using a `TOOL::INTERFACE` declaration).

Note - setting build targets

  The `Makefile` generated by the build system contains a list of targets that can be built. The build system allows you to build (or perform the actions of) any targets that are present in the generated `Makefile`. There are two ways to specify the targets to be built.

  Firstly, you can use the `TARGET` declarations in your build configuration file to specify the default targets to be built. These targets will be set as dependencies of the `all` target in the generated `Makefile`, which is the default target to be built when `make` is invoked by FCM. It is worth noting that `TARGET` declarations are cumulative. A later declaration does not override an earlier one - it simply adds more targets to the list.

  Alternatively, you can use the `-t` option when you invoke the `fcm build` command. The option takes an argument, which should be a colon `:` separated list of targets to be built. When the `-t` option is set, FCM invokes `make` to build these targets instead. (E.g. if we invoke the build system with the command `fcm build -t foo.exe:bar.exe`, it will invoke `make` to build `foo.exe` and `bar.exe`.)

  If you do not specify any explicit targets, the system will search your source tree for main programs:

  - If there are main programs in your source tree, they will be set as the default targets automatically.
  - Otherwise, the default is to build the top level library archive containing objects compiled from the source files in the current source tree. (For more information on building library archives, please see the section on Creating library archives.)

## Advanced Features

### Further dependency features

Apart from the usual dependency patterns described in the previous sub-section, the automatic dependency scanner also recognises two special directives when they are inserted into a source file:

- The directive `DEPENDS ON: <object>` in a comment line of a Fortran/C source file: It states that the current file is dependent on the declared external object. The executable depending on the current file needs to link with this external object in order to resolve all its external references. It needs to be re-linked if the declared external object (and its dependencies) has changed.
- The directive `CALLS: <executable>` in a comment line of a script: It states that the current script is dependent on the declared executable file, which can be another script or a binary executable. The current script can only function correctly if the declared executable is found in the search path. This directive is useful to ensure that all dependent executables are built or copied to the correct path.

Another way to specify external dependency is to use the `EXE_DEP` declaration to declare extra dependencies. The declaration normally applies to all main programs, but if the form `EXE_DEP::<target>` is used, it will only apply to <target>, (which must be the name of a main program target). If the declaration is made without a value, the main programs will be set to depend on all object files. Otherwise, the value can be supplied as a space delimited list of items. Each item can be either the name of a sub-package or an object target. For the former, the main programs will be set to depend on all object files within the sub-package. For the latter, the main programs will be set to depend on the object target. The following are some examples:

    # Example 5
    # ----------------------------------------------------------------------
    cfg::type          ext
    cfg::version       1.0

    bld::exe_dep::foo.exe  foo/bar egg.o # line 4
    bld::exe_dep                         # line 5
    # ... some other declarations ...

Here is an explanation of what each line does:

- line 4: this line declares the dependency on the sub-package `foo/bar` and the object target `egg.o` for building the main program target `foo.exe`. The target `foo.exe` will now depends on all object files in the `foo/bar` sub-package as well as the object target `egg.o`.
- line 5: this line declares that all other main program targets will depend on all (non-program) object files in the build.

Note - naming of object files

  By default, object files are named with the suffix `.o`. For a Fortran source file, the build system uses the lower case name of the first program unit within the file to name its object file. For example, if the first program unit in the Fortran source file `foo.f90` is `PROGRAM Bar`, the object file will be `bar.o`. For a C source file, the build system uses the lower case root name of the source file to name its object file. For example, a C source file called `egg.c` will have its object file named `egg.o`.

  The reason for using lower case to name the object files is because Fortran is a case insensitive language. Its symbols can either be in lower or upper case. E.g. the `SUBROUTINE Foo` is the same as the `SUBROUTINE foo`. It can be rather confusing if the subroutines are stored in different files. When they are compiled and archived into a library, there will be a clash of namespace, as the Fortran compiler thinks they are the same. However, this type of error does not normally get reported. If `Foo` and `foo` are very different code, the user may end up using the wrong subroutine, which may lead to a very long debugging session. By naming all object files in lower case, this type of situation can be avoided. If there is a clash in names due to the use of upper/lower cases, it will be reported as warnings by the build system, (as duplicated targets for building `foo.o`).

It is realised that there are situations when an automatically detected dependency should not be written into the `Makefile`. For example, the dependency may be a standard module provided by the Fortran compiler, and does not need to be built in the usual way. In such case, we need to have a way to exclude this module during an automatic dependency scan.

The `EXCL_DEP` declaration can be used to do just that. The following extract configuration contains some examples of the basic usage of the `EXCL_DEP` declaration:

    # Example 6
    # ----------------------------------------------------------------------
    cfg::type          ext
    cfg::version       1.0

    bld::excl_dep  USE::YourFortranMod             # line 4
    bld::excl_dep  INTERFACE::HerFortran.interface # line 5
    bld::excl_dep  INC::HisFortranInc.inc          # line 6
    bld::excl_dep  H::TheirHeader.h                # line 7
    bld::excl_dep  OBJ::ItsObject.o                # line 8

    # ... some other declarations ...

Here is an explanation of what each line does:

- line 4: this line declares that the Fortran module `YourFortranMod` should be excluded. The value of each `EXCL_DEP` declaration has two parts. The first part is a label that is used to define the type of dependency to be excluded. For a full list of these labels, please see the dependency types table in the Annex: Declarations in FCM build configuration file. The label `USE` denotes a Fortran module. The second part of the label is the dependency itself. For instance, if a Fortran source file contains the line: `USE YourFortranMod`, the dependency scanner will ignore it.
- line 5: this line declares that the include statement for the Fortran 9X interface file `HerFortran.interface` should be excluded. The label `INTERFACE` denotes a Fortran INCLUDE statement for a Fortran 9X interface block file. For example, if a Fortran source file contains the line: `INCLUDE 'HerFortran.interface'`, the dependency scanner will ignore it.
- line 6: this line declares that the include statement for `HisFortranInc.inc` should be excluded. The label `INC` denotes a Fortran INCLUDE statement other than an INCLUDE statement for an interface block file. For example, if a Fortran source file contains the line: `INCLUDE 'HisFortranInc.inc'`, the dependency scanner will ignore it.
- line 7: this line declares that the header include statement `TheirHeader.h` should be excluded. The label `H` denotes a pre-processing #include statement. For example, if a source file contains the line: `#include 'TheirHeader.h'`, the dependency scanner will ignore it.
- line 8: this line declares that the external dependency for `ItsObject.o` should be excluded. The label `OBJ` denotes a compiled binary object. These dependencies are normally inserted into the source files as special comments. For example, if a source file contains the line: `! depends on: ItsObject.o`, the dependency scanner will ignore it.

An `EXCL_DEP` declaration normally applies to all files in the build. However, you can suffix it with the name of a sub-package, i.e. `EXCL_DEP::<pcks>`. In such case, the declaration will only apply while scanning for dependencies in the source files in the sub-package named <pcks>.

You can also exclude all dependency scan of a particular type. To do so, simply declare the type in the value. For example, if you do not want the build system to scan for the `CALLS: <executable>` directive in the comment lines of your scripts, you can make the following declaration:

    bld::excl_dep  EXE

The opposite of the `EXCL_DEP` declaration is the `DEP::<pcks>` declaration, which you can use to add a dependency to a source file (in the package name `<pcks>`). The syntax of the declaration is similar to that of `EXCL_DEP`, but you must specify the package name of a source file for DEP declarations. Please also note that a `DEP` declaration only works if the particular dependency is supported for the particular source file - as it makes no sense, for example, to specify a USE dependency for a shell script.

If you need to switch off dependency checking completely, you can use the `NO_DEP` declaration. For example, to switch off dependency checking for all but the `foo/bar` sub-package, you can do:

    bld::no_dep           true
    bld::no_dep::foo/bar  false

### Linking a Fortran executable with a BLOCKDATA program unit

If it is required to link Fortran executables with BLOCKDATA program units, you must declare the executable targets and the objects containing the BLOCKDATA program units using the `BLOCKDATA::<target>` declarations. For example, if `foo.exe` is an executable target depending on the objects of the BLOCKDATA program units `blkdata.o` and `fbk.o`, you will make the following declarations:

    bld::blockdata::foo.exe  blkdata fbk

If all your executables are dependent on `blkdata.o` and `fbk.o`, you will make the following declarations:

    bld::blockdata  blkdata fbk

### Creating library archives

If you are interested in building library archives, the build system allows you to do it in a relatively simple way. For each sub-package in the source tree, there is a target to build a library containing all the objects compiled from the source files (that are not main programs) within the sub-package. If the sub-package contains children sub-packages, the object files of the children will also be included recursively. By default, the library archive is named after the sub-package, in the format `lib<pcks>.a`. (For example, the library archive for the package `foo/bar/egg` will be named `libfoo__bar__egg.a` by default.) If you do not like the default name for the sub-package library, you can use the `LIB::<pcks>` declaration to rename it, as long as the new name does not clash with other targets. For example, to rename `libfoo__bar__egg.a` to `libham.a`, you will make the following declaration in your extract configuration file:

    bld::lib::foo/bar/egg  ham

In addition to sub-package libraries, you can also build a global library archive for the whole source tree. By default, the library is named `libfcm_default.a`, but you can rename it using the `LIB` declaration as above. For example, to rename the library to `libmy-lib.a`, you will make the following declaration in your extract configuration file:

    bld::lib  my-lib

When a library archive is created successfully, the build system will automatically generate the relevant exclude dependency configurations in the `etc/` sub-directory of the build root. You will be able to include these configurations in subsequent builds that utilise the library. The root names of the configuration files match those of the library archives that you can create in the current build, but the extension `*.a` is replaced with `*.cfg`. For example, the exclude dependency configuration for `libmy-lib.a` is `libmy-lib.cfg`.

### Pre-processing

As most modern compilers can handle pre-processing, the build system leaves pre-processing to the compiler by default. However, it is recognised that there are code written with pre-processor directives that can alter the argument list of procedures and/or their dependencies. If a source file requires pre-processing in such a way, we have to pre-process before running the interface block generator and the dependency scanner. The `PP` declaration can be used to switch on this pre-processing stage. The pre-processing stage can be switched on globally or for individual sub-packages only. The following is an example, using an extract configuration file:

    # Example 7
    # ----------------------------------------------------------------------
    cfg::type          ext
    cfg::version       1.0

    bld::pp::gen       true                  # line 4
    bld::pp::var/foo   true                  # line 5

    bld::tool::cppkeys GOOD WEATHER FORECAST # line 7
    bld::tool::fppkeys FOO BAR EGG HAM       # line 8

    # ... some other declarations ...

Here is an explanation of what each line does:

- line 4-5: these switches on the pre-processing stage for all sub-packages under `gen` and `var/foo`.
- line 7: this declares a list of pre-defined macros `GOOD`, `WEATHER` and `FORECAST` for pre-processing all C files.
- line 8: this declares a list of pre-defined macros `FOO`, `BAR`, `EGG` and `HAM` for pre-processing all Fortran files that require processing.

Source files requiring pre-processing may contain `#include` statements to include header files. For including a local file, its name should be embedded within a pair of quotes, i.e. `'file.h'` or `"file.h"`. If the header file is embedded within a pair of <file.h> angle brackets, the system will assume that the file can be found in a standard location.

The build system allows header files to be placed anywhere within the declared source tree. The system uses the dependency scanner, as described in the previous sub-section to scan for any header file dependencies. All source files requiring pre-processing and all header files are scanned. Header files that are required are copied to the `inc/` subdirectory of the build root, which is automatically added to the pre-processor search path via the `-I<dir>` option. The build system uses an internal logic similar to `make` to perform pre-processing. Header files are only copied to the `inc/` sub-directory if they are used in `#include` statements.

Unlike `make`, which only uses the timestamp to determine whether an item is out of date, the internal logic of the build system does this by inspecting the content of the file as well. In an incremental build, the pre-processed file is only updated if its content has changed. This avoids unnecessary updates (and hence unnecessary re-compilation) in an incremental build if the changed section of the code does not affect the output file.

Pre-processed code generated during the pre-processing stage are sent to the `ppsrc/` sub-directory of the build root. It will have a relative path that reflects the name of the declared sub-package. The pre-processed source file will have the same root name as the original source file. For C files, the same extension `.c` will be used. For Fortran files, the case of the extension will normally be dropped, e.g. from `.F90` to `.f90`.

Following pre-processing, the system will use the pre-processed source file as if it is the original source file. The interface generator will generate the interface file using the pre-processed file, the dependency scanner will scan the pre-processed file for dependencies, and the compiler will compile the pre-processed source.

The `TOOL::CPPKEYS` and `TOOL::FPPKEYS` declarations are used to pre-define macros in the C and Fortran pre-processor respectively. This is implemented by the build system using the pre-processor `-D` option on each word in the list. The use of these declarations are not confined to the pre-process stage. If any source files requiring pre-processing are left to the compiler, the declarations will be used to set up the commands for compiling these source files.

The `TOOL::CPPKEYS` and `TOOL::FPPKEYS` declarations normally applies globally, but like any other `TOOL` declarations, they can be suffixed with sub-package names. In such cases, the declarations will apply only to the specified sub-packages.

Note - changing pre-processor flags

  As for compiler flags, the build system detects changes in pre-processor flags (`TOOL::CPPFLAGS` and `TOOL::FPPFLAGS`) and macro definitions (`TOOL::CPPKEYS` and `TOOL::FPPKEYS`). If the pre-processor flags or the macro definitions have changed in an incremental build, the system will re-do all the necessary pre-processing. The following hierarchy is followed:

  - If the pre-processor flags or macro definitions for a particular source file change, only that source file will be pre-processed again.
  - If the pre-processor flags or macro definitions for a particular container package change, only source files within that container will be pre-processed again.
  - If the global pre-processor flags or macro definitions change, all source files will be pre-processed again.
  - If the pre-processor command changes, all source files are pre-processed again.

### File type

The build system only knows what to do with an input source file if it knows what type of file it is. The type of a source file is normally determined automatically using one of the following three methods (in order):

1. If the file is named with an extension, its extension will be matched against a set of registered file extensions. If a match is found, the file type will be set according to the register.
2. If a file does not have an extension or does not match with a registered extension, its name is compared with a set of pre-defined patterns. If a match is found, the file type will be set according to the file type associated with the pattern.
3. If the above two methods failed and if the file is a text file, the system will attempt to read the first line of the file. If the first line begins with a `#!` pattern, the line will be compared with a set of pre-defined patterns. If a match is found, the file type will be set according to the file type associated with the pattern.

In addition to the above, if a file is a Fortran or C source file, the system will attempt to open the source file to determine whether it contains a main program, module (Fortran only) or just standalone procedures. All these information will be used later by the build system to process the source file.

The build system registers a file type with a set of type flags delimited by the double colons `::`. For example, a Fortran 9X source file is registered as `FORTRAN::FORTRAN9X::SOURCE`. (Please note that the order of the type flags in the list is insignificant. For example, `FORTRAN::SOURCE` is the same as `SOURCE::FORTRAN`.) For a list of all the type flags used by the build system, please see the input file extension type flags table in the Annex: Declarations in FCM build configuration file.

The following is a list of default input file extensions and their associated types:

`.f .for .ftn .f77`
  `FORTRAN::SOURCE` Fortran 77 source file (assumed to be fixed format)
`.f90 .f95`
  `FORTRAN::FORTRAN9X::SOURCE` Fortran 9X source file (assumed to be free format)
`.F .FOR .FTN .F77`
  `FPP::SOURCE` Fortran 77 source file (assumed to be fixed format) that requires pre-processing
`.F90 .F95`
  `FPP::FPP9X::SOURCE` Fortran 9X source file (assumed to be free format) that requires pre-processing
`.c`
  `C::SOURCE` C source file
`.h .h90`
  `CPP::INCLUDE` Pre-processor `#include` header file
`.o .obj`
  `BINARY::OBJ` Compiled binary object
`.exe`
  `BINARY::EXE` Binary executable
`.a`
  `BINARY::LIB` Binary object library archive
`.sh .ksh .bash .csh`
  `SHELL::SCRIPT` Unix shell script
`.pl .pm`
  `PERL::SCRIPT` Perl script
`.py`
  `PYTHON::SCRIPT` Python script
`.tcl`
  `TCL::SCRIPT` Tcl/Tk script
`.pro`
  `PVWAVE::SCRIPT` IDL/PVWave program
`.cfg`
  `CFGFILE` FCM configuration file
`.inc`
  `FORTRAN::FORTRAN9X::INCLUDE` Fortran INCLUDE file
`.interface`
  `FORTRAN::FORTRAN9X::INCLUDE::INTERFACE` Fortran 9X INCLUDE interface block file

N.B. The extension must be unique. For example, the system does not support the use of `.inc` files for both `#include` and Fortran `INCLUDE`.

The following is a list of supported file name patterns and their associated types:

`*Scr_* *Comp_* *IF_* *Suite_* *Interface_*`
  `SHELL::SCRIPT` Unix shell script, GEN-based project naming conventions
`*List_*`
  `SHELL::SCRIPT::GENLIST` Unix shell script, GEN list file
`*Sql_*`
  `SCRIPT::SQL` SQL script, GEN-based project naming conventions

The following is a list of supported `#!` line patterns and their associated types:

`*sh* *ksh* *bash* *csh*`
  `SHELL::SCRIPT` Unix shell script
`*perl*`
  `PERL::SCRIPT` Perl script
`*python*`
  `PYTHON::SCRIPT` Python script
`*tclsh* *wish*`
  `TCL::SCRIPT` Tcl/Tk script

The build system allows you to add or modify the register for input file extensions and their associated type using the `INFILE_EXT::<ext>` declaration, where <ext> is a file name extension without the leading dot. If file extension alone is insufficient for defining the type of your source file, you can use the `SRC_TYPE::<pcks>` declaration, (where <pcks> is the package name of the source file). For example, in an extract configuration file, you may have:

    # Example 8
    # ----------------------------------------------------------------------
    cfg::type                ext
    cfg::version             1.0

    bld::infile_ext::foo     CPP::INCLUDE                # line 4
    bld::infile_ext::bar     FORTRAN::FORTRAN9X::INCLUDE # line 5
    bld::src_type::egg/ham.f FORTRAN::FORTRAN9X::INCLUDE # line 6

    # ... some other declarations ...

Here is an explanation of what each line does:

- line 4: this line registers the extension `.foo` to be of type `CPP::INCLUDE`. This means that any input files with `.foo` extension will be treated as if they are pre-processor header files.
- line 5: this line registers the extension `.bar` to be of type `FORTRAN::FORTRAN9X::INCLUDE`. This means that any input file with `.bar` extension will be treated as if they are Fortran 9X INCLUDE files.
- line 6: this line declares the type for the source file in the package `egg::ham.f` to be `FORTRAN::FORTRAN9X::INCLUDE`. Without this declaration, this file would normally be given the type `FORTRAN::SOURCE`.

The `INFILE_EXT` declarations deal with extensions of input files. There is also a `OUTFILE_EXT::<type>` declaration that deals with extensions of output files. The declaration is opposite that of `INFILE_EXT`. The file <type> is now declared with the label, and the extension is declared as the value. It is worth noting that `OUTFILE_EXT` declarations use very different syntax for <type>, and the declared extension must include the leading dot. For a list of output types used by the build system, please see the output file extension types table in the Annex: Declarations in FCM build configuration file. An example is given below:

    # Example 9
    # ----------------------------------------------------------------------
    cfg::type                   ext
    cfg::version                1.0

    bld::outfile_ext::mod       .MOD   # line 4
    bld::outfile_ext::interface .intfb # line 5

    # ... some other declarations ...

Here is an explanation of what each line does:

- line 4: this line modifies the extension of compiled Fortran 9X module information files from the default `.mod` to `.MOD`.
- line 5: this line modifies the extension of INCLUDE Fortran 9X interface block files from the default `.interface` to `.intfb`.

N.B. If you have made changes to the file type registers, whether it is for input files or output files, it is always worth re-building your code in full-build mode to avoid unexpected behaviour.

### Inherit from a previous build

As you can inherit from previous extracts, you can inherit from previous builds. The very same `USE` statement can be used to declare a build, which the current build will depend on. The only difference is that the declared location must contain a valid build configuration file. In fact, if you use the extract system to obtain your build configuration file, any `USE` declarations in the extract configuration file will also be `USE` declarations in the output build configuration file.

By declaring a previous build with a `USE` statement, the current build automatically inherits settings from it. The following points are worth noting:

- Build targets are not normally inherited. However, you can switch on inheritance of build targets using an `INHERIT::TARGET` declaration, such as:

      inherit::target  true

- The build root directory and its sub-directories of the inherited build are placed into the search paths. For example, if we have an inherited build at `/path/to/inherited`, and it is used by a build at `/path/to/my_build`, the search path of executable files will become `/path/to/my_build/bin:/path/to/inherited/bin`, so that the `bin/` sub-directory of the current build is searched before the `bin/` sub-directory of the inherited build. If two or more `USE` statements are declared, the `USE` statement declared last will have higher priority. For example, if the current build is C, and it USEs build A before build B, the search path will be `C:B:A`.
- Source files are inherited by default. If a source file is declared in the current build that has the same package name as a source file of the inherited build, it will override that in the inherited build. Any source files missing from the current build will be taken from the inherited build.

  You can switch off inheritance of source files using an `INHERIT::SRC` declaration. This declaration can be suffixed with the name of a sub-package. In such case, the declaration applies only to the inheritance of the sub-package. Otherwise, it applies globally. For example:

      # Switch off inheritance of source files in the gen sub-package
      inherit::src::gen  false

- `BLOCKDATA`, `DEP`, `EXCL_DEP`, `EXE_DEP`, `INFILE_EXT`, `LIB`, `OUTFILE_EXT`, `PP`, `TOOL` and `SRC_TYPE` declarations are automatically inherited. If the same setting is declared in the current incremental build, it overrides the inherited declaration.

As an example, suppose we have already performed an extract and build based on the configuration in example 2, we can set up an extract configuration file as follows:

    # Example 10
    # ----------------------------------------------------------------------
    cfg::type            ext
    cfg::version         1.0

    use                  $HOME/example               # line 4

    dest                 $HOME/example10             # line 6

    bld::inherit::target true                        # line 8
    bld::target          ham.exe egg.exe             # line 9

    bld::tool::fflags    -O2 -w                      # line 11
    bld::tool::cflags    -O2                         # line 12

    # ... and other declarations for repositories and source directories ...

Here is an explanation of what each line does:

- line 4: this line declares a previous extract at `$HOME/example` which the current extract will inherit from. The same line will be written to the output build configuration file. The subsequent build will then inherit from the build at `$HOME/example`.
- line 6: this declares the destination root directory of the current extract, which will become the root directory of the current build. Search paths of the build sub-directories will be set automatically. For example, the search path for executable files created by the current build will be `$HOME/example10/bin:$HOME/example/bin`.
- line 8: this line switches on inheritance of build targets. The build targets in example 1, i.e. `foo.exe` and `bar.exe` will be built as part of the current build.
- line 9: this declares two new build targets `ham.exe` and `egg.exe` to be added to the inherited ones. The default build targets of the current build will now be `foo.exe`, `bar.exe`, `ham.exe` and `egg.exe`.
- line 11-12: these lines modify options used by the Fortran and the C compilers, overriding those inherited from example 1.

Build inheritance limitation: handling of include files

  The build system uses the compiler/pre-processor's `-I` option to specify the search path for include files. For example, it uses the option to specify the `inc/` sub-directories of the current build and its inherited build.

  However, some compilers/pre-processors (e.g. `cpp`) search for include files from the container directory of the source file before searching for the paths specified by the `-I` options. This behaviour may cause the build to behave incorrectly.

  Consider a source file `egg/hen.c` that includes `fried.h`. If the directory structure looks like:

      # Sources in inherited build:
      egg/hen.c
      egg/fried.h

      # Sources in current build:
      egg/fried.h

  The system will correctly identify that `fried.h` is out of date, and trigger a re-compilation of `egg/hen.c`. However, if the compiler searches for the include files from the container directory of the source file first, it will wrongly use the include file in the inherited build instead of the current one.

  Some compilers (e.g. `gfortran`) do not behave this way and others (e.g. `ifort`) have options to prevent include file search in the container directory of the source file. If you are using such a compiler you can avoid the problem for Fortran compilation although this does not fix the problem entirely if you have switched on the pre-processing stage. Otherwise you may have to work around the problem, (e.g. by making a comment change in the source file, or by not using an inherited build at all).

### Building data files

While the usual targets to be built are the executables associated with source files containing main programs, libraries or scripts, the build system also allows you to build data files. All files with no registered type are considered to be data files. For each container sub-package, there is an automatic target for copying all data files to the `etc/` sub-directory of the build root. The name of the target has the form `<pcks>.etc`, where <pcks> is the name of the sub-package (with package names delimited by the double underscore `__`). For example, the target name for sub-package `foo/bar` is `foo__bar.etc`. This target is particularly useful for copying, say, all namelists in a sub-package to the `etc/` sub-directory of the build root.

At the end of a successful build, if the `etc/` sub-directory is not empty, the `fcm_env.sh` script will export the environment variable FCM_ETCDIR to point to the `etc/` sub-directory. You should be able to use this environment variable to locate your data files.

## Diagnostic verbose level

The amount of diagnostic messages generated by the build system is normally set to a level suitable for normal everyday operation. This is the default diagnostic verbose level 1. If you want a minimum amount of diagnostic messages, you should set the verbose level to 0. If you want more diagnostic messages, you can set the verbose level to 2 or 3. You can modify the verbose level in two ways. The first way is to set the environment variable FCM_VERBOSE to the desired verbose level. The second way is to invoke the build system with the `-v <level>` option. (If set, the command line option overrides the environment variable.)

The following is a list of diagnostic output at each verbose level:

Level 0
  - Report the time taken at the end of each stage of the build process.
  - Run the `make` command in silent mode.
Level 1
  - Everything at verbose level 0.
  - Report the name of the build configuration file.
  - Report the location of the build destination.
  - Report date/time at the beginning of each stage of the build process.
  - Report removed directories.
  - Report number of pre-processed files.
  - Report number of generated F9X interface files.
  - Report number of source files scanned for dependencies.
  - Report name of updated `Makefile`.
  - Print compiler/linker commands.
  - Report total time.
Level 2
  - Everything at verbose level 1.
  - For incremental build in archive mode, report the commands used to extract the archives.
  - Report creation and removal of directories.
  - Report pre-processor commands.
  - Print compiler/linker commands with timestamps.
Level 3
  - Everything at verbose level 2.
  - Report update of dummy files.
  - Report all shell commands.
  - Report pre-processor commands with timestamps.
  - Report any F9X interface files generated.
  - Report number of lines and number of automatic dependencies for each source file which is scanned.
  - Run `make` on normal mode (as opposed to silent mode).
  - Report start date/time and time taken of `make` commands.

## Overview of the build process

The FCM build process can be summarised in five stages. Here is a summary of what is done in each stage:

1. Parse configuration and setup destination: in this pre-requisite stage, the build system parses the configuration file. The `src/` sub-directory is searched recursively for source files. For full builds, it ensures that the sub-directories and files created by the build system are removed. If you invoke `fcm build` with a `--clean` option, the system will not go any further.
2. Setup build: in this first stage, the system determines whether any settings have changed by using the cache. If so, the cache is updated with the current settings.
3. Pre-process: if any files in any source files require pre-processing, they will be pre-processed at this stage. The resulting pre-processed source files will be sent to the `ppsrc/` sub-directory of the build root.
4. Generate dependency: the system scans source files of registered types for dependency information. For an incremental build, the information is only updated if a source file is changed. The system then uses the information to write a `Makefile` for the main build.
5. Generate interface: if there are Fortran 9X source files with standalone subroutines and functions, the build system generates interface blocks for them. The result of which will be written to the interface files in the `inc/` sub-directory of the build root.
6. Make: the system invokes `make` on the `Makefile` generated in the previous stage to perform the main build. Following a build, the root directory of the build may contain the following sub-directories (empty ones are removed automatically at the end of the build process):
   `.cache/.bld/`
     Cache files, used internally by FCM.
   `bin/`
     Executable binaries and scripts.
   `cfg/`
     Configuration files.
   `done/`
     Dummy done files used internally by the `Makefile` generated by FCM.
   `etc/`
     Miscellaneous data files.
   `flags/`
     Dummy flags files used internally by the `Makefile` generated by FCM.
   `inc/`
     Include files, such as `*.h`, `*.inc`, `*.interface`, and `*.mod`.
   `lib/`
     Object library archives.
   `obj/`
     Compiled object files.
   `ppsrc/`
     Source directories with pre-processed files.
   `src/`
     Source directories. This directory is not changed by the build system.
   `tmp/`
     Temporary objects and binaries. Files generated by the compiler/linker may be left here.