# Bam Manual

This manual is a work in progress and is not complete.

Copyright © 2010 Magnus Auvinen. Freely available under the terms of the zlib/libpng license.

## Contents

- 1 - Introduction

- 2 - Building Bam

- 2.1 - Unix-like Systems with GCC (Linux, *BSD, Mac OS X)

- 2.2 - Windows

- 2.3 - BeOS and Haiku

- 2.4 - Running the Test Suite

- 3 - Quick Start

- 3.1 - Your first script

- 4 - Custom Actions

- 4.1 - Dependency Graph

- 4.2 - Jobs

- 4.3 - Dependencies

- 4.4 - All Together

- 4.5 - Examples

- 5 - Command Line Reference

- 5.1 - Targets ( name )

- 5.2 - Script Arguments ( name=value )

- 5.3 - Abort on error ( -a )

- 5.4 - Lua execute ( -e )

- 5.5 - Clean ( -c )

- 5.6 - Force ( -f )

- 5.7 - Dependent build ( -d )

- 5.8 - Dry Run ( --dry )

- 5.9 - Threading ( -j N )

- 5.10 - Script File ( -s FILENAME )

- 5.11 - Script Locals ( -l )

- 5.12 - Script Backtrace ( -t )

- 5.13 - Report Format ( -r [b][s][c] )

- 5.14 - Verbose ( -v )

- 5.15 - No cache ( -n )

- 5.16 - Ignore script timestamp ( -g )

- 5.17 - Help ( -h, --help )

- 5.18 - Debug Help ( --help-debug )

- 5.19 - Debug: Dump Nodes ( --debug-nodes )

- 5.20 - Debug: Dump Nodes Detailed ( --debug-detail )

- 5.21 - Debug: Dump Jobs ( --debug-jobs )

- 5.22 - Debug: Dump Joblist ( --debug-joblist )

- 5.23 - Debug: Dump Dot ( --debug-dot )

- 5.24 - Debug: Dump Jobs Dot ( --debug-jobs-dot )

- 5.25 - Debug: Trace VM ( --debug-trace-vm )

- 5.26 - Debug: Event Log ( --debug-eventlog FILENAME )

- 5.27 - Debug: Event Log Flush ( --debug-eventlog-flush )

- 5.28 - Debug: Dump Internal Scripts ( --debug-dump-int )

- 6 - Function Reference

- 6.1 - Common

- 6.1.1 - CheckVersion

- 6.1.2 - Execute

- 6.1.3 - ExecuteSilent

- 6.2 - Files and Directories

- 6.2.1 - Collect

- 6.2.2 - CollectRecursive

- 6.2.3 - CollectDirs

- 6.2.4 - CollectDirsRecursive

- 6.2.5 - MakeDirectory

- 6.2.6 - MakeDirectories

- 6.3 - Job and Dependencies

- 6.3.1 - AddJob

- 6.3.2 - AddDependency

- 6.3.3 - AddDependencySearch

- 6.4 - Modules

- 6.4.1 - Import

- 6.4.2 - ModuleFilename

- 6.5 - Path Manipulation

- 6.5.1 - Path

- 6.5.2 - PathBase

- 6.5.3 - PathFileExt

- 6.5.4 - PathFilename

- 6.5.5 - PathJoin

- 6.5.6 - PathDir

- 6.6 - Settings

- 6.6.1 - AddTool

- 6.6.2 - NewSettings

- 6.7 - Tables

- 6.7.1 - TableDeepCopy

- 6.7.2 - TableFlatten

- 6.7.3 - TableLock

- 6.7.4 - TableToString

- 6.7.5 - TableWalk

- 6.8 - Targets

- 6.8.1 - DefaultTarget

- 6.8.2 - PseudoTarget

- 7 - Tool Reference

- 7.1 - Common Settings (settings)

- 7.1.1 - config_name

- 7.1.2 - config_ext

- 7.1.3 - labelprefix

- 7.2 - Compile

- 7.2.1 - Compile

- 7.2.2 - Compile Settings

- 7.2.3 - C/C++ Settings

- 7.3 - Link

- 7.3.1 - Link

- 7.3.2 - Settings

- 7.4 - Other

- 7.4.1 - CopyFile

- 7.4.2 - CopyToDirectory

- 7.5 - SharedLibrary

- 7.6 - StaticLibrary

- 7.6.1 - StaticLibrary

- 8 - Quirks

- 8.1 - C++ dependency checker is greedy

- 8.2 - Bad job command: Not returning error correctly

- 8.3 - Bad job command: Not updating output timestamp

# 1 - Introduction

Bam is a build system with the focus on being having fast build times and flexiable build scripts. Instead of having a custom language it uses Lua to describe the build steps. It's written in clean C and is distrubuted under the liberal zlib licence. Available on many platforms including but not limited to Linux, Mac OS X and Windows.

# 2 - Building Bam

## 2.1 - Unix-like Systems with GCC (Linux, *BSD, Mac OS X)

Building bam is easy and have very few dependencies. Make sure that you have GCC and the pthreads library installed and run this command:

    sh make_unix.sh

This will build Bam with optimizations turned on. If you want a debug version of Bam, you can invoke the script like this:

    sh make_unix.sh -O0 -g

That will turn off the optimizations and include debug information in the build.

## 2.2 - Windows

To build on windows you need to have either Microsoft Visual C++ or MinGW.

To build with Visual C++, just run press Start, Run and enter cmd.exe and press enter. Then navigate to the Bam directory and type this:

    make_win32_msvc.bat

The batch file will automaticly detect the Visual C++ version and setup the compile environment and then build an optimized version of Bam. You should find the exe in the base directory of Bam. If it fails to detect Visual C++ you should have a shortcut in the start-menu for a Visual C++ command-line that setups the compile environment.

To build with MinGW, make sure that you have a correct build environment setup in a console and then run this:

    make_win32_mingw.bat

An optimized version of Bam will be built and the exe can be located in the base directory of Bam.

## 2.3 - BeOS and Haiku

Building Bam works the same way as on Unix-like systems but the script is called make_beos.sh instead. You can build Bam by running:

    sh make_beos.sh

## 2.4 - Running the Test Suite

Bam comes with a test suite to verify that Bam is working like it should. The suite requires Python to be installed. This test suite is used during the development of Bam to verify that nothing breaks between releases. You can run the test suite by typing this:

    python scripts/test.py

It will either print out that all tests where successfull or what tests it didn't pass.

# 3 - Quick Start

This section is a short introduction to bam and is designed to get you started quickly.

## 3.1 - Your first script

    1: settings = NewSettings()
    2: source = Collect("src/*.c")
    3: objects = Compile(settings, source)
    4: exe = Link(settings, "my_app", objects)

Line 1 creates a new settings object. This contains all the settings on how to compile, link etc. See NewSettings.

Line 2 gathers all the files under the src/ directory which has .c as extention. Collect returns a table of strings which are the files. See Collect.

Line 3 compiles the source using the specified settings and returns a table of the object files. See Compile.

Line 4 links the object files to an executable named "my_app", using the specified settings. See Link.

# 4 - Custom Actions

This section describes how you can make your own custom actions needed for your perticular project. An action is just a normal Lua function that adds a series of jobs and dependencies.

For the sake of demonstration, we are going to compile an application without using the supplied Compile and Link actions. Instead we are going to add the jobs and dependencies by hand. We assume that we are running under a Unix like operating system with GCC as tool chain for sake of simplicity.

## 4.1 - Dependency Graph

At the core of Bam there is a dependency graphs that tells one files are dependent on other files. This graph is used to figure out what order to build the target.

Here is a simple graph over and application called app which has the src1.c, src2.c, header1.h, header2.h and shared.h files.

The commands encased in brackets are the commands that is executed in order to build that node.

## 4.2 - Jobs

A job is a command line that needs to be executed to generate an output file. All actions results in a series of jobs being added to the node graph.

To compile src1.c into obj1.o we simply add this to our bam file:

    AddJob("obj1.o", "compiling src1.c", "gcc -c src1.c -o obj1.o")

Bam now knows that inorder to get obj1.o, it must run "gcc -c src1.c -o obj1.o". It will print "compiling app.c" when this is happening as a nice indicator instead of spewing out the whole command line that it ran. See AddJob for a complete reference of the function.

Now that we can compile our object, we need to link it as well. To link the application, we add this to our bam file:

    AddJob("app", "linking app", "gcc obj1.o obj2.o -o app")

Bam now knows that inorder to get app, it must run "gcc obj1.o obj2.o -o app".

We can now build our application by running these commands from a shell:

    # bam obj1.o
    # bam obj2.o
    # bam app

We must run all three commands because Bam does not yet know that it needs to build obj1.o and obj2.o before it can build app. This is where dependencies comes in.

## 4.3 - Dependencies

To tell Bam that app needs obj1.o and obj2.o, we simply add a dependency. This is done with the AddDependency function like this:

    AddDependency("app", "obj1.o", "obj2.o")
    AddDependency("obj1.o", "src1.c")
    AddDependency("obj2.o", "src2.c")
    AddDependency("src1.c", "header1.h", "header2.h", "shared.h")
    AddDependency("src2.c", "shared.h")
    AddDependency("shared.h", "header2.h")

This tells Bam that app needs obj1.o and obj2.o inorder to build. We also added the source files as dependencies for the object files. This will make sure that bam rebuilds obj1.o when src1.c changes. Also, we added the dependencies for the source and header files so Bam can figure out want needs to be built if one of those changes. See AddDependency for a complete reference of the function.

## 4.4 - All Together

Here is an example of it all working together.

    AddJob("myapp.o", "compiling myapp.c", "gcc -c myapp.c -o myapp.o")
    AddJob("myapp", "linking myapp", "gcc myapp.o -o myapp")
    AddDependency("myapp", "myapp.o")
    AddDependency("myapp.o", "myapp.c")
    DefaultTarget("myapp")

There is also a shortcut that you can use

## 4.5 - Examples

TODO: Some nice text about this

Here is a small function that takes one C file as a string and returns the object file as one string. This is an over simplification of the supplied Compile function and serves just as an example.

    function Compile(cfile)
     	output = PathBase(cfile) .. ".o"
     	AddJob(
     		output,
     		"Compiling " .. cfile,
     		"gcc -c " ..  cfile .. " -o " .. output
     	)
     	AddDependency(output, cfile)
     	return output
    end

# 5 - Command Line Reference

## 5.1 - Targets ( name )

Specify a target to be built. A target can be any output specified to the AddJob function.

If no targets are specified, the default target will be built If there are no default target and there is only one target specified with the Target function, it will be built. Otherwise bam will report an error.

There is a special pseudo target named all that represents all targets specified by the Target function.

## 5.2 - Script Arguments ( name=value )

Sets a script argument. These arguments can be fetched form the build script by accessing the ScriptArgs table.

## 5.3 - Abort on error ( -a )

Setting this will cause bam to abort the build process when an error has occured. Normally it would continue as far as it can.

## 5.4 - Lua execute ( -e )

Executes a lua file without running the build system.

## 5.5 - Clean ( -c )

Cleans the specified targets or the default target.

## 5.6 - Force ( -f )

Forces all the jobs to be dirty

## 5.7 - Dependent build ( -d )

Builds all targets that are dependent on the given targets. If no targets are given this option doesn't do anything.

## 5.8 - Dry Run ( --dry )

Does everything that it normally would do but does not execute any commands.

## 5.9 - Threading ( -j N )

Sets the number of threads used when building. A good value for N is the same number as logical cores on the machine. Set to 0 to disable.

## 5.10 - Script File ( -s FILENAME )

Bam file to use. In normal operation, Bam executes bam.lua. This option allows you to specify another bam file.

## 5.11 - Script Locals ( -l )

Prints local and up values in the backtrace when there is a script error

## 5.12 - Script Backtrace ( -t )

Prints backtrace when there is a script error

## 5.13 - Report Format ( -r [b][s][c] )

Sets the format of the progress report when building.

- b
- Use a progress bar showing the percentage.
- s
- Show each step when building. (default)
- c
- Use ANSI colors.

## 5.14 - Verbose ( -v )

Prints all commands that are runned when building.

## 5.15 - No cache ( -n )

Do not use cache when building.

## 5.16 - Ignore script timestamp ( -g )

Ignores the timestamp on the script when doing dirty checking. Enabling this causes the output not to be rebuilt when the build script changes.

## 5.17 - Help ( -h, --help )

Prints out a short reference of the command line options and quits directly after.

## 5.18 - Debug Help ( --help-debug )

Prints out a reference over the debugging options.

## 5.19 - Debug: Dump Nodes ( --debug-nodes )

Dumps all nodes in the dependency graph.

## 5.20 - Debug: Dump Nodes Detailed ( --debug-detail )

Dumps all nodes in the dependency graph, their state and their dependent nodes. This is useful if you are writing your own actions to verify that dependencies are correctly added.

## 5.21 - Debug: Dump Jobs ( --debug-jobs )

## 5.22 - Debug: Dump Joblist ( --debug-joblist )

## 5.23 - Debug: Dump Dot ( --debug-dot )

Dumps all nodes in the dependency graph into a dot file that can be rendered with graphviz.

## 5.24 - Debug: Dump Jobs Dot ( --debug-jobs-dot )

Dumps all jobs and their dependent jobs into a dot file that can be rendered with graphviz.

## 5.25 - Debug: Trace VM ( --debug-trace-vm )

Prints a the function and source line for every instruction that the vm makes.

## 5.26 - Debug: Event Log ( --debug-eventlog FILENAME )

Outputs an build event log that contains all the events with timing information.

## 5.27 - Debug: Event Log Flush ( --debug-eventlog-flush )

Flushes the event log after each write.

## 5.28 - Debug: Dump Internal Scripts ( --debug-dump-int )

# 6 - Function Reference

## 6.1 - Common

### 6.1.1 - CheckVersion

Tells bam what version this script is written for. It will either make sure that it behaves like that version or print out an error.

    CheckVersion("0.1.0")

### 6.1.2 - Execute (command)

Executes the command in the shell and returns the error code.

### 6.1.3 - ExecuteSilent (command)

Does the same as Execute(command) but supresses stdout and stderr of that command.

## 6.2 - Files and Directories

### 6.2.1 - Collect (...)

Gathers a set of files using wildcard. Accepts strings and tables of strings as input and returns a table of all the files that matches A single wildcard * may be used in each string to collect a set of files.

Example:

    source_files = Collect("src/*.c", "lib/*.c")

Note. This version collects files, non-recursive.

### 6.2.2 - CollectRecursive (...)

Collects files as the Collect but does so recursivly.

### 6.2.3 - CollectDirs (...)

Collects directories in the same fashion as Collect but returns directories instead.

### 6.2.4 - CollectDirsRecursive (...)

Collects directories in the same fashion as Collect but does so recursivly and returns directories instead.

### 6.2.5 - MakeDirectory (path)

Creates the requested directory.

### 6.2.6 - MakeDirectories (filename)

Creates the path upto the filename.

Example:

    MakeDirectories("output/directory/object.o")

This will create the complete "output/directory" path.

## 6.3 - Job and Dependencies

### 6.3.1 - AddJob (output, label, command, ...)

Adds a job to be done. The output string specifies the file that will be created by the command line specified in command string. The label is printed out before command is runned. You can also add extra parameters, those will become for dependencies for the job.

    AddJob("myapp.o", "compiling myapp.c", "gcc -c myapp.c -o myapp.o")
    AddDependency("myapp.o", "myapp.c")

This is the same as this:

    AddJob("myapp.o", "compiling myapp.c", "gcc -c myapp.c -o myapp.o", "myapp.c")

You can also add several dependencies at once like this:

    AddJob("myapp", "linking myapp", "gcc myapp1.o myapp2.o myapp3.o -o myapp.o", {"myapp1.o", "myapp2.o"}, "myapp3.o")

### 6.3.2 - AddDependency (filename, ...)

Adds dependencies to a job. The files specified in the argument list gets added. Strings and nested tables of strings are accepted.

### 6.3.3 - AddDependencySearch (filename, paths, ...)

Searches for dependencies in the specified paths and adds them to the file.

## 6.4 - Modules

### 6.4.1 - Import (filename)

Imports a script specified by filename. A search for the script will be done by first checking the current directory and then the paths specified by the BAM_PACKAGES environment variable. Several paths can be specified in the variable by separating them by a ':' character.

The importing script can figure out it's path by calling the ModuleFilename function.

### 6.4.2 - ModuleFilename ()

Returns the filename of the current script being imported (by Import) as relative to the current working directory.

## 6.5 - Path Manipulation

### 6.5.1 - Path (str)

Normalizes the path str by removing ".." and "." from it.

    Path("test/./path/../file.name.ext") -- Returns "test/file.name.ext"
    Path("../test/../path/file.name.ext") -- Returns "../path/file.name.ext"

### 6.5.2 - PathBase (path)

Returns the everthing except the extention in the path.

    PathBase("test/path/file.name.ext") -- Returns "test/path/file.name"
    PathBase("test/path/file.name") -- Returns "test/path/file"
    PathBase("test/path/file") -- Returns "test/path/file"

### 6.5.3 - PathFileExt (str)

Returns the extension of the filename in str.

    PathFileExt("test/path/file.name.ext") -- Returns "ext"

### 6.5.4 - PathFilename (str)

Returns the filename of the path in str.

    PathFilename("test/path/file.name.ext") -- Returns "file.name.ext"

### 6.5.5 - PathJoin (base, add)

Joins the two paths base and add together and returns a normalized path. This function haldes trailing path separators in the base argument.

    PathJoin("test/path/", "../filename.ext") -- Returns "test/filename.ext"
    PathJoin("../test", "path/filename.ext") -- Returns "../test/path/filename.ext"

### 6.5.6 - PathDir (str)

Returns the path of the filename in str.

    PathDir("test/path/file.name.ext") -- Returns "test/path"

## 6.6 - Settings

### 6.6.1 - AddTool (func)

Adds a new tool called name to bam. The func will be called when NewSettings function is invoked with the settings object as first parameter.

### 6.6.2 - NewSettings ()

Create a new settings table with the settings for all the registered tools. This table is passed to many of the tools and contains how they should act.

## 6.7 - Tables

### 6.7.1 - TableDeepCopy (tbl)

Makes a deep copy of the table tbl resulting in a complete separate table.

### 6.7.2 - TableFlatten (tbl)

Does a deep walk of the tbl table for strings and generates a new flat table with the strings. If it occurs anything else then a table or string, it will generate an error.

    -- Returns {"a", "b", "c", "d", "e", "f"}
    TableFlatten({"a", {"b", {"c"}, "d"}, "e", {}, {"f"}})

### 6.7.3 - TableLock (tbl)

Locks the table tbl so no new keys can be added. Trying to add a new key will result in an error.

### 6.7.4 - TableToString (tbl, prefix, postfix)

Takes every string element in the tbl table, prepends prefix and appends postfix to each element and returns the result.

    TableToString({"a", "b"}, "", "") -- Returns "ab"

### 6.7.5 - TableWalk (tbl)

Returns an iterator that does a deep walk of a table looking for strings. Only checks numeric keys and anything else then table and strings will cause an error.

    for filename in TableWalk({...}) do
    print(filename)
    end

## 6.8 - Targets

### 6.8.1 - DefaultTarget (filename)

Specifies the default target use build when no targets are specified when bam is invoked.

### 6.8.2 - PseudoTarget (name, ...)

Creates a pseudo target named name and assigns a set of dependencies specified by ....

# 7 - Tool Reference

## 7.1 - Common Settings (settings)

### 7.1.1 - config_name

Name of the settings. TODO: explain when you could use it

### 7.1.2 - config_ext

A short postfix that you can append to files that have been built by this configuration.

### 7.1.3 - labelprefix

Prefix to use for all jobs that are added. TODO: this option feels a bit strange

## 7.2 - Compile

### 7.2.1 - Compile (settings, ...)

Compiles a set of files using the supplied settings. It uses settings.compile.mappings to map the input extension to a compiler function. A compiler functions should look like Compiler(settings, input) where settings is the settings object and input is the filename of the file to compile. The function should return a string that contains the object file that it will generate.

    function MyCompiler(settings, input)
        -- compile stuff
        return output
    endsettings.compile.mappings".my" = MyCompiler
    objects = Compile(settings, "code.my") -- Invokes the MyCompiler function

### 7.2.2 - Compile Settings (settings.compile)

mappings | Table that matches extentions to a compiler function. See the Compile function for a reference how this table is used.

### 7.2.3 - C/C++ Settings (settings.cc)

defines | Table of defines that should be set when compiling.
    settings.cc.defines:Add("ANSWER=42")
DriverC | Function that drives the C compiler. Function is responsible for building the command line and adding the job to compile the input file.
DriverCXX | Same as DriverC but for the C++ compiler.
exe_c | Name (and path) of the executable that is the C compiler
exe_cxx | Same as c_exe but for the C++ compiler
extension | Extention that the object files should have. Usally ".o" or ".obj" depending on compiler tool chain.
flags | Table of flags that will be appended to the command line to the C/C++ compiler. These flags are used for both the C and C++ compiler.
    settings.cc.flags:Add("-O2", "-g")
flags_c | Same as flags but specific for the C compiler.
flags_cxx | Same as flags but specific for the C++ compiler.
frameworks | Mac OS X specific. What frameworks to use when compiling.
includes | Table of paths where to find headers.
    settings.cc.includes:Add("my/include/directory")
Output(settings, path) | Function that should transform the input path into the output path. The appending of the extension is done automaticly.
    settings.cc.Output = function(settings, input)
        return PathBase(input) .. settings.config_ext
    end
systemincludes | Mac OS X specific. Table of paths where to find system headers.

## 7.3 - Link

### 7.3.1 - Link (settings, output, ...)

TODO

### 7.3.2 - Settings (settings.link)

Driver | Function that drives the linker. Function is responsible for building the command line and adding the job to link the input files into an executable.
exe | Path to the executable to use as linker.
extension | Extention of the executable. Usally "" on most platform but can be ".exe" on platforms like Windows.
extrafiles | A table of additional files that should be linked against. These files will be treated as normal objects when linking.
flags | Table of raw flags to send to the linker.
    settings.link.flags:Add("-v")
frameworks | Mac OS X specific. A table of frameworks to link against.
frameworkpath | Mac OS X specific. A table of paths were to find frameworks.
libs | Table of library files to link with.
    settings.link.libs:Add("pthread")
libpath | A table of paths of where to find library files that could be included in the linking process.
Output | Function that should transform the input path into the output path. The appending of the extension is done automaticly.
    settings.link.Output = function(settings, input)
        return PathBase(input) .. settings.config_ext
    end
inputflags (REMOVE?) |

## 7.4 - Other

### 7.4.1 - CopyFile (dst, src)

### 7.4.2 - CopyToDirectory (dst, ...)

## 7.5 - SharedLibrary

## 7.6 - StaticLibrary

### 7.6.1 - StaticLibrary (settings, output, ...)

TODO

# 8 - Quirks

This section is a collection of quirks and ...

## 8.1 - C++ dependency checker is greedy

The C++ dependency checker in bam is very simple and can in exotic cases cause problems. The parser only checks for include lines and ignores all ifdefs. This will cause that more files then nessesary will be compiled when a header changes but is seldom a problem.

## 8.2 - Bad job command: Not returning error correctly

If a job command doesn't return an error code when it failed, bam will consider it a success and move on as usual. This can become even worse if the command touches the output file and fails without returning a proper error code. In this case, bam will never know that it has to rebuild that target again.

This problem can be solved by wrapping the bad commandline in some script that parses the output of it, detects errors and returns a correct error code.

## 8.3 - Bad job command: Not updating output timestamp

If a job command is successful but doesn't touch and update the time stamp on the output file, bam will try to build that again every time you run it.

This problem can be solved by telling bam that it needs to touch the output file after a successful to update the timestamp.

    function MyTool(input, output)
     	AddJob(output, "running mytool", "mytool " .. input .. " " .. output)
     	SetTouch(output)
    end

Generated at Tue Jul 16 15:05:41 2024.