Ruffus Decorators

See also

Indicator objects

Core

Decorator	Examples
@originate (Summary / Manual) Creates (originates) a set of starting file without dependencies from scratch (ex nihilo!) Only called to create files which do not exist. Invoked onces (a job created) per item in the output_files list.	@originate ( output_files, [extra_parameters,...] )
@split (Summary / Manual) Splits a single input into multiple output Globs in output can specify an indeterminate number of files.	@split ( tasks_or_file_names, output_files, [extra_parameters,...] )
@transform (Summary / Manual) Applies the task function to transform input data to output.	@transform ( tasks_or_file_names, suffix(suffix_string), output_pattern, [extra_parameters,...] ) @transform ( tasks_or_file_names, regex(regex_pattern), output_pattern, [extra_parameters,...] ) @transform ( tasks_or_file_names, formatter(regex_pattern), output_pattern, [extra_parameters,...] )
@merge (Summary / Manual) Merges multiple input files into a single output.	@merge (tasks_or_file_names, output, [extra_parameters,...] )

Combinatorics

Decorator	Examples
@product (Summary / Manual) Generates the product, i.e. all vs all comparisons, between sets of input files.	@product ( tasks_or_file_names, formatter ([ regex_pattern ]) ,*[* tasks_or_file_names, formatter ([ regex_pattern ]), ], output_pattern, [extra_parameters,...] )
@permutations (Summary / Manual) Generates the permutations, between all the elements of a set of Input Analogous to the python itertools.permutations permutations(‘ABCD’, 2) –> AB AC AD BA BC BD CA CB CD DA DB DC	@permutations ( tasks_or_file_names, formatter ([ regex_pattern ]), tuple_size, output_pattern, [extra_parameters,...] )
@combinations (Summary / Manual) Generates the permutations, between all the elements of a set of Input Analogous to the python itertools.combinations combinations(‘ABCD’, 3) –> ABC ABD ACD BCD Generates the combinations, between all the elements of a set of Input: i.e. r-length tuples of input elements with no repeated elements (A A) and where order of the tuples is irrelevant (either A B or B A, not both).	@combinations ( tasks_or_file_names, formatter ([ regex_pattern ]), tuple_size, output_pattern, [extra_parameters,...] )
@combinations_with_replacement (Summary / Manual) Generates the permutations, between all the elements of a set of Input Analogous to the python itertools.permutations combinations(‘ABCD’, 3) –> ABC ABD ACD BCD Generates the combinations_with_replacement, between all the elements of a set of Input: i.e. r-length tuples of input elements with no repeated elements (A A) and where order of the tuples is irrelevant (either A B or B A, not both).	@combinations_with_replacement ( tasks_or_file_names, formatter ([ regex_pattern ]), tuple_size, output_pattern, [extra_parameters,...] )

Advanced

Decorator	Examples
@subdivide (Summary / Manual) - Subdivides a set of Inputs each further into multiple Outputs. - The number of files in each Output can be set at runtime by the use of globs. - Many to Even More operator. - The use of split is a synonym for subdivide is deprecated.	@subdivide ( tasks_or_file_names, regex(regex_pattern), [ inputs | add_inputs(input_pattern), ] output_pattern, [extra_parameters,...] ) @subdivide ( tasks_or_file_names, formatter([regex_pattern] ), [ inputs | add_inputs(input_pattern), ] output_pattern, [extra_parameters,...] )
@transform (Summary / Manual) Infers input as well as output from regular expression substitutions Useful for adding additional file dependencies	@transform ( tasks_or_file_names, regex(regex_pattern), [ inputs | add_inputs(input_pattern), ] output_pattern, [extra_parameters,...] ) @transform ( tasks_or_file_names, formatter(regex_pattern), [ inputs | add_inputs(input_pattern), ] output_pattern, [extra_parameters,...] )
@collate (Summary / Manual) Groups multiple input files using regular expression matching Input resulting in the same output after substitution will be collated together.	@collate (tasks_or_file_names, regex(regex_pattern), output_pattern, [extra_parameters,...] ) @collate (tasks_or_file_names, regex(regex_pattern), inputs | add_inputs(input_pattern), output_pattern, [extra_parameters,...] ) @collate (tasks_or_file_names, formatter(formatter_pattern), output_pattern, [extra_parameters,...] ) @collate (tasks_or_file_names, formatter(formatter_pattern), inputs | add_inputs(input_pattern), output_pattern, [extra_parameters,...] )
@follows (Summary / Manual) Indicates task dependency optional mkdir prerequisite (see Manual)	@follows ( task1, 'task2' )) @follows ( task1, mkdir( 'my/directory/' ))
@posttask (Summary / Manual) Calls function after task completes Optional touch_file indicator (Manual)	@posttask ( signal_task_completion_function ) @posttask (touch_file( 'task1.completed' ))
@active_if (Summary / Manual) Switches tasks on and off at run time depending on its parameters Evaluated each time pipeline_run(...), pipeline_printout(...) or pipeline_printout_graph(...) is called. Dormant tasks behave as if they are up to date and have no output.	@active_if ( on_or_off1, [on_or_off2, ...] )
@jobs_limit (Summary / Manual) Limits the amount of multiprocessing for the specified task Ensures that fewer than N jobs for this task are run in parallel Overrides multiprocess parameter in pipeline_run(...)	@jobs_limit ( NUMBER_OF_JOBS_RUNNING_CONCURRENTLY )
@mkdir (Summary / Manual) Generates paths for os.makedirs	@mkdir ( tasks_or_file_names, suffix(suffix_string), output_pattern ) @mkdir ( tasks_or_file_names, regex(regex_pattern), output_pattern ) @mkdir ( tasks_or_file_names, formatter(regex_pattern), output_pattern)
@graphviz (Summary / Manual) Customise the graphic for each task in printed flowcharts	@graphviz ( graphviz_parameter = XXX, [graphviz_parameter2 = YYY ...])

Esoteric!

Decorator	Examples
@files (Summary / Manual) I/O parameters skips up-to-date jobs Should use @transform etc instead	@files( parameter_list ) @files( parameter_generating_function ) @files ( input_file, output_file, other_params, ... )
@parallel (Summary / Manual) By default, does not check if jobs are up to date Best used in conjuction with @check_if_uptodate	@parallel ( parameter_list ) (see Manual) @parallel ( parameter_generating_function ) (see Manual)
@check_if_uptodate (Summary / Manual) Custom function to determine if jobs need to be run	@check_if_uptodate ( is_task_up_to_date_function )
Tip The use of this overly complicated function is discouraged. @files_re (Summary) I/O file names via regular expressions start from lists of file names or glob results skips up-to-date jobs	@files_re ( tasks_or_file_names, matching_regex, [input_pattern,] output_pattern, ... ) input_pattern/output_pattern are regex patterns used to create input/output file names from the starting list of either glob_str or file names