Command Line Tool - ftc


Fuego comes with a command line tool, called ftc, that is used to perform a variety of tasks.

ftc has many sub-commands for performing different operations, but they can be grouped into different categories, including the following:

  • Essential commands

  • Commands for inspecting results

  • Commands for managing boards

  • Commands for working with Jenkins

  • Commands for working with a Fuego (or other) server

  • Miscelaneous commands

Note that you can get a list of all the commands supported by ftc, by using ftc help.

For help on any individual sub-command, use ftc help <command>.

General FTC options

Some options to ftc commands are used consistently throughout the tool. That is, there are some global options, which apply to any sub-command, such as ‘-v’ for verbose mode. Or, some options are always specified with the same argument letter, such as ‘-b’ to specify the board for the operation.

Here are options that are commonly used with ftc commands:

  • -v = verbose mode - this means to report more information than usual

  • -q = quiet mode - this means to report less information than usual, sometimes making the command completely silent. Quiet mode is often used to omit header data and make the output suitable for parsing by other tools.

  • -h = help - show help for the specified sub-command

  • –debug = run in debug mode - this makes ftc produce debug information while it runs

  • -b = specify the board - used when a command takes a board argument

  • -t = specify the test - used when a command test a test argument.


For commands that require a board, you can specify the board a few different ways. The most common is to use the ‘-b’ option followed by the board name. You can omit this if you have a board name specified in the environment variable FUEGO_BOARD. Also, you can set a default board name in your fuego configuration, which is found in fuego-ro/conf/fuego.conf.

Some commands (such as add-board, add-job, or run-test) can operate on multiple boards. In this case, use the ‘-b’ option and specify multiple board names separated by commas after the ‘-b’, like this: ftc add-board -b board1,board2

In some cases, you don’t need to specify the full board name, but can provide a short string, as long as the string is sufficient to identify the desired board uniquely. For example, if you had only a single board name that started with a ‘q’ (like ‘qemu-arm’), you could use something like this: ftc add-board -b q


A test name is specified using either its full name, which includes the test type, or its shortened name, which is the portion of the name after the type. For example, for the test Functional.hello_world, the full test name is “Functional.hello_world”, and the short name is just “hello_world”. Either name may be specified with the ‘-t’ option, although it is usually more convenient to use the short name.

In case there is both a Benchmark and Functional test with the same short name, the full name must be used with ‘-t’ to make it clear which test is referred to.

Commands groups

Essential commands

Here are the commands that are essential for adding information to the Jenkins interface (nodes and jobs), for querying the available boards, tests, and test specs, and for executing a test.

  • add-nodes

  • add-jobs

  • list-boards

  • list-tests

  • list-specs

  • run-test

These core commands are used for setup of jobs in Jenkins, and for finding out (or trying to remember), the test elements that are available on your system for test exection. Use ftc add-nodes and ftc add-jobs to populate the Jenkins interface with nodes and jobs (respectively. See the section Jenkins-related commands below for more information about these commands, or see Adding a Board or Adding test jobs to Jenkins for more details.

The most important ftc command is ftc run-test. This is the command used to actually run a test. Details about ftc run-test can be found in the section ftc run-test below.

Commands for inspecting results

ftc provides a few commands for inspecting the results from operations. Usually, Jenkins is used to show visualization of results. However, you can also see what tests have been run, and what the results from those tests were, using the following commands:

  • list-runs

  • gen-report

Use the list-runs command to see a list of all test runs on the system. The list of runs can be filtered by various criteria.

FIXTHIS - reference –where clause description section

Use the gen-report command to generate reports of test results data.

Usually, test results are examined in the Jenkins User interface. However, you can also generate lists of results at the command line using ftc gen-report

This command gives you control over the results that are reported, as well as the content (exact fields and headers) and format of the report.

In summary, ftc gen-report can:

  • select the test runs to report results from

  • select the header fields to show in the report

  • select the data (result) fields to show in the report

  • filter the data by results (for example showing only failures)

  • select the format of the report

  • specify the output location for the report

See the section Generating Reports for details about this command and its options, and overall information about generating reports from test run data.

Board management

These commands have to do with managing boards (defined on the local machine):

  • list-boards

  • query-board

  • set-var

  • delete-var

  • power-cycle

  • power-off

  • power-on

In Fuego, boards are defined and configured in a board file, found in the fuego-ro/boards directory.

You can use ftc list-boards to see a list of the currently configured boards in the Fuego system.

Board attributes (or variables)

Usually, to change the configuration of a board, you manually edit the file for that board and adjust its base settings directly. However, Fuego also allows for viewing board attributes (also referred to as board ‘variables’), and for setting and removing attributes of a board using ftc (that is, without having to manually read or edting the board configuration file).

The variables defined in the board configuration file are considered its ‘base’ settings or base attributes. These attributes are considered statically defined for a board. Fuego also allows you to store information about a board that is considered dyanmic. This information is stored in a board configuration file in the fuego-rw/boards directory.

Also, Fuego automatically assigns certain functions to a board based on the value of the DISTRIB variable for the the board. These functions are called overlay functions, because they can be overridden (or “overlayed”) with functions from the board configuration file.

ftc query-board

You can use the ftc query-board command to view any of the configured or calculated information about a board. This includes its base variables, dynamic variables, and overlay functions.

To see all of the attributes of a board, use ftc query-board and specify the board to inspect, like this:

ftc query-board -b beaglebone

The output may be quite verbose. To see just a list of attributes names, (ie without their values), use:

ftc query-board -q -b beaglebone

To see the value of a single attribute, use the -n option, and specify the attribute name:

ftc query-board -b beaglebone -n TOOLCHAIN

The set-var and delete-var commands are used to set or delete an individual dynamic variable for a board. These ftc commands are intended for programs that automatically configure attributes of a board, and are not usually used by users directly.

ftc set-var and delete-var

Here are some examples of using set-var and delete-var on a board:

ftc set-var -b beaglebone FOO_COUNT=5

ftc delete-var -b beaglebone FOO_COUNT

These would set FOO_COUNT (to the value of ‘5’) in the beaglebone board attributes or remove FOO_COUNT from the beaglebone board attributes, respectively.

Finally, ftc includes commands for performing power control of a board. When Fuego detects that a board is not responding, it tries to automatically restart the board by doing a power reset.

ftc power commands

The three commands that can be used to manipulate board power are: power-cycle, power-off, and power-on

Here is an example of a power-related command for a board:

ftc power-cycle -b beaglebone


In order for Fuego to be able to manipulate the power for a board, the board must have a supported BOARD_CONTROL system in its configuration.

Commands for working with a Fuego (or other) server

The Fuego server feature supports executing tests, and sharing test definitions and test run results, between multiple test sites. This feature is currently under construction.

The following commands are related to using Fuego in conjunction with a Fuego server:

  • get-board

  • get-run

  • install-run

  • install-test

  • list-requests

  • package-run

  • package-test

  • put-binary-package

  • put-request

  • put-run

  • put-test

  • query-request

  • rm-request

  • update-board

The following commands support remote operations (using the ‘-r’ or ‘–remote’ flags):

  • list-boards

  • list-tests

  • list-runs

These commands are used for performing operations with a Fuego server. A Fuego server supports registering boards, and storing test packages, binary test packages, run data, and test execution requests. the package-test and package-run allow for creating packages for a test and a run, respectively. These can be uploaded to the server, or sent directly to another developer, who can install them on their system.

Users can download tests, binary-packaged tests, and runs from a server.

A Fuego server allows a collection of Fuego sites to share tests and test results (runs) with each other. It also allows users to request tests to be executed on boards at another site. These test ‘requests’ can be submitted, viewed, and processed by users interacting with the central server.


The Fuego server feature is currently under construction. You may experiment with it if you would like, but the features are not robust and the documentation is not finished for it yet. Proceed at your own risk with this feature (and these commands).

Miscellaneous commands

The following commands are for various utility functions, unrelated to the other categories of operations:

  • config

  • help

  • version

The ftc config command allow quering the current ftc config file, (located in the fuego-ro\conf directory). Use ftc config -l to get a list of all config items, and ftc config <config_name> to get the value of the indicated config item. Usually, humans will not use this, as they can inspect the file manually. The config command is intended for use for external tools that want to determine the value for a specific Fuego configuration item.

The ftc help command is used to get online usage help for ftc and for individual ftc commands.


ftc help - will show a list of all available ftc commands

ftc help list-boards - will show the help for the 'list-boards' command

ftc version shows the current version of ftc.

ftc run-test

One of the most important commands that ftc can execute is the ‘run-test’ command. This is the command actually used to perform a test on a board. A test can be executed either from the command line (using the ftc command, or it can be exected from Jenkins, via a job definition (which is made with the ftc add-jobs command). Even when running from Jenkins, the ftc run-test command is used to actually execute the test.

When running a test, multiple arguments and options are supported.

Arguments that are required for this are the board name and the test name. The board is specified using ‘-b’ and the test is specified using ‘-t’. foo bar

Here is the ftc run-test usage:

Usage: ftc run-test -b <board> -t <test> [-s <spec>] [-p <phases>]
  [--timeout <timeout>]
  [--rebuild <true|false>]
  [--reboot <true|false>]
  [--precleanup <true|false>]
  [--postcleanup <true|false>]
  [--dynamic-vars <variable assignments or python_dict>]

Choosing a test spec

Some Fuego tests include different variants of a test, that can be selected using the test spec. You can see the list of specs for a test using the ftc list-specs command. If no spec is specified for run-test, then the “default” spec is used, which is usually the test executed in its most common configuration.

Test control options

Various other flags control aspects of test execution:

  • timeout: specify the maximum time the test is allowed to run (default: 30m = 30 minutes)

  • rebuild: if ‘true’ rebuild the test source even if it was already built. (default: ‘false’)

  • reboot: if ‘true’ reboot the board before the test. (default: ‘false’)

  • precleanup: if ‘false’, do not clean up the board’s test folder before the test. (default: ‘true’)

  • postcleanup: if ‘false’ do not clean up the board’s test folder after the test. (default: ‘true’)

  • batch: generate a batch id for this test

Each of the boolean test control flags can set be ‘true’ or ‘false’.

The control flags are used to pre-reboot the board being tested, or to prevent or force cleaning up the test directory. Usually, Fuego removes all traces of the test upon test completion. When debugging a test, it is often useful to set --postcleanup to ‘false’, so that Fuego won’t remove the test directory on the board at the end of the test. This allows you to inspect the test materials on the board, or run the test manually. Setting --precleanup to ‘false’ is sometimes useful when you want to avoid the deploy phase. (See phases below.)

The timeout value is specified as an integer and a suffix (one of s, m, h, or d). The suffix corresponds to one of: seconds, minutes, hours, days. For example, a 10-minute timeout would be specified as --timeout 10m.

The batch id is a number used to group tests together for reporting purposes. if --batch is specified, Fuego will select a new batch id for the test, and set the FUEGO_BATCH_ID environment variable. This will be recorded for this test and any sub-tests called during execution of the test). You can filter tests using the batch id in a --where clause, in the ftc gen-report command.


If you would like to specify your own batch id for a test, you can do so by setting the FUEGO_BATCH_ID environment variable to your own value before calling ftc run-test.

Test variables

It is possible to override one or more test variables on the ftc run-test command line, using the --dynamic-vars option.

This allows overriding the variables in a test spec from the ftc command line. For example, the Benchmark.Dhrystone test uses a test variable of LOOPS to indicated the number of times to execute the Dhrystone operations. The value of this variable in the default spec for this test is 10000000 (10 million). You could override this value at the command line, using

ftc run-test -b bbb -t Dhrystone --dynamic-vars "LOOPS=40000000"

You can specify the variable value using a python dictionary expression or using simple NAME=VALUE syntax. Here is an example using python dictionary syntax:

ftc run-test -b bbb -t Benchmark.Dhrystone --dynamic-vars "{'LOOPS':'400000000'}"

Note that both of these syntaxes allow for multiple variables to be specified. In the case of NAME=VALUE pairs, separate the pairs with a comma, like this:

--dynamic-vars "VAR1=value1,VAR2=Another value"


A test is normally run in phases, one sequentially after the other. However, in special circumstances (such as when debugging a test during test development) it may be useful to only execute certain phases of the test. Some phases take a long time, and it can be helpful to skip them when debugging a test.

When specifying a set of phases with the run-test ‘-p’ option, each phases is represented by a single character:

p = pre_test
c = pre_check
b = build
d = deploy
s = snapshot
r = run
t = post_test
a = processing
m = make binary package

To control the phases executed during a test run, use the ‘-p’ option, and specify a list of characters corresponding to the phases you want to execute.

For example:

ftc run-test -b myboard -t mytest -p pcbd

would run the pre_test, pre_check, build and deploy phases of the test ‘mytest’.

This is useful during development of a test (ie. for testing tests). Use caution running later phases of a test without their normal precursors (e.g. specifying to execute the run phase, without also specify to execute the pre_test, build or deploy phases). This can lead to undefined behavior.


It is almost always desirable to run the pre_test phase (‘-p’), so use caution omitting that phase from your list. In general, using phase selection is quite tricky, and unless you know what each phase does (and its side effects), it may lead to unexpected results.

Results and Result code

When a test is run, log files and results are placed in a log directory. The log directory is based on a set of attributes for the test run, underneath the fuego-rw/logs directory. The pattern for the directory name is:


So a full log directory name might look like this:


For more details about the log files that are produced during a test run, see Log files.

In addition to populating the Fuego log directory, if Jenkins is being used and a corresponding Jenkins job is defined for the test, then the Jenkins interface will be populated with information for that test run (referred to as a “build” in Jenkins). A visualization of the test results (for example a chart or a table), may be prepared for display in the Jenkins interface.

Also, although a test may have multiple individual test cases that it executes, the overall status of the test is reported via the return code from ftc run-test. This will be 0 for success, and something else for test failure. Usually, a non-zero result will be the value that was returned by the main test program that was run on the board.