benchbuild.experiment module

Experiments

An experiment in benchbuild is a simple list of actions that need to be executed on every project that is part of the experiment. Every callable can serve as an action.

However, benchbuild provides many predefined actions that can be reused by implementations in the benchbuild.utils.actions module. Furthermore, if you do not need to control the default order of actions for a run-time or a compile-time experiment you can reuse the Experiment.default_runtime_actions or Experiment.default_compiletime_actions.

Besides the list of actions, it is also the responsibility of an experiment to configure each single project that should take part in an experiment. This includes setting appropriate CFLAGS, LDFLAGS and any additional metadata that has to be added to binary runs for later evaluation.

Example

```python class HelloExperiment(Experiment):

pass

```

class benchbuild.experiment.Configuration(project=None, config=None)

Bases: object

Build a set of experiment actions out of a list of configurations.

class benchbuild.experiment.Experiment(name=NOTHING, projects=NOTHING, id=NOTHING, schema=NOTHING)

Bases: object

A series of commands executed on a project that form an experiment.

The default implementation should provide a sane environment for all derivates.

One important task executed by the basic implementation is setting up the default set of projects that belong to this project. As every project gets registered in the ProjectFactory, the experiment gets a list of experiment names that work as a filter.

name

str – The name of the experiment, defaults to NAME

projects

list of benchbuild.project.Project – A list of projects that is assigned to this experiment.

id

str – A uuid encoded as str used to identify this instance of experiment. Equivalent to the experiment_group in the database scheme.

NAME = None

SCHEMA = None

actions()

actions_for_project(project)

Get the actions a project wants to run.

Parameters:project (benchbuild.Project) – the project we want to run.

static default_compiletime_actions(project)

Return a series of actions for a compile time experiment.

default_id()

static default_runtime_actions(project)

Return a series of actions for a run time experiment.

default_schema()

validate_id(_, new_id)

validate_schema(_, new_schema)

class benchbuild.experiment.ExperimentRegistry(name, bases, _dict)

Bases: type

Registry for benchbuild experiments.

experiments = {'empty': <class 'benchbuild.experiments.empty.Empty'>, 'no-measurement': <class 'benchbuild.experiments.empty.NoMeasurement'>, 'raw': <class 'benchbuild.experiments.raw.RawRuntime'>}