benchbuild.utils package

Module handler that makes sure the modules for our commands are build similar to plumbum. The built modules are only active during a run of an experiment and get deleted afterwards.

benchbuild.utils.cmd

Module-hack, adapted from plumbum.

Submodules

benchbuild.utils.actions module

This defines classes that can be used to implement a series of Actions.

class benchbuild.utils.actions.Any(actions)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Just run all actions, no questions asked.'
NAME = 'ANY'
class benchbuild.utils.actions.Build(project)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Build the project'
NAME = 'BUILD'
class benchbuild.utils.actions.Clean(project_or_experiment, action_fn=None, check_empty=False)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Cleans the build directory'
NAME = 'CLEAN'
class benchbuild.utils.actions.CleanExtra(project_or_experiment, action_fn=None)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Cleans the extra directories.'
NAME = 'CLEAN EXTRA'
class benchbuild.utils.actions.Configure(project)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Configure project source files'
NAME = 'CONFIGURE'
class benchbuild.utils.actions.Download(project)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Download project source files'
NAME = 'DOWNLOAD'
class benchbuild.utils.actions.Echo(message)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Print a message.'
NAME = 'ECHO'
class benchbuild.utils.actions.Experiment(experiment, actions)[source]

Bases: benchbuild.utils.actions.Any

DESCRIPTION = 'Run a experiment, wrapped in a db transaction'
NAME = 'EXPERIMENT'
begin_transaction()[source]
end_transaction(experiment, session)[source]
class benchbuild.utils.actions.MakeBuildDir(project_or_experiment, action_fn=None)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Create the build directory'
NAME = 'MKDIR'
class benchbuild.utils.actions.Prepare(project)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Prepare project build folder'
NAME = 'PREPARE'
class benchbuild.utils.actions.RequireAll(actions)[source]

Bases: benchbuild.utils.actions.Step

class benchbuild.utils.actions.RetrieveFile(project_or_experiment, filename, run_group)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Retrieve a file from the database'
NAME = 'RETRIEVEFILE'
class benchbuild.utils.actions.Run(project)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Execute the run action'
NAME = 'RUN'
class benchbuild.utils.actions.SaveProfile(project_or_experiment, filename)[source]

Bases: benchbuild.utils.actions.Step

DESCRIPTION = 'Save a profile in llvm format in the DB'
NAME = 'SAVEPROFILE'
class benchbuild.utils.actions.Step(project_or_experiment, action_fn=None)[source]

Bases: object

DESCRIPTION = None
NAME = None
ON_STEP_BEGIN = []
ON_STEP_END = []
onerror()[source]
status
class benchbuild.utils.actions.StepClass[source]

Bases: abc.ABCMeta

class benchbuild.utils.actions.StepResult[source]

Bases: enum.IntEnum

An enumeration.

CAN_CONTINUE = 2
ERROR = 3
OK = 1
UNSET = 0
benchbuild.utils.actions.log_before_after(name: str, desc: str)[source]
benchbuild.utils.actions.notify_step_begin_end(f)[source]
benchbuild.utils.actions.prepend_status(f)[source]
benchbuild.utils.actions.to_step_result(f)[source]

benchbuild.utils.bootstrap module

Helper functions for bootstrapping external dependencies.

benchbuild.utils.bootstrap.check_uchroot_config()[source]
benchbuild.utils.bootstrap.find_package(binary)[source]
benchbuild.utils.bootstrap.install_package(pkg_name)[source]
benchbuild.utils.bootstrap.install_uchroot()[source]
benchbuild.utils.bootstrap.linux_distribution_major()[source]
benchbuild.utils.bootstrap.provide_package(pkg_name)[source]
benchbuild.utils.bootstrap.provide_packages(pkg_names)[source]

benchbuild.utils.compiler module

Helper functions for dealing with compiler replacement.

This provides a few key functions to deal with varying/measuring the compilers used inside the benchbuild study. From a high-level view, there are 2 interesting functions:

  • lt_clang(cflags, ldflags, func)
  • lt_clang_cxx(cflags, ldflags, func)

These generate a wrapped clang/clang++ in the current working directory and hide the given cflags/ldflags from the calling build system. Both will give you a working plumbum command and call a python script that redirects to the real clang/clang++ given the additional cflags&ldflags.

The wrapper-script generated for both functions can be found inside:
  • wrap_cc()
The remaining methods:
  • llvm()
  • llvm_libs()
  • clang()
  • clang_cxx()

Are just convencience methods that can be used when interacting with the configured llvm/clang source directories.

class benchbuild.utils.compiler.ExperimentCommand(cmd, args, exp_args)[source]

Bases: plumbum.commands.base.BoundCommand

args
cmd
benchbuild.utils.compiler.clang()[source]

Get a usable clang plumbum command.

This searches for a usable clang in the llvm binary path (See llvm()) and returns a plumbum command to call it.

Returns:
plumbum Command that executes clang++
benchbuild.utils.compiler.clang_cxx()[source]

Get a usable clang++ plumbum command.

This searches for a usable clang++ in the llvm binary path (See llvm()) and returns a plumbum command to call it.

Returns:
plumbum Command that executes clang++
benchbuild.utils.compiler.llvm()[source]

Get the path where all llvm binaries can be found.

Environment variable:
BB_LLVM_DIR
Returns:
LLVM binary path.
benchbuild.utils.compiler.llvm_libs()[source]

Get the path where all llvm libraries can be found.

Environment variable:
BB_LLVM_DIR
Returns:
LLVM library path.
benchbuild.utils.compiler.lt_clang(cflags, ldflags, func=None)[source]

Return a clang that hides CFLAGS and LDFLAGS.

This will generate a wrapper script in the current directory and return a complete plumbum command to it.

Args:

cflags: The CFLAGS we want to hide. ldflags: The LDFLAGS we want to hide. func (optional): A function that will be pickled alongside the compiler.

It will be called before the actual compilation took place. This way you can intercept the compilation process with arbitrary python code.
Returns (benchbuild.utils.cmd):
Path to the new clang command.
benchbuild.utils.compiler.lt_clang_cxx(cflags, ldflags, func=None)[source]

Return a clang++ that hides CFLAGS and LDFLAGS.

This will generate a wrapper script in the current directory and return a complete plumbum command to it.

Args:

cflags: The CFLAGS we want to hide. ldflags: The LDFLAGS we want to hide. func (optional): A function that will be pickled alongside the compiler.

It will be called before the actual compilation took place. This way you can intercept the compilation process with arbitrary python code.
Returns (benchbuild.utils.cmd):
Path to the new clang command.
benchbuild.utils.compiler.wrap_cc_in_uchroot(cflags, ldflags, func=None, cc_name='clang')[source]

Generate a clang wrapper that may be called from within a uchroot.

This basically does the same as lt_clang/lt_clang_cxx. However, we do not create a valid plumbum command. The generated script will only work inside a uchroot environment that has is root at the current working directory, when calling this function.

Args:

cflags: The CFLAGS we want to hide ldflags: The LDFLAGS we want to hide func (optional): A function that will be pickled alongside the compiler.

It will be called before the actual compilation took place. This way you can intercept the compilation process with arbitrary python code.

uchroot_path: Prefix path of the compiler inside the uchroot. cc_name: Name of the generated script.

benchbuild.utils.compiler.wrap_cxx_in_uchroot(cflags, ldflags, func=None)[source]

Delegate to wrap_cc_in_uchroot).

benchbuild.utils.container module

Container utilites.

class benchbuild.utils.container.Container[source]

Bases: object

filename
local

Finds the current location of a container. Also unpacks the project if necessary.

Returns:
target: The path, where the container lies in the end.
name = 'container'
remote
class benchbuild.utils.container.Gentoo[source]

Bases: benchbuild.utils.container.Container

latest_src_uri(*args, **kwargs)[source]
name = 'gentoo'
remote

Get a remote URL of the requested container.

class benchbuild.utils.container.Ubuntu[source]

Bases: benchbuild.utils.container.Container

name = 'ubuntu'
remote

Get a remote URL of the requested container.

benchbuild.utils.container.cached(func)[source]
benchbuild.utils.container.is_valid_container(container, path)[source]

Checks if a container exists and is unpacked.

Args:
path: The location where the container is expected.
Returns:
True if the container is valid, False if the container needs to unpacked or if the path does not exist yet.
benchbuild.utils.container.unpack_container(container, path)[source]

Unpack a container usable by uchroot.

Method that checks if a directory for the container exists, checks if erlent support is needed and then unpacks the container accordingly.

Args:
path: The location where the container is, that needs to be unpacked.

benchbuild.utils.db module

Database support module for the benchbuild study.

benchbuild.utils.db.create_run(cmd, project, exp, grp)[source]

Create a new ‘run’ in the database.

This creates a new transaction in the database and creates a new run in this transaction. Afterwards we return both the transaction as well as the run itself. The user is responsible for committing it when the time comes.

Args:
cmd: The command that has been executed. prj: The project this run belongs to. exp: The experiment this run belongs to. grp: The run_group (uuid) we blong to.
Returns:
The inserted tuple representing the run and the session opened with the new run. Don’t forget to commit it at some point.
benchbuild.utils.db.create_run_group(prj)[source]

Create a new ‘run_group’ in the database.

This creates a new transaction in the database and creates a new run_group within this transaction. Afterwards we return both the transaction as well as the run_group itself. The user is responsible for committing it when the time comes.

Args:
prj - The project for which we open the run_group.
Returns:
A tuple (group, session) containing both the newly created run_group and the transaction object.
benchbuild.utils.db.extract_file(filename, outfile, exp_id, run_group)[source]

Extract a previously stored file from the database.

Args:
filename (str):
The name of the file associated to the content in the database.
outfile (str):
The filepath we want to store the content to.
exp_id (uuid):
The experiment uuid the file was stored in.
run_group (uuid):
The run_group the file was stored in.
benchbuild.utils.db.persist_compilestats(run, session, stats)[source]

Persist the run results in the database.

Args:
run: The run we attach the compilestats to. session: The db transaction we belong to. stats: The stats we want to store in the database.
benchbuild.utils.db.persist_config(run, session, cfg)[source]

Persist the configuration in as key-value pairs.

Args:
run: The run we attach the config to. session: The db transaction we belong to. cfg: The configuration we want to persist.
benchbuild.utils.db.persist_experiment(experiment)[source]

Persist this experiment in the benchbuild database.

Args:
experiment: The experiment we want to persist.
benchbuild.utils.db.persist_file(f, experiment_id, run_group)[source]

Persist a file in the FileContent relation.

Args:
f (str):
The filename we want to persist.
experiment_id (uuid):
The experiment uuid this file needs to be assigned to.
run_group (uuid):
The run group uuid this file needs to be assigned to.
benchbuild.utils.db.persist_likwid(run, session, measurements)[source]

Persist all likwid results.

Args:
run: The run we attach our measurements to. session: The db transaction we belong to. measurements: The likwid measurements we want to store.
benchbuild.utils.db.persist_perf(run, session, svg_path)[source]

Persist the flamegraph in the database.

The flamegraph exists as a SVG image on disk until we persist it in the database.

Args:
run: The run we attach these perf measurements to. session: The db transaction we belong to. svg_path: The path to the SVG file we want to store.
benchbuild.utils.db.persist_project(project)[source]

Persist this project in the benchbuild database.

Args:
project: The project we want to persist.
benchbuild.utils.db.persist_time(run, session, *args, **kwargs)[source]
benchbuild.utils.db.validate(func)[source]

benchbuild.utils.downloader module

Downloading helper functions for benchbuild.

The helpers defined in this module provide access to some common Downloading methods for the source code of benchbuild projects. All downloads will be cached in BB_TMP_DIR and locked down with a hash that is generated after the first download. If the hash matches the file/folder found in BB_TMP_DIR, nothing will be downloaded at all.

Supported methods:
Copy, CopyNoFail, Wget, Git, Svn, Rsync
benchbuild.utils.downloader.Copy(From, To)[source]

Small copy wrapper.

Args:
From (str): Path to the SOURCE. To (str): Path to the TARGET.
benchbuild.utils.downloader.CopyNoFail(src, root=None)[source]

Just copy fName into the current working directory, if it exists.

No action is executed, if fName does not exist. No Hash is checked.

Args:

src: The filename we want to copy to ‘.’. root: The optional source dir we should pull fName from. Defaults

to benchbuild.settings.CFG[“tmpdir”].
Returns:
True, if we copied something.
benchbuild.utils.downloader.Git(src_url, tgt_name, tgt_root=None)[source]

Get a shallow clone of the given repo

Args:

src_url (str): Git URL of the SOURCE repo. tgt_name (str): Name of the repo folder on disk. tgt_root (str): TARGET folder for the git repo.

Defaults to CFG["tmpdir"]
benchbuild.utils.downloader.Rsync(url, tgt_name, tgt_root=None)[source]

RSync a folder.

Args:

url (str): The url of the SOURCE location. fname (str): The name of the TARGET. to (str): Path of the target location.

Defaults to CFG["tmpdir"].
benchbuild.utils.downloader.Svn(url, fname, to=None)[source]

Checkout the SVN repo.

Args:

url (str): The SVN SOURCE repo. fname (str): The name of the repo on disk. to (str): The name of the TARGET folder on disk.

Defaults to CFG["tmpdir"]
benchbuild.utils.downloader.Wget(src_url, tgt_name, tgt_root=None)[source]

Download url, if required.

Args:

src_url (str): Our SOURCE url. tgt_name (str): The filename we want to have on disk. tgt_root (str): The TARGET directory for the download.

Defaults to CFG["tmpdir"].
benchbuild.utils.downloader.get_hash_of_dirs(directory)[source]

Recursively hash the contents of the given directory.

Args:
directory (str): The root directory we want to hash.
Returns:
A hash of all the contents in the directory.
benchbuild.utils.downloader.source_required(src_file, src_root)[source]

Check, if a download is required.

Args:
src_file: The filename to check for. src_root: The path we find the file in.
Returns:
True, if we need to download something, False otherwise.
benchbuild.utils.downloader.update_hash(src, root)[source]

Update the hash for the given file.

Args:
src: The file name. root: The path of the given file.

benchbuild.utils.log module

benchbuild.utils.log.configure()[source]

Load logging configuration from our own defaults.

benchbuild.utils.log.set_defaults()[source]

Configure the loggers default settings.

benchbuild.utils.path module

Path utilities for benchbuild.

benchbuild.utils.path.determine_path()[source]

Borrowed from wxglade.py

benchbuild.utils.path.list_to_path(pathlist)[source]

Convert a list of path elements to a path string.

benchbuild.utils.path.mkdir_uchroot(dirpath, root='.')[source]

Create a file inside a uchroot env.

You will want to use this when you need to create a file with apropriate rights inside a uchroot container with subuid/subgid handling enabled.

Args:
dirpath:
The dirpath that should be created. Absolute inside the uchroot container.
root:
The root PATH of the container filesystem as seen outside of the container.
benchbuild.utils.path.mkfile_uchroot(filepath, root='.')[source]

Create a file inside a uchroot env.

You will want to use this when you need to create a file with apropriate rights inside a uchroot container with subuid/subgid handling enabled.

Args:
filepath:
The filepath that should be created. Absolute inside the uchroot container.
root:
The root PATH of the container filesystem as seen outside of the container.
benchbuild.utils.path.path_to_list(pathstr)[source]

Conver a path string to a list of path elements.

benchbuild.utils.path.template_files(path, exts=[])[source]

Return a list of filenames found at @path.

The list of filenames can be filtered by extensions.

Arguments:
path: Existing filepath we want to list. exts: List of extensions to filter by.
Returns:
A list of filenames found in the path.
benchbuild.utils.path.template_path(template)[source]

Return path to template file.

benchbuild.utils.path.template_str(template)[source]

Read a template file from the resources and return it as str.

benchbuild.utils.run module

Experiment helpers.

class benchbuild.utils.run.RunInfo(**kwargs)[source]

Bases: object

commit()[source]
has_failed

Check, whether this run failed.

class benchbuild.utils.run.UchrootEC[source]

Bases: enum.Enum

An enumeration.

MNT_DEV_FAILED = 253
MNT_FAILED = 255
MNT_PROC_FAILED = 254
MNT_PTS_FAILED = 251
MNT_SYS_FAILED = 252
exception benchbuild.utils.run.UnmountError[source]

Bases: BaseException

benchbuild.utils.run.begin_run_group(project)[source]

Begin a run_group in the database.

A run_group groups a set of runs for a given project. This models a series of runs that form a complete binary runtime test.

Args:
project: The project we begin a new run_group for.
Returns:
(group, session) where group is the created group in the database and session is the database session this group lives in.
benchbuild.utils.run.end_run_group(group, session)[source]

End the run_group successfully.

Args:
group: The run_group we want to complete. session: The database transaction we will finish.
benchbuild.utils.run.exit_code_from_run_infos(run_infos: typing.List[benchbuild.utils.run.RunInfo])[source]
benchbuild.utils.run.fail_run_group(group, session)[source]

End the run_group unsuccessfully.

Args:
group: The run_group we want to complete. session: The database transaction we will finish.
benchbuild.utils.run.fetch_time_output(marker, format_s, ins)[source]

Fetch the output /usr/bin/time from a.

Args:
marker: The marker that limits the time output format_s: The format string used to parse the timings ins: A list of lines we look for the output.
Returns:
A list of timing tuples
benchbuild.utils.run.in_builddir(sub='.')[source]

Decorate a project phase with a local working directory change.

Args:
sub: An optional subdirectory to change into.
benchbuild.utils.run.retry(pb_cmd, retries=0, max_retries=10, retcode=0, retry_retcodes=None)[source]
benchbuild.utils.run.run(command, retcode=0)[source]

Execute a plumbum command, depending on the user’s settings.

Args: command & TEE(retcode=retcode)

command: The plumbumb command to execute.
benchbuild.utils.run.store_config(func)[source]

Decorator for storing the configuration in the project’s builddir.

benchbuild.utils.run.track_execution(cmd, project, experiment, **kwargs)[source]

Guard the execution of the given command.

Args:
cmd: the command we guard. pname: the database we run under. ename: the database session this run belongs to. run_group: the run group this execution will belong to.
Raises:
RunException: If the cmd encounters an error we wrap the exception
in a RunException and re-raise. This ends the run unsuccessfully.
benchbuild.utils.run.uchroot(*args, **kwargs)[source]

Return a customizable uchroot command.

Args:
args: List of additional arguments for uchroot (typical: mounts)
Return:
chroot_cmd
benchbuild.utils.run.uchroot_env(mounts)[source]

Compute the environment of the change root for the user.

Args:
mounts: The mountpoints of the current user.
Return:
paths ld_libs
benchbuild.utils.run.uchroot_mounts(prefix, mounts)[source]

Compute the mountpoints of the current user.

Args:
prefix: Define where the job was running if it ran on a cluster. mounts: All mounts the user currently uses in his file system.
Return:
mntpoints
benchbuild.utils.run.uchroot_no_args()[source]

Return the uchroot command without any customizations.

benchbuild.utils.run.uchroot_no_llvm(*args, **kwargs)[source]

Return a customizable uchroot command.

The command will be executed inside a uchroot environment.

Args:
args: List of additional arguments for uchroot (typical: mounts)
Return:
chroot_cmd
benchbuild.utils.run.uchroot_with_mounts(*args, **kwargs)[source]

Return a uchroot command with all mounts enabled.

benchbuild.utils.run.unionfs(base_dir='./base', image_dir='./image', image_prefix=None, mountpoint='./union')[source]

Decorator for the UnionFS feature.

This configures a unionfs for projects. The given base_dir and/or image_dir are layered as follows:

image_dir=RW:base_dir=RO

All writes go to the image_dir, while base_dir delivers the (read-only) versions of the rest of the filesystem.

The unified version will be provided in the project’s builddir. Unmouting is done as soon as the function completes.

Args:
base_dir:The unpacked container of a project delievered by a method
out of the container utils.
image_dir: Virtual image of the actual file system represented by the
build_dir of a project.
image_prefix: Useful prefix if the projects run on a cluster,
to identify where the job came from and where it runs.
mountpoint: Location where the filesystems merge, currently per default
as ‘./union’.
benchbuild.utils.run.unionfs_is_active(root)[source]
benchbuild.utils.run.unionfs_set_up(ro_base, rw_image, mountpoint)[source]

Setup a unionfs via unionfs-fuse.

Args:
ro_base: base_directory of the project rw_image: virtual image of actual file system mountpoint: location where ro_base and rw_image merge
benchbuild.utils.run.uretry(cmd, retcode=0)[source]
benchbuild.utils.run.with_env_recursive(cmd, **envvars)[source]

Recursively updates the environment of cmd and all its subcommands.

Args:
cmd - A plumbum command-like object **envvars - The environment variables to update
Returns:
The updated command.

benchbuild.utils.schema module

Database schema for benchbuild

The schema should initialize itself on an empty database. For now, we do not support automatic upgrades on schema changes. You might encounter some roadbumps when using an older version of benchbuild.

Furthermore, for now, we are restricted to postgresql databases, although we already support arbitrary connection strings via config.

If you want to use reports that use one of our SQL functions, you need to initialize the functions first using the following command:

> BB_DB_CREATE_FUNCTIONS=true benchbuild run -E empty -l

After that you (normally) do not need to do this agains, unless we supply a new version that you are interested in. As soon as we have alembic running, we can provide automatic up/downgrade paths for you.

class benchbuild.utils.schema.CompileStat(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store compilestats as given by LLVM’s ‘-stats’ commoand.

component
id
name
run_id
value
class benchbuild.utils.schema.Config(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store customized information about a run.

You can store arbitrary configuration information about a run here. Use it for extended filtering against the run table.

name
run_id
value
class benchbuild.utils.schema.Event(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store PAPI profiling based events.

duration
id
name
run_id
start
tid
type
class benchbuild.utils.schema.Experiment(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store metadata about experiments.

begin
description
end
id
name
class benchbuild.utils.schema.FileContent(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store content of a file for being able to retrieve it later

content
experience_id
filename
rungroup_id
class benchbuild.utils.schema.GUID(*args, as_uuid=False, **kwargs)[source]

Bases: sqlalchemy.sql.type_api.TypeDecorator

Platform-independent GUID type.

Uses Postgresql’s UUID type, otherwise uses CHAR(32), storing as stringified hex values.

as_uuid = False
impl

alias of CHAR

load_dialect_impl(dialect)[source]
process_bind_param(value, dialect)[source]
process_result_value(value, dialect)[source]
class benchbuild.utils.schema.GlobalConfig(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store customized information about a run.

You can store arbitrary configuration information about a run here. Use it for extended filtering against the run table.

experiment_group
name
value
class benchbuild.utils.schema.IslAst(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store default metrics, simple name value store.

ast
function
run_id
class benchbuild.utils.schema.Likwid(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store measurement results of likwid based experiments.

core
metric
region
run_id
value
class benchbuild.utils.schema.Metadata(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store metadata information for every run.

If you happen to have some free-form data that belongs to the database, this is the place for it.

name
run_id
value
class benchbuild.utils.schema.Metric(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store default metrics, simple name value store.

name
run_id
value
class benchbuild.utils.schema.Project(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store project metadata.

description
domain
group_name
name
src_url
version
class benchbuild.utils.schema.Regions(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store region metadata generated by libpjit.

duration
events
id
name
run_id
class benchbuild.utils.schema.RegressionTest(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store regression tests for all projects.

This relation is filled from the PolyJIT side, not from benchbuild-study. We collect all JIT SCoPs found by PolyJIT in this relation for regression-test generation.

module
name
project_name
run_id
class benchbuild.utils.schema.Run(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store a run for each executed test binary.

begin
command
end
experiment_group
experiment_name
id
project_group
project_name
run_group
status
class benchbuild.utils.schema.RunGroup(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store information about a run group.

begin
end
experiment
id
status
class benchbuild.utils.schema.RunLog(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store log information for every run.

Properties like, start time, finish time, exit code, stderr, stdout are stored here.

begin
config
end
run_id
status
stderr
stdout
class benchbuild.utils.schema.Schedule(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store default metrics, simple name value store.

function
run_id
schedule
class benchbuild.utils.schema.ScopDetection(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store results of polli-profile-scops

count
invalid_reason
run_id
class benchbuild.utils.schema.Sequence(**kwargs)[source]

Bases: sqlalchemy.ext.declarative.api.Base

Store a the fittest sequence of an opt command and its fitness value.

name
run_id
value
benchbuild.utils.schema.Session()
class benchbuild.utils.schema.SessionManager[source]

Bases: object

get()[source]
benchbuild.utils.schema.init_functions(connection)[source]

Initialize all SQL functions in the database.

benchbuild.utils.slurm module

SLURM support for the benchbuild study.

This module can be used to generate bash scripts that can be executed by the SLURM controller either as batch or interactive script.

benchbuild.utils.slurm.dump_slurm_script(script_name, benchbuild, experiment, projects)[source]

Dump a bash script that can be given to SLURM.

Args:

script_name (str): name of the bash script. commands (list(benchbuild.utils.cmd)):

List of plumbum commands to write to the bash script.
**kwargs: Dictionary with all environment variable bindings we should
map in the bash script.
benchbuild.utils.slurm.prepare_directories(dirs)[source]

Make sure that the required directories exist.

Args:
dirs - the directories we want.
benchbuild.utils.slurm.prepare_slurm_script(experiment, projects)[source]

Prepare a slurm script that executes the experiment for a given project.

Args:
experiment: The experiment we want to execute projects: All projects we generate an array job for.

benchbuild.utils.user_interface module

User interface helpers for benchbuild.

benchbuild.utils.user_interface.ask(question, default_answer=False, default_answer_str='no')[source]
benchbuild.utils.user_interface.query_yes_no(question, default='yes')[source]

Ask a yes/no question via raw_input() and return their answer.

Args:

question (str): Question hat is presented to the user. default (str): The presumed answer, if the user just hits <Enter>.

It must be “yes” (the default), “no” or None (meaning an answer is required of the user).
Returns (boolean):
True, if ‘yes’, False otherwise.

benchbuild.utils.versions module

Gather version information for BB.

benchbuild.utils.versions.get_git_hash(from_url)[source]

Get the git commit hash of HEAD from :from_url.

Args:
from_url: The file system url of our git repository.
Returns:
git commit hash of HEAD, or empty string.
benchbuild.utils.versions.get_version_from_cache_dir(src_file)[source]

Creates a version for a project out of the hash.

The hash is taken from the directory of the source file.

Args:
src_file: The source file of the project using this function.
Returns:
Either returns the first 8 digits of the hash as string, the entire hash as a string if the hash consists out of less than 7 digits or None if the path is incorrect.

benchbuild.utils.wrapping module

Wrapper utilities for benchbuild.

This module provides methods to wrap binaries with extensions that are pickled alongside the original binary. In place of the original binary a new python module is generated that loads the pickle and redirects the program call with all its arguments to it. This allows interception of arbitrary programs for experimentation.

Examples:
TODO
Compiler Wrappers:
The compiler wrappers substitute the compiler call with a script that produces the expected output from the original compiler call first. Afterwards the pickle is loaded and the original call is forwarded to the pickle. This way the user is not obligated to produce valid output during his own experiment.
Runtime Wrappers:
These directly forward the binary call to the pickle without any execution of the binary. We cannot guarantee that repeated execution is valid, therefore, we let the user decide what the program should do.
benchbuild.utils.wrapping.strip_path_prefix(ipath, prefix)[source]

Strip prefix from path.

Args:
ipath: input path prefix: the prefix to remove, if it is found in :ipath:
Examples:
>>> strip_path_prefix("/foo/bar", "/bar")
'/foo/bar'
>>> strip_path_prefix("/foo/bar", "/")
'foo/bar'
>>> strip_path_prefix("/foo/bar", "/foo")
'/bar'
>>> strip_path_prefix("/foo/bar", "None")
'/foo/bar'
benchbuild.utils.wrapping.unpickle(pickle_file)[source]

Unpickle a python object from the given path.

benchbuild.utils.wrapping.wrap(name, runner, sprefix=None, python='/home/docs/checkouts/readthedocs.org/user_builds/pprof-study/envs/v2.0.3/bin/python')[source]

Wrap the binary :name: with the function :runner:.

This module generates a python tool that replaces :name: The function in runner only accepts the replaced binaries name as argument. We use the cloudpickle package to perform the serialization, make sure :runner: can be serialized with it and you’re fine.

Args:
name: Binary we want to wrap runner: Function that should run instead of :name:
Returns:
A plumbum command, ready to launch.
benchbuild.utils.wrapping.wrap_cc(filepath, cflags, ldflags, compiler, extension, compiler_ext_name=None, python='/home/docs/checkouts/readthedocs.org/user_builds/pprof-study/envs/v2.0.3/bin/python')[source]

Substitute a compiler with a script that hides CFLAGS & LDFLAGS.

This will generate a wrapper script in the current directory and return a complete plumbum command to it.

Args:

filepath (str): Path to the wrapper script. cflags (list(str)): The CFLAGS we want to hide. ldflags (list(str)): The LDFLAGS we want to hide. compiler (benchbuild.utils.cmd): Real compiler command we should call in the

script.
extension: A function that will be pickled alongside the compiler.
It will be called before the actual compilation took place. This way you can intercept the compilation process with arbitrary python code.
compiler_ext_name: The name that we should give to the generated
dill blob for :func:
Returns (benchbuild.utils.cmd):
Command of the new compiler we can call.
benchbuild.utils.wrapping.wrap_dynamic(self, name, runner, sprefix=None, python='/home/docs/checkouts/readthedocs.org/user_builds/pprof-study/envs/v2.0.3/bin/python', name_filters=None)[source]

Wrap the binary :name with the function :runner.

This module generates a python tool :name: that can replace a yet unspecified binary. It behaves similar to the :wrap: function. However, the first argument is the actual binary name.

Args:

name: name of the python module runner: Function that should run the real binary sprefix: Prefix that should be used for commands. python: The python executable that should be used. name_filters:

List of regex expressions that are used to filter the real project name. Make sure to include a match group named ‘name’ in the regex, e.g., [

r’foo(?P<name>.)-flt’

]

Returns: plumbum command, readty to launch.

benchbuild.utils.wrapping.wrap_dynamic_in_uchroot(self, name, runner, sprefix=None)[source]
benchbuild.utils.wrapping.wrap_in_uchroot(name, runner, sprefix=None)[source]