benchbuild.project module¶
Project
A project in benchbuild is an abstract representation of a software system that can live in various stages throughout an experiment. It defines two extension points for an experiment to attach on, the compile-time phase and the (optional) run-time phase.
An experiment can intercept the compilation phase of a project and perform any experiment that requires the source artifacts as input.
Furthermore, it is possible to intercept a project’s run-time pahse with a measurement.
The project definition ensures that all experiments run through the same series of commands in both phases and that all experiments run inside a separate build directory in isolation of one another.
-
class
benchbuild.project.
Project
(experiment, name=NOTHING, domain=NOTHING, group=NOTHING, src_file=NOTHING, container=NOTHING, version=NOTHING, builddir=NOTHING, testdir=NOTHING, cflags=NOTHING, ldflags=NOTHING, run_f=NOTHING, run_uuid=NOTHING, compiler_extension=NOTHING, runtime_extension=None)[source]¶ Bases:
object
Abstract class for benchbuild projects.
A project is an arbitrary software system usable by benchbuild in experiments. Subclasses of Project are registered automatically by benchbuild, if imported in the same interpreter session. For this to happen, you must list the in the settings under plugins -> projects.
- A project implementation must provide the following method:
- compile: Downloads & Compiles the source.
- A project implementation may provide the following functions:
- run_tests: Wrap any binary that has to be run under the
- runtime_extension wrapper and execute an implementation defined set of run-time tests. Defaults to a call of a binary with the name run_f in the build directory without arguments.
- clean: Clean the project’s build directory. Defaults to
- recursive ‘rm’ on the build directory and can be disabled
by setting the environment variable
BB_CLEAN=false
.
Raises: AttributeError
– Class definition raises an attribute error, if the implementation does not provide a value for the attributes NAME, DOMAIN, and GROUPTypeError
– Validation of properties may throw a TypeError.
-
experiment
¶ benchbuild.experiment.Experiment – The experiment this project is assigned to.
-
name
¶ str, optional – The name of this project. Defaults to NAME.
-
domain
¶ str, optional – The application domain of this project. Defaults to DOMAIN.
-
group
¶ str, optional – The group this project belongs to. Defaults to GROUP.
-
src_file
¶ str, optional – A main src_file this project is assigned to. Defaults to SRC_FILE
-
container
¶ benchbuild.utils.container.Container, optional – A uchroot compatible container that we can use for this project. Defaults to benchbuild.utils.container.Gentoo.
-
version
¶ str, optional – A version information for this project. Defaults to VERSION.
-
builddir
¶ str, optional – The build directory for this project. Auto generated, if not set.
-
testdir
¶ str, optional – The location of any additional test-files for this project, usually stored out of tree. Auto generated, if not set. Usually a project implementation will define this itself.
-
cflags
¶ list
ofstr
, optional – A list of cflags used, for compilation of this project.
-
ldflags
¶ list
ofstr
, optional – A list of ldflags used, for compilation of this project.
-
run_f
¶ str, optional – A filename that points to the binary we want to track. Usually a project implementation will define this itself.
-
run_uuid
¶ uuid.UUID, optional – An UUID that identifies all binaries executed by a single run of this project. In the database schema this is named the ‘run_group’.
-
compiler_extension
¶ Callable[str, iterable[str], RunInfo], optional – A composable extension that will be used in place of the real compiler. Defaults to running the compiler with a timeout command wrapped around it.
-
runtime_extension
¶ Callable[str, iterable[str], RunInfo], optional – A composable extension that will be used in place of any binary this project wants to execute. Which binaries to replace is defined by the implementation using benchbuild.utils.wrapping.wrap. Defaults to None.
-
CONTAINER
= None¶
-
DOMAIN
= None¶
-
GROUP
= None¶
-
NAME
= None¶
-
SRC_FILE
= None¶
-
VERSION
= None¶
-
id
¶
-
class
benchbuild.project.
ProjectDecorator
(name, bases, attrs)[source]¶ Bases:
benchbuild.project.ProjectRegistry
Decorate the interface of a project with the in_builddir decorator.
This is just a small safety net for benchbuild users, because we make sure to run every project method in the project’s build directory.
-
decorated_methods
= ['redirect', 'compile', 'run_tests']¶
-
-
class
benchbuild.project.
ProjectRegistry
(name, bases, attrs)[source]¶ Bases:
type
Registry for benchbuild projects.
-
projects
= Trie((('ffmpeg/benchbuild', <class 'benchbuild.projects.benchbuild.ffmpeg.LibAV'>), ('syr2k/polybench', <class 'benchbuild.projects.polybench.polybench.Syr2k'>), ('syr2k/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Syr2k'>), ('srad-1/rodinia', <class 'benchbuild.projects.apollo.rodinia.SRAD1'>), ('ludcmp/polybench', <class 'benchbuild.projects.polybench.polybench.LuDCMP'>), ('ludcmp/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.LuDCMP'>), ('cfd/rodinia', <class 'benchbuild.projects.apollo.rodinia.CFD'>), ('mcrypt/benchbuild', <class 'benchbuild.projects.benchbuild.mcrypt.MCrypt'>), ('durbin/polybench', <class 'benchbuild.projects.polybench.polybench.Durbin'>), ('durbin/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Durbin'>), ('lulesh-omp/benchbuild', <class 'benchbuild.projects.benchbuild.lulesh.LuleshOMP'>), ('lammps/gentoo', <class 'benchbuild.projects.gentoo.lammps.Lammps'>), ('lammps/benchbuild', <class 'benchbuild.projects.benchbuild.lammps.Lammps'>), ('correlation/polybench', <class 'benchbuild.projects.polybench.polybench.Correlation'>), ('correlation/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Correlation'>), ('kmeans/rodinia', <class 'benchbuild.projects.apollo.rodinia.KMeans'>), ('doitgen/polybench', <class 'benchbuild.projects.polybench.polybench.Doitgen'>), ('doitgen/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Doitgen'>), ('gesummv/polybench', <class 'benchbuild.projects.polybench.polybench.Gesummv'>), ('gesummv/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Gesummv'>), ('gzip/gentoo', <class 'benchbuild.projects.gentoo.gzip.GZip'>), ('gzip/benchbuild', <class 'benchbuild.projects.benchbuild.gzip.Gzip'>), ('lu/polybench', <class 'benchbuild.projects.polybench.polybench.Lu'>), ('lu/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Lu'>), ('jacobi-1d/polybench', <class 'benchbuild.projects.polybench.polybench.Jacobi1D'>), ('jacobi-1d/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Jacobi1D'>), ('bicg/polybench', <class 'benchbuild.projects.polybench.polybench.BicG'>), ('bicg/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.BicG'>), ('Rasdaman/benchbuild', <class 'benchbuild.projects.benchbuild.rasdaman.Rasdaman'>), ('tcc/benchbuild', <class 'benchbuild.projects.benchbuild.tcc.TCC'>), ('heartwall/rodinia', <class 'benchbuild.projects.apollo.rodinia.HeartWall'>), ('alignment/bots', <class 'benchbuild.projects.benchbuild.bots.Alignment'>), ('scimark/apollo', <class 'benchbuild.projects.apollo.scimark.SciMark'>), ('heat-3d/polybench', <class 'benchbuild.projects.polybench.polybench.Heat3D'>), ('heat-3d/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Heat3D'>), ('mvt/polybench', <class 'benchbuild.projects.polybench.polybench.Mvt'>), ('mvt/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Mvt'>), ('hotspot3D/rodinia', <class 'benchbuild.projects.apollo.rodinia.Hotspot3D'>), ('gramschmidt/polybench', <class 'benchbuild.projects.polybench.polybench.Gramschmidt'>), ('gramschmidt/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Gramschmidt'>), ('nussinov/polybench', <class 'benchbuild.projects.polybench.polybench.Nussinov'>), ('nussinov/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Nussinov'>), ('jacobi-2d/polybench', <class 'benchbuild.projects.polybench.polybench.Jacobi2Dimper'>), ('jacobi-2d/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Jacobi2Dimper'>), ('sdcc/benchbuild', <class 'benchbuild.projects.benchbuild.sdcc.SDCC'>), ('floorplan/bots', <class 'benchbuild.projects.benchbuild.bots.FloorPlan'>), ('srad-2/rodinia', <class 'benchbuild.projects.apollo.rodinia.SRAD2'>), ('SingleSourceBenchmarks/lnt', <class 'benchbuild.projects.lnt.lnt.SingleSourceBenchmarks'>), ('hotspot/rodinia', <class 'benchbuild.projects.apollo.rodinia.Hotspot'>), ('libressl/benchbuild', <class 'benchbuild.projects.benchbuild.openssl.LibreSSL'>), ('openblas/benchbuild', <class 'benchbuild.projects.benchbuild.lapack.OpenBlas'>), ('povray/benchbuild', <class 'benchbuild.projects.benchbuild.povray.Povray'>), ('MultiSourceBenchmarks/lnt', <class 'benchbuild.projects.lnt.lnt.MultiSourceBenchmarks'>), ('js/benchbuild', <class 'benchbuild.projects.benchbuild.js.SpiderMonkey'>), ('lud/rodinia', <class 'benchbuild.projects.apollo.rodinia.LUD'>), ('leveldb/benchbuild', <class 'benchbuild.projects.benchbuild.leveldb.LevelDB'>), ('minisat/benchbuild', <class 'benchbuild.projects.benchbuild.minisat.Minisat'>), ('crafty/gentoo', <class 'benchbuild.projects.gentoo.crafty.Crafty'>), ('crafty/benchbuild', <class 'benchbuild.projects.benchbuild.crafty.Crafty'>), ('covariance/polybench', <class 'benchbuild.projects.polybench.polybench.Covariance'>), ('covariance/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Covariance'>), ('p7zip/gentoo', <class 'benchbuild.projects.gentoo.sevenz.SevenZip'>), ('nw/rodinia', <class 'benchbuild.projects.apollo.rodinia.NW'>), ('sort/bots', <class 'benchbuild.projects.benchbuild.bots.Sort'>), ('x264/gentoo', <class 'benchbuild.projects.gentoo.x264.X264'>), ('x264/benchbuild', <class 'benchbuild.projects.benchbuild.x264.X264'>), ('symm/polybench', <class 'benchbuild.projects.polybench.polybench.Symm'>), ('symm/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Symm'>), ('health/bots', <class 'benchbuild.projects.benchbuild.bots.Health'>), ('deriche/polybench', <class 'benchbuild.projects.polybench.polybench.Deriche'>), ('deriche/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Deriche'>), ('knapsack/bots', <class 'benchbuild.projects.benchbuild.bots.Knapsack'>), ('lulesh/benchbuild', <class 'benchbuild.projects.benchbuild.lulesh.Lulesh'>), ('2mm/polybench', <class 'benchbuild.projects.polybench.polybench.TwoMM'>), ('2mm/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.TwoMM'>), ('floyd-warshall/polybench', <class 'benchbuild.projects.polybench.polybench.FloydWarshall'>), ('floyd-warshall/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.FloydWarshall'>), ('leukocyte/rodinia', <class 'benchbuild.projects.apollo.rodinia.Leukocyte'>), ('bfs/rodinia', <class 'benchbuild.projects.apollo.rodinia.BFS'>), ('python/benchbuild', <class 'benchbuild.projects.benchbuild.python.Python'>), ('atax/polybench', <class 'benchbuild.projects.polybench.polybench.Atax'>), ('atax/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Atax'>), ('sqlite3/benchbuild', <class 'benchbuild.projects.benchbuild.sqlite3.SQLite3'>), ('strassen/bots', <class 'benchbuild.projects.benchbuild.bots.Strassen'>), ('nn/rodinia', <class 'benchbuild.projects.apollo.rodinia.NN'>), ('lavaMD/rodinia', <class 'benchbuild.projects.apollo.rodinia.LavaMD'>), ('gemm/polybench', <class 'benchbuild.projects.polybench.polybench.Gemm'>), ('gemm/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Gemm'>), ('myocyte/rodinia', <class 'benchbuild.projects.apollo.rodinia.Myocyte'>), ('adi/polybench', <class 'benchbuild.projects.polybench.polybench.Adi'>), ('adi/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Adi'>), ('syrk/polybench', <class 'benchbuild.projects.polybench.polybench.Syrk'>), ('syrk/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Syrk'>), ('streamcluster/rodinia', <class 'benchbuild.projects.apollo.rodinia.StreamCluster'>), ('ruby/benchbuild', <class 'benchbuild.projects.benchbuild.ruby.Ruby'>), ('test/test', <class 'benchbuild.projects.test.test.TestProject'>), ('SPEC2006/lnt', <class 'benchbuild.projects.lnt.lnt.SPEC2006'>), ('fft/bots', <class 'benchbuild.projects.benchbuild.bots.FFT'>), ('linpack/benchbuild', <class 'benchbuild.projects.benchbuild.linpack.Linpack'>), ('3mm/polybench', <class 'benchbuild.projects.polybench.polybench.ThreeMM'>), ('3mm/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.ThreeMM'>), ('uts/bots', <class 'benchbuild.projects.benchbuild.bots.UTS'>), ('gemver/polybench', <class 'benchbuild.projects.polybench.polybench.Gemver'>), ('gemver/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Gemver'>), ('backprop/rodinia', <class 'benchbuild.projects.apollo.rodinia.Backprop'>), ('7z/benchbuild', <class 'benchbuild.projects.benchbuild.sevenz.SevenZip'>), ('pathfinder/rodinia', <class 'benchbuild.projects.apollo.rodinia.PathFinder'>), ('MultiSourceApplications/lnt', <class 'benchbuild.projects.lnt.lnt.MultiSourceApplications'>), ('fib/bots', <class 'benchbuild.projects.benchbuild.bots.Fib'>), ('Povray/lnt', <class 'benchbuild.projects.lnt.lnt.Povray'>), ('nqueens/bots', <class 'benchbuild.projects.benchbuild.bots.NQueens'>), ('b+tree/rodinia', <class 'benchbuild.projects.apollo.rodinia.BPlusTree'>), ('particlefilter/rodinia', <class 'benchbuild.projects.apollo.rodinia.ParticleFilter'>), ('info/gentoo', <class 'benchbuild.projects.gentoo.info.Info'>), ('seidel-2d/polybench', <class 'benchbuild.projects.polybench.polybench.Seidel2D'>), ('seidel-2d/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Seidel2D'>), ('trisolv/polybench', <class 'benchbuild.projects.polybench.polybench.Trisolv'>), ('trisolv/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Trisolv'>), ('bzip2/gentoo', <class 'benchbuild.projects.gentoo.bzip2.BZip2'>), ('bzip2/benchbuild', <class 'benchbuild.projects.benchbuild.bzip2.Bzip2'>), ('fdtd-2d/polybench', <class 'benchbuild.projects.polybench.polybench.FDTD2D'>), ('fdtd-2d/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.FDTD2D'>), ('crocopat/benchbuild', <class 'benchbuild.projects.benchbuild.crocopat.Crocopat'>), ('postgresql/gentoo', <class 'benchbuild.projects.gentoo.postgresql.Postgresql'>), ('xz/gentoo', <class 'benchbuild.projects.gentoo.xz.XZ'>), ('xz/benchbuild', <class 'benchbuild.projects.benchbuild.xz.XZ'>), ('lapack/benchbuild', <class 'benchbuild.projects.benchbuild.lapack.Lapack'>), ('trmm/polybench', <class 'benchbuild.projects.polybench.polybench.Trmm'>), ('trmm/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Trmm'>), ('cholesky/polybench', <class 'benchbuild.projects.polybench.polybench.Cholesky'>), ('cholesky/polybench-mod', <class 'benchbuild.projects.polybench.polybench-mod.Cholesky'>), ('test-fail/test', <class 'benchbuild.projects.test.test.TestProjectRuntimeFail'>), ('eix/gentoo', <class 'benchbuild.projects.gentoo.eix.Eix'>), ('ccrypt/benchbuild', <class 'benchbuild.projects.benchbuild.ccrypt.Ccrypt'>), ('sparselu/bots', <class 'benchbuild.projects.benchbuild.bots.SparseLU'>),))¶
-
-
benchbuild.project.
populate
(projects_to_filter=None, group=None)[source]¶ Populate the list of projects that belong to this experiment.
Parameters: - projects_to_filter (list(Project)) – List of projects we want to assign to this experiment. We intersect the list of projects with the list of supported projects to get the list of projects that belong to this experiment.
- group (list(str)) – In addition to the project filter, we provide a way to filter whole groups.