benchbuild.utils.download module

# Downloading helper functions for benchbuild.

The helpers defined in this module provide access to some common Downloading methods for the source code of benchbuild projects. All downloads will be cached in BB_TMP_DIR and locked down with a hash that is generated after the first download. If the hash matches the file/folder found in BB_TMP_DIR, nothing will be downloaded at all.

Supported methods:

Copy, CopyNoFail, Wget, Git, Svn, Rsync

benchbuild.utils.download.Copy(From, To)

Small copy wrapper.

Parameters:

• From (str) – Path to the SOURCE.
• To (str) – Path to the TARGET.

benchbuild.utils.download.CopyNoFail(src, root=None)

Just copy fName into the current working directory, if it exists.

No action is executed, if fName does not exist. No Hash is checked.

Parameters:

• src – The filename we want to copy to ‘.’.
• root – The optional source dir we should pull fName from. Defaults to benchbuild.settings.CFG[“tmpdir”].

Returns:

True, if we copied something.

benchbuild.utils.download.Git(repository, directory, rev=None, prefix=None, shallow_clone=True)

Get a clone of the given repo

Parameters:

• repository (str) – Git URL of the SOURCE repo.
• directory (str) – Name of the repo folder on disk.
• tgt_root (str) – TARGET folder for the git repo. Defaults to CFG["tmpdir"]
• shallow_clone (bool) – Only clone the repository shallow Defaults to true

benchbuild.utils.download.Rsync(url, tgt_name, tgt_root=None)

RSync a folder.

Parameters:

• url (str) – The url of the SOURCE location.
• fname (str) – The name of the TARGET.
• to (str) – Path of the target location. Defaults to CFG["tmpdir"].

benchbuild.utils.download.Svn(url, fname, to=None)

Checkout the SVN repo.

Parameters:

• url (str) – The SVN SOURCE repo.
• fname (str) – The name of the repo on disk.
• to (str) – The name of the TARGET folder on disk. Defaults to CFG["tmpdir"]

benchbuild.utils.download.Wget(src_url, tgt_name, tgt_root=None)

Download url, if required.

Parameters:

• src_url (str) – Our SOURCE url.
• tgt_name (str) – The filename we want to have on disk.
• tgt_root (str) – The TARGET directory for the download. Defaults to CFG["tmpdir"].

benchbuild.utils.download.get_hash_of_dirs(directory)

Recursively hash the contents of the given directory.

Parameters:directory (str) – The root directory we want to hash. Returns:A hash of all the contents in the directory.

benchbuild.utils.download.source_required(src_file)

Check, if a download is required.

Parameters:

• src_file – The filename to check for.
• src_root – The path we find the file in.

Returns:

True, if we need to download something, False otherwise.

benchbuild.utils.download.update_hash(src_file)

Update the hash for the given file.

Parameters:

• src – The file name.
• root – The path of the given file.

benchbuild.utils.download.with_git(repo, target_dir=None, limit=None, refspec='HEAD', clone=True, rev_list_args=None)

Decorate a project class with git-based version information.

This adds two attributes to a project class:

• A versions method that returns a list of available versions for this project.
• A repository attribute that provides a repository string to download from later.

We use the git rev-list subcommand to list available versions.

Parameters:

• repo (str) – Repository to download from, this will be stored in the repository attribute of the decorated class.
• target_dir (str) – An optional path where we should put the clone. If unspecified, we will use the SRC_FILE attribute of the decorated class.
• limit (int) – Limit the number of commits to consider for available versions. Versions are ‘ordered’ from latest to oldest.
• refspec (str) – A git refspec string to start listing the versions from.
• clone (bool) – Should we clone the repo if it isn’t already available in our tmp dir? Defaults to True. You can set this to False to avoid time consuming clones, when the project has not been accessed at least once in your installation.
• ref_list_args (list of str) – Additional arguments you want to pass to git rev-list.

benchbuild.utils.download.with_wget(url_dict=None, target_file=None)

Decorate a project class with wget-based version information.

This adds two attributes to a project class:

• A versions method that returns a list of available versions for this project.
• A repository attribute that provides a repository string to download from later.

We use the git rev-list subcommand to list available versions.

Parameters:

• url_dict (dict) – A dictionary that assigns a version to a download URL.
• target_file (str) – An optional path where we should put the clone. If unspecified, we will use the SRC_FILE attribute of the decorated class.