Bucket

Module for bucket object.

class qarnot.bucket.Bucket(connection, name, create=True)

Bases: qarnot.storage.Storage

Represents a resource/result bucket.

This class is the interface to manage resources or results from a qarnot.bucket.Bucket.

Note

A Bucket must be created with qarnot.connection.Connection.create_bucket() or retrieved with qarnot.connection.Connection.buckets(), qarnot.connection.Connection.retrieve_bucket(), or qarnot.connection.Connection.retrieve_or_create_bucket().

Note

Paths given as ‘remote’ arguments, (or as path arguments for Bucket.directory()) must be valid unix-like paths.

__init__(connection, name, create=True)

x.__init__(…) initializes x; see help(type(x)) for signature

delete()

Delete the bucket represented by this Bucket.

list_files()

List files in the bucket

Return type:list(S3.ObjectSummary) Returns:A list of ObjectSummary resources

directory(directory='')

List files in a directory of the bucket according to prefix.

Return type:list(S3.ObjectSummary) Returns:A list of ObjectSummary resources

sync_directory(directory, verbose=False, remote=None)

Synchronize a local directory with the remote buckets.

Parameters:

• directory (str) – The local directory to use for synchronization
• verbose (bool) – Print information about synchronization operations
• remote (str) – path of the directory on remote node (defaults to local)

Warning

Local changes are reflected on the server, a file present on the bucket but not in the local directory will be deleted from the bucket.

A file present in the directory but not in the bucket will be uploaded.

Note

The following parameters are used to determine whether synchronization is required :

• name
• size
• sha1sum

sync_files(files, verbose=False, remote=None)

Synchronize files with the remote buckets.

Parameters:

• files (dict) – Dictionary of synchronized files
• verbose (bool) – Print information about synchronization operations
• remote (str) – path of the directory on remote node (defaults to local)

Dictionary key is the remote file path while value is the local file path.

Warning

Local changes are reflected on the server, a file present on the bucket but not in the local directory will be deleted from the bucket.

A file present in the directory but not in the bucket will be uploaded.

Note

The following parameters are used to determine whether synchronization is required :

• name
• size
• sha1sum

add_file(local_or_file, remote=None)

Add a local file or a Python File on the storage.

Note

You can also use object[remote] = local

Parameters:

• local_or_file (str or File) – path of the local file or an opened Python File
• remote (str) – name of the remote file (defaults to local_or_file)

get_all_files(output_dir, progress=None)

Get all files from the storage.

Parameters:

• output_dir (str) – local directory for the retrieved files.
• progress (bool or function(float, float, str)) – can be a callback (read,total,filename) or True to display a progress bar

Raises:

• qarnot.exceptions.MissingBucketException – the bucket is not on the server
• qarnot.exceptions.QarnotGenericException – API general error, see message for details
• qarnot.exceptions.UnauthorizedException – invalid credentials

Warning

Will override output_dir content.

get_file(remote, local=None, progress=None)

Get a file from the storage. Create needed subfolders.

Parameters:

• remote (str) – the name of the remote file
• local (str) – local name of the retrieved file (defaults to remote)
• progress (bool or function(float, float, str)) – can be a callback (read,total,filename) or True to display a progress bar

Return type:

str

Returns:

The name of the output file.

Raises:

ValueError – no such file

add_directory(local, remote='')

Add a directory to the storage. Does not follow symlinks. File hierarchy is preserved.

Parameters:

• local (str) – path of the local directory to add
• remote (str) – path of the directory on remote node (defaults to local)

Raises:

IOError – not a valid directory

copy_file(source, dest)

Create a copy of a file

Parameters:

• source (str) – name of the existing file to duplicate
• dest (str) – name of the created file

flush()

Ensure all background uploads are complete

update(flush=False)

Update object from remote endpoint

Parameters:flush (bool) – bypass cache

delete_file(remote)

Delete a file from the storage.

Parameters:remote (str) – the name of the remote file

uuid

Bucket identifier

description

Bucket identifier