Globus Transfer

Client

The primary interface for the Globus Transfer API is the TransferClient class.

class globus_sdk.TransferClient(authorizer=None, **kwargs)[source]

Bases: globus_sdk.base.BaseClient

Client for the Globus Transfer API.

This class provides helper methods for most common resources in the REST API, and basic get, put, post, and delete methods from the base rest client that can be used to access any REST resource.

There are two types of helper methods: list methods which return an iterator of GlobusResponse objects, and simple methods that return a single TransferResponse object.

Some calls are paginated. If a call returns a PaginatedResource object, the result is an iterator which can only be walked once. If you need to do multiple passes over the result, call list() on the PaginatedResource or call the original method again to get fresh results.

Detailed documentation is available in the official REST API documentation, which is linked to from the method documentation. Methods that allow arbitrary keyword arguments will pass the extra arguments as query parameters.

Parameters

authorizer (GlobusAuthorizer) – An authorizer instance used for all calls to Globus Transfer

Methods

get_endpoint(endpoint_id, **params)[source]

GET /endpoint/<endpoint_id>

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> endpoint = tc.get_endpoint(endpoint_id)
>>> print("Endpoint name:",
>>>       endpoint["display_name"] or endpoint["canonical_name"])

External Documentation

See Get Endpoint by ID in the REST documentation for details.

update_endpoint(endpoint_id, data, **params)[source]

PUT /endpoint/<endpoint_id>

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> epup = dict(display_name="My New Endpoint Name",
>>>             description="Better Description")
>>> update_result = tc.update_endpoint(endpoint_id, epup)

External Documentation

See Update Endpoint by ID in the REST documentation for details.

create_endpoint(data)[source]

POST /endpoint/<endpoint_id>

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> ep_data = {
>>>   "DATA_TYPE": "endpoint",
>>>   "display_name": display_name,
>>>   "DATA": [
>>>     {
>>>       "DATA_TYPE": "server",
>>>       "hostname": "gridftp.example.edu",
>>>     },
>>>   ],
>>> }
>>> create_result = tc.create_endpoint(ep_data)
>>> endpoint_id = create_result["id"]

External Documentation

See Create endpoint in the REST documentation for details.

delete_endpoint(endpoint_id)[source]

DELETE /endpoint/<endpoint_id>

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> delete_result = tc.delete_endpoint(endpoint_id)

External Documentation

See Delete endpoint by id in the REST documentation for details.

GET /endpoint_search?filter_fulltext=<filter_fulltext>&filter_scope=<filter_scope>
Parameters
  • filter_fulltext (str, optional) – The string to use in a full text search on endpoints. Effectively, the “search query” which is being requested. May be omitted with specific filter_scope values.

  • filter_scope (str, optional) – A “scope” within which to search for endpoints. This must be one of the limited and known names known to the service, which can be found documented in the External Documentation below. Defaults to searching all endpoints (in which case filter_fulltext is required)

  • num_results (int or None) – The number of search results to fetch from the service. May be set to None to request the maximum allowable number of results. [Default: 25]

  • params (dict) – Any additional parameters will be passed through as query params.

Return type

PaginatedResource, an iterable of GlobusResponse

Examples

Search for a given string as a fulltext search:

>>> tc = globus_sdk.TransferClient(...)
>>> for ep in tc.endpoint_search('String to search for!'):
>>>     print(ep['display_name'])

Search for a given string, but only on endpoints that you own:

>>> for ep in tc.endpoint_search('foo', filter_scope='my-endpoints'):
>>>     print('{0} has ID {1}'.format(ep['display_name'], ep['id']))

Search results are capped at a number of elements equal to the num_results parameter. If you want more than the default, 25, elements, do like so:

>>> for ep in tc.endpoint_search('String to search for!',
>>>                             num_results=120):
>>>     print(ep['display_name'])

It is important to be aware that the Endpoint Search API limits you to 1000 results for any search query. You can request the maximum number of results either explicitly, with num_results=1000, or by stating that you want no limit by setting it to None:

>>> for ep in tc.endpoint_search('String to search for!',
>>>                             num_results=None):
>>>     print(ep['display_name'])

External Documentation

For additional information, see Endpoint Search. in the REST documentation for details.

endpoint_autoactivate(endpoint_id, **params)[source]

POST /endpoint/<endpoint_id>/autoactivate

Return type

TransferResponse

The following example will try to “auto” activate the endpoint using a credential available from another endpoint or sign in by the user with the same identity provider, but only if the endpoint is not already activated or going to expire within an hour (3600 seconds). If that fails, direct the user to the globus website to perform activation:

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> r = tc.endpoint_autoactivate(ep_id, if_expires_in=3600)
>>> while (r["code"] == "AutoActivationFailed"):
>>>     print("Endpoint requires manual activation, please open "
>>>           "the following URL in a browser to activate the "
>>>           "endpoint:")
>>>     print("https://app.globus.org/file-manager?origin_id=%s"
>>>           % ep_id)
>>>     # For python 2.X, use raw_input() instead
>>>     input("Press ENTER after activating the endpoint:")
>>>     r = tc.endpoint_autoactivate(ep_id, if_expires_in=3600)

This is the recommended flow for most thick client applications, because many endpoints require activation via OAuth MyProxy, which must be done in a browser anyway. Web based clients can link directly to the URL.

You also might want messaging or logging depending on why and how the operation succeeded, in which case you’ll need to look at the value of the “code” field and either decide on your own messaging or use the response’s “message” field.

>>> tc = globus_sdk.TransferClient(...)
>>> r = tc.endpoint_autoactivate(ep_id, if_expires_in=3600)
>>> if r['code'] == 'AutoActivationFailed':
>>>     print('Endpoint({}) Not Active! Error! Source message: {}'
>>>           .format(ep_id, r['message']))
>>>     sys.exit(1)
>>> elif r['code'] == 'AutoActivated.CachedCredential':
>>>     print('Endpoint({}) autoactivated using a cached credential.'
>>>           .format(ep_id))
>>> elif r['code'] == 'AutoActivated.GlobusOnlineCredential':
>>>     print(('Endpoint({}) autoactivated using a built-in Globus '
>>>            'credential.').format(ep_id))
>>> elif r['code'] = 'AlreadyActivated':
>>>     print('Endpoint({}) already active until at least {}'
>>>           .format(ep_id, 3600))

External Documentation

See Autoactivate endpoint in the REST documentation for details.

endpoint_deactivate(endpoint_id, **params)[source]

POST /endpoint/<endpoint_id>/deactivate

Return type

TransferResponse

External Documentation

See Deactive endpoint in the REST documentation for details.

endpoint_activate(endpoint_id, requirements_data, **params)[source]

POST /endpoint/<endpoint_id>/activate

Return type

TransferResponse

Consider using autoactivate and web activation instead, described in the example for endpoint_autoactivate().

External Documentation

See Activate endpoint in the REST documentation for details.

endpoint_get_activation_requirements(endpoint_id, **params)[source]

GET /endpoint/<endpoint_id>/activation_requirements

Return type

ActivationRequirementsResponse

External Documentation

See Get activation requirements in the REST documentation for details.

my_effective_pause_rule_list(endpoint_id, **params)[source]

GET /endpoint/<endpoint_id>/my_effective_pause_rule_list

Return type

IterableTransferResponse

External Documentation

See Get my effective endpoint pause rules in the REST documentation for details.

my_shared_endpoint_list(endpoint_id, **params)[source]

GET /endpoint/<endpoint_id>/my_shared_endpoint_list

Return type

IterableTransferResponse

External Documentation

See Get shared endpoint list in the REST documentation for details.

create_shared_endpoint(data)[source]

POST /shared_endpoint

Parameters

data (dict) – A python dict representation of a shared_endpoint document

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> shared_ep_data = {
>>>   "DATA_TYPE": "shared_endpoint",
>>>   "host_endpoint": host_endpoint_id,
>>>   "host_path": host_path,
>>>   "display_name": display_name,
>>>   # optionally specify additional endpoint fields
>>>   "description": "my test share"
>>> }
>>> create_result = tc.create_shared_endpoint(shared_ep_data)
>>> endpoint_id = create_result["id"]

External Documentation

See Create shared endpoint in the REST documentation for details.

endpoint_server_list(endpoint_id, **params)[source]

GET /endpoint/<endpoint_id>/server_list

Return type

IterableTransferResponse

External Documentation

See Get endpoint server list in the REST documentation for details.

get_endpoint_server(endpoint_id, server_id, **params)[source]

GET /endpoint/<endpoint_id>/server/<server_id>

Return type

TransferResponse

External Documentation

See Get endpoint server by id in the REST documentation for details.

add_endpoint_server(endpoint_id, server_data)[source]

POST /endpoint/<endpoint_id>/server

Return type

TransferResponse

External Documentation

See Add endpoint server in the REST documentation for details.

update_endpoint_server(endpoint_id, server_id, server_data)[source]

PUT /endpoint/<endpoint_id>/server/<server_id>

Return type

TransferResponse

External Documentation

See Update endpoint server by id in the REST documentation for details.

delete_endpoint_server(endpoint_id, server_id)[source]

DELETE /endpoint/<endpoint_id>/server/<server_id>

Return type

TransferResponse

External Documentation

See Delete endpoint server by id in the REST documentation for details.

endpoint_role_list(endpoint_id, **params)[source]

GET /endpoint/<endpoint_id>/role_list

Return type

IterableTransferResponse

External Documentation

See Get list of endpoint roles in the REST documentation for details.

add_endpoint_role(endpoint_id, role_data)[source]

POST /endpoint/<endpoint_id>/role

Return type

TransferResponse

External Documentation

See Create endpoint role in the REST documentation for details.

get_endpoint_role(endpoint_id, role_id, **params)[source]

GET /endpoint/<endpoint_id>/role/<role_id>

Return type

TransferResponse

External Documentation

See Get endpoint role by id in the REST documentation for details.

delete_endpoint_role(endpoint_id, role_id)[source]

DELETE /endpoint/<endpoint_id>/role/<role_id>

Return type

TransferResponse

External Documentation

See Delete endpoint role by id in the REST documentation for details.

endpoint_acl_list(endpoint_id, **params)[source]

GET /endpoint/<endpoint_id>/access_list

Return type

IterableTransferResponse

External Documentation

See Get list of access rules in the REST documentation for details.

get_endpoint_acl_rule(endpoint_id, rule_id, **params)[source]

GET /endpoint/<endpoint_id>/access/<rule_id>

Return type

TransferResponse

External Documentation

See Get access rule by id in the REST documentation for details.

add_endpoint_acl_rule(endpoint_id, rule_data)[source]

POST /endpoint/<endpoint_id>/access

Parameters
  • endpoint_id (str) – ID of endpoint to which to add the acl

  • rule_data (dict) – A python dict representation of an access document

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> rule_data = {
>>>   "DATA_TYPE": "access",
>>>   "principal_type": "identity",
>>>   "principal": identity_id,
>>>   "path": "/dataset1/",
>>>   "permissions": "rw",
>>> }
>>> result = tc.add_endpoint_acl_rule(endpoint_id, rule_data)
>>> rule_id = result["access_id"]

Note that if this rule is being created on a shared endpoint the “path” field is relative to the “host_path” of the shared endpoint.

External Documentation

See Create access rule in the REST documentation for details.

update_endpoint_acl_rule(endpoint_id, rule_id, rule_data)[source]

PUT /endpoint/<endpoint_id>/access/<rule_id>

Return type

TransferResponse

External Documentation

See Update access rule in the REST documentation for details.

delete_endpoint_acl_rule(endpoint_id, rule_id)[source]

DELETE /endpoint/<endpoint_id>/access/<rule_id>

Return type

TransferResponse

External Documentation

See Delete access rule in the REST documentation for details.

bookmark_list(**params)[source]

GET /bookmark_list

Return type

IterableTransferResponse

External Documentation

See Get list of bookmarks in the REST documentation for details.

create_bookmark(bookmark_data)[source]

POST /bookmark

Return type

TransferResponse

External Documentation

See Create bookmark in the REST documentation for details.

get_bookmark(bookmark_id, **params)[source]

GET /bookmark/<bookmark_id>

Return type

TransferResponse

External Documentation

See Get bookmark by id in the REST documentation for details.

update_bookmark(bookmark_id, bookmark_data)[source]

PUT /bookmark/<bookmark_id>

Return type

TransferResponse

External Documentation

See Update bookmark in the REST documentation for details.

delete_bookmark(bookmark_id)[source]

DELETE /bookmark/<bookmark_id>

Return type

TransferResponse

External Documentation

See Delete bookmark by id in the REST documentation for details.

operation_ls(endpoint_id, **params)[source]

GET /operation/endpoint/<endpoint_id>/ls

Return type

IterableTransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> for entry in tc.operation_ls(ep_id, path="/~/project1/"):
>>>     print(entry["name"], entry["type"])

External Documentation

See List Directory Contents in the REST documentation for details.

operation_mkdir(endpoint_id, path, **params)[source]

POST /operation/endpoint/<endpoint_id>/mkdir

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> tc.operation_mkdir(ep_id, path="/~/newdir/")

External Documentation

See Make Directory in the REST documentation for details.

operation_rename(endpoint_id, oldpath, newpath, **params)[source]

POST /operation/endpoint/<endpoint_id>/rename

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> tc.operation_rename(ep_id, oldpath="/~/file1.txt",
>>>                     newpath="/~/project1data.txt")

External Documentation

See Rename in the REST documentation for details.

POST /operation/endpoint/<endpoint_id>/symlink

Return type

TransferResponse

The path is the name of the symlink, and the symlink_target is the path referenced by the symlink.

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> tc.operation_symlink(ep_id, symlink_target="/~/file1.txt",
>>>                      path="/~/link-to-file1.txt")

External Documentation

See Symlink in the REST documentation for details.

get_submission_id(**params)[source]

GET /submission_id

Return type

TransferResponse

Submission IDs are required to submit tasks to the Transfer service via the submit_transfer and submit_delete methods.

Most users will not need to call this method directly, as the convenience classes TransferData and DeleteData will call it automatically if they are not passed a submission_id explicitly.

External Documentation

See Get a submission id in the REST documentation for more details.

submit_transfer(data)[source]

POST /transfer

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> tdata = globus_sdk.TransferData(tc, source_endpoint_id,
>>>                                 destination_endpoint_id,
>>>                                 label="SDK example",
>>>                                 sync_level="checksum")
>>> tdata.add_item("/source/path/dir/", "/dest/path/dir/",
>>>                recursive=True)
>>> tdata.add_item("/source/path/file.txt",
>>>                "/dest/path/file.txt")
>>> transfer_result = tc.submit_transfer(tdata)
>>> print("task_id =", transfer_result["task_id"])

The data parameter can be a normal Python dictionary, or a TransferData object.

External Documentation

See Submit a transfer task in the REST documentation for more details.

submit_delete(data)[source]

POST /delete

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> ddata = globus_sdk.DeleteData(tc, endpoint_id, recursive=True)
>>> ddata.add_item("/dir/to/delete/")
>>> ddata.add_item("/file/to/delete/file.txt")
>>> delete_result = tc.submit_delete(ddata)
>>> print("task_id =", delete_result["task_id"])

The data parameter can be a normal Python dictionary, or a DeleteData object.

External Documentation

See Submit a delete task in the REST documentation for details.

task_list(num_results=10, **params)[source]

Get an iterable of task documents owned by the current user.

GET /task_list

Parameters
  • num_results (int or none) – The number of tasks to fetch from the service. May be set to None to request the maximum allowable number of results. [Default: 10]

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

PaginatedResource, an iterable of GlobusResponse

Examples

Fetch the default number (10) of tasks and print some basic info:

>>> tc = TransferClient(...)
>>> for task in tc.task_list():
>>>     print("Task({}): {} -> {}".format(
>>>         task["task_id"], task["source_endpoint"],
>>>         task["destination_endpoint"]))

External Documentation

See Task list in the REST documentation for details.

task_event_list(task_id, num_results=10, **params)[source]

List events (for example, faults and errors) for a given Task.

GET /task/<task_id>/event_list

Parameters
  • task_id (str) – The ID of the task to inspect.

  • num_results (int or None) – The number of events to fetch from the service. May be set to None to request the maximum allowable number of results. [Default: 10]

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

PaginatedResource, an iterable of GlobusResponse

Examples

Fetch the default number (10) of events and print some basic info:

>>> tc = TransferClient(...)
>>> task_id = ...
>>> for event in tc.task_event_list(task_id):
>>>     print("Event on Task({}) at {}:\n{}".format(
>>>         task_id, event["time"], event["description"])

External Documentation

See Get event list in the REST documentation for details.

get_task(task_id, **params)[source]

GET /task/<task_id>

Return type

TransferResponse

External Documentation

See Get task by id in the REST documentation for details.

update_task(task_id, data, **params)[source]

PUT /task/<task_id>

Return type

TransferResponse

External Documentation

See Update task by id in the REST documentation for details.

cancel_task(task_id)[source]

POST /task/<task_id>/cancel

Return type

TransferResponse

External Documentation

See Cancel task by id in the REST documentation for details.

task_wait(task_id, timeout=10, polling_interval=10)[source]

Wait until a Task is complete or fails, with a time limit. If the task is “ACTIVE” after time runs out, returns False. Otherwise returns True.

Parameters
  • task_id (str) – ID of the Task to wait on for completion

  • timeout (int, optional) – Number of seconds to wait in total. Minimum 1. [Default: 10]

  • polling_interval (int, optional) – Number of seconds between queries to Globus about the Task status. Minimum 1. [Default: 10]

Examples

If you want to wait for a task to terminate, but want to warn every minute that it doesn’t terminate, you could:

>>> tc = TransferClient(...)
>>> while not tc.task_wait(task_id, timeout=60):
>>>     print("Another minute went by without {0} terminating"
>>>           .format(task_id))

Or perhaps you want to check on a task every minute for 10 minutes, and give up if it doesn’t complete in that time:

>>> tc = TransferClient(...)
>>> done = tc.task_wait(task_id, timeout=600, polling_interval=60):
>>> if not done:
>>>     print("{0} didn't successfully terminate!"
>>>           .format(task_id))
>>> else:
>>>     print("{0} completed".format(task_id))

You could print dots while you wait for a task by only waiting one second at a time:

>>> tc = TransferClient(...)
>>> while not tc.task_wait(task_id, timeout=1, polling_interval=1):
>>>     print(".", end="")
>>> print("\n{0} completed!".format(task_id))
task_pause_info(task_id, **params)[source]

GET /task/<task_id>/pause_info

Return type

TransferResponse

External Documentation

See Get task pause info in the REST documentation for details.

task_successful_transfers(task_id, num_results=100, **params)[source]

Get the successful file transfers for a completed Task.

Note

Only files that were actually transferred are included. This does not include directories, files that were checked but skipped as part of a sync transfer, or files which were skipped due to skip_source_errors being set on the task.

GET /task/<task_id>/successful_transfers

Parameters
  • task_id (str) – The ID of the task to inspect.

  • num_results (int or None, optional) – The number of file transfer records to fetch from the service. May be set to None to request the maximum allowable number of results. [Default: 100]

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

PaginatedResource, an iterable of GlobusResponse

Examples

Fetch all transferred files for a task and print some basic info:

>>> tc = TransferClient(...)
>>> task_id = ...
>>> for info in tc.task_successful_transfers(task_id):
>>>     print("{} -> {}".format(
>>>         info["source_path"], info["destination_path"]))

External Documentation

See Get Task Successful Transfers in the REST documentation for details.

task_skipped_errors(task_id, num_results=100, **params)[source]

Get path and error information for all paths that were skipped due to skip_source_errors being set on a completed transfer Task.

GET /task/<task_id>/skipped_errors

Parameters
  • task_id (str) – The ID of the task to inspect.

  • num_results (int or None, optional) – The number of file transfer records to fetch from the service. May be set to None to request the maximum allowable number of results. [Default: 100]

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

PaginatedResource, an iterable of GlobusResponse

Examples

Fetch all skipped errors for a task and print some basic info:

>>> tc = TransferClient(...)
>>> task_id = ...
>>> for info in tc.task_skipped_errors(task_id):
>>>     print("{} -> {}".format(
>>>         info["error_code"], info["source_path"]))

External Documentation

See Get Task Skipped Errors in the REST documentation for details.

endpoint_manager_monitored_endpoints(**params)[source]

Get endpoints the current user is a monitor or manager on.

GET endpoint_manager/monitored_endpoints

Return type

iterable of GlobusResponse

See Get monitored endpoints in the REST documentation for details.

endpoint_manager_hosted_endpoint_list(endpoint_id, **params)[source]

Get shared endpoints hosted on the given endpoint.

GET /endpoint_manager/endpoint/<endpoint_id>/hosted_endpoint_list

Return type

iterable of GlobusResponse

See Get hosted endpoint list in the REST documentation for details.

endpoint_manager_get_endpoint(endpoint_id, **params)[source]

Get endpoint details as an admin.

GET /endpoint_manager/endpoint/<endpoint_id>

Return type

TransferResponse

External Documentation

See Get endpoint as admin in the REST documentation for details.

endpoint_manager_acl_list(endpoint_id, **params)[source]

Get a list of access control rules on specified endpoint as an admin.

GET endpoint_manager/endpoint/<endpoint_id>/access_list

Return type

IterableTransferResponse

External Documentation

See Get endpoint access list as admin in the REST documentation for details.

endpoint_manager_task_list(num_results=10, **params)[source]

Get a list of tasks visible via activity_monitor role, as opposed to tasks owned by the current user.

GET endpoint_manager/task_list

Parameters
  • num_results (int or None, optional) – The number of tasks to fetch from the service. May be set to None to request the maximum allowable number of results. [Default: 10]

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

PaginatedResource, an iterable of GlobusResponse

Examples

Fetch the default number (10) of tasks and print some basic info:

>>> tc = TransferClient(...)
>>> for task in tc.endpoint_manager_task_list():
>>>     print("Task({}): {} -> {}\n  was submitted by\n  {}".format(
>>>         task["task_id"], task["source_endpoint"],
>>>         task["destination_endpoint"], task["owner_string"]))

Do that same operation on all tasks visible via activity_monitor status:

>>> tc = TransferClient(...)
>>> for task in tc.endpoint_manager_task_list(num_results=None):
>>>     print("Task({}): {} -> {}\n  was submitted by\n  {}".format(
>>>         task["task_id"], task["source_endpoint"],
>>>         task["destination_endpoint"], task["owner_string"]))

External Documentation

See Advanced Endpoint Management: Get tasks in the REST documentation for details.

endpoint_manager_get_task(task_id, **params)[source]

Get task info as an admin. Requires activity monitor effective role on the destination endpoint of the task.

GET /endpoint_manager/task/<task_id>

Return type

TransferResponse

External Documentation

See Get task as admin in the REST documentation for details.

endpoint_manager_task_event_list(task_id, num_results=10, **params)[source]

List events (for example, faults and errors) for a given task as an admin. Requires activity monitor effective role on the destination endpoint of the task.

GET /task/<task_id>/event_list

Parameters
  • task_id (str) – The ID of the task to inspect.

  • num_results (int or None, optional) – The number of events to fetch from the service. May be set to None to request the maximum allowable number of results. [Default: 10]

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

PaginatedResource, an iterable of GlobusResponse

External Documentation

See Get task events as admin in the REST documentation for details.

endpoint_manager_task_pause_info(task_id, **params)[source]

Get details about why a task is paused as an admin. Requires activity monitor effective role on the destination endpoint of the task.

GET /endpoint_manager/task/<task_id>/pause_info

Return type

TransferResponse

External Documentation

See Get task pause info as admin in the REST documentation for details.

endpoint_manager_task_successful_transfers(task_id, num_results=100, **params)[source]

Get the successful file transfers for a completed Task as an admin.

GET /endpoint_manager/task/<task_id>/successful_transfers

Parameters
  • task_id (str) – The ID of the task to inspect.

  • num_results (int or None, optional) – The number of file transfer records to fetch from the service. May be set to None to request the maximum allowable number of results. [Default: 100]

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

PaginatedResource, an iterable of GlobusResponse

External Documentation

See Get task successful transfers as admin in the REST documentation for details.

endpoint_manager_task_skipped_errors(task_id, num_results=100, **params)[source]

Get skipped errors for a completed Task as an admin.

GET /endpoint_manager/task/<task_id>/skipped_errors

Parameters
  • task_id (str) – The ID of the task to inspect.

  • num_results (int or None, optional) – The number of skipped error records to fetch from the service. May be set to None to request the maximum allowable number of results. [Default: 100]

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

PaginatedResource, an iterable of GlobusResponse

External Documentation

See Get task skipped errors as admin in the REST documentation for details.

endpoint_manager_cancel_tasks(task_ids, message, **params)[source]

Cancel a list of tasks as an admin. Requires activity manager effective role on the task(s) source or destination endpoint(s).

POST /endpoint_manager/admin_cancel

Parameters
  • task_ids (iterable of str) – List of task ids to cancel.

  • message (str) – Message given to all users who’s tasks have been canceled.

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

TransferResponse

External Documentation

See Cancel tasks as admin in the REST documentation for details.

endpoint_manager_cancel_status(admin_cancel_id, **params)[source]

Get the status of an an admin cancel (result of endpoint_manager_cancel_tasks).

GET /endpoint_manager/admin_cancel/<admin_cancel_id>

Parameters
  • admin_cancel_id (str) – The ID of the the cancel job to inspect.

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

TransferResponse

External Documentation

See Get cancel status by id in the REST documentation for details.

endpoint_manager_pause_tasks(task_ids, message, **params)[source]

Pause a list of tasks as an admin. Requires activity manager effective role on the task(s) source or destination endpoint(s).

POST /endpoint_manager/admin_pause

Parameters
  • task_ids (iterable of str) – List of task ids to pause.

  • message (str) – Message given to all users who’s tasks have been paused.

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

TransferResponse

External Documentation

See Pause tasks as admin in the REST documentation for details.

endpoint_manager_resume_tasks(task_ids, **params)[source]

Resume a list of tasks as an admin. Requires activity manager effective role on the task(s) source or destination endpoint(s).

POST /endpoint_manager/admin_resume

Parameters
  • task_ids (iterable of str) – List of task ids to resume.

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

TransferResponse

External Documentation

See Resume tasks as admin in the REST documentation for details.

endpoint_manager_pause_rule_list(filter_endpoint=None, **params)[source]

Get a list of pause rules on endpoints that the current user has the activity monitor effective role on.

GET /endpoint_manager/pause_rule_list

Parameters
  • filter_endpoint (str) – An endpoint ID. Limit results to rules on endpoints hosted by this endpoint. Must be activity monitor on this endpoint, not just the hosted endpoints.

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

IterableTransferResponse

External Documentation

See Get pause rules in the REST documentation for details.

endpoint_manager_create_pause_rule(data)[source]

Create a new pause rule. Requires the activity manager effective role on the endpoint defined in the rule.

POST /endpoint_manager/pause_rule

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> rule_data = {
>>>   "DATA_TYPE": "pause_rule",
>>>   "message": "Message to users explaining why tasks are paused",
>>>   "endpoint_id": "339abc22-aab3-4b45-bb56-8d40535bfd80",
>>>   "identity_id": None,  # affect all users on endpoint
>>>   "start_time": None  # start now
>>> }
>>> create_result = tc.endpoint_manager_create_pause_rule(ep_data)
>>> rule_id = create_result["id"]

External Documentation

See Create pause rule in the REST documentation for details.

endpoint_manager_get_pause_rule(pause_rule_id, **params)[source]

Get an existing pause rule by ID. Requires the activity manager effective role on the endpoint defined in the rule.

GET /endpoint_manager/pause_rule/<pause_rule_id>

Parameters
  • pause_rule_id (str) – ID of pause rule to get.

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

TransferResponse

External Documentation

See Get pause rule in the REST documentation for details.

endpoint_manager_update_pause_rule(pause_rule_id, data)[source]

Update an existing pause rule by ID. Requires the activity manager effective role on the endpoint defined in the rule. Note that non update-able fields in data will be ignored.

PUT /endpoint_manager/pause_rule/<pause_rule_id>

Return type

TransferResponse

Examples

>>> tc = globus_sdk.TransferClient(...)
>>> rule_data = {
>>>   "message": "Update to pause, reads are now allowed.",
>>>   "pause_ls": False,
>>>   "pause_task_transfer_read": False
>>> }
>>> update_result = tc.endpoint_manager_update_pause_rule(ep_data)

External Documentation

See Update pause rule in the REST documentation for details.

endpoint_manager_delete_pause_rule(pause_rule_id, **params)[source]

Delete an existing pause rule by ID. Requires the user to see the “editible” field of the rule as True. Any tasks affected by this rule will no longer be once it is deleted.

DELETE /endpoint_manager/pause_rule/<pause_rule_id>

Parameters
  • pause_rule_id (str) – The ID of the pause rule to delete.

  • params (dict, optional) – Any additional parameters will be passed through as query params.

Return type

TransferResponse

External Documentation

See Delete pause rule in the REST documentation for details.

Helper Objects

These helper objects make it easier to correctly create data for consumption by a TransferClient.

class globus_sdk.TransferData(transfer_client, source_endpoint, destination_endpoint, label=None, submission_id=None, sync_level=None, verify_checksum=False, preserve_timestamp=False, encrypt_data=False, deadline=None, skip_source_errors=False, fail_on_quota_errors=False, recursive_symlinks='ignore', **kwargs)[source]

Bases: dict

Convenience class for constructing a transfer document, to use as the data parameter to submit_transfer.

At least one item must be added using add_item.

If submission_id isn’t passed, one will be fetched automatically. The submission ID can be pulled out of here to inspect, but the document can be used as-is multiple times over to retry a potential submission failure (so there shouldn’t be any need to inspect it).

Parameters
  • transfer_client (TransferClient) – A TransferClient instance which will be used to get a submission ID if one is not supplied. Should be the same instance that is used to submit the transfer.

  • source_endpoint (str) – The endpoint ID of the source endpoint

  • destination_endpoint (str) – The endpoint ID of the destination endpoint

  • label (str, optional) – A string label for the Task

  • submission_id (str, optional) – A submission ID value fetched via get_submission_id . Defaults to using transfer_client.get_submission_id

  • sync_level (int or str, optional) – The method used to compare items between the source and destination. One of "exists", "size", "mtime", or "checksum" See the section below on sync-level for an explanation of values.

  • verify_checksum (bool, optional) – When true, after transfer verify that the source and destination file checksums match. If they don’t, re-transfer the entire file and keep trying until it succeeds. This will create CPU load on both the origin and destination of the transfer, and may even be a bottleneck if the network speed is high enough. [default: False]

  • preserve_timestamp (bool, optional) – When true, Globus Transfer will attempt to set file timestamps on the destination to match those on the origin. [default: False]

  • encrypt_data (bool, optional) – When true, all files will be TLS-protected during transfer. [default: False]

  • deadline (str or datetime, optional) – An ISO-8601 timestamp (as a string) or a datetime object which defines a deadline for the transfer. At the deadline, even if the data transfer is not complete, the job will be canceled. We recommend ensuring that the timestamp is in UTC to avoid confusion and ambiguity. Examples of ISO-8601 timestamps include 2017-10-12 09:30Z, 2017-10-12 12:33:54+00:00, and 2017-10-12

  • recursive_symlinks (str) – Specify the behavior of recursive directory transfers when encountering symlinks. One of "ignore", "keep", or "copy". "ignore" skips symlinks, "keep" creates symlinks at the destination matching the source (without modifying the link path at all), and "copy" follows symlinks on the source, failing if the link is invalid. [default: "ignore"]

  • skip_source_errors (bool, optional) – When true, source permission denied and file not found errors from the source endpoint will cause the offending path to be skipped. [default: False]

  • fail_on_quota_errors (bool, optional) – When true, quota exceeded errors will cause the task to fail. [default: False]

Any additional parameters are fed into the dict being created verbatim.

Sync Levels

The values for sync_level are used to determine how comparisons are made between files found both on the source and the destination. When files match, no data transfer will occur.

For compatibility, this can be an integer 0, 1, 2, or 3 in addition to the string values.

The meanings are as follows:

value

behavior

0, exists

Determine whether or not to transfer based on file existence. If the destination file is absent, do the transfer.

1, size

Determine whether or not to transfer based on the size of the file. If destination file size does not match the source, do the transfer.

2, mtime

Determine whether or not to transfer based on modification times. If source has a newer modififed time than the destination, do the transfer.

3, checksum

Determine whether or not to transfer based on checksums of file contents. If source and destination contents differ, as determined by a checksum of their contents, do the transfer.

Examples

See the submit_transfer documentation for example usage.

External Documentation

See the Task document definition and Transfer specific fields in the REST documentation for more details on Transfer Task documents.

Methods

add_item(source_path, destination_path, recursive=False, external_checksum=None, checksum_algorithm=None, **params)[source]

Add a file or directory to be transfered. If the item is a symlink to a file or directory, the file or directory at the target of the symlink will be transfered.

Appends a transfer_item document to the DATA key of the transfer document.

Note

The full path to the destination file must be provided for file items. Parent directories of files are not allowed. See task submission documentation for more details.

Parameters
  • source_path (str) – Path to the source directory or file to be transfered

  • destination_path (str) – Path to the source directory or file will be transfered to

  • recursive (bool) – Set to True if the target at source path is a directory

  • external_checksum (str, optional) – A checksum to verify source file integrity before the transfer and destination file integrity after the transfer. Cannot be used with directories. Assumed to be an MD5 checksum unless checksum_algorithm is also given.

  • checksum_algorithm (str, optional) – Specifies the checksum algorithm to be used when verify_checksum is True, sync_level is “checksum” or 3, or an external_checksum is given.

Add a symlink to be transfered as a symlink rather than as the target of the symlink.

Appends a transfer_symlink_item document to the DATA key of the transfer document.

Parameters
  • source_path (str) – Path to the source symlink

  • destination_path (str) – Path to which the source symlink will be transfered

class globus_sdk.DeleteData(transfer_client, endpoint, label=None, submission_id=None, recursive=False, deadline=None, **kwargs)[source]

Bases: dict

Convenience class for constructing a delete document, to use as the data parameter to submit_delete.

At least one item must be added using add_item.

If submission_id isn’t passed, one will be fetched automatically. The submission ID can be pulled out of here to inspect, but the document can be used as-is multiple times over to retry a potential submission failure (so there shouldn’t be any need to inspect it).

Parameters
  • transfer_client (TransferClient) – A TransferClient instance which will be used to get a submission ID if one is not supplied. Should be the same instance that is used to submit the deletion.

  • endpoint (str) – The endpoint ID which is targeted by this deletion Task

  • label (str, optional) – A string label for the Task

  • submission_id (str, optional) – A submission ID value fetched via get_submission_id. Defaults to using transfer_client.get_submission_id

  • recursive (bool) – Recursively delete subdirectories on the target endpoint [default: False]

  • deadline (str or datetime, optional) – An ISO-8601 timestamp (as a string) or a datetime object which defines a deadline for the deletion. At the deadline, even if the data deletion is not complete, the job will be canceled. We recommend ensuring that the timestamp is in UTC to avoid confusion and ambiguity. Examples of ISO-8601 timestamps include 2017-10-12 09:30Z, 2017-10-12 12:33:54+00:00, and 2017-10-12

Examples

See the submit_delete documentation for example usage.

External Documentation

See the Task document definition and Delete specific fields in the REST documentation for more details on Delete Task documents.

Methods

add_item(path, **params)[source]

Add a file or directory or symlink to be deleted. If any of the paths are directories, recursive must be set True on the top level DeleteData. Symlinks will never be followed, only deleted.

Appends a delete_item document to the DATA key of the delete document.

Client Errors

When an error occurs, a TransferClient will raise this specialized type of error, rather than a generic GlobusAPIError.

class globus_sdk.exc.TransferAPIError(r)[source]

Bases: globus_sdk.exc.GlobusAPIError

Error class for the Transfer API client. In addition to the inherited code and message instance variables, provides:

Variables

request_id – Unique identifier for the request, which should be provided when contacting support@globus.org.

Transfer Responses

class globus_sdk.transfer.response.ActivationRequirementsResponse(*args, **kwargs)[source]

Bases: globus_sdk.transfer.response.base.TransferResponse

Response class for Activation Requirements responses.

All Activation Requirements documents refer to a specific Endpoint, from whence they were acquired. References to “the Endpoint” implicitly refer to that originating Endpoint, and not to some other Endpoint.

External Documentation

See Activation Requirements Document in the API documentation for details.

active_until(time_seconds, relative_time=True)[source]

Check if the Endpoint will be active until some time in the future, given as an integer number of seconds. When relative_time=False, the time_seconds is interpreted as a POSIX timestamp.

This supports queries using both relative and absolute timestamps to better support a wide range of use cases. For example, if I have a task that I know will typically take N seconds, and I want an M second safety margin:

>>> num_secs_allowed = N + M
>>> tc = TransferClient(...)
>>> reqs_doc = tc.endpoint_get_activation_requirements(...)
>>> if not reqs_doc.active_until(num_secs_allowed):
>>>     raise Exception("Endpoint won't be active long enough")
>>> ...

or, alternatively, if I know that the endpoint must be active until October 18th, 2016 for my tasks to complete:

>>> oct18_2016 = 1476803436
>>> tc = TransferClient(...)
>>> reqs_doc = tc.endpoint_get_activation_requirements(...)
>>> if not reqs_doc.active_until(oct18_2016, relative_time=False):
>>>     raise Exception("Endpoint won't be active long enough")
>>> ...
Parameters
  • time_seconds (int) – Number of seconds into the future.

  • relative_time (bool) – Defaults to True. When False, time_seconds is treated as a POSIX timestamp (i.e. seconds since epoch as an integer) instead of its ordinary behavior.

Returns

True if the Endpoint will be active until the deadline, False otherwise

Return type

bool

property always_activated

Returns True if the endpoint activation never expires (e.g. shared endpoints, globus connect personal endpoints).

Return type

bool

property supports_auto_activation

Check if the document lists Auto-Activation as an available type of activation. Typically good to use when you need to catch endpoints that require web activation before proceeding.

>>> endpoint_id = "..."
>>> tc = TransferClient(...)
>>> reqs_doc = tc.endpoint_get_activation_requirements(endpoint_id)
>>> if not reqs_doc.supports_auto_activation:
>>>     # use `from __future__ import print_function` in py2
>>>     print(("This endpoint requires web activation. "
>>>            "Please login and activate the endpoint here:\n"
>>>            "https://app.globus.org/file-manager?origin_id={}")
>>>           .format(endpoint_id), file=sys.stderr)
>>>     # py3 calls it `input()` in py2, use `raw_input()`
>>>     input("Please Hit Enter When You Are Done")
Return type

bool

property supports_web_activation

Check if the document lists known types of activation that can be done through the web. If this returns False, it means that the endpoint is of a highly unusual type, and you should directly inspect the response’s data attribute to see what is required. Sending users to the web page for activation is also a fairly safe action to take. Note that ActivationRequirementsResponse.supports_auto_activation directly implies ActivationRequirementsResponse.supports_web_activation, so these are not exclusive.

For example,

>>> tc = TransferClient(...)
>>> reqs_doc = tc.endpoint_get_activation_requirements(...)
>>> if not reqs_doc.supports_web_activation:
>>>     # use `from __future__ import print_function` in py2
>>>     print("Highly unusual endpoint. " +
>>>           "Cannot webactivate. Raw doc: " +
>>>           str(reqs_doc), file=sys.stderr)
>>>     print("Sending user to web anyway, just in case.",
>>>           file=sys.stderr)
>>> ...
Return type

bool

class globus_sdk.transfer.response.IterableTransferResponse(http_response, client=None)[source]

Bases: globus_sdk.transfer.response.base.TransferResponse

Response class for non-paged list oriented resources. Allows top level fields to be accessed normally via standard item access, and also provides a convenient way to iterate over the sub-item list in the DATA key:

>>> print("Path:", r["path"])
>>> # Equivalent to: for item in r["DATA"]
>>> for item in r:
>>>     print(item["name"], item["type"])
class globus_sdk.transfer.response.TransferResponse(http_response, client=None)[source]

Bases: globus_sdk.response.GlobusHTTPResponse

Base class for TransferClient responses.

PaginatedResource Responses

The PaginatedResource class should not typically be instantiated directly, but is returned from several TransferClient methods. It is an iterable of GlobusRepsonse objects.

class globus_sdk.transfer.paging.PaginatedResource(client_method, path, client_kwargs, num_results=None, max_results_per_call=1000, max_total_results=None, offset=0, paging_style=0)[source]

Bases: globus_sdk.response.GlobusResponse, object

A PaginatedResource is an iterable response which implements the Python iterator interface. As such, you can only iterate over PaginatedResources once. Future iterations will be empty.

If you need fresh results, make a call for a new PaginatedResource, and if you want to cache and reuse results, convert to a list or other structure. You may also want to read the docs on the data property.

Because paginated data can be large, you will tend to get the best performance by being sure to only iterate over the results once.

property data

To get the “data” on a PaginatedResource, fetch all pages and convert them into the only python data structure that makes sense: a list.

Note that this forces iteration/evaluation of all pages from the API. It therefore may cause singificant IO spikes when used. You should avoid using the PaginatedResource.data property whenever possible.