Globus Transfer¶
Client¶
The primary interface for the Globus Transfer API is the TransferClient
class.
-
class
globus_sdk.
TransferClient
(authorizer=None, **kwargs)[source]¶ Bases:
globus_sdk.base.BaseClient
Client for the Globus Transfer API.
This class provides helper methods for most common resources in the REST API, and basic
get
,put
,post
, anddelete
methods from the base rest client that can be used to access any REST resource.There are two types of helper methods: list methods which return an iterator of
GlobusResponse
objects, and simple methods that return a singleTransferResponse
object.Some calls are paginated. If a call returns a
PaginatedResource
object, the result is an iterator which can only be walked once. If you need to do multiple passes over the result, calllist()
on thePaginatedResource
or call the original method again to get fresh results.Detailed documentation is available in the official REST API documentation, which is linked to from the method documentation. Methods that allow arbitrary keyword arguments will pass the extra arguments as query parameters.
- Parameters
authorizer (
GlobusAuthorizer
) – An authorizer instance used for all calls to Globus Transfer
Methods
-
get_endpoint
(endpoint_id, **params)[source]¶ GET /endpoint/<endpoint_id>
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> endpoint = tc.get_endpoint(endpoint_id) >>> print("Endpoint name:", >>> endpoint["display_name"] or endpoint["canonical_name"])
External Documentation
See Get Endpoint by ID in the REST documentation for details.
-
update_endpoint
(endpoint_id, data, **params)[source]¶ PUT /endpoint/<endpoint_id>
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> epup = dict(display_name="My New Endpoint Name", >>> description="Better Description") >>> update_result = tc.update_endpoint(endpoint_id, epup)
External Documentation
See Update Endpoint by ID in the REST documentation for details.
-
create_endpoint
(data)[source]¶ POST /endpoint/<endpoint_id>
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> ep_data = { >>> "DATA_TYPE": "endpoint", >>> "display_name": display_name, >>> "DATA": [ >>> { >>> "DATA_TYPE": "server", >>> "hostname": "gridftp.example.edu", >>> }, >>> ], >>> } >>> create_result = tc.create_endpoint(ep_data) >>> endpoint_id = create_result["id"]
External Documentation
See Create endpoint in the REST documentation for details.
-
delete_endpoint
(endpoint_id)[source]¶ DELETE /endpoint/<endpoint_id>
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> delete_result = tc.delete_endpoint(endpoint_id)
External Documentation
See Delete endpoint by id in the REST documentation for details.
-
endpoint_search
(filter_fulltext=None, filter_scope=None, num_results=25, **params)[source]¶ GET /endpoint_search?filter_fulltext=<filter_fulltext>&filter_scope=<filter_scope>
- Parameters
filter_fulltext (str, optional) – The string to use in a full text search on endpoints. Effectively, the “search query” which is being requested. May be omitted with specific
filter_scope
values.filter_scope (str, optional) – A “scope” within which to search for endpoints. This must be one of the limited and known names known to the service, which can be found documented in the External Documentation below. Defaults to searching all endpoints (in which case
filter_fulltext
is required)num_results (int or None) – The number of search results to fetch from the service. May be set to
None
to request the maximum allowable number of results. [Default:25
]params (dict) – Any additional parameters will be passed through as query params.
- Return type
PaginatedResource
, an iterable ofGlobusResponse
Examples
Search for a given string as a fulltext search:
>>> tc = globus_sdk.TransferClient(...) >>> for ep in tc.endpoint_search('String to search for!'): >>> print(ep['display_name'])
Search for a given string, but only on endpoints that you own:
>>> for ep in tc.endpoint_search('foo', filter_scope='my-endpoints'): >>> print('{0} has ID {1}'.format(ep['display_name'], ep['id']))
Search results are capped at a number of elements equal to the
num_results
parameter. If you want more than the default, 25, elements, do like so:>>> for ep in tc.endpoint_search('String to search for!', >>> num_results=120): >>> print(ep['display_name'])
It is important to be aware that the Endpoint Search API limits you to 1000 results for any search query. You can request the maximum number of results either explicitly, with
num_results=1000
, or by stating that you want no limit by setting it toNone
:>>> for ep in tc.endpoint_search('String to search for!', >>> num_results=None): >>> print(ep['display_name'])
External Documentation
For additional information, see Endpoint Search. in the REST documentation for details.
-
endpoint_autoactivate
(endpoint_id, **params)[source]¶ POST /endpoint/<endpoint_id>/autoactivate
- Return type
The following example will try to “auto” activate the endpoint using a credential available from another endpoint or sign in by the user with the same identity provider, but only if the endpoint is not already activated or going to expire within an hour (3600 seconds). If that fails, direct the user to the globus website to perform activation:
Examples
>>> tc = globus_sdk.TransferClient(...) >>> r = tc.endpoint_autoactivate(ep_id, if_expires_in=3600) >>> while (r["code"] == "AutoActivationFailed"): >>> print("Endpoint requires manual activation, please open " >>> "the following URL in a browser to activate the " >>> "endpoint:") >>> print("https://app.globus.org/file-manager?origin_id=%s" >>> % ep_id) >>> # For python 2.X, use raw_input() instead >>> input("Press ENTER after activating the endpoint:") >>> r = tc.endpoint_autoactivate(ep_id, if_expires_in=3600)
This is the recommended flow for most thick client applications, because many endpoints require activation via OAuth MyProxy, which must be done in a browser anyway. Web based clients can link directly to the URL.
You also might want messaging or logging depending on why and how the operation succeeded, in which case you’ll need to look at the value of the “code” field and either decide on your own messaging or use the response’s “message” field.
>>> tc = globus_sdk.TransferClient(...) >>> r = tc.endpoint_autoactivate(ep_id, if_expires_in=3600) >>> if r['code'] == 'AutoActivationFailed': >>> print('Endpoint({}) Not Active! Error! Source message: {}' >>> .format(ep_id, r['message'])) >>> sys.exit(1) >>> elif r['code'] == 'AutoActivated.CachedCredential': >>> print('Endpoint({}) autoactivated using a cached credential.' >>> .format(ep_id)) >>> elif r['code'] == 'AutoActivated.GlobusOnlineCredential': >>> print(('Endpoint({}) autoactivated using a built-in Globus ' >>> 'credential.').format(ep_id)) >>> elif r['code'] = 'AlreadyActivated': >>> print('Endpoint({}) already active until at least {}' >>> .format(ep_id, 3600))
External Documentation
See Autoactivate endpoint in the REST documentation for details.
-
endpoint_deactivate
(endpoint_id, **params)[source]¶ POST /endpoint/<endpoint_id>/deactivate
- Return type
External Documentation
See Deactive endpoint in the REST documentation for details.
-
endpoint_activate
(endpoint_id, requirements_data, **params)[source]¶ POST /endpoint/<endpoint_id>/activate
- Return type
Consider using autoactivate and web activation instead, described in the example for
endpoint_autoactivate()
.External Documentation
See Activate endpoint in the REST documentation for details.
-
endpoint_get_activation_requirements
(endpoint_id, **params)[source]¶ GET /endpoint/<endpoint_id>/activation_requirements
- Return type
External Documentation
See Get activation requirements in the REST documentation for details.
-
my_effective_pause_rule_list
(endpoint_id, **params)[source]¶ GET /endpoint/<endpoint_id>/my_effective_pause_rule_list
- Return type
External Documentation
See Get my effective endpoint pause rules in the REST documentation for details.
GET /endpoint/<endpoint_id>/my_shared_endpoint_list
- Return type
External Documentation
See Get shared endpoint list in the REST documentation for details.
POST /shared_endpoint
- Parameters
data (dict) – A python dict representation of a
shared_endpoint
document- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> shared_ep_data = { >>> "DATA_TYPE": "shared_endpoint", >>> "host_endpoint": host_endpoint_id, >>> "host_path": host_path, >>> "display_name": display_name, >>> # optionally specify additional endpoint fields >>> "description": "my test share" >>> } >>> create_result = tc.create_shared_endpoint(shared_ep_data) >>> endpoint_id = create_result["id"]
External Documentation
See Create shared endpoint in the REST documentation for details.
-
endpoint_server_list
(endpoint_id, **params)[source]¶ GET /endpoint/<endpoint_id>/server_list
- Return type
External Documentation
See Get endpoint server list in the REST documentation for details.
-
get_endpoint_server
(endpoint_id, server_id, **params)[source]¶ GET /endpoint/<endpoint_id>/server/<server_id>
- Return type
External Documentation
See Get endpoint server by id in the REST documentation for details.
-
add_endpoint_server
(endpoint_id, server_data)[source]¶ POST /endpoint/<endpoint_id>/server
- Return type
External Documentation
See Add endpoint server in the REST documentation for details.
-
update_endpoint_server
(endpoint_id, server_id, server_data)[source]¶ PUT /endpoint/<endpoint_id>/server/<server_id>
- Return type
External Documentation
See Update endpoint server by id in the REST documentation for details.
-
delete_endpoint_server
(endpoint_id, server_id)[source]¶ DELETE /endpoint/<endpoint_id>/server/<server_id>
- Return type
External Documentation
See Delete endpoint server by id in the REST documentation for details.
-
endpoint_role_list
(endpoint_id, **params)[source]¶ GET /endpoint/<endpoint_id>/role_list
- Return type
External Documentation
See Get list of endpoint roles in the REST documentation for details.
-
add_endpoint_role
(endpoint_id, role_data)[source]¶ POST /endpoint/<endpoint_id>/role
- Return type
External Documentation
See Create endpoint role in the REST documentation for details.
-
get_endpoint_role
(endpoint_id, role_id, **params)[source]¶ GET /endpoint/<endpoint_id>/role/<role_id>
- Return type
External Documentation
See Get endpoint role by id in the REST documentation for details.
-
delete_endpoint_role
(endpoint_id, role_id)[source]¶ DELETE /endpoint/<endpoint_id>/role/<role_id>
- Return type
External Documentation
See Delete endpoint role by id in the REST documentation for details.
-
endpoint_acl_list
(endpoint_id, **params)[source]¶ GET /endpoint/<endpoint_id>/access_list
- Return type
External Documentation
See Get list of access rules in the REST documentation for details.
-
get_endpoint_acl_rule
(endpoint_id, rule_id, **params)[source]¶ GET /endpoint/<endpoint_id>/access/<rule_id>
- Return type
External Documentation
See Get access rule by id in the REST documentation for details.
-
add_endpoint_acl_rule
(endpoint_id, rule_data)[source]¶ POST /endpoint/<endpoint_id>/access
- Parameters
endpoint_id (str) – ID of endpoint to which to add the acl
rule_data (dict) – A python dict representation of an
access
document
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> rule_data = { >>> "DATA_TYPE": "access", >>> "principal_type": "identity", >>> "principal": identity_id, >>> "path": "/dataset1/", >>> "permissions": "rw", >>> } >>> result = tc.add_endpoint_acl_rule(endpoint_id, rule_data) >>> rule_id = result["access_id"]
Note that if this rule is being created on a shared endpoint the “path” field is relative to the “host_path” of the shared endpoint.
External Documentation
See Create access rule in the REST documentation for details.
-
update_endpoint_acl_rule
(endpoint_id, rule_id, rule_data)[source]¶ PUT /endpoint/<endpoint_id>/access/<rule_id>
- Return type
External Documentation
See Update access rule in the REST documentation for details.
-
delete_endpoint_acl_rule
(endpoint_id, rule_id)[source]¶ DELETE /endpoint/<endpoint_id>/access/<rule_id>
- Return type
External Documentation
See Delete access rule in the REST documentation for details.
-
bookmark_list
(**params)[source]¶ GET /bookmark_list
- Return type
External Documentation
See Get list of bookmarks in the REST documentation for details.
-
create_bookmark
(bookmark_data)[source]¶ POST /bookmark
- Return type
External Documentation
See Create bookmark in the REST documentation for details.
-
get_bookmark
(bookmark_id, **params)[source]¶ GET /bookmark/<bookmark_id>
- Return type
External Documentation
See Get bookmark by id in the REST documentation for details.
-
update_bookmark
(bookmark_id, bookmark_data)[source]¶ PUT /bookmark/<bookmark_id>
- Return type
External Documentation
See Update bookmark in the REST documentation for details.
-
delete_bookmark
(bookmark_id)[source]¶ DELETE /bookmark/<bookmark_id>
- Return type
External Documentation
See Delete bookmark by id in the REST documentation for details.
-
operation_ls
(endpoint_id, **params)[source]¶ GET /operation/endpoint/<endpoint_id>/ls
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> for entry in tc.operation_ls(ep_id, path="/~/project1/"): >>> print(entry["name"], entry["type"])
External Documentation
See List Directory Contents in the REST documentation for details.
-
operation_mkdir
(endpoint_id, path, **params)[source]¶ POST /operation/endpoint/<endpoint_id>/mkdir
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> tc.operation_mkdir(ep_id, path="/~/newdir/")
External Documentation
See Make Directory in the REST documentation for details.
-
operation_rename
(endpoint_id, oldpath, newpath, **params)[source]¶ POST /operation/endpoint/<endpoint_id>/rename
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> tc.operation_rename(ep_id, oldpath="/~/file1.txt", >>> newpath="/~/project1data.txt")
External Documentation
See Rename in the REST documentation for details.
-
operation_symlink
(endpoint_id, symlink_target, path, **params)[source]¶ POST /operation/endpoint/<endpoint_id>/symlink
- Return type
The
path
is the name of the symlink, and thesymlink_target
is the path referenced by the symlink.Examples
>>> tc = globus_sdk.TransferClient(...) >>> tc.operation_symlink(ep_id, symlink_target="/~/file1.txt", >>> path="/~/link-to-file1.txt")
External Documentation
See Symlink in the REST documentation for details.
-
get_submission_id
(**params)[source]¶ GET /submission_id
- Return type
Submission IDs are required to submit tasks to the Transfer service via the
submit_transfer
andsubmit_delete
methods.Most users will not need to call this method directly, as the convenience classes
TransferData
andDeleteData
will call it automatically if they are not passed asubmission_id
explicitly.External Documentation
See Get a submission id in the REST documentation for more details.
-
submit_transfer
(data)[source]¶ POST /transfer
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> tdata = globus_sdk.TransferData(tc, source_endpoint_id, >>> destination_endpoint_id, >>> label="SDK example", >>> sync_level="checksum") >>> tdata.add_item("/source/path/dir/", "/dest/path/dir/", >>> recursive=True) >>> tdata.add_item("/source/path/file.txt", >>> "/dest/path/file.txt") >>> transfer_result = tc.submit_transfer(tdata) >>> print("task_id =", transfer_result["task_id"])
The data parameter can be a normal Python dictionary, or a
TransferData
object.External Documentation
See Submit a transfer task in the REST documentation for more details.
-
submit_delete
(data)[source]¶ POST /delete
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> ddata = globus_sdk.DeleteData(tc, endpoint_id, recursive=True) >>> ddata.add_item("/dir/to/delete/") >>> ddata.add_item("/file/to/delete/file.txt") >>> delete_result = tc.submit_delete(ddata) >>> print("task_id =", delete_result["task_id"])
The data parameter can be a normal Python dictionary, or a
DeleteData
object.External Documentation
See Submit a delete task in the REST documentation for details.
-
task_list
(num_results=10, **params)[source]¶ Get an iterable of task documents owned by the current user.
GET /task_list
- Parameters
num_results (int or none) – The number of tasks to fetch from the service. May be set to
None
to request the maximum allowable number of results. [Default:10
]params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
PaginatedResource
, an iterable ofGlobusResponse
Examples
Fetch the default number (10) of tasks and print some basic info:
>>> tc = TransferClient(...) >>> for task in tc.task_list(): >>> print("Task({}): {} -> {}".format( >>> task["task_id"], task["source_endpoint"], >>> task["destination_endpoint"]))
External Documentation
See Task list in the REST documentation for details.
-
task_event_list
(task_id, num_results=10, **params)[source]¶ List events (for example, faults and errors) for a given Task.
GET /task/<task_id>/event_list
- Parameters
task_id (str) – The ID of the task to inspect.
num_results (int or None) – The number of events to fetch from the service. May be set to
None
to request the maximum allowable number of results. [Default:10
]params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
PaginatedResource
, an iterable ofGlobusResponse
Examples
Fetch the default number (10) of events and print some basic info:
>>> tc = TransferClient(...) >>> task_id = ... >>> for event in tc.task_event_list(task_id): >>> print("Event on Task({}) at {}:\n{}".format( >>> task_id, event["time"], event["description"])
External Documentation
See Get event list in the REST documentation for details.
-
get_task
(task_id, **params)[source]¶ GET /task/<task_id>
- Return type
External Documentation
See Get task by id in the REST documentation for details.
-
update_task
(task_id, data, **params)[source]¶ PUT /task/<task_id>
- Return type
External Documentation
See Update task by id in the REST documentation for details.
-
cancel_task
(task_id)[source]¶ POST /task/<task_id>/cancel
- Return type
External Documentation
See Cancel task by id in the REST documentation for details.
-
task_wait
(task_id, timeout=10, polling_interval=10)[source]¶ Wait until a Task is complete or fails, with a time limit. If the task is “ACTIVE” after time runs out, returns
False
. Otherwise returnsTrue
.- Parameters
task_id (str) – ID of the Task to wait on for completion
timeout (int, optional) – Number of seconds to wait in total. Minimum 1. [Default:
10
]polling_interval (int, optional) – Number of seconds between queries to Globus about the Task status. Minimum 1. [Default:
10
]
Examples
If you want to wait for a task to terminate, but want to warn every minute that it doesn’t terminate, you could:
>>> tc = TransferClient(...) >>> while not tc.task_wait(task_id, timeout=60): >>> print("Another minute went by without {0} terminating" >>> .format(task_id))
Or perhaps you want to check on a task every minute for 10 minutes, and give up if it doesn’t complete in that time:
>>> tc = TransferClient(...) >>> done = tc.task_wait(task_id, timeout=600, polling_interval=60): >>> if not done: >>> print("{0} didn't successfully terminate!" >>> .format(task_id)) >>> else: >>> print("{0} completed".format(task_id))
You could print dots while you wait for a task by only waiting one second at a time:
>>> tc = TransferClient(...) >>> while not tc.task_wait(task_id, timeout=1, polling_interval=1): >>> print(".", end="") >>> print("\n{0} completed!".format(task_id))
-
task_pause_info
(task_id, **params)[source]¶ GET /task/<task_id>/pause_info
- Return type
External Documentation
See Get task pause info in the REST documentation for details.
-
task_successful_transfers
(task_id, num_results=100, **params)[source]¶ Get the successful file transfers for a completed Task.
Note
Only files that were actually transferred are included. This does not include directories, files that were checked but skipped as part of a sync transfer, or files which were skipped due to skip_source_errors being set on the task.
GET /task/<task_id>/successful_transfers
- Parameters
task_id (str) – The ID of the task to inspect.
num_results (int or None, optional) – The number of file transfer records to fetch from the service. May be set to
None
to request the maximum allowable number of results. [Default:100
]params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
PaginatedResource
, an iterable ofGlobusResponse
Examples
Fetch all transferred files for a task and print some basic info:
>>> tc = TransferClient(...) >>> task_id = ... >>> for info in tc.task_successful_transfers(task_id): >>> print("{} -> {}".format( >>> info["source_path"], info["destination_path"]))
External Documentation
See Get Task Successful Transfers in the REST documentation for details.
-
task_skipped_errors
(task_id, num_results=100, **params)[source]¶ Get path and error information for all paths that were skipped due to skip_source_errors being set on a completed transfer Task.
GET /task/<task_id>/skipped_errors
- Parameters
task_id (str) – The ID of the task to inspect.
num_results (int or None, optional) – The number of file transfer records to fetch from the service. May be set to
None
to request the maximum allowable number of results. [Default:100
]params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
PaginatedResource
, an iterable ofGlobusResponse
Examples
Fetch all skipped errors for a task and print some basic info:
>>> tc = TransferClient(...) >>> task_id = ... >>> for info in tc.task_skipped_errors(task_id): >>> print("{} -> {}".format( >>> info["error_code"], info["source_path"]))
External Documentation
See Get Task Skipped Errors in the REST documentation for details.
-
endpoint_manager_monitored_endpoints
(**params)[source]¶ Get endpoints the current user is a monitor or manager on.
GET endpoint_manager/monitored_endpoints
- Return type
iterable of
GlobusResponse
See Get monitored endpoints in the REST documentation for details.
-
endpoint_manager_hosted_endpoint_list
(endpoint_id, **params)[source]¶ Get shared endpoints hosted on the given endpoint.
GET /endpoint_manager/endpoint/<endpoint_id>/hosted_endpoint_list
- Return type
iterable of
GlobusResponse
See Get hosted endpoint list in the REST documentation for details.
-
endpoint_manager_get_endpoint
(endpoint_id, **params)[source]¶ Get endpoint details as an admin.
GET /endpoint_manager/endpoint/<endpoint_id>
- Return type
External Documentation
See Get endpoint as admin in the REST documentation for details.
-
endpoint_manager_acl_list
(endpoint_id, **params)[source]¶ Get a list of access control rules on specified endpoint as an admin.
GET endpoint_manager/endpoint/<endpoint_id>/access_list
- Return type
External Documentation
See Get endpoint access list as admin in the REST documentation for details.
-
endpoint_manager_task_list
(num_results=10, **params)[source]¶ Get a list of tasks visible via
activity_monitor
role, as opposed to tasks owned by the current user.GET endpoint_manager/task_list
- Parameters
num_results (int or None, optional) – The number of tasks to fetch from the service. May be set to
None
to request the maximum allowable number of results. [Default:10
]params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
PaginatedResource
, an iterable ofGlobusResponse
Examples
Fetch the default number (10) of tasks and print some basic info:
>>> tc = TransferClient(...) >>> for task in tc.endpoint_manager_task_list(): >>> print("Task({}): {} -> {}\n was submitted by\n {}".format( >>> task["task_id"], task["source_endpoint"], >>> task["destination_endpoint"], task["owner_string"]))
Do that same operation on all tasks visible via
activity_monitor
status:>>> tc = TransferClient(...) >>> for task in tc.endpoint_manager_task_list(num_results=None): >>> print("Task({}): {} -> {}\n was submitted by\n {}".format( >>> task["task_id"], task["source_endpoint"], >>> task["destination_endpoint"], task["owner_string"]))
External Documentation
See Advanced Endpoint Management: Get tasks in the REST documentation for details.
-
endpoint_manager_get_task
(task_id, **params)[source]¶ Get task info as an admin. Requires activity monitor effective role on the destination endpoint of the task.
GET /endpoint_manager/task/<task_id>
- Return type
External Documentation
See Get task as admin in the REST documentation for details.
-
endpoint_manager_task_event_list
(task_id, num_results=10, **params)[source]¶ List events (for example, faults and errors) for a given task as an admin. Requires activity monitor effective role on the destination endpoint of the task.
GET /task/<task_id>/event_list
- Parameters
task_id (str) – The ID of the task to inspect.
num_results (int or None, optional) – The number of events to fetch from the service. May be set to
None
to request the maximum allowable number of results. [Default:10
]params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
PaginatedResource
, an iterable ofGlobusResponse
External Documentation
See Get task events as admin in the REST documentation for details.
-
endpoint_manager_task_pause_info
(task_id, **params)[source]¶ Get details about why a task is paused as an admin. Requires activity monitor effective role on the destination endpoint of the task.
GET /endpoint_manager/task/<task_id>/pause_info
- Return type
External Documentation
See Get task pause info as admin in the REST documentation for details.
-
endpoint_manager_task_successful_transfers
(task_id, num_results=100, **params)[source]¶ Get the successful file transfers for a completed Task as an admin.
GET /endpoint_manager/task/<task_id>/successful_transfers
- Parameters
task_id (str) – The ID of the task to inspect.
num_results (int or None, optional) – The number of file transfer records to fetch from the service. May be set to
None
to request the maximum allowable number of results. [Default:100
]params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
PaginatedResource
, an iterable ofGlobusResponse
External Documentation
See Get task successful transfers as admin in the REST documentation for details.
-
endpoint_manager_task_skipped_errors
(task_id, num_results=100, **params)[source]¶ Get skipped errors for a completed Task as an admin.
GET /endpoint_manager/task/<task_id>/skipped_errors
- Parameters
task_id (str) – The ID of the task to inspect.
num_results (int or None, optional) – The number of skipped error records to fetch from the service. May be set to
None
to request the maximum allowable number of results. [Default:100
]params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
PaginatedResource
, an iterable ofGlobusResponse
External Documentation
See Get task skipped errors as admin in the REST documentation for details.
-
endpoint_manager_cancel_tasks
(task_ids, message, **params)[source]¶ Cancel a list of tasks as an admin. Requires activity manager effective role on the task(s) source or destination endpoint(s).
POST /endpoint_manager/admin_cancel
- Parameters
task_ids (iterable of str) – List of task ids to cancel.
message (str) – Message given to all users who’s tasks have been canceled.
params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
External Documentation
See Cancel tasks as admin in the REST documentation for details.
-
endpoint_manager_cancel_status
(admin_cancel_id, **params)[source]¶ Get the status of an an admin cancel (result of endpoint_manager_cancel_tasks).
GET /endpoint_manager/admin_cancel/<admin_cancel_id>
- Parameters
admin_cancel_id (str) – The ID of the the cancel job to inspect.
params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
External Documentation
See Get cancel status by id in the REST documentation for details.
-
endpoint_manager_pause_tasks
(task_ids, message, **params)[source]¶ Pause a list of tasks as an admin. Requires activity manager effective role on the task(s) source or destination endpoint(s).
POST /endpoint_manager/admin_pause
- Parameters
task_ids (iterable of str) – List of task ids to pause.
message (str) – Message given to all users who’s tasks have been paused.
params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
External Documentation
See Pause tasks as admin in the REST documentation for details.
-
endpoint_manager_resume_tasks
(task_ids, **params)[source]¶ Resume a list of tasks as an admin. Requires activity manager effective role on the task(s) source or destination endpoint(s).
POST /endpoint_manager/admin_resume
- Parameters
task_ids (iterable of str) – List of task ids to resume.
params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
External Documentation
See Resume tasks as admin in the REST documentation for details.
-
endpoint_manager_pause_rule_list
(filter_endpoint=None, **params)[source]¶ Get a list of pause rules on endpoints that the current user has the activity monitor effective role on.
GET /endpoint_manager/pause_rule_list
- Parameters
filter_endpoint (str) – An endpoint ID. Limit results to rules on endpoints hosted by this endpoint. Must be activity monitor on this endpoint, not just the hosted endpoints.
params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
External Documentation
See Get pause rules in the REST documentation for details.
-
endpoint_manager_create_pause_rule
(data)[source]¶ Create a new pause rule. Requires the activity manager effective role on the endpoint defined in the rule.
POST /endpoint_manager/pause_rule
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> rule_data = { >>> "DATA_TYPE": "pause_rule", >>> "message": "Message to users explaining why tasks are paused", >>> "endpoint_id": "339abc22-aab3-4b45-bb56-8d40535bfd80", >>> "identity_id": None, # affect all users on endpoint >>> "start_time": None # start now >>> } >>> create_result = tc.endpoint_manager_create_pause_rule(ep_data) >>> rule_id = create_result["id"]
External Documentation
See Create pause rule in the REST documentation for details.
-
endpoint_manager_get_pause_rule
(pause_rule_id, **params)[source]¶ Get an existing pause rule by ID. Requires the activity manager effective role on the endpoint defined in the rule.
GET /endpoint_manager/pause_rule/<pause_rule_id>
- Parameters
pause_rule_id (str) – ID of pause rule to get.
params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
External Documentation
See Get pause rule in the REST documentation for details.
-
endpoint_manager_update_pause_rule
(pause_rule_id, data)[source]¶ Update an existing pause rule by ID. Requires the activity manager effective role on the endpoint defined in the rule. Note that non update-able fields in data will be ignored.
PUT /endpoint_manager/pause_rule/<pause_rule_id>
- Return type
Examples
>>> tc = globus_sdk.TransferClient(...) >>> rule_data = { >>> "message": "Update to pause, reads are now allowed.", >>> "pause_ls": False, >>> "pause_task_transfer_read": False >>> } >>> update_result = tc.endpoint_manager_update_pause_rule(ep_data)
External Documentation
See Update pause rule in the REST documentation for details.
-
endpoint_manager_delete_pause_rule
(pause_rule_id, **params)[source]¶ Delete an existing pause rule by ID. Requires the user to see the “editible” field of the rule as True. Any tasks affected by this rule will no longer be once it is deleted.
DELETE /endpoint_manager/pause_rule/<pause_rule_id>
- Parameters
pause_rule_id (str) – The ID of the pause rule to delete.
params (dict, optional) – Any additional parameters will be passed through as query params.
- Return type
External Documentation
See Delete pause rule in the REST documentation for details.
Helper Objects¶
These helper objects make it easier to correctly create data for consumption by
a TransferClient
.
-
class
globus_sdk.
TransferData
(transfer_client, source_endpoint, destination_endpoint, label=None, submission_id=None, sync_level=None, verify_checksum=False, preserve_timestamp=False, encrypt_data=False, deadline=None, skip_source_errors=False, fail_on_quota_errors=False, recursive_symlinks='ignore', **kwargs)[source]¶ Bases:
dict
Convenience class for constructing a transfer document, to use as the data parameter to
submit_transfer
.At least one item must be added using
add_item
.If
submission_id
isn’t passed, one will be fetched automatically. The submission ID can be pulled out of here to inspect, but the document can be used as-is multiple times over to retry a potential submission failure (so there shouldn’t be any need to inspect it).- Parameters
transfer_client (
TransferClient
) – ATransferClient
instance which will be used to get a submission ID if one is not supplied. Should be the same instance that is used to submit the transfer.source_endpoint (str) – The endpoint ID of the source endpoint
destination_endpoint (str) – The endpoint ID of the destination endpoint
label (str, optional) – A string label for the Task
submission_id (str, optional) – A submission ID value fetched via
get_submission_id
. Defaults to usingtransfer_client.get_submission_id
sync_level (int or str, optional) – The method used to compare items between the source and destination. One of
"exists"
,"size"
,"mtime"
, or"checksum"
See the section below on sync-level for an explanation of values.verify_checksum (bool, optional) – When true, after transfer verify that the source and destination file checksums match. If they don’t, re-transfer the entire file and keep trying until it succeeds. This will create CPU load on both the origin and destination of the transfer, and may even be a bottleneck if the network speed is high enough. [default:
False
]preserve_timestamp (bool, optional) – When true, Globus Transfer will attempt to set file timestamps on the destination to match those on the origin. [default:
False
]encrypt_data (bool, optional) – When true, all files will be TLS-protected during transfer. [default:
False
]deadline (str or datetime, optional) – An ISO-8601 timestamp (as a string) or a datetime object which defines a deadline for the transfer. At the deadline, even if the data transfer is not complete, the job will be canceled. We recommend ensuring that the timestamp is in UTC to avoid confusion and ambiguity. Examples of ISO-8601 timestamps include
2017-10-12 09:30Z
,2017-10-12 12:33:54+00:00
, and2017-10-12
recursive_symlinks (str) – Specify the behavior of recursive directory transfers when encountering symlinks. One of
"ignore"
,"keep"
, or"copy"
."ignore"
skips symlinks,"keep"
creates symlinks at the destination matching the source (without modifying the link path at all), and"copy"
follows symlinks on the source, failing if the link is invalid. [default:"ignore"
]skip_source_errors (bool, optional) – When true, source permission denied and file not found errors from the source endpoint will cause the offending path to be skipped. [default:
False
]fail_on_quota_errors (bool, optional) – When true, quota exceeded errors will cause the task to fail. [default:
False
]
Any additional parameters are fed into the dict being created verbatim.
Sync Levels
The values for
sync_level
are used to determine how comparisons are made between files found both on the source and the destination. When files match, no data transfer will occur.For compatibility, this can be an integer
0
,1
,2
, or3
in addition to the string values.The meanings are as follows:
value
behavior
0
,exists
Determine whether or not to transfer based on file existence. If the destination file is absent, do the transfer.
1
,size
Determine whether or not to transfer based on the size of the file. If destination file size does not match the source, do the transfer.
2
,mtime
Determine whether or not to transfer based on modification times. If source has a newer modififed time than the destination, do the transfer.
3
,checksum
Determine whether or not to transfer based on checksums of file contents. If source and destination contents differ, as determined by a checksum of their contents, do the transfer.
Examples
See the
submit_transfer
documentation for example usage.External Documentation
See the Task document definition and Transfer specific fields in the REST documentation for more details on Transfer Task documents.
Methods
-
add_item
(source_path, destination_path, recursive=False, external_checksum=None, checksum_algorithm=None, **params)[source]¶ Add a file or directory to be transfered. If the item is a symlink to a file or directory, the file or directory at the target of the symlink will be transfered.
Appends a transfer_item document to the DATA key of the transfer document.
Note
The full path to the destination file must be provided for file items. Parent directories of files are not allowed. See task submission documentation for more details.
- Parameters
source_path (str) – Path to the source directory or file to be transfered
destination_path (str) – Path to the source directory or file will be transfered to
recursive (bool) – Set to True if the target at source path is a directory
external_checksum (str, optional) – A checksum to verify source file integrity before the transfer and destination file integrity after the transfer. Cannot be used with directories. Assumed to be an MD5 checksum unless checksum_algorithm is also given.
checksum_algorithm (str, optional) – Specifies the checksum algorithm to be used when verify_checksum is True, sync_level is “checksum” or 3, or an external_checksum is given.
-
add_symlink_item
(source_path, destination_path)[source]¶ Add a symlink to be transfered as a symlink rather than as the target of the symlink.
Appends a transfer_symlink_item document to the DATA key of the transfer document.
- Parameters
source_path (str) – Path to the source symlink
destination_path (str) – Path to which the source symlink will be transfered
-
class
globus_sdk.
DeleteData
(transfer_client, endpoint, label=None, submission_id=None, recursive=False, deadline=None, **kwargs)[source]¶ Bases:
dict
Convenience class for constructing a delete document, to use as the data parameter to
submit_delete
.At least one item must be added using
add_item
.If
submission_id
isn’t passed, one will be fetched automatically. The submission ID can be pulled out of here to inspect, but the document can be used as-is multiple times over to retry a potential submission failure (so there shouldn’t be any need to inspect it).- Parameters
transfer_client (
TransferClient
) – ATransferClient
instance which will be used to get a submission ID if one is not supplied. Should be the same instance that is used to submit the deletion.endpoint (str) – The endpoint ID which is targeted by this deletion Task
label (str, optional) – A string label for the Task
submission_id (str, optional) – A submission ID value fetched via
get_submission_id
. Defaults to usingtransfer_client.get_submission_id
recursive (bool) – Recursively delete subdirectories on the target endpoint [default:
False
]deadline (str or datetime, optional) – An ISO-8601 timestamp (as a string) or a datetime object which defines a deadline for the deletion. At the deadline, even if the data deletion is not complete, the job will be canceled. We recommend ensuring that the timestamp is in UTC to avoid confusion and ambiguity. Examples of ISO-8601 timestamps include
2017-10-12 09:30Z
,2017-10-12 12:33:54+00:00
, and2017-10-12
Examples
See the
submit_delete
documentation for example usage.External Documentation
See the Task document definition and Delete specific fields in the REST documentation for more details on Delete Task documents.
Methods
Client Errors¶
When an error occurs, a TransferClient
will raise this specialized type of
error, rather than a generic GlobusAPIError
.
-
class
globus_sdk.exc.
TransferAPIError
(r)[source]¶ Bases:
globus_sdk.exc.GlobusAPIError
Error class for the Transfer API client. In addition to the inherited
code
andmessage
instance variables, provides:- Variables
request_id – Unique identifier for the request, which should be provided when contacting support@globus.org.
Transfer Responses¶
-
class
globus_sdk.transfer.response.
ActivationRequirementsResponse
(*args, **kwargs)[source]¶ Bases:
globus_sdk.transfer.response.base.TransferResponse
Response class for Activation Requirements responses.
All Activation Requirements documents refer to a specific Endpoint, from whence they were acquired. References to “the Endpoint” implicitly refer to that originating Endpoint, and not to some other Endpoint.
External Documentation
See Activation Requirements Document in the API documentation for details.
-
active_until
(time_seconds, relative_time=True)[source]¶ Check if the Endpoint will be active until some time in the future, given as an integer number of seconds. When
relative_time=False
, thetime_seconds
is interpreted as a POSIX timestamp.This supports queries using both relative and absolute timestamps to better support a wide range of use cases. For example, if I have a task that I know will typically take N seconds, and I want an M second safety margin:
>>> num_secs_allowed = N + M >>> tc = TransferClient(...) >>> reqs_doc = tc.endpoint_get_activation_requirements(...) >>> if not reqs_doc.active_until(num_secs_allowed): >>> raise Exception("Endpoint won't be active long enough") >>> ...
or, alternatively, if I know that the endpoint must be active until October 18th, 2016 for my tasks to complete:
>>> oct18_2016 = 1476803436 >>> tc = TransferClient(...) >>> reqs_doc = tc.endpoint_get_activation_requirements(...) >>> if not reqs_doc.active_until(oct18_2016, relative_time=False): >>> raise Exception("Endpoint won't be active long enough") >>> ...
- Parameters
time_seconds (int) – Number of seconds into the future.
relative_time (bool) – Defaults to True. When False,
time_seconds
is treated as a POSIX timestamp (i.e. seconds since epoch as an integer) instead of its ordinary behavior.
- Returns
True if the Endpoint will be active until the deadline, False otherwise
- Return type
bool
-
property
always_activated
¶ Returns True if the endpoint activation never expires (e.g. shared endpoints, globus connect personal endpoints).
- Return type
bool
-
property
supports_auto_activation
¶ Check if the document lists Auto-Activation as an available type of activation. Typically good to use when you need to catch endpoints that require web activation before proceeding.
>>> endpoint_id = "..." >>> tc = TransferClient(...) >>> reqs_doc = tc.endpoint_get_activation_requirements(endpoint_id) >>> if not reqs_doc.supports_auto_activation: >>> # use `from __future__ import print_function` in py2 >>> print(("This endpoint requires web activation. " >>> "Please login and activate the endpoint here:\n" >>> "https://app.globus.org/file-manager?origin_id={}") >>> .format(endpoint_id), file=sys.stderr) >>> # py3 calls it `input()` in py2, use `raw_input()` >>> input("Please Hit Enter When You Are Done")
- Return type
bool
-
property
supports_web_activation
¶ Check if the document lists known types of activation that can be done through the web. If this returns
False
, it means that the endpoint is of a highly unusual type, and you should directly inspect the response’sdata
attribute to see what is required. Sending users to the web page for activation is also a fairly safe action to take. Note thatActivationRequirementsResponse.supports_auto_activation
directly impliesActivationRequirementsResponse.supports_web_activation
, so these are not exclusive.For example,
>>> tc = TransferClient(...) >>> reqs_doc = tc.endpoint_get_activation_requirements(...) >>> if not reqs_doc.supports_web_activation: >>> # use `from __future__ import print_function` in py2 >>> print("Highly unusual endpoint. " + >>> "Cannot webactivate. Raw doc: " + >>> str(reqs_doc), file=sys.stderr) >>> print("Sending user to web anyway, just in case.", >>> file=sys.stderr) >>> ...
- Return type
bool
-
-
class
globus_sdk.transfer.response.
IterableTransferResponse
(http_response, client=None)[source]¶ Bases:
globus_sdk.transfer.response.base.TransferResponse
Response class for non-paged list oriented resources. Allows top level fields to be accessed normally via standard item access, and also provides a convenient way to iterate over the sub-item list in the
DATA
key:>>> print("Path:", r["path"]) >>> # Equivalent to: for item in r["DATA"] >>> for item in r: >>> print(item["name"], item["type"])
-
class
globus_sdk.transfer.response.
TransferResponse
(http_response, client=None)[source]¶ Bases:
globus_sdk.response.GlobusHTTPResponse
Base class for
TransferClient
responses.
PaginatedResource Responses¶
The PaginatedResource
class should not typically be instantiated directly,
but is returned from several TransferClient
methods.
It is an iterable of GlobusRepsonse
objects.
-
class
globus_sdk.transfer.paging.
PaginatedResource
(client_method, path, client_kwargs, num_results=None, max_results_per_call=1000, max_total_results=None, offset=0, paging_style=0)[source]¶ Bases:
globus_sdk.response.GlobusResponse
,object
A
PaginatedResource
is an iterable response which implements the Python iterator interface. As such, you can only iterate over PaginatedResources once. Future iterations will be empty.If you need fresh results, make a call for a new
PaginatedResource
, and if you want to cache and reuse results, convert to a list or other structure. You may also want to read the docs on thedata
property.Because paginated data can be large, you will tend to get the best performance by being sure to only iterate over the results once.
-
property
data
¶ To get the “data” on a PaginatedResource, fetch all pages and convert them into the only python data structure that makes sense: a list.
Note that this forces iteration/evaluation of all pages from the API. It therefore may cause singificant IO spikes when used. You should avoid using the
PaginatedResource.data
property whenever possible.
-
property