Setting a Relative Deadline for a Transfer¶

When transferring or deleting data via Globus Transfer, users are able to set a deadline for their tasks. This allows you to declare a time by which the task must be completed – if the deadline is reached and the task is still in progress, it will be cancelled.

The deadline field in a Transfer task takes a date and time as a string, with support for common ISO 8601 format. Because of the use of standard formats, it is easy to use the Python datetime module to compute a relative deadline at some point in the future. You can use this to easily submit tasks with deadlines limited to the next minute, hour, or day. You can use this, for example, to enforce that a Transfer Task which takes too long results in errors (even if it is making slow progress).

Note

Not all ISO 8601 syntaxes are supported. We recommend sticking to the restrictive subset defined by RFC 3339, in which date-time data is typically formatted as “YYYY-MM-DD’T’HH:mm:SSZ”.

datetime.isoformat() follows this format and is therefore a great choice for Python programmers!

For readers who prefer to start with complete working examples, jump ahead to the example script.

Computing and Formatting a Deadline¶

We need to compute a relative deadline – some point into the future – and format it as a string. We’ll express that idea in a function which takes a datetime.timedelta as an offset, an amount of time into the future. This gives us a generic phrasing of getting a future date:

import datetime


def make_relative_deadline(offset: datetime.timedelta) -> str:
    now = datetime.datetime.now(tz=datetime.timezone.utc)
    deadline = now + offset
    return deadline.isoformat()

We can then see that this works by testing it out:

>>> make_relative_deadline(datetime.timedelta(minutes=10))
'2003-09-21T18:58:09.279314+00:00'

Creating a Task with the Deadline¶

deadline is an initialization parameter to TransferData and DeleteData. Along with all of our other parameters to create the Transfer Task, here’s a sample task document with a deadline set for “an hour from now”:

# Globus Tutorial Collection 1
# https://app.globus.org/file-manager/collections/6c54cade-bde5-45c1-bdea-f4bd71dba2cc
SRC_COLLECTION = "6c54cade-bde5-45c1-bdea-f4bd71dba2cc"
SRC_PATH = "/share/godata/file1.txt"

# Globus Tutorial Collection 2
# https://app.globus.org/file-manager/collections/31ce9ba0-176d-45a5-add3-f37d233ba47d
DST_COLLECTION = "31ce9ba0-176d-45a5-add3-f37d233ba47d"
DST_PATH = "/~/example-transfer-script-destination.txt"

# create a Transfer Task request document, including a relative deadline
transfer_request = globus_sdk.TransferData(
    source_endpoint=SRC_COLLECTION,
    destination_endpoint=DST_COLLECTION,
    deadline=make_relative_deadline(datetime.timedelta(hours=1)),
)
transfer_request.add_item(SRC_PATH, DST_PATH)

This is then valid to submit with a TransferClient:

tc = globus_sdk.TransferClient(...)

tc.submit_transfer(transfer_request)

Summary: Complete Example¶

For a complete example script, we will need to also include a GlobusApp for login, so that we can setup the TransferClient correctly. We’ll also take a small step to make the script work with mapped collections which require the data_access scope, like the tutorial collections. With those small additions, the above examples can be turned into a working script!

This example is complete. It should run without errors “as is”.

submit_transfer_relative_deadline.py [download]¶

import datetime

import globus_sdk
from globus_sdk.globus_app import UserApp

# Tutorial Client ID - <replace this with your own client>
NATIVE_CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
USER_APP = UserApp("relative-deadline-transfer", client_id=NATIVE_CLIENT_ID)

# Globus Tutorial Collection 1
# https://app.globus.org/file-manager/collections/6c54cade-bde5-45c1-bdea-f4bd71dba2cc
SRC_COLLECTION = "6c54cade-bde5-45c1-bdea-f4bd71dba2cc"
SRC_PATH = "/home/share/godata/file1.txt"

# Globus Tutorial Collection 2
# https://app.globus.org/file-manager/collections/31ce9ba0-176d-45a5-add3-f37d233ba47d
DST_COLLECTION = "31ce9ba0-176d-45a5-add3-f37d233ba47d"
DST_PATH = "/~/example-transfer-script-destination.txt"


def make_relative_deadline(offset: datetime.timedelta) -> str:
    now = datetime.datetime.now(tz=datetime.timezone.utc)
    deadline = now + offset
    return deadline.isoformat()


transfer_client = globus_sdk.TransferClient(app=USER_APP)

# Comment out each of these lines if the referenced collection is either
#   (1) A guest collection or (2) high assurance.
transfer_client.add_app_data_access_scope(SRC_COLLECTION)
transfer_client.add_app_data_access_scope(DST_COLLECTION)

transfer_request = globus_sdk.TransferData(
    source_endpoint=SRC_COLLECTION,
    destination_endpoint=DST_COLLECTION,
    deadline=make_relative_deadline(datetime.timedelta(hours=1)),
)
transfer_request.add_item(SRC_PATH, DST_PATH)

task = transfer_client.submit_transfer(transfer_request)
print(f"Submitted transfer. Task ID: {task['task_id']}")