Transport Layer#

The transport consists of a transport object ( RequestsTransport), but also tooling for handling retries. It is possible to either register custom retry check methods, or to override the Transport used by a client in order to customize this behavior.

Transport#

class globus_sdk.transport.RequestsTransport(verify_ssl=None, http_timeout=None, retry_backoff=<function _exponential_backoff>, retry_checks=None, max_sleep=10, max_retries=None)[source]#

The RequestsTransport handles HTTP request sending and retries.

It receives raw request information from a client class, and then performs the following steps - encode the data in a prepared request - repeatedly send the request until no retry is requested - return the last response or reraise the last exception

Retry checks are registered as hooks on the Transport. Additional hooks can be passed to the constructor via retry_checks. Or hooks can be added to an existing transport via a decorator.

If the maximum number of retries is reached, the final response or exception will be returned or raised.

Parameters:

  • verify_ssl (bool | str | pathlib.Path | None) – Explicitly enable or disable SSL verification, or configure the path to a CA certificate bundle to use for SSL verification

  • http_timeout (float | None) – Explicitly set an HTTP timeout value in seconds. This parameter defaults to 60s but can be set via the GLOBUS_SDK_HTTP_TIMEOUT environment variable. Any value set via this parameter takes precedence over the environment variable.

  • retry_backoff (t.Callable[[RetryContext], float]) – A function which determines how long to sleep between calls based on the RetryContext. Defaults to exponential backoff with jitter based on the context attempt number.

  • retry_checks (list[RetryCheck] | None) – A list of initial retry checks. Any hooks registered, including the default hooks, will run after these checks.

  • max_sleep (float | int) – The maximum sleep time between retries (in seconds). If the computed sleep time or the backoff requested by a retry check exceeds this value, this amount of time will be used instead

  • max_retries (int | None) – The maximum number of retries allowed by this transport

DEFAULT_MAX_RETRIES = 5#

default maximum number of retries

RETRY_AFTER_STATUS_CODES: tuple[int, ...] = (429, 503)#

status codes for responses which may have a Retry-After header

TRANSIENT_ERROR_STATUS_CODES: tuple[int, ...] = (429, 500, 502, 503, 504)#

status codes for error responses which should generally be retried

EXPIRED_AUTHORIZATION_STATUS_CODES: tuple[int, ...] = (401,)#

status codes indicating that authorization info was missing or expired

encoders: dict[str, RequestEncoder] = {'form': <globus_sdk.transport.encoders.FormRequestEncoder object>, 'json': <globus_sdk.transport.encoders.JSONRequestEncoder object>, 'text': <globus_sdk.transport.encoders.RequestEncoder object>}#

the encoders are a mapping of encoding names to encoder objects

tune(*, verify_ssl=None, http_timeout=None, retry_backoff=None, max_sleep=None, max_retries=None)[source]#

Temporarily adjust some of the request sending settings of the transport. This method works as a context manager, and will reset settings to their original values after it exits.

In particular, this can be used to temporarily adjust request-sending minutiae like the http_timeout used.

Parameters:

  • verify_ssl (bool | str | Path | None) – Explicitly enable or disable SSL verification, or configure the path to a CA certificate bundle to use for SSL verification

  • http_timeout (float | None) – Explicitly set an HTTP timeout value in seconds

  • retry_backoff (Callable[[RetryContext], float] | None) – A function which determines how long to sleep between calls based on the RetryContext

  • max_sleep (float | int | None) – The maximum sleep time between retries (in seconds). If the computed sleep time or the backoff requested by a retry check exceeds this value, this amount of time will be used instead

  • max_retries (int | None) – The maximum number of retries allowed by this transport

Return type:

Iterator[None]

Examples

This can be used with any client class to temporarily set values in the context of one or more HTTP requests. To increase the HTTP request timeout from the default of 60 to 120 seconds,

>>> client = ...  # any client class
>>> with client.transport.tune(http_timeout=120):
>>>     foo = client.get_foo()

or to disable retries (note that this also disables the retry on expired-and-refreshed credentials):

>>> client = ...  # any client class
>>> with client.transport.tune(max_retries=0):
>>>     foo = client.get_foo()
request(method, url, query_params=None, data=None, headers=None, encoding=None, authorizer=None, allow_redirects=True, stream=False)[source]#

Send an HTTP request

Parameters:

  • url (str) – URL for the request

  • method (str) – HTTP request method, as an all caps string

  • query_params (dict[str, Any] | None) – Parameters to be encoded as a query string

  • headers (dict[str, str] | None) – HTTP headers to add to the request

  • data (dict[str, Any] | list[Any] | PayloadWrapper | str | bytes | None) – Data to send as the request body. May pass through encoding.

  • encoding (str | None) – A way to encode request data. “json”, “form”, and “text” are all valid values. Custom encodings can be used only if they are registered with the transport. By default, strings get “text” behavior and all other objects get “json”.

  • authorizer (GlobusAuthorizer | None) – The authorizer which is used to get or update authorization information for the request

  • allow_redirects (bool) – Follow Location headers on redirect response automatically. Defaults to True

  • stream (bool) – Do not immediately download the response content. Defaults to False

Returns:

requests.Response object

Return type:

Response

register_retry_check(func)[source]#

Register a retry check with this transport.

A retry checker is a callable responsible for implementing check(RetryContext) -> RetryCheckResult

check should not perform any sleeps or delays. Multiple checks should be chainable and callable in any order.

Parameters:

func (Callable[[RetryContext], RetryCheckResult]) – The function or other callable to register as a retry check

Return type:

Callable[[RetryContext], RetryCheckResult]

register_default_retry_checks()[source]#

This hook is called during transport initialization. By default, it registers the following hooks:

  • default_check_expired_authorization

  • default_check_request_exception

  • default_check_retry_after_header

  • default_check_transient_error

It can be overridden to register additional hooks or to remove the default hooks.

default_check_request_exception(ctx)[source]#

Check if a network error was encountered

Parameters:

ctx (RetryContext) – The context object which describes the state of the request and the retries which may already have been attempted.

Return type:

RetryCheckResult

default_check_retry_after_header(ctx)[source]#

Check for a retry-after header if the response had a matching status

Parameters:

ctx (RetryContext) – The context object which describes the state of the request and the retries which may already have been attempted.

Return type:

RetryCheckResult

default_check_transient_error(ctx)[source]#

Check for transient error status codes which could be resolved by retrying the request

Parameters:

ctx (RetryContext) – The context object which describes the state of the request and the retries which may already have been attempted.

Return type:

RetryCheckResult

default_check_expired_authorization(ctx)[source]#

This check evaluates whether or not there is invalid or expired authorization information which could be updated with some action – most typically a token refresh for an expired access token.

The check is flagged to only run once per request.

Parameters:

ctx (RetryContext) – The context object which describes the state of the request and the retries which may already have been attempted.

Return type:

RetryCheckResult

Retries#

These are the components used by the RequestsTransport to implement retry logic.

class globus_sdk.transport.RetryContext(attempt, *, authorizer=None, response=None, exception=None)[source]#

The RetryContext is an object passed to retry checks in order to determine whether or not a request should be retried. The context is constructed after each request, regardless of success or failure.

If an exception was raised, the context will contain that exception object. Otherwise, the context will contain a response object. Exactly one of response or exception will be present.

Parameters:

  • attempt (int) – The request attempt number, starting at 0.

  • response (requests.Response | None) – The response on a successful request

  • exception (Exception | None) – The error raised when trying to send the request

  • authorizer (GlobusAuthorizer | None) – The authorizer object from the client making the request

class globus_sdk.transport.RetryCheckResult(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#
do_retry = 1#

yes, retry the request

do_not_retry = 2#

no, do not retry the request

no_decision = 3#

“I don’t know”, ask other checks for an answer

globus_sdk.transport.RetryCheck#

The type for a retry check, a callable which takes a RetryContext and returns a RetryCheckResult. Equivalent to Callable[[globus_sdk.transport.RetryContext], globus_sdk.transport.RetryCheckResult]

class globus_sdk.transport.RetryCheckRunner(checks)[source]#

A RetryCheckRunner is an object responsible for running retry checks over the lifetime of a request. Unlike the checks or the retry context, the runner persists between retries. It can therefore implement special logic for checks like “only try this check once”.

Its primary responsibility is to answer the question “should_retry(context)?” with a boolean.

It takes as its input a list of checks. Checks may be paired with flags to indicate their configuration options. When not paired with flags, the flags are taken to be “NONE”.

Supported flags:

RUN_ONCE

The check will run at most once for a given request. Once it has run, it is recorded as “has_run” and will not be run again on that request.

class globus_sdk.transport.RetryCheckFlags(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#
NONE = 1#

no flags (default)

RUN_ONCE = 2#

only run this check once per request

@globus_sdk.transport.set_retry_check_flags[source]#

A decorator for setting retry check flags on a retry check function. Usage:

>>> @set_retry_check_flags(RetryCheckFlags.RUN_ONCE)
>>> def foo(ctx): ...
Parameters:

flag (RetryCheckFlags) – The flag to set on the check

Return type:

Callable[[C], C]

Data Encoders#

class globus_sdk.transport.RequestEncoder[source]#

A RequestEncoder takes input parameters and outputs a requests.Requests object.

The default encoder requires that the data is text and is a no-op. It can also be referred to as the "text" encoder.

class globus_sdk.transport.JSONRequestEncoder[source]#

This encoder prepares the data as JSON. It also ensures that content-type is set, so that APIs requiring a content-type of “application/json” are able to read the data.

class globus_sdk.transport.FormRequestEncoder[source]#

This encoder formats data as a form-encoded body. It requires that the input data is a dict – any other datatype will result in errors.