Transport Layer#
The transport consists of a transport object (
RequestsTransport
), but also
tooling for handling retries. It is possible to either register custom retry
check methods, or to override the Transport used by a client in order to
customize this behavior.
Transport#
- class globus_sdk.transport.RequestsTransport(verify_ssl: bool | None = None, http_timeout: float | None = None, retry_backoff: ~typing.Callable[[~globus_sdk.transport.retry.RetryContext], float] = <function _exponential_backoff>, retry_checks: list[typing.Callable[[globus_sdk.transport.retry.RetryContext], globus_sdk.transport.retry.RetryCheckResult]] | None = None, max_sleep: float | int = 10, max_retries: int | None = None)[source]#
The RequestsTransport handles HTTP request sending and retries.
It receives raw request information from a client class, and then performs the following steps - encode the data in a prepared request - repeatedly send the request until no retry is requested - return the last response or reraise the last exception
Retry checks are registered as hooks on the Transport. Additional hooks can be passed to the constructor via retry_checks. Or hooks can be added to an existing transport via a decorator.
If the maximum number of retries is reached, the final response or exception will be returned or raised.
- Parameters:
verify_ssl (bool, optional) – Explicitly enable or disable SSL verification. This parameter defaults to True, but can be set via the
GLOBUS_SDK_VERIFY_SSL
environment variable. Any non-None
setting via this parameter takes precedence over the environment variable.http_timeout (float, optional) – Explicitly set an HTTP timeout value in seconds. This parameter defaults to 60s but can be set via the
GLOBUS_SDK_HTTP_TIMEOUT
environment variable. Any value set via this parameter takes precedence over the environment variable.retry_backoff (callable, optional) – A function which determines how long to sleep between calls based on the RetryContext. Defaults to exponential backoff with jitter based on the context
attempt
number.retry_checks (list of callable, optional) – A list of initial retry checks. Any hooks registered, including the default hooks, will run after these checks.
max_sleep (float or int, optional) – The maximum sleep time between retries (in seconds). If the computed sleep time or the backoff requested by a retry check exceeds this value, this amount of time will be used instead
max_retries (int, optional) – The maximum number of retries allowed by this transport
- DEFAULT_MAX_RETRIES = 5#
default maximum number of retries
- RETRY_AFTER_STATUS_CODES: tuple[int, ...] = (429, 503)#
status codes for responses which may have a Retry-After header
- TRANSIENT_ERROR_STATUS_CODES: tuple[int, ...] = (429, 500, 502, 503, 504)#
status codes for error responses which should generally be retried
- EXPIRED_AUTHORIZATION_STATUS_CODES: tuple[int, ...] = (401,)#
status codes indicating that authorization info was missing or expired
- encoders: dict[str, globus_sdk.transport.encoders.RequestEncoder] = {'form': <globus_sdk.transport.encoders.FormRequestEncoder object>, 'json': <globus_sdk.transport.encoders.JSONRequestEncoder object>, 'text': <globus_sdk.transport.encoders.RequestEncoder object>}#
the encoders are a mapping of encoding names to encoder objects
- tune(*, verify_ssl: bool | None = None, http_timeout: float | None = None, retry_backoff: Callable[[RetryContext], float] | None = None, max_sleep: float | int | None = None, max_retries: int | None = None) Iterator[None] [source]#
Temporarily adjust some of the request sending settings of the transport. This method works as a context manager, and will reset settings to their original values after it exits.
In particular, this can be used to temporarily adjust request-sending minutiae like the
http_timeout
used.- Parameters:
verify_ssl (bool, optional) – Explicitly enable or disable SSL verification
http_timeout (float, optional) – Explicitly set an HTTP timeout value in seconds
retry_backoff (callable, optional) – A function which determines how long to sleep between calls based on the RetryContext
max_sleep (float or int, optional) – The maximum sleep time between retries (in seconds). If the computed sleep time or the backoff requested by a retry check exceeds this value, this amount of time will be used instead
max_retries (int, optional) – The maximum number of retries allowed by this transport
Examples
This can be used with any client class to temporarily set values in the context of one or more HTTP requests. To increase the HTTP request timeout from the default of 60 to 120 seconds,
>>> client = ... # any client class >>> with client.transport.tune(http_timeout=120): >>> foo = client.get_foo()
or to disable retries (note that this also disables the retry on expired-and-refreshed credentials):
>>> client = ... # any client class >>> with client.transport.tune(max_retries=0): >>> foo = client.get_foo()
- request(method: str, url: str, query_params: dict[str, Any] | None = None, data: dict[str, Any] | str | None = None, headers: dict[str, str] | None = None, encoding: str | None = None, authorizer: GlobusAuthorizer | None = None, allow_redirects: bool = True, stream: bool = False) Response [source]#
Send an HTTP request
- Parameters:
url (str) – URL for the request
method (str) – HTTP request method, as an all caps string
query_params (dict, optional) – Parameters to be encoded as a query string
headers (dict) – HTTP headers to add to the request
data (dict or str) – Data to send as the request body. May pass through encoding.
encoding (str) – A way to encode request data. “json”, “form”, and “text” are all valid values. Custom encodings can be used only if they are registered with the transport. By default, strings get “text” behavior and all other objects get “json”.
authorizer (GlobusAuthorizer, optional) – The authorizer which is used to get or update authorization information for the request
allow_redirects (bool) – Follow Location headers on redirect response automatically. Defaults to
True
stream (bool) – Do not immediately download the response content. Defaults to
False
- Returns:
requests.Response
object
- register_retry_check(func: Callable[[RetryContext], RetryCheckResult]) Callable[[RetryContext], RetryCheckResult] [source]#
Register a retry check with this transport.
A retry checker is a callable responsible for implementing check(RetryContext) -> RetryCheckResult
check should not perform any sleeps or delays. Multiple checks should be chainable and callable in any order.
- register_default_retry_checks() None [source]#
This hook is called during transport initialization. By default, it registers the following hooks:
default_check_expired_authorization
default_check_request_exception
default_check_retry_after_header
default_check_transient_error
It can be overridden to register additional hooks or to remove the default hooks.
- default_check_request_exception(ctx: RetryContext) RetryCheckResult [source]#
check if a network error was encountered
- default_check_retry_after_header(ctx: RetryContext) RetryCheckResult [source]#
check for a retry-after header if the response had a matching status
- default_check_transient_error(ctx: RetryContext) RetryCheckResult [source]#
check for transient error status codes which could be resolved by retrying the request
- default_check_expired_authorization(ctx: RetryContext) RetryCheckResult [source]#
This check evaluates whether or not there is invalid or expired authorization information which could be updated with some action – most typically a token refresh for an expired access token.
The check is flagged to only run once per request.
Retries#
These are the components used by the RequestsTransport
to implement retry
logic.
- class globus_sdk.transport.RetryContext(attempt: int, *, authorizer: GlobusAuthorizer | None = None, response: Response | None = None, exception: Exception | None = None)[source]#
The RetryContext is an object passed to retry checks in order to determine whether or not a request should be retried. The context is constructed after each request, regardless of success or failure.
If an exception was raised, the context will contain that exception object. Otherwise, the context will contain a response object. Exactly one of
response
orexception
will be present.- Parameters:
attempt (int) – The request attempt number, starting at 0.
response (requests.Response) – The response on a successful request
exception (Exception) – The error raised when trying to send the request
authorizer (
GlobusAuthorizer
) – The authorizer object from the client making the request
- class globus_sdk.transport.RetryCheckResult(value)[source]#
An enumeration.
- do_retry = 1#
yes, retry the request
- do_not_retry = 2#
no, do not retry the request
- no_decision = 3#
“I don’t know”, ask other checks for an answer
- globus_sdk.transport.RetryCheck#
The type for a retry check, a callable which takes a
RetryContext
and returns aRetryCheckResult
. Equivalent toCallable[[globus_sdk.transport.RetryContext], globus_sdk.transport.RetryCheckResult]
- class globus_sdk.transport.RetryCheckRunner(checks: list[Callable[[globus_sdk.transport.retry.RetryContext], globus_sdk.transport.retry.RetryCheckResult]])[source]#
A RetryCheckRunner is an object responsible for running retry checks over the lifetime of a request. Unlike the checks or the retry context, the runner persists between retries. It can therefore implement special logic for checks like “only try this check once”.
Its primary responsibility is to answer the question “should_retry(context)?” with a boolean.
It takes as its input a list of checks. Checks may be paired with flags to indicate their configuration options. When not paired with flags, the flags are taken to be “NONE”.
Supported flags:
RUN_ONCE
The check will run at most once for a given request. Once it has run, it is recorded as “has_run” and will not be run again on that request.
- class globus_sdk.transport.RetryCheckFlags(value)[source]#
An enumeration.
- NONE = 1#
no flags (default)
- RUN_ONCE = 2#
only run this check once per request
- @globus_sdk.transport.set_retry_check_flags(flag: RetryCheckFlags) Callable[[C], C] [source]#
A decorator for setting retry check flags on a retry check function. Usage:
>>> @set_retry_check_flags(RetryCheckFlags.RUN_ONCE) >>> def foo(ctx): ...
Data Encoders#
- class globus_sdk.transport.RequestEncoder[source]#
A RequestEncoder takes input parameters and outputs a requests.Requests object.
The default encoder requires that the data is text and is a no-op. It can also be referred to as the
"text"
encoder.