Globus Search#

Bases: BaseClient

Client for the Globus Search API

This class provides helper methods for most common resources in the API, and basic get, put, post, and delete methods from the base client that can be used to access any API resource.

Parameters:: authorizer (GlobusAuthorizer ) – An authorizer instance used for all calls to Globus Search

Methods

batch_delete_by_subject()
create_entry()
create_index()
create_role()
delete_by_query()
delete_entry()
delete_index()
delete_role()
delete_subject()
get_entry()
get_index()
get_role_list()
get_subject()
get_task()
get_task_list()
ingest()
post_search(), paginated.post_search()
reopen_index()
scroll(), paginated.scroll()
search(), paginated.search()
update_entry()

scopes: ScopeBuilder | None = <globus_sdk.scopes.builder.ScopeBuilder object>#: the scopes for this client may be present as a ScopeBuilder

create_index(display_name: str, description: str) → GlobusHTTPResponse[source]#

Create a new index.

Parameters:

display_name (str) – the name of the index
description (str) – a description of the index

New indices default to trial status. For subscribers with a subscription ID, indices can be converted to non-trial by sending a request to support@globus.org

Example Usage

sc = globus_sdk.SearchClient(...)
r = sc.create_index(
    "History and Witchcraft",
    "Searchable information about history and witchcraft",
)
print(f"index ID: {r['id']}")

Example Response Data

{
  "@datatype": "GSearchIndex",
  "@version": "2017-09-01",
  "creation_date": "2021-04-05 15:05:18",
  "display_name": "Awesome Index of Awesomeness",
  "description": "An index so awesome that it simply cannot be described",
  "id": "395c510f-74f0-4221-8dab-615f4cb263db",
  "is_trial": true,
  "subscription_id": null,
  "max_size_in_mb": 1,
  "num_entries": 0,
  "num_subjects": 0,
  "size_in_mb": 0,
  "status": "open"
}

API Info

POST /v1/index

See Index Create in the API documentation for details.

delete_index(index_id: UUID | str) → GlobusHTTPResponse[source]#

Mark an index for deletion.

Globus Search does not immediately delete indices. Instead, this API sets the index status to "delete-pending". Search will move pending tasks on the index to the CANCELLED state and will eventually delete the index.

If the index is a trial index, it will be deleted a few minutes after being marked for deletion. If the index is non-trial, it will be kept for 30 days and will be eligible for use with the reopen API (see reopen_index()) during that time.

Parameters:: index_id (str or UUID) – the ID of the index

Example Usage

sc = globus_sdk.SearchClient(...)
sc.delete_index(index_id)

Example Response Data

{
  "index_id": "489c9a61-ad3e-4276-984c-33a12c4d39d4",
  "acknowledged": true
}

API Info

DELETE /v1/index/<index_id>

See Index Delete in the API documentation for details.

reopen_index(index_id: UUID | str) → GlobusHTTPResponse[source]#

Reopen an index that has been marked for deletion, cancelling the deletion.

Parameters:: index_id (str or UUID) – the ID of the index

Example Usage

sc = globus_sdk.SearchClient(...)
sc.reopen_index(index_id)

Example Response Data

{
  "index_id": "df4a7306-20b2-45c4-aee1-1448ed648a39",
  "acknowledged": true
}

API Info

POST /v1/index/<index_id>/reopen

See Index Reopen in the API documentation for details.

get_index(index_id: UUID | str, *, query_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

Get descriptive data about a Search index, including its title and description and how much data it contains.

Parameters:

index_id (str or UUID) – the ID of the index
query_params (dict, optional) – additional parameters to pass as query params

Example Usage

sc = globus_sdk.SearchClient(...)
index = sc.get_index(index_id)
assert index["id"] == index_id
print(index["display_name"], "(" + index_id + "):", index["description"])

API Info

GET /v1/index/<index_id>

See Index Show in the API documentation for details.

search(index_id: UUID | str, q: str, *, offset: int = 0, limit: int = 10, advanced: bool = False, query_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

Execute a simple Search Query, described by the query string q.

Parameters:

index_id (str or UUID) – the ID of the index
q (str) – the query string
offset (int) – an offset for pagination
limit (int) – the size of a page of results
advanced (bool) – enable ‘advanced’ query mode, which has sophisticated syntax but may result in BadRequest errors when used if the query is invalid
query_params (dict, optional) – additional parameters to pass as query params

Example Usage

sc = globus_sdk.SearchClient(...)
result = sc.search(index_id, "query string")
advanced_result = sc.search(index_id, 'author: "Ada Lovelace"', advanced=True)

Paginated Usage

This method supports paginated access. To use the paginated variant, give the same arguments as normal, but prefix the method name with paginated, as in

client.paginated.search(...)

For more information, see how to make paginated calls.

API Info

GET /v1/index/<index_id>/search

See GET Search Query in the API documentation for details.

Example Response Data

{
  "@datatype": "GSearchResult",
  "@version": "2017-09-01",
  "count": 1,
  "gmeta": [
    {
      "@datatype": "GMetaResult",
      "@version": "2019-08-27",
      "entries": [
        {
          "content": {
            "foo": "bar"
          },
          "entry_id": null,
          "matched_principal_sets": []
        }
      ],
      "subject": "foo-bar"
    }
  ],
  "has_next_page": true,
  "offset": 0,
  "total": 10
}

post_search(index_id: UUID | str, data: dict[str, Any] | SearchQuery, *, offset: int | None = None, limit: int | None = None) → GlobusHTTPResponse[source]#

Execute a complex Search Query, using a query document to express filters, facets, sorting, field boostring, and other behaviors.

Parameters:

index_id (str or UUID) – The index on which to search
data (dict or SearchQuery) – A Search Query document containing the query and any other fields
offset (int, optional) – offset used in paging (overwrites any offset in data)
limit (int, optional) – limit the number of results (overwrites any limit in data)

Example Usage

sc = globus_sdk.SearchClient(...)
query_data = {
    "q": "user query",
    "filters": [
        {
            "type": "range",
            "field_name": "path.to.date",
            "values": [{"from": "*", "to": "2014-11-07"}],
        }
    ],
    "facets": [
        {
            "name": "Publication Date",
            "field_name": "path.to.date",
            "type": "date_histogram",
            "date_interval": "year",
        }
    ],
    "sort": [{"field_name": "path.to.date", "order": "asc"}],
}
search_result = sc.post_search(index_id, query_data)

Paginated Usage

This method supports paginated access. To use the paginated variant, give the same arguments as normal, but prefix the method name with paginated, as in

client.paginated.post_search(...)

For more information, see how to make paginated calls.

API Info

POST /v1/index/<index_id>/search

See POST Search Query in the API documentation for details.

scroll(index_id: UUID | str, data: dict[str, Any] | SearchScrollQuery, *, marker: str | None = None) → GlobusHTTPResponse[source]#

Scroll all data in a Search index. The paginated version of this API should typically be preferred, as it is the intended mode of usage.

Note that if data is written or deleted during scrolling, it is possible for scrolling to not include results or show other unexpected behaviors.

Parameters:

index_id (str or UUID) – The index on which to search
data (dict or SearchScrollQuery) – A Search Scroll Query document
marker (str, optional) – marker used in paging (overwrites any marker in data)

Example Usage

sc = globus_sdk.SearchClient(...)
scroll_result = sc.scroll(index_id, {"q": "*"})

Paginated Usage

This method supports paginated access. To use the paginated variant, give the same arguments as normal, but prefix the method name with paginated, as in

client.paginated.scroll(...)

For more information, see how to make paginated calls.

API Info

POST /v1/index/<index_id>/scroll

See Scroll Query in the API documentation for details.

ingest(index_id: UUID | str, data: dict[str, Any]) → GlobusHTTPResponse[source]#

Write data to a Search index as an asynchronous task. The data can be provided as a single document or list of documents, but only one task_id value will be included in the response.

Parameters:

index_id (str or UUID) – The index into which to write data
data (dict) – an ingest document

Example Usage

sc = globus_sdk.SearchClient(...)
ingest_data = {
    "ingest_type": "GMetaEntry",
    "ingest_data": {
        "subject": "https://example.com/foo/bar",
        "visible_to": ["public"],
        "content": {"foo/bar": "some val"},
    },
}
sc.ingest(index_id, ingest_data)

or with multiple entries at once via a GMetaList:

sc = globus_sdk.SearchClient(...)
ingest_data = {
    "ingest_type": "GMetaList",
    "ingest_data": {
        "gmeta": [
            {
                "subject": "https://example.com/foo/bar",
                "visible_to": ["public"],
                "content": {"foo/bar": "some val"},
            },
            {
                "subject": "https://example.com/foo/bar",
                "id": "otherentry",
                "visible_to": ["public"],
                "content": {"foo/bar": "some otherval"},
            },
        ]
    },
}
sc.ingest(index_id, ingest_data)

API Info

POST /v1/index/<index_id>/ingest

See Ingest in the API documentation for details.

delete_by_query(index_id: UUID | str, data: dict[str, Any]) → GlobusHTTPResponse[source]#

Delete data in a Search index as an asynchronous task, deleting all documents which match a given query. The query uses a restricted subset of the syntax available for complex queries, as it is not meaningful to boost, sort, or otherwise rank data in this case.

A task_id value will be included in the response.

Parameters:

index_id (str or UUID) – The index in which to delete data
data (dict) – a query document for documents to delete

Example Usage

sc = globus_sdk.SearchClient(...)
query_data = {
    "q": "user query",
    "filters": [
        {
            "type": "range",
            "field_name": "path.to.date",
            "values": [{"from": "*", "to": "2014-11-07"}],
        }
    ],
}
sc.delete_by_query(index_id, query_data)

API Info

POST /v1/index/<index_id>/delete_by_query

See Delete By Query in the API documentation for details.

batch_delete_by_subject(index_id: UUID | str, subjects: Iterable[str], additional_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

Delete data in a Search index as an asynchronous task, deleting multiple documents based on their subject values.

A task_id value will be included in the response.

Parameters:

index_id (str or UUID) – The index in which to delete data
subjects (iterable of str) – The subjects to delete, as an iterable of strings
additional_params (dict, optional) – Additional parameters to include in the request body

Example Usage

sc = globus_sdk.SearchClient(...)
sc.batch_delete_by_subject(
    index_id,
    subjects=[
        "very-cool-document",
        "less-cool-document",
        "document-wearing-sunglasses",
    ],
)

Example Response Data

{
  "task_id": "a97ada52-8eda-11ee-b1d2-0242ac110002"
}

API Info

POST /v1/index/<index_id>/batch_delete_by_subject

See Delete By Subject in the API documentation for details.

get_subject(index_id: UUID | str, subject: str, *, query_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

Fetch exactly one Subject document from Search, containing one or more Entries.

Parameters:

index_id (str or UUID) – the index containing this Subject
subject (str) – the subject string to fetch
query_params (dict, optional) – additional parameters to pass as query params

Example Usage

Fetch the data for subject http://example.com/abc from index index_id:

sc = globus_sdk.SearchClient(...)
subject_data = sc.get_subject(index_id, "http://example.com/abc")

API Info

GET /v1/index/<index_id>/subject

See Get By Subject in the API documentation for details.

delete_subject(index_id: UUID | str, subject: str, *, query_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

Delete exactly one Subject document from Search, containing one or more Entries, as an asynchronous task.

A task_id value will be included in the response.

Parameters:

index_id (str or UUID) – the index in which data will be deleted
subject (str) – the subject string for the Subject document to delete
query_params (dict, optional) – additional parameters to pass as query params

Example Usage

Delete all data for subject http://example.com/abc from index index_id, even data which is not visible to the current user:

sc = globus_sdk.SearchClient(...)
response = sc.delete_subject(index_id, "http://example.com/abc")
task_id = response["task_id"]

API Info

DELETE /v1/index/<index_id>/subject

See Delete By Subject in the API documentation for details.

get_entry(index_id: UUID | str, subject: str, *, entry_id: str | None = None, query_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

Fetch exactly one Entry document from Search, identified by the combination of subject string and entry_id, which defaults to null.

Parameters:

index_id (str or UUID) – the index containing this Entry
subject (str) – the subject string for the Subject document containing this Entry
entry_id (str, optional) – the entry_id for this Entry, which defaults to null
query_params (dict, optional) – additional parameters to pass as query params

Example Usage

Lookup the entry with a subject of https://example.com/foo/bar and a null entry_id:

sc = globus_sdk.SearchClient(...)
entry_data = sc.get_entry(index_id, "http://example.com/foo/bar")

Lookup the entry with a subject of https://example.com/foo/bar and an entry_id of foo/bar:

sc = globus_sdk.SearchClient(...)
entry_data = sc.get_entry(index_id, "http://example.com/foo/bar", entry_id="foo/bar")

API Info

GET /v1/index/<index_id>/entry

See Get Entry in the API documentation for details.

create_entry(index_id: UUID | str, data: dict[str, Any]) → GlobusHTTPResponse[source]#

This API method is in effect an alias of ingest and is deprecated. Users are recommended to use ingest() instead.

Create or update one Entry document in Search.

The API does not enforce that the document does not exist, and will overwrite any existing data.

Parameters:

index_id (str or UUID) – the index containing this Entry
data (dict) – the entry document to write

Example Usage

Create an entry with a subject of https://example.com/foo/bar and a null entry_id:

sc = globus_sdk.SearchClient(...)
sc.create_entry(
    index_id,
    {
        "subject": "https://example.com/foo/bar",
        "visible_to": ["public"],
        "content": {"foo/bar": "some val"},
    },
)

Create an entry with a subject of https://example.com/foo/bar and an entry_id of foo/bar:

sc = globus_sdk.SearchClient(...)
sc.create_entry(
    index_id,
    {
        "subject": "https://example.com/foo/bar",
        "visible_to": ["public"],
        "id": "foo/bar",
        "content": {"foo/bar": "some val"},
    },
)

API Info

POST /v1/index/<index_id>/entry

See Create Entry in the API documentation for details.

update_entry(index_id: UUID | str, data: dict[str, Any]) → GlobusHTTPResponse[source]#

This API method is in effect an alias of ingest and is deprecated. Users are recommended to use ingest() instead.

Create or update one Entry document in Search.

This does not do a partial update, but replaces the existing document.

Parameters:

index_id (str or UUID) – the index containing this Entry
data (dict) – the entry document to write

Example Usage

Update an entry with a subject of https://example.com/foo/bar and a null entry_id:

sc = globus_sdk.SearchClient(...)
sc.update_entry(
    index_id,
    {
        "subject": "https://example.com/foo/bar",
        "visible_to": ["public"],
        "content": {"foo/bar": "some val"},
    },
)

API Info

PUT /v1/index/<index_id>/entry

See Update Entry in the API documentation for details.

delete_entry(index_id: UUID | str, subject: str, *, entry_id: str | None = None, query_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

Delete exactly one Entry document in Search as an asynchronous task.

A task_id value will be included in the response.

Parameters:

index_id (str or UUID) – the index in which data will be deleted
subject (str) – the subject string for the Subject of the document to delete
entry_id (str) – the ID string for the Entry to delete
query_params (dict, optional) – additional parameters to pass as query params

Example Usage

Delete an entry with a subject of https://example.com/foo/bar and a null entry_id:

sc = globus_sdk.SearchClient(...)
sc.delete_entry(index_id, "https://example.com/foo/bar")

Delete an entry with a subject of https://example.com/foo/bar and an entry_id of “foo/bar”:

sc = globus_sdk.SearchClient(...)
sc.delete_entry(index_id, "https://example.com/foo/bar", entry_id="foo/bar")

API Info

DELETE /v1/index/<index_id>/entry

See Delete Entry in the API documentation for details.

get_task(task_id: UUID | str, *, query_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

Fetch a Task document by ID, getting task details and status.

Parameters:

task_id (str or UUID) – the task ID from the original task submission
query_params (dict, optional) – additional parameters to pass as query params

Example Usage

sc = globus_sdk.SearchClient(...)
task = sc.get_task(task_id)
assert task["index_id"] == known_index_id
print(task["task_id"], "|", task["state"])

API Info

GET /v1/task/<task_id>

See Get Task in the API documentation for details.

get_task_list(index_id: UUID | str, *, query_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

Fetch a list of recent Task documents for an index, getting task details and status.

Parameters:

index_id (str or UUID) – the index to query
query_params (dict, optional) – additional parameters to pass as query params

Example Usage

sc = globus_sdk.SearchClient(...)
task_list = sc.get_task_list(index_id)
for task in task_list["tasks"]:
    print(task["task_id"], "|", task["state"])

API Info

GET /v1/task_list/<index_id>

See Task List in the API documentation for details.

create_role(index_id: UUID | str, data: dict[str, Any], *, query_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

Create a new role on an index. You must already have the owner or admin role on an index to create additional roles.

Roles are specified as a role name (one of "owner", "admin", or "writer") and a Principal URN.

Parameters:

index_id (uuid or str) – The index on which to create the role
data (dict) – The partial role document to use for creation
query_params (dict, optional) – Any additional query params to pass

Example Usage

identity_id = "46bd0f56-e24f-11e5-a510-131bef46955c"
sc = globus_sdk.SearchClient(...)
sc.create_role(
    index_id,
    {"role_name": "writer", "principal": f"urn:globus:auth:identity:{identity_id}"},
)

API Info

POST /v1/index/<index_id>/role

See Create Role in the API documentation for details.

get_role_list(index_id: UUID | str, *, query_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

List all roles on an index. You must have the owner or admin role on an index to list roles.

Parameters:

index_id (uuid or str) – The index on which to list roles
query_params (dict, optional) – Any additional query params to pass

API Info

GET /v1/index/<index_id>/role_list

See Get Role List in the API documentation for details.

delete_role(index_id: UUID | str, role_id: str, *, query_params: dict[str, Any] | None = None) → GlobusHTTPResponse[source]#

Delete a role from an index. You must have the owner or admin role on an index to delete roles. You cannot remove the last owner from an index.

Parameters:

index_id (uuid or str) – The index from which to delete a role
role_id (str) – The role to delete
query_params (dict, optional) – Any additional query params to pass

API Info

DELETE /v1/index/<index_id>/role/<role_id>

See Role Delete in the API documentation for details.

Helper Objects#

Note that you should not use SearchQueryBase directly, and it is not importable from the top level of the SDK. It is included in documentation only to document the methods it provides to its subclasses.

class globus_sdk.services.search.data.SearchQueryBase(dict=None, /, **kwargs)[source]#

Bases: PayloadWrapper

The base class for all Search query helpers.

Search has multiple types of query documents. Not all of their supported attributes are shared, and they therefore do not inherit from one another. This class implements common methods to all of them.

Query objects have a chainable API, in which methods return the query object after modification. This allows usage like

>>> query = ...
>>> query = query.set_limit(10).set_advanced(False)

add_filter(field_name: str, values: list[str], *, type: str = 'match_all', additional_fields: dict[str, Any] | None = None) → SearchQueryT[source]#

Add a filter subdocument to the query.

Parameters:

field_name (str) – the field on which to filter
values (list of str) – the values to use in the filter
type (str) – the type of filter to apply, defaults to “match_all”
additional_fields (dict, optional) – additional data to include in the filter document

set_advanced(advanced: bool) → SearchQueryT[source]#

Enable or disable advanced query string processing.

Parameters:: advanced (bool) – whether to enable (True) or not (False)

set_limit(limit: int) → SearchQueryT[source]#

Set the limit for the query document.

Parameters:: limit (int) – a limit on the number of results returned in a single page

set_query(query: str) → SearchQueryT[source]#

Set the query string for the query document.

Parameters:: query (str) – the new query string

Bases: SearchQueryBase

A specialized dict which has helpers for creating and modifying a Search Query document.

Parameters:

q (str, optional) – The query string. Required unless filters are used.
limit (int) – A limit on the number of results returned in a single page
offset (int) – An offset into the set of all results for the query
advanced (bool, optional) – Whether to enable (True) or not to enable (False) advanced parsing of query strings. The default of False is robust and guarantees that the query will not error with “bad query string” errors
additional_fields (dict, optional) – additional data to include in the query document

Example usage:

>>> from globus_sdk import SearchClient, SearchQuery
>>> sc = SearchClient(...)
>>> index_id = ...
>>> query = (SearchQuery(q='example query')
>>>          .set_limit(100).set_offset(10)
>>>          .add_filter('path.to.field1', ['foo', 'bar']))
>>> result = sc.post_search(index_id, query)

add_boost(field_name: str, factor: str | int | float, *, additional_fields: dict[str, Any] | None = None) → SearchQuery[source]#

Add a boost subdocument to the query.

Parameters:

field_name (str) – the field to boost in result weighting
factor (str, int, or float) – the factor by which to adjust the field weight (where 1.0 is the default weight)
additional_fields (dict, optional) – additional data to include in the boost document

add_facet(name: str, field_name: str, *, type: str = 'terms', size: int | None = None, date_interval: str | None = None, histogram_range: tuple[Any, Any] | None = None, additional_fields: dict[str, Any] | None = None) → SearchQuery[source]#

Add a facet subdocument to the query.

Parameters:

name (str) – the name for the facet in the result
field_name (str) – the field on which to build the facet
type (str) – the type of facet to apply, defaults to “terms”
size (int, optional) – the size parameter for the facet
date_interval (str, optional) – the date interval for a date histogram facet
histogram_range (tuple, optional) – a low and high bound for a numeric histogram facet
additional_fields (dict, optional) – additional data to include in the facet document

add_sort(field_name: str, *, order: str | None = None, additional_fields: dict[str, Any] | None = None) → SearchQuery[source]#

Add a sort subdocument to the query.

Parameters:

field_name (str) – the field on which to sort
order (str, optional) – ascending or descending order, given as "asc" or "desc"
additional_fields (dict, optional) – additional data to include in the sort document

set_offset(offset: int) → SearchQuery[source]#

Set the offset for the query document.

Parameters:: offset (int) – an offset into the set of all results for the query

Bases: SearchQueryBase

A scrolling query type, for scrolling the full result set for an index.

Scroll queries have more limited capabilities than general searches. They cannot boost fields, sort, or apply facets. They can, however, still apply the same filtering mechanisms which are available to normal queries.

Scrolling also differs in that it supports the use of the marker field, which is used to paginate results.

Parameters:

q (str, optional) – The query string
limit (int) – A limit on the number of results returned in a single page
advanced (bool, optional) – Whether to enable (True) or not to enable (False) advanced parsing of query strings. The default of False is robust and guarantees that the query will not error with “bad query string” errors
marker (str, optional) – the marker value
additional_fields (dict, optional) – additional data to include in the query document

set_marker(marker: str) → SearchScrollQuery[source]#

Set the marker on a scroll query.

Parameters:: marker (str) – the marker value

Client Errors#

When an error occurs, a SearchClient will raise this specialized type of error, rather than a generic GlobusAPIError.

class globus_sdk.SearchAPIError(r: Response)[source]#

Bases: GlobusAPIError

Error class for the Search API client. In addition to the inherited instance variables, provides error_data.

Variables:: error_data – Additional object returned in the error response. May be a dict, list, or None.