Globus Search#

class globus_sdk.SearchClient(*, environment: Optional[str] = None, base_url: Optional[str] = None, authorizer: Optional[GlobusAuthorizer] = None, app_name: Optional[str] = None, transport_params: Optional[dict[str, Any]] = None)[source]#

Bases: BaseClient

Client for the Globus Search API

This class provides helper methods for most common resources in the API, and basic get, put, post, and delete methods from the base client that can be used to access any API resource.

Parameters:: authorizer (GlobusAuthorizer ) – An authorizer instance used for all calls to Globus Search

Methods

create_entry()
create_role()
delete_by_query()
delete_entry()
delete_role()
delete_subject()
get_entry()
get_index()
get_role_list()
get_subject()
get_task()
get_task_list()
ingest()
post_search(), paginated.post_search()
scroll(), paginated.scroll()
search(), paginated.search()
update_entry()

scopes: ScopeBuilder | None = <globus_sdk.scopes.builder.ScopeBuilder object>#: the scopes for this client may be present as a ScopeBuilder

get_index(index_id: Union[UUID, str], *, query_params: Optional[dict[str, Any]] = None) → GlobusHTTPResponse[source]#

GET /v1/index/<index_id>

Get descriptive data about a Search index, including its title and description and how much data it contains.

Parameters:

index_id (str or UUID) – the ID of the index
query_params (dict, optional) – additional parameters to pass as query params

Examples

>>> sc = globus_sdk.SearchClient(...)
>>> index = sc.get_index(index_id)
>>> assert index['id'] == index_id
>>> print(index["display_name"],
>>>       "(" + index_id + "):",
>>>       index["description"])

External Documentation

See Get Index Metadata in the API documentation for details.

search(index_id: Union[UUID, str], q: str, *, offset: int = 0, limit: int = 10, advanced: bool = False, query_params: Optional[dict[str, Any]] = None) → GlobusHTTPResponse[source]#

Execute a simple Search Query, described by the query string q.

Parameters:

index_id (str or UUID) – the ID of the index
q (str) – the query string
offset (int) – an offset for pagination
limit (int) – the size of a page of results
advanced (bool) – enable ‘advanced’ query mode, which has sophisticated syntax but may result in BadRequest errors when used if the query is invalid
query_params (dict, optional) – additional parameters to pass as query params

Example Usage

sc = globus_sdk.SearchClient(...)
result = sc.search(index_id, "query string")
advanced_result = sc.search(index_id, 'author: "Ada Lovelace"', advanced=True)

Paginated Usage

This method supports paginated access. To use the paginated variant, give the same arguments as normal, but prefix the method name with paginated, as in

client.paginated.search(...)

For more information, see how to make paginated calls.

API Info

GET /v1/index/<index_id>/search

See GET Search Query in the API documentation for details.

Example Response Data

{
  "@datatype": "GSearchResult",
  "@version": "2017-09-01",
  "count": 1,
  "gmeta": [
    {
      "@datatype": "GMetaResult",
      "@version": "2019-08-27",
      "entries": [
        {
          "content": {
            "foo": "bar"
          },
          "entry_id": null,
          "matched_principal_sets": []
        }
      ],
      "subject": "foo-bar"
    }
  ],
  "has_next_page": true,
  "offset": 0,
  "total": 10
}

post_search(index_id: Union[UUID, str], data: dict[str, Any] | globus_sdk.services.search.data.SearchQuery, *, offset: Optional[int] = None, limit: Optional[int] = None) → GlobusHTTPResponse[source]#

Execute a complex Search Query, using a query document to express filters, facets, sorting, field boostring, and other behaviors.

Parameters:

index_id (str or UUID) – The index on which to search
data (dict or SearchQuery) – A Search Query document containing the query and any other fields
offset (int, optional) – offset used in paging (overwrites any offset in data)
limit (int, optional) – limit the number of results (overwrites any limit in data)

Example Usage

sc = globus_sdk.SearchClient(...)
query_data = {
    "q": "user query",
    "filters": [
        {
            "type": "range",
            "field_name": "path.to.date",
            "values": [{"from": "*", "to": "2014-11-07"}],
        }
    ],
    "facets": [
        {
            "name": "Publication Date",
            "field_name": "path.to.date",
            "type": "date_histogram",
            "date_interval": "year",
        }
    ],
    "sort": [{"field_name": "path.to.date", "order": "asc"}],
}
search_result = sc.post_search(index_id, query_data)

Paginated Usage

This method supports paginated access. To use the paginated variant, give the same arguments as normal, but prefix the method name with paginated, as in

client.paginated.post_search(...)

For more information, see how to make paginated calls.

API Info

POST /v1/index/<index_id>/search

See POST Search Query in the API documentation for details.

scroll(index_id: Union[UUID, str], data: dict[str, Any] | globus_sdk.services.search.data.SearchScrollQuery, *, marker: Optional[str] = None) → GlobusHTTPResponse[source]#

Scroll all data in a Search index. The paginated version of this API should typically be preferred, as it is the intended mode of usage.

Note that if data is written or deleted during scrolling, it is possible for scrolling to not include results or show other unexpected behaviors.

Parameters:

index_id (str or UUID) – The index on which to search
data (dict or SearchScrollQuery) – A Search Scroll Query document
marker (str, optional) – marker used in paging (overwrites any marker in data)

Example Usage

sc = globus_sdk.SearchClient(...)
scroll_result = sc.scroll(index_id, {"q": "*"})

Paginated Usage

This method supports paginated access. To use the paginated variant, give the same arguments as normal, but prefix the method name with paginated, as in

client.paginated.scroll(...)

For more information, see how to make paginated calls.

API Info

POST /v1/index/<index_id>/scroll

See Scroll Query in the API documentation for details.

ingest(index_id: Union[UUID, str], data: dict[str, Any]) → GlobusHTTPResponse[source]#

POST /v1/index/<index_id>/ingest

Write data to a Search index as an asynchronous task. The data can be provided as a single document or list of documents, but only one task_id value will be included in the response.

Parameters:

index_id (str or UUID) – The index into which to write data
data (dict) – an ingest document

Examples

>>> sc = globus_sdk.SearchClient(...)
>>> ingest_data = {
>>>   "ingest_type": "GMetaEntry",
>>>   "ingest_data": {
>>>     "subject": "https://example.com/foo/bar",
>>>     "visible_to": ["public"],
>>>     "content": {
>>>       "foo/bar": "some val"
>>>     }
>>>   }
>>> }
>>> sc.ingest(index_id, ingest_data)

or with multiple entries at once via a GMetaList:

>>> sc = globus_sdk.SearchClient(...)
>>> ingest_data = {
>>>   "ingest_type": "GMetaList",
>>>   "ingest_data": {
>>>     "gmeta": [
>>>       {
>>>         "subject": "https://example.com/foo/bar",
>>>         "visible_to": ["public"],
>>>         "content": {
>>>           "foo/bar": "some val"
>>>         }
>>>       },
>>>       {
>>>         "subject": "https://example.com/foo/bar",
>>>         "id": "otherentry",
>>>         "visible_to": ["public"],
>>>         "content": {
>>>           "foo/bar": "some otherval"
>>>         }
>>>       }
>>>     ]
>>>   }
>>> }
>>> sc.ingest(index_id, ingest_data)

External Documentation

See Ingest in the API documentation for details.

delete_by_query(index_id: Union[UUID, str], data: dict[str, Any]) → GlobusHTTPResponse[source]#

POST /v1/index/<index_id>/delete_by_query

Delete data in a Search index as an asynchronous task, deleting all documents which match a given query. The query uses a restricted subset of the syntax available for complex queries, as it is not meaningful to boost, sort, or otherwise rank data in this case.

A task_id value will be included in the response.

Parameters:

index_id (str or UUID) – The index in which to delete data
data (dict) – a query document for documents to delete

Examples

>>> sc = globus_sdk.SearchClient(...)
>>> query_data = {
>>>   "q": "user query",
>>>   "filters": [
>>>     {
>>>       "type": "range",
>>>       "field_name": "path.to.date",
>>>       "values": [
>>>         {"from": "*",
>>>          "to": "2014-11-07"}
>>>       ]
>>>     }
>>>   ]
>>> }
>>> sc.delete_by_query(index_id, query_data)

External Documentation

See Delete By Query in the API documentation for details.

get_subject(index_id: Union[UUID, str], subject: str, *, query_params: Optional[dict[str, Any]] = None) → GlobusHTTPResponse[source]#

GET /v1/index/<index_id>/subject

Fetch exactly one Subject document from Search, containing one or more Entries.

Parameters:

index_id (str or UUID) – the index containing this Subject
subject (str) – the subject string to fetch
query_params (dict, optional) – additional parameters to pass as query params

Examples

Fetch the data for subject http://example.com/abc from index index_id:

>>> sc = globus_sdk.SearchClient(...)
>>> subject_data = sc.get_subject(index_id, 'http://example.com/abc')

External Documentation

See Get Subject in the API documentation for details.

delete_subject(index_id: Union[UUID, str], subject: str, *, query_params: Optional[dict[str, Any]] = None) → GlobusHTTPResponse[source]#

DELETE /v1/index/<index_id>/subject

Delete exactly one Subject document from Search, containing one or more Entries, as an asynchronous task.

A task_id value will be included in the response.

Parameters:

index_id (str or UUID) – the index in which data will be deleted
subject (str) – the subject string for the Subject document to delete
query_params (dict, optional) – additional parameters to pass as query params

Examples

Delete all data for subject http://example.com/abc from index index_id, even data which is not visible to the current user:

>>> sc = globus_sdk.SearchClient(...)
>>> subject_data = sc.delete_subject(index_id, 'http://example.com/abc')

External Documentation

See Delete Subject in the API documentation for details.

get_entry(index_id: Union[UUID, str], subject: str, *, entry_id: Optional[str] = None, query_params: Optional[dict[str, Any]] = None) → GlobusHTTPResponse[source]#

GET /v1/index/<index_id>/entry

Fetch exactly one Entry document from Search, identified by the combination of subject string and entry_id, which defaults to null.

Parameters:

index_id (str or UUID) – the index containing this Entry
subject (str) – the subject string for the Subject document containing this Entry
entry_id (str, optional) – the entry_id for this Entry, which defaults to null
query_params (dict, optional) – additional parameters to pass as query params

Examples

Lookup the entry with a subject of https://example.com/foo/bar and a null entry_id:

>>> sc = globus_sdk.SearchClient(...)
>>> entry_data = sc.get_entry(index_id, 'http://example.com/foo/bar')

Lookup the entry with a subject of https://example.com/foo/bar and an entry_id of foo/bar:

>>> sc = globus_sdk.SearchClient(...)
>>> entry_data = sc.get_entry(index_id, 'http://example.com/foo/bar',
>>>                           entry_id='foo/bar')

External Documentation

See Get Entry in the API documentation for details.

create_entry(index_id: Union[UUID, str], data: dict[str, Any]) → GlobusHTTPResponse[source]#

This API method is in effect an alias of ingest and is deprecated. Users are recommended to use ingest() instead.

POST /v1/index/<index_id>/entry

Create or update one Entry document in Search.

The API does not enforce that the document does not exist, and will overwrite any existing data.

Parameters:

index_id (str or UUID) – the index containing this Entry
data (dict) – the entry document to write

Examples

Create an entry with a subject of https://example.com/foo/bar and a null entry_id:

>>> sc = globus_sdk.SearchClient(...)
>>> sc.create_entry(index_id, {
>>>     "subject": "https://example.com/foo/bar",
>>>     "visible_to": ["public"],
>>>     "content": {
>>>         "foo/bar": "some val"
>>>     }
>>> })

Create an entry with a subject of https://example.com/foo/bar and an entry_id of foo/bar:

>>> sc = globus_sdk.SearchClient(...)
>>> sc.create_entry(index_id, {
>>>     "subject": "https://example.com/foo/bar",
>>>     "visible_to": ["public"],
>>>     "id": "foo/bar",
>>>     "content": {
>>>         "foo/bar": "some val"
>>>     }
>>> })

External Documentation

See Create Entry in the API documentation for details.

update_entry(index_id: Union[UUID, str], data: dict[str, Any]) → GlobusHTTPResponse[source]#

This API method is in effect an alias of ingest and is deprecated. Users are recommended to use ingest() instead.

PUT /v1/index/<index_id>/entry

Create or update one Entry document in Search.

This does not do a partial update, but replaces the existing document.

Parameters:

index_id (str or UUID) – the index containing this Entry
data (dict) – the entry document to write

Examples

Update an entry with a subject of https://example.com/foo/bar and a null entry_id:

>>> sc = globus_sdk.SearchClient(...)
>>> sc.update_entry(index_id, {
>>>     "subject": "https://example.com/foo/bar",
>>>     "visible_to": ["public"],
>>>     "content": {
>>>         "foo/bar": "some val"
>>>     }
>>> })

External Documentation

See Update Entry in the API documentation for details.

delete_entry(index_id: Union[UUID, str], subject: str, *, entry_id: Optional[str] = None, query_params: Optional[dict[str, Any]] = None) → GlobusHTTPResponse[source]#

DELETE /v1/index/<index_id>/entry

Delete exactly one Entry document in Search as an asynchronous task.

A task_id value will be included in the response.

Parameters:

index_id (str or UUID) – the index in which data will be deleted
subject (str) – the subject string for the Subject of the document to delete
entry_id (str) – the ID string for the Entry to delete
query_params (dict, optional) – additional parameters to pass as query params

Examples

Delete an entry with a subject of https://example.com/foo/bar and a null entry_id:

>>> sc = globus_sdk.SearchClient(...)
>>> sc.delete_entry(index_id, "https://example.com/foo/bar")

Delete an entry with a subject of https://example.com/foo/bar and an entry_id of “foo/bar”:

>>> sc = globus_sdk.SearchClient(...)
>>> sc.delete_entry(index_id, "https://example.com/foo/bar",
>>>                 entry_id="foo/bar")

External Documentation

See Delete Entry in the API documentation for details.

get_task(task_id: Union[UUID, str], *, query_params: Optional[dict[str, Any]] = None) → GlobusHTTPResponse[source]#

GET /v1/task/<task_id>

Fetch a Task document by ID, getting task details and status.

Parameters:

task_id (str or UUID) – the task ID from the original task submission
query_params (dict, optional) – additional parameters to pass as query params

Examples

>>> sc = globus_sdk.SearchClient(...)
>>> task = sc.get_task(task_id)
>>> assert task['index_id'] == known_index_id
>>> print(task["task_id"] + " | " + task['state'])

External Documentation

See Get Task in the API documentation for details.

get_task_list(index_id: Union[UUID, str], *, query_params: Optional[dict[str, Any]] = None) → GlobusHTTPResponse[source]#

GET /v1/task_list/<index_id>

Fetch a list of recent Task documents for an index, getting task details and status.

Parameters:

index_id (str or UUID) – the index to query
query_params (dict, optional) – additional parameters to pass as query params

Examples

>>> sc = globus_sdk.SearchClient(...)
>>> task_list = sc.get_task_list(index_id)
>>> for task in task_list['tasks']:
>>>     print(task["task_id"] + " | " + task['state'])

External Documentation

See Task List in the API documentation for details.

create_role(index_id: Union[UUID, str], data: dict[str, Any], *, query_params: Optional[dict[str, Any]] = None) → GlobusHTTPResponse[source]#

POST /v1/index/<index_id>/role

Create a new role on an index. You must already have the owner or admin role on an index to create additional roles.

Roles are specified as a role name (one of "owner", "admin", or "writer") and a Principal URN.

Parameters:

index_id (uuid or str) – The index on which to create the role
data (dict) – The partial role document to use for creation
query_params (dict, optional) – Any additional query params to pass

Examples

>>> identity_id = "46bd0f56-e24f-11e5-a510-131bef46955c"
>>> sc = globus_sdk.SearchClient(...)
>>> sc.create_role(
>>>     index_id,
>>>     {
>>>         "role_name": "writer",
>>>         "principal": f"urn:globus:auth:identity:{identity_id}"
>>>     }
>>> )

External Documentation

See Create Role in the API documentation for details.

get_role_list(index_id: Union[UUID, str], *, query_params: Optional[dict[str, Any]] = None) → GlobusHTTPResponse[source]#

GET /v1/index/<index_id>/role_list

List all roles on an index. You must have the owner or admin role on an index to list roles.

Parameters:

index_id (uuid or str) – The index on which to list roles
query_params (dict, optional) – Any additional query params to pass

External Documentation

See Get Role List in the API documentation for details.

delete_role(index_id: Union[UUID, str], role_id: str, *, query_params: Optional[dict[str, Any]] = None) → GlobusHTTPResponse[source]#

DELETE /v1/index/<index_id>/role/<role_id>

Delete a role from an index. You must have the owner or admin role on an index to delete roles. You cannot remove the last owner from an index.

Parameters:

index_id (uuid or str) – The index from which to delete a role
role_id (str) – The role to delete
query_params (dict, optional) – Any additional query params to pass

External Documentation

See Role Delete in the API documentation for details.

Helper Objects#

Note that you should not use SearchQueryBase directly, and it is not importable from the top level of the SDK. It is included in documentation only to document the methods it provides to its subclasses.

class globus_sdk.services.search.data.SearchQueryBase(dict=None, /, **kwargs)[source]#

Bases: PayloadWrapper

The base class for all Search query helpers.

Search has multiple types of query documents. Not all of their supported attributes are shared, and they therefore do not inherit from one another. This class implements common methods to all of them.

Query objects have a chainable API, in which methods return the query object after modification. This allows usage like

>>> query = ...
>>> query = query.set_limit(10).set_advanced(False)

add_filter(field_name: str, values: list[str], *, type: str = 'match_all', additional_fields: Optional[dict[str, Any]] = None) → SearchQueryT[source]#

Add a filter subdocument to the query.

Parameters:

field_name (str) – the field on which to filter
values (list of str) – the values to use in the filter
type (str) – the type of filter to apply, defaults to “match_all”
additional_fields (dict, optional) – additional data to include in the filter document

set_advanced(advanced: bool) → SearchQueryT[source]#

Enable or disable advanced query string processing.

Parameters:: advanced (bool) – whether to enable (True) or not (False)

set_limit(limit: int) → SearchQueryT[source]#

Set the limit for the query document.

Parameters:: limit (int) – a limit on the number of results returned in a single page

set_query(query: str) → SearchQueryT[source]#

Set the query string for the query document.

Parameters:: query (str) – the new query string

class globus_sdk.SearchQuery(q: Optional[str] = None, *, limit: Optional[int] = None, offset: Optional[int] = None, advanced: Optional[bool] = None, additional_fields: Optional[dict[str, Any]] = None)[source]#

Bases: SearchQueryBase

A specialized dict which has helpers for creating and modifying a Search Query document.

Parameters:

q (str, optional) – The query string. Required unless filters are used.
limit (int) – A limit on the number of results returned in a single page
offset (int) – An offset into the set of all results for the query
advanced (bool, optional) – Whether to enable (True) or not to enable (False) advanced parsing of query strings. The default of False is robust and guarantees that the query will not error with “bad query string” errors
additional_fields (dict, optional) – additional data to include in the query document

Example usage:

>>> from globus_sdk import SearchClient, SearchQuery
>>> sc = SearchClient(...)
>>> index_id = ...
>>> query = (SearchQuery(q='example query')
>>>          .set_limit(100).set_offset(10)
>>>          .add_filter('path.to.field1', ['foo', 'bar']))
>>> result = sc.post_search(index_id, query)

add_boost(field_name: str, factor: str | int | float, *, additional_fields: Optional[dict[str, Any]] = None) → SearchQuery[source]#

Add a boost subdocument to the query.

Parameters:

field_name (str) – the field to boost in result weighting
factor (str, int, or float) – the factor by which to adjust the field weight (where 1.0 is the default weight)
additional_fields (dict, optional) – additional data to include in the boost document

add_facet(name: str, field_name: str, *, type: str = 'terms', size: Optional[int] = None, date_interval: Optional[str] = None, histogram_range: Optional[tuple[Any, Any]] = None, additional_fields: Optional[dict[str, Any]] = None) → SearchQuery[source]#

Add a facet subdocument to the query.

Parameters:

name (str) – the name for the facet in the result
field_name (str) – the field on which to build the facet
type (str) – the type of facet to apply, defaults to “terms”
size (int, optional) – the size parameter for the facet
date_interval (str, optional) – the date interval for a date histogram facet
histogram_range (tuple, optional) – a low and high bound for a numeric histogram facet
additional_fields (dict, optional) – additional data to include in the facet document

add_sort(field_name: str, *, order: Optional[str] = None, additional_fields: Optional[dict[str, Any]] = None) → SearchQuery[source]#

Add a sort subdocument to the query.

Parameters:

field_name (str) – the field on which to sort
order (str, optional) – ascending or descending order, given as "asc" or "desc"
additional_fields (dict, optional) – additional data to include in the sort document

set_offset(offset: int) → SearchQuery[source]#

Set the offset for the query document.

Parameters:: offset (int) – an offset into the set of all results for the query

class globus_sdk.SearchScrollQuery(q: Optional[str] = None, *, limit: Optional[int] = None, advanced: Optional[bool] = None, marker: Optional[str] = None, additional_fields: Optional[dict[str, Any]] = None)[source]#

Bases: SearchQueryBase

A scrolling query type, for scrolling the full result set for an index.

Scroll queries have more limited capabilities than general searches. They cannot boost fields, sort, or apply facets. They can, however, still apply the same filtering mechanisms which are available to normal queries.

Scrolling also differs in that it supports the use of the marker field, which is used to paginate results.

Parameters:

q (str, optional) – The query string
limit (int) – A limit on the number of results returned in a single page
advanced (bool, optional) – Whether to enable (True) or not to enable (False) advanced parsing of query strings. The default of False is robust and guarantees that the query will not error with “bad query string” errors
marker (str, optional) – the marker value
additional_fields (dict, optional) – additional data to include in the query document

set_marker(marker: str) → SearchScrollQuery[source]#

Set the marker on a scroll query.

Parameters:: marker (str) – the marker value

Client Errors#

When an error occurs, a SearchClient will raise this specialized type of error, rather than a generic GlobusAPIError.

class globus_sdk.SearchAPIError(r: Response)[source]#

Bases: GlobusAPIError

Error class for the Search API client. In addition to the inherited code and message instance variables, provides error_data.

Variables:: error_data – Additional object returned in the error response. May be a dict, list, or None.