Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Changelog

## [8.44.0](https://github.com/Arize-ai/arize/compare/arize-python-sdk/v8.43.1...arize-python-sdk/v8.44.0) (2026-08-04)


### 🎁 New Features

* **experiments:** Enable standalone experiment creation ([#81022](https://github.com/Arize-ai/arize/issues/81022)) ([a1a11b2](https://github.com/Arize-ai/arize/commit/a1a11b212e6cb49d9dad1522afa1e08a549c6c64))
* **integrations:** add integrations subclient (llm+agent CRUD) + agent_call task support ([#79759](https://github.com/Arize-ai/arize/issues/79759)) ([9ab254a](https://github.com/Arize-ai/arize/commit/9ab254a9c086b57d7f746fb94872d7245831a3c5))

## [8.43.1](https://github.com/Arize-ai/arize/compare/arize-python-sdk/v8.43.0...arize-python-sdk/v8.43.1) (2026-07-27)


Expand Down
164 changes: 155 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,13 @@
- [Get an AI Integration](#get-an-ai-integration)
- [Update an AI Integration](#update-an-ai-integration)
- [Delete an AI Integration](#delete-an-ai-integration)
- [Operations on Integrations](#operations-on-integrations)
- [List Integrations](#list-integrations)
- [Get an Integration](#get-an-integration)
- [Create an LLM Integration](#create-an-llm-integration)
- [Create an Agent Integration](#create-an-agent-integration)
- [Update an Integration](#update-an-integration)
- [Delete an Integration](#delete-an-integration)
- [SDK Configuration](#sdk-configuration)
- [Logging](#logging)
- [In Code](#in-code)
Expand Down Expand Up @@ -617,7 +624,7 @@ not_deleted = resp.not_deleted_example_ids

### List Experiments

You can list all experiments that the user has access to using `client.experiments.list()`. You can use the `limit` parameter to specify the maximum number of experiments desired in the response and you can specify the `dataset` to target the list operation to a particular dataset.
You can list all experiments that the user has access to using `client.experiments.list()`. You can use the `limit` parameter to specify the maximum number of experiments desired in the response, and narrow the results by scope: pass `dataset` for only the experiments run on that dataset, or `space` for every experiment in that space — including those with no dataset. The two are mutually exclusive.

```python
resp = client.experiments.list(
Expand Down Expand Up @@ -687,22 +694,39 @@ It is possible that you have run the experiment yourself without the above funct

> NOTE: If you don't have experiment data and want to run an experiment, see the `client.experiments.run()` section above.

In addition, you must specify which columns are the `example_id` and the `result` using `ExperimentTaskFieldNames`. If you have evaluation data, indicate the evaluation columns using `EvaluationResultFieldNames`.
An experiment belongs to a space and may optionally be associated with a dataset. Pass exactly one of:

If the number of runs is too large, the client SDK will try to send the data via Arrow Flight via gRPC for better performance. If you want to force the data transfer to HTTP you can use the `force_http` flag. The response is an `Experiment` object.
- `dataset` — associates the experiment with a dataset, so its runs can reference that dataset's examples.
- `space` — creates the experiment directly in a space, with no dataset.

In addition, you must specify which column holds the `output` using `ExperimentTaskFieldNames`. Also set `example_id` there when you pass `dataset`, so each run can be matched to the example it ran against; it isn't used for an experiment with no dataset. If you have evaluation data, indicate the evaluation columns using `EvaluationResultFieldNames`.

If the number of runs is too large, the client SDK will try to send the data via Arrow Flight via gRPC for better performance. If you want to force the data transfer to HTTP you can use the `force_http` flag. Experiments with no dataset always upload via HTTP, since the gRPC + Flight path only supports dataset-associated experiments. The response is an `Experiment` object.

```python
from arize.experiments.types import ExperimentTaskFieldNames, EvaluationResultFieldNames

# Associated with a dataset: runs reference the dataset's examples.
created_experiment = client.experiments.create(
name="<your-experiment-name>", # Name must be unique within a dataset
name="<your-experiment-name>", # Name must be unique within the dataset
dataset="<your-dataset-id-or-name>",
space=..., # Optional, space ID or name
space=..., # Optional, space ID or name, used to resolve `dataset` by name
experiment_runs=..., # List of dictionaries or pandas dataframe
task_fields=ExperimentTaskFieldNames(...),
task_fields=ExperimentTaskFieldNames(
output="<your-output-column>",
example_id="<your-example-id-column>",
),
evaluator_columns=... # Optional
# force_http=... # Optionally pass force_http to create experiments via HTTP instead of gRPC, defaults to False
)

# No dataset: created directly in a space, and runs need no example ID.
standalone_experiment = client.experiments.create(
name="<your-experiment-name>", # Name must be unique within the space
space="<your-space-id-or-name>",
experiment_runs=...,
task_fields=ExperimentTaskFieldNames(output="<your-output-column>"),
)
```

### Get an Experiment
Expand All @@ -712,8 +736,10 @@ To get an experiment by its ID or name use `client.experiments.get()`. The retur
```python
experiment = client.experiments.get(
experiment=... # The experiment ID or name
dataset=... # Optional, dataset ID or name (required when looking up by experiment name)
space=... # Optional, space ID or name
dataset=... # Optional, dataset ID or name, to look up by name within a dataset
space=... # Optional, space ID or name; looks up by name within the space when
# `dataset` is omitted, which is the only option for an experiment
# with no dataset
)
```

Expand Down Expand Up @@ -758,7 +784,7 @@ resp_df = resp.to_df()

### Append Experiment Runs

Append between 1 and 1000 new runs to an existing experiment using `client.experiments.append_runs()`. Each run must include `example_id` (the ID of an example from the experiment's dataset) and `output`. The response includes the updated experiment and the generated run IDs in input order (`run_ids`).
Append between 1 and 1000 new runs to an existing experiment using `client.experiments.append_runs()`. Each run must include `output`; `example_id` (the ID of an example from the experiment's dataset) is required only when the target experiment is associated with a dataset. The response includes the updated experiment and the generated run IDs in input order (`run_ids`).

```python
result = client.experiments.append_runs(
Expand Down Expand Up @@ -1758,6 +1784,126 @@ client.ai_integrations.delete(
)
```

## Operations on Integrations

Use `client.integrations` to manage **agent** and **LLM** integrations. Integrations are polymorphic: LLM integrations configure a model provider, while agent integrations connect a customer-hosted agent exposed at an HTTPS endpoint. Names are only unique per `(account, type)`, so an `integration_type` is required to resolve a name (an ID needs no type); `space` is only an optional visibility filter.

> **Note:** This is distinct from `client.ai_integrations` (the legacy AI integrations used by the Playground and online evaluations).

> **Note:** Integrations are an **alpha** endpoint. Enable it via the pre-release opt-in; the surface may change without notice.

### List Integrations

When `integration_type` is omitted, integrations of every type are returned in one merged list; each item carries its `type`.

```python
from arize.integrations.types import IntegrationType

resp = client.integrations.list(
integration_type=..., # Optional, IntegrationType.LLM or IntegrationType.AGENT
name=..., # Optional, case-insensitive substring filter
space=..., # Optional, space ID or name (visibility filter)
limit=..., # Optional, defaults to 50
cursor=..., # Optional, pagination cursor from a previous response
)
integration_list = resp.integrations
```

### Get an Integration

```python
from arize.integrations.types import IntegrationType

integration = client.integrations.get(
integration="<integration-id-or-name>",
integration_type=IntegrationType.LLM, # Required to resolve a name; not needed for an ID
space=..., # Optional, visibility filter
)
```

### Create an LLM Integration

LLM integrations configure access to a model provider. Construct the generated config that matches the provider you want — all 7 are supported: `CreateOpenAiConfig`, `CreateAnthropicConfig`, `CreateGeminiConfig`, `CreateAwsBedrockConfig`, `CreateCustomConfig`, `CreateVertexAiConfig`, and `CreateNvidiaNimConfig`.

```python
from arize.integrations.types import CreateOpenAiConfig

integration = client.integrations.create_llm(
name="my-openai-integration", # Must be unique within the account per type
config=CreateOpenAiConfig(
provider="OPEN_AI",
api_key="<your-provider-api-key>",
),
scopings=..., # Optional, visibility scoping rules (defaults to account-wide)
)
```

### Create an Agent Integration

Agent integrations connect a customer-hosted agent exposed at an HTTPS endpoint.

```python
from arize.integrations.types import CreateAgentRequestPresetInput

integration = client.integrations.create_agent(
name="my-agent", # Must be unique within the account per type
endpoint="https://my-agent.example.com/replay", # Validated for SSRF server-side
input_schema={
"type": "object",
"properties": {"input": {"type": "string"}},
"required": ["input"],
},
description=..., # Optional
headers=..., # Optional, custom headers (encrypted at rest, never returned)
request_presets=[ # Optional, list of CreateAgentRequestPresetInput
CreateAgentRequestPresetInput(name="default", config={"input": "hello"}),
],
scopings=..., # Optional, visibility scoping rules
)
```

### Update an Integration

Only the fields you pass are sent; omitted fields are left unchanged. Use the method matching the integration's type (the provider/type is immutable). Nullable fields accept an explicit `None` to clear them.

```python
# Update an LLM integration (e.g. rotate the API key)
integration = client.integrations.update_llm(
integration="<integration-id-or-name>",
space=..., # Optional, visibility filter
name=..., # Optional
api_key=..., # Optional, pass None to clear
function_calling_enabled=..., # Optional
# Provider-conditional fields are also accepted: base_url, headers,
# model_names, is_default_models_enabled, auth (AWS Bedrock),
# project_id, location, project_access_label (Vertex AI).
)

# Update an agent integration
integration = client.integrations.update_agent(
integration="<integration-id-or-name>",
space=..., # Optional
name=..., # Optional
description=..., # Optional, pass None to clear
endpoint=..., # Optional
input_schema=..., # Optional
headers=..., # Optional, pass None to clear
request_presets=..., # Optional, replaces existing presets (matched by name)
)
```

### Delete an Integration

```python
from arize.integrations.types import IntegrationType

client.integrations.delete(
integration="<integration-id-or-name>",
integration_type=IntegrationType.AGENT, # Required to resolve a name; not needed for an ID
space=..., # Optional
)
```

# SDK Configuration

## Logging
Expand Down
8 changes: 4 additions & 4 deletions docs/source/_static/switcher.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@
"url": "https://arize-client-python.readthedocs.io/en/latest/",
"preferred": true
},
{
"version": "v8.44.0",
"url": "https://arize-client-python.readthedocs.io/en/v8.44.0/"
},
{
"version": "v8.43.0",
"url": "https://arize-client-python.readthedocs.io/en/v8.43.0/"
Expand Down Expand Up @@ -40,10 +44,6 @@
"version": "v8.35.0",
"url": "https://arize-client-python.readthedocs.io/en/v8.35.0/"
},
{
"version": "v8.34.0",
"url": "https://arize-client-python.readthedocs.io/en/v8.34.0/"
},
{
"version": "v7.52.0",
"url": "https://arize-client-python.readthedocs.io/en/v7.52.0/"
Expand Down
1 change: 1 addition & 0 deletions docs/source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ api_keys
datasets
evaluators
experiments
integrations
ml
organizations
projects
Expand Down
16 changes: 16 additions & 0 deletions docs/source/integrations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Integrations

```{eval-rst}
.. currentmodule:: arize.integrations.client
.. autoclass:: IntegrationsClient
:members:
:member-order: bysource
```

## Response Types

```{eval-rst}
.. automodule:: arize.integrations.types
:members:
:member-order: bysource
```
9 changes: 6 additions & 3 deletions src/arize/_generated/api_client/api/annotation_configs_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@ def delete_annotation_config(
) -> None:
"""Delete an annotation config

Delete an annotation config by its ID. This operation is irreversible. <Note>This endpoint is in beta, read more [here](https://arize.com/docs/ax/rest-reference#api-version-stages).</Note>
Delete an annotation config by its ID. The annotation config must not be associated with an active annotation queue; remove it from those queues before deleting it. This operation is irreversible. <Note>This endpoint is in beta, read more [here](https://arize.com/docs/ax/rest-reference#api-version-stages).</Note>

:param annotation_config_id: The unique annotation config identifier (base64) (required)
:type annotation_config_id: str
Expand Down Expand Up @@ -397,6 +397,7 @@ def delete_annotation_config(
'401': "Problem",
'403': "Problem",
'404': "Problem",
'409': "Problem",
'429': "Problem",
}
response_data = self.api_client.call_api(
Expand Down Expand Up @@ -429,7 +430,7 @@ def delete_annotation_config_with_http_info(
) -> ApiResponse[None]:
"""Delete an annotation config

Delete an annotation config by its ID. This operation is irreversible. <Note>This endpoint is in beta, read more [here](https://arize.com/docs/ax/rest-reference#api-version-stages).</Note>
Delete an annotation config by its ID. The annotation config must not be associated with an active annotation queue; remove it from those queues before deleting it. This operation is irreversible. <Note>This endpoint is in beta, read more [here](https://arize.com/docs/ax/rest-reference#api-version-stages).</Note>

:param annotation_config_id: The unique annotation config identifier (base64) (required)
:type annotation_config_id: str
Expand Down Expand Up @@ -469,6 +470,7 @@ def delete_annotation_config_with_http_info(
'401': "Problem",
'403': "Problem",
'404': "Problem",
'409': "Problem",
'429': "Problem",
}
response_data = self.api_client.call_api(
Expand Down Expand Up @@ -501,7 +503,7 @@ def delete_annotation_config_without_preload_content(
) -> RESTResponseType:
"""Delete an annotation config

Delete an annotation config by its ID. This operation is irreversible. <Note>This endpoint is in beta, read more [here](https://arize.com/docs/ax/rest-reference#api-version-stages).</Note>
Delete an annotation config by its ID. The annotation config must not be associated with an active annotation queue; remove it from those queues before deleting it. This operation is irreversible. <Note>This endpoint is in beta, read more [here](https://arize.com/docs/ax/rest-reference#api-version-stages).</Note>

:param annotation_config_id: The unique annotation config identifier (base64) (required)
:type annotation_config_id: str
Expand Down Expand Up @@ -541,6 +543,7 @@ def delete_annotation_config_without_preload_content(
'401': "Problem",
'403': "Problem",
'404': "Problem",
'409': "Problem",
'429': "Problem",
}
response_data = self.api_client.call_api(
Expand Down
Loading