Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 44 additions & 1 deletion guides/data-apps.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ Every prompt you send creates a new version of the app. The agent keeps its work

If a build is taking too long or going in the wrong direction, you can cancel it and try a different prompt.

Versions that were uploaded with a [custom dependency set](#declared-dependencies) show a small package chip in the version timeline. Hover the chip to see the `name@version` list that was installed in the build sandbox for that version — handy for confirming what changed when you carry an app forward or restore an earlier version.

### Restoring an earlier version

If a recent prompt took the app in the wrong direction, you can roll back to any earlier ready version and keep iterating from there.
Expand Down Expand Up @@ -322,11 +324,52 @@ You can also set `LIGHTDASH_URL` and `LIGHTDASH_API_KEY` instead of running `lig
Before you rely on data apps as code in a workflow, keep the following in mind:

- **Enterprise only.** Both the source and destination instances must have data apps enabled, including the build sandbox. See [Self-hosting Data apps](/guides/data-apps/self-hosting) for the configuration required on self-hosted instances.
- **Fixed dependency set.** The sandbox rebuilds against a pre-installed library set. You can edit any source file, but you can't add libraries that the sandbox doesn't already provide, and edits to root config files (`package.json`, `vite.config.js`, `tsconfig.json`, etc.) have no effect on the deployed app.
- **Declared dependencies only.** The sandbox rebuilds against a trusted template. You can declare extra npm packages by editing `package.json` (see [Declared dependencies](#declared-dependencies)), but all other root config files (`vite.config.js`, `tsconfig.json`, `tailwind.config.js`, etc.) are read-only — edits to them have no effect on the deployed app.
- **Semantic layer coupling.** A moved app's queries run against the target project's fields **by name**. If a field the app references is missing in the target project, the upload still succeeds and the build still finishes — but the app surfaces query errors at runtime when a user opens it. Check the app in the destination after uploading.
- **Concurrent build rate limit.** Each project has a cap on how many app builds can run at once. If you upload a large batch and exceed the cap, the server returns HTTP 429 and the CLI reports the error for that app. Re-run the upload for the failed apps once earlier builds complete.
- **Bulk download cap.** `--include-apps` returns at most the first 10 apps in a space. Use `--apps <uuids>` to fetch more or to reach apps that aren't in a space.

### Declared dependencies

An app can declare its own npm dependencies alongside the sandbox's built-in library set. This lets a locally-authored app pull in libraries the template doesn't ship — for example, adding `date-fns` to a small ecommerce dashboard for date formatting, or a niche chart library for a one-off visualization.

Custom dependencies are gated because adding third-party npm packages is a supply-chain capability. Three checks must all pass for an upload to declare dependencies:

1. **Instance flag on** — `LIGHTDASH_APP_CUSTOM_DEPENDENCIES_ENABLED=true` (see [Self-hosting Data apps](/guides/data-apps/self-hosting#custom-dependencies)).
2. **Organization flag on** — the `enable-data-app-custom-dependencies` feature flag, enabled per organization. On single-org self-hosted instances an admin can enable it globally via `LIGHTDASH_ENABLE_FEATURE_FLAGS`.
3. **User has `manage:DataAppDependency` scope** — a dedicated admin-only scope required specifically to add or change custom dependencies. Everyone with `create:DataApp` or `manage:DataApp` can still upload template-only apps; they just can't declare new dependencies.

If any of these is missing, the upload is rejected at the API with a clear error. Template-only uploads (no declared dependencies) are unaffected.

Additional upload-time security screens may also run before your version is accepted (see [Upload-time dependency checks](/guides/data-apps/self-hosting#upload-time-dependency-checks)): a known-malware feed screen (OSV, default on) and an optional minimum release-age guard.

**Adding a dependency locally**

Work inside a downloaded app folder (see [Downloading an app](#downloading-an-app)):

```bash
cd lightdash/apps/ecom-dashboard
pnpm add date-fns
```

`pnpm add` updates both `package.json` and `pnpm-lock.yaml`. Commit both — uploads reject new dependencies without a matching lockfile. If you edit `package.json` by hand, run `pnpm install --lockfile-only` to regenerate the lockfile.

Rules the CLI validates on upload:

- **Registry packages only.** Plain semver versions (`"date-fns": "^3.0.0"`); no git, file, tarball, or URL specs.
- **Up to 60 direct dependencies.**
- **`pnpm-lock.yaml` must be committed** and consistent with `package.json`.
- **`.npmrc` sets `ignore-scripts=true`.** Downloaded apps may be authored by someone else — this stops their dependencies' install/lifecycle scripts from running on your machine when you `pnpm install` locally. Leave it in place. Explicit `pnpm dev` / `pnpm build` still work. The pinned `@lightdash/query-sdk` is exempt from pnpm's release-age policies because its releases are often less than three days old.
- **Root config outside `package.json` and `pnpm-lock.yaml` stays read-only** — the server rebuilds against its trusted template.

The AI builder and in-app UI cannot change the dependency set. Declared dependencies are set by editing `package.json` and re-uploading with `lightdash upload --apps`.

On upload the CLI warns which packages will be installed in the build sandbox. Once the version is built, its `name@version` set appears as a package chip on the version timeline, and the assistant bubble in the chat UI shows the same chip so you can confirm what was actually installed.

**Blob-URL workers**

The app-serving Content Security Policy allows `blob:` sources in `script-src`, `worker-src`, and `child-src`. This means libraries that spin up workers or animation runtimes from blob URLs — for example `canvas-confetti`, `deck.gl`, or `comlink` — work in a deployed app without any extra configuration.

## Data context and the app model

Data apps don't ship with their own copy of your data. They run inside a sandboxed iframe with no direct access to the Lightdash API, so every query the app wants to run is proxied through the Lightdash UI hosting the iframe.
Expand Down
29 changes: 29 additions & 0 deletions guides/data-apps/self-hosting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,33 @@ Restart the backend. The "Data apps" entry will appear in the **New** menu for u
| --- | --- | --- |
| `E2B_TEMPLATE_TAG` | Running Lightdash version (e.g. `0.2917.1`) | Pins the sandbox image to a specific tag of the E2B template. Each Lightdash release publishes a matching tag so the backend and sandbox stay in sync. You can override this to roll back to a previous build (e.g. `E2B_TEMPLATE_TAG=0.2916.0`) or set it to an empty string to use the template's `default` tag. Most operators don't need to touch this. |

### Custom dependencies

Data apps can [declare their own npm dependencies](/guides/data-apps#declared-dependencies) alongside the sandbox template's built-in library set. Because installing third-party packages is a supply-chain capability, the feature is gated at three layers — all three must pass for an upload to declare dependencies:

1. **Instance env flag** (this page): `LIGHTDASH_APP_CUSTOM_DEPENDENCIES_ENABLED=true`. Off by default. Also acts as an instance-wide kill-switch — see below.
2. **Per-organization feature flag**: `enable-data-app-custom-dependencies`. Lets Lightdash Cloud enable the feature for specific orgs without instance-wide exposure. On single-org self-hosted instances, add it to `LIGHTDASH_ENABLE_FEATURE_FLAGS` to turn it on globally.
3. **Admin-only scope**: `manage:DataAppDependency` (see [Permissions](#permissions)). Required to add or change custom dependencies specifically; template-only uploads are unaffected.

Template-only uploads always succeed regardless of these settings.

| Variable | Default | Purpose |
| --- | --- | --- |
| `LIGHTDASH_APP_CUSTOM_DEPENDENCIES_ENABLED` | `false` | Master switch for custom dependencies. When `false`, uploads that declare a non-empty dependency set are rejected at the API, and builds or AI iterations of versions that already have stored custom dependencies also refuse to run — flipping the toggle back to `false` is a hard stop that also blocks sandbox restores of previously-approved dependency sets. Template-only uploads are always accepted. |
| `LIGHTDASH_APP_DEPENDENCY_REGISTRY_HOSTS` | `registry.npmjs.org` | Comma-separated list of npm registry hostnames added to the sandbox egress allowlist when a version has custom dependencies. Use this to point at an internal mirror. Template-only builds never gain these hosts. |
| `LIGHTDASH_APP_DEPENDENCY_INSTALL_TIMEOUT_MS` | `120000` (2 minutes) | Timeout in milliseconds for the `pnpm install` step that runs before the Vite build when a version has custom dependencies. Raise this if your registry is slow, or lower it to fail faster in CI-style setups. |

Turning `LIGHTDASH_APP_CUSTOM_DEPENDENCIES_ENABLED` back off is a full kill-switch: it stops installs on every path, including new uploads, AI-iterate carry-forward builds, and sandbox restores of versions whose dependency sets were previously approved. Use it if you need to disable third-party installs on the instance without redeploying app source.

### Upload-time dependency checks

Two optional guards run on uploads that pass the gates above. Both apply to the resolved lockfile (direct + transitive), so a malicious transitive dependency is caught even if the top-level package looks fine.

| Variable | Default | Purpose |
| --- | --- | --- |
| `LIGHTDASH_APP_DEPENDENCY_MALWARE_CHECK_ENABLED` | `true` | Screens every resolved package in the lockfile against the [OSV](https://osv.dev/) malicious-packages feed and rejects the upload if any version carries a `MAL-` advisory. Precise (near-zero false positives) and defaults on for that reason. **Fails closed** — if `api.osv.dev` can't be reached the upload is rejected. Set to `false` on air-gapped instances or during an OSV outage to keep uploading. |
| `LIGHTDASH_APP_DEPENDENCY_MIN_RELEASE_AGE_DAYS` | `0` (off) | Minimum age in days a resolved dependency's version must have since publication on npm. Guards against freshly published, potentially compromised versions before advisory feeds catch up. Off by default because it adds registry round-trips and can block legitimate recent releases; `3` is a sensible starting point if you want it on. Fails closed when a publish date can't be verified. |

## Costs

Self-hosting Data apps means you pay E2B and your Claude provider directly:
Expand All @@ -72,6 +99,8 @@ Both providers expose usage dashboards. We recommend setting spend limits on bot

Data apps follow the same space-based permission model as charts and dashboards. The relevant scopes (`view:DataApp`, `create:DataApp`, `manage:DataApp`) are bundled into the default system roles - but on enterprise instances using custom roles, you'll need to grant them explicitly. See [Custom roles](/references/workspace/custom-roles) for details.

Adding [custom npm dependencies](#custom-dependencies) to an app requires an additional scope, `manage:DataAppDependency`, which is granted to admins only in the default role set. This is deliberate — declaring third-party packages is a supply-chain capability distinct from uploading a template-only app. Grant it to a custom role only when you're comfortable with those users choosing which packages your build sandbox pulls from your configured registry.

## Troubleshooting

**The "Data apps" entry doesn't appear in the New menu.**
Expand Down
5 changes: 5 additions & 0 deletions self-host/customize-deployment/environment-variables.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -664,6 +664,11 @@ Configure the sandboxed runtime that powers [data apps](/guides/data-apps/self-h
| `E2B_TEMPLATE_TAG` | E2B template tag. Defaults to the running Lightdash version so the template matches the release. |
| `E2B_AI_WRITEBACK_TEMPLATE_NAME` | E2B template used for AI writeback sandboxes. (default=`lightdash-ai-writeback`) |
| `E2B_AI_WRITEBACK_TEMPLATE_TAG` | E2B AI writeback template tag. Defaults to the running Lightdash version. |
| `LIGHTDASH_APP_CUSTOM_DEPENDENCIES_ENABLED` | Instance-wide switch for [custom app dependencies](/guides/data-apps#declared-dependencies). When `false`, uploads with declared dependencies are rejected and builds/iterations of versions with stored custom deps refuse to run (including sandbox restores). Template-only uploads are unaffected. (default=false) |
| `LIGHTDASH_APP_DEPENDENCY_REGISTRY_HOSTS` | Comma-separated npm registry hostnames added to the sandbox egress allowlist when a version has custom dependencies. Point at an internal mirror if needed. (default=`registry.npmjs.org`) |
| `LIGHTDASH_APP_DEPENDENCY_INSTALL_TIMEOUT_MS` | Timeout in milliseconds for the `pnpm install` step that runs before the Vite build when a version has custom dependencies. (default=120000) |
| `LIGHTDASH_APP_DEPENDENCY_MALWARE_CHECK_ENABLED` | Screens the resolved lockfile of custom-dependency uploads against the OSV malicious-packages feed and rejects the upload on any `MAL-` advisory. Fails closed — set to `false` on air-gapped instances or during an OSV outage. (default=true) |
| `LIGHTDASH_APP_DEPENDENCY_MIN_RELEASE_AGE_DAYS` | Minimum age in days a resolved dependency version must have since publication on npm; `0` disables. Guards against freshly published, potentially compromised versions. Fails closed when a publish date can't be verified. (default=0) |

## Managed agent

Expand Down
Loading