Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/llm-benchmark-update.yml
Original file line number Diff line number Diff line change
Expand Up @@ -302,4 +302,11 @@ jobs:
GH_TOKEN: ${{ secrets.CLOCKWORK_LABS_BOT_PAT }}
run: |
git remote set-url origin "https://x-access-token:${GH_TOKEN}@github.com/${{ steps.pr.outputs.head_repo_full_name }}.git"
# Fetch and rebase in case branch moved since workflow started (e.g., previous benchmark run)
git fetch origin "${{ steps.pr.outputs.head_ref }}"
if ! git rebase "origin/${{ steps.pr.outputs.head_ref }}"; then
git rebase --abort
echo "::error::Rebase failed due to conflicts. The PR branch may have been updated during the benchmark run. Please re-run /update-llm-benchmark."
exit 1
fi
git push origin "HEAD:${{ steps.pr.outputs.head_ref }}"
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Traditional stack: SpacetimeDB:

## Everything is Persistent

You will ask, does everything need to be persistent? Won't that be a lot of data? Won't that be slow. The answers are yes, no, and no. SpacetimeDB persists everything by default, even the full history of everything that has every changed. It should be **your choice** to delete data, not the databases. SpacetimeDB gives you that choice.
You will ask, does everything need to be persistent? Won't that be a lot of data? Won't that be slow. The answers are yes, no, and no. SpacetimeDB persists everything by default, even the full history of everything that has every changed. It should be **your choice** to delete data, not the database's. SpacetimeDB gives you that choice.

SpacetimeDB holds all your data in memory for blazing-fast access, but automatically persists everything to disk. You get the speed of in-memory computing with the durability of a traditional database.

Expand Down
149 changes: 148 additions & 1 deletion docs/docs/00200-core-concepts/00300-tables/00500-schedule-tables.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,156 @@ fn send_reminder(ctx: &ReducerContext, reminder: Reminder) -> Result<(), String>
</TabItem>
</Tabs>

## Inserting Schedules

To schedule an action, insert a row into the schedule table with a `scheduled_at` value. You can schedule actions to run:

- **At intervals** - Execute repeatedly at fixed time intervals (e.g., every 5 seconds)
- **At specific times** - Execute once at an absolute timestamp

### Scheduling at Intervals

Use intervals for periodic tasks like game ticks, heartbeats, or recurring maintenance:

<Tabs groupId="server-language" queryString>
<TabItem value="typescript" label="TypeScript">

```typescript
import { ScheduleAt } from 'spacetimedb';

// Schedule to run every 5 seconds (5,000,000 microseconds)
ctx.db.reminder.insert({
scheduled_id: 0n,
scheduled_at: ScheduleAt.interval(5_000_000n),
message: "Check for updates",
});

// Schedule to run every 100 milliseconds
ctx.db.reminder.insert({
scheduled_id: 0n,
scheduled_at: ScheduleAt.interval(100_000n), // 100ms in microseconds
message: "Game tick",
});
```

</TabItem>
<TabItem value="csharp" label="C#">

```csharp
// Schedule to run every 5 seconds
ctx.Db.Reminder.Insert(new Reminder
{
Message = "Check for updates",
ScheduleAt = new ScheduleAt.Interval(TimeSpan.FromSeconds(5))
});

// Schedule to run every 100 milliseconds
ctx.Db.Reminder.Insert(new Reminder
{
Message = "Game tick",
ScheduleAt = new ScheduleAt.Interval(TimeSpan.FromMilliseconds(100))
});
```

</TabItem>
<TabItem value="rust" label="Rust">

```rust
use spacetimedb::{ScheduleAt, Duration};

// Schedule to run every 5 seconds
ctx.db.reminder().insert(Reminder {
id: 0,
message: "Check for updates".to_string(),
scheduled_at: ScheduleAt::Interval(Duration::from_secs(5).into()),
});

// Schedule to run every 100 milliseconds
ctx.db.reminder().insert(Reminder {
id: 0,
message: "Game tick".to_string(),
scheduled_at: ScheduleAt::Interval(Duration::from_millis(100).into()),
});
```

</TabItem>
</Tabs>

### Scheduling at Specific Times

Use specific times for one-shot actions like sending a reminder at a particular moment or expiring content:

<Tabs groupId="server-language" queryString>
<TabItem value="typescript" label="TypeScript">

```typescript
import { ScheduleAt } from 'spacetimedb';

// Schedule for 10 seconds from now
const tenSecondsFromNow = ctx.timestamp.microseconds + 10_000_000n;
ctx.db.reminder.insert({
scheduled_id: 0n,
scheduled_at: ScheduleAt.time(tenSecondsFromNow),
message: "Your auction has ended",
});

// Schedule for a specific Unix timestamp (microseconds since epoch)
const targetTime = 1735689600_000_000n; // Jan 1, 2025 00:00:00 UTC
ctx.db.reminder.insert({
scheduled_id: 0n,
scheduled_at: ScheduleAt.time(targetTime),
message: "Happy New Year!",
});
```

</TabItem>
<TabItem value="csharp" label="C#">

```csharp
// Schedule for 10 seconds from now
ctx.Db.Reminder.Insert(new Reminder
{
Message = "Your auction has ended",
ScheduleAt = new ScheduleAt.Time(DateTimeOffset.UtcNow.AddSeconds(10))
});

// Schedule for a specific time
var targetTime = new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero);
ctx.Db.Reminder.Insert(new Reminder
{
Message = "Happy New Year!",
ScheduleAt = new ScheduleAt.Time(targetTime)
});
```

</TabItem>
<TabItem value="rust" label="Rust">

```rust
use spacetimedb::{ScheduleAt, Duration};

// Schedule for 10 seconds from now
let ten_seconds_from_now = ctx.timestamp + Duration::from_secs(10);
ctx.db.reminder().insert(Reminder {
id: 0,
message: "Your auction has ended".to_string(),
scheduled_at: ScheduleAt::Time(ten_seconds_from_now),
});

// Schedule for immediate execution (current timestamp)
ctx.db.reminder().insert(Reminder {
id: 0,
message: "Process now".to_string(),
scheduled_at: ScheduleAt::Time(ctx.timestamp.clone()),
});
```

</TabItem>
</Tabs>

## How It Works

1. **Insert a row** with a `schedule_at` time
1. **Insert a row** with a `ScheduleAt` value
2. **SpacetimeDB monitors** the schedule table
3. **When the time arrives**, the specified reducer/procedure is automatically called with the row as a parameter
4. **The row is typically deleted** or updated by the reducer after processing
Expand Down
88 changes: 36 additions & 52 deletions docs/llms/docs-benchmark-analysis.md
Original file line number Diff line number Diff line change
@@ -1,80 +1,64 @@
# Benchmark Failure Analysis

Generated from: `C:\Users\Tyler\Developer\SpacetimeDB\tools\xtask-llm-benchmark\../../docs/llms/docs-benchmark-details.json`
Generated from: `/__w/SpacetimeDB/SpacetimeDB/tools/xtask-llm-benchmark/../../docs/llms/docs-benchmark-details.json`

## Summary

- **Total failures analyzed**: 26
- **Total failures analyzed**: 59

## Analysis

# SpacetimeDB Benchmark Test Failures Analysis
# Analysis of SpacetimeDB Benchmark Test Failures: Rust and C#

## Rust Failures

### 1. Root Causes
- **Compile/Publish Errors (3 failures)**:
- The primary issue across the failures is related to the use of `ScheduleAt::every_micros` versus `ScheduleAt::RepeatMicros`, which indicates a lack of clarity in the documentation about the correct method of using scheduled types.
- Another issue is the incorrect implementation of `pub` for some fields and missing `#[derive(SpacetimeType)]` for structs, which has led to schema mismatches.

- **Other Failures (1 failure)**:
- The test `t_003_struct_in_table` has a mismatch where the expected reducer setup differs from what's provided. This highlights insufficient documentation around initial setup requirements for reducers.
- **Inconsistent Table Names**: Many failures arise from incorrect or inconsistent table names used in the code segments compared to what is expected in the benchmarks. Tables such as `users`, `drawings`, `event`, etc., are referenced with incorrect names leading to errors.
- **Lack of `pub` Keyword**: Many struct fields are missing the `pub` keyword, causing issues with accessing these fields outside the module.
- **Unstable API Warnings**: Numerous tests are failing due to reliance on unstable methods or API changes.
- **Missing Error Handling**: Functions that should return `Result` types do not, leading to issues when error handling is assumed.

### 2. Recommendations
- **Documentation Updates**:
- **Scheduled Types Documentation**: Enhance the section on scheduled types in the documentation to clarify the use of `ScheduleAt::every_micros` and `ScheduleAt::RepeatMicros`. Example for addition:
```markdown
### Scheduled Types
- Use `ScheduleAt::every_micros(interval)` for non-repeating intervals.
- Use `ScheduleAt::RepeatMicros(interval)` for repeating intervals. Ensure proper usage to avoid publishing errors.
```

- **Section on Structs and Reducers**: Update the section dealing with struct fields to illustrate the necessity of using `pub` where it applies and clarifying how reducers must align:
```markdown
### Struct Definitions
- Struct fields must be marked as `pub` to ensure they are accessible within the SpacetimeDB context.
- Reducers must be defined properly; ensure that each reducer matches expected configurations in your schemas.
```

- **Example Code Alignment**: Revise example code throughout documentation to align with the latest syntax and ensure that all required attributes are included.
- **Update Table Names for Consistency**:
- In the table definitions and usages, ensure names are consistent throughout the documentation. For instance:
- `event` should replace all instances of `events`.
- `primitive` should replace all instances of `primitives`.
- `drawing` should replace all instances of `drawings`.
- **Add `pub` Keyword for Structs and Fields**:
- Documentation should specify that structs and their fields must be public for access.
- **Document API Stability**:
- Clearly mark all APIs that are unstable and subject to change. Provide a dedicated section for upcoming breaking changes, if possible.
- **Error Handling**:
- Example code should consistently include error handling to return results or handle errors gracefully. This should be highlighted in the documentation.

### 3. Priority
- **High Impact Fixes**:
1. Scheduled Types Documentation (to prevent compile errors).
2. Structs and Reducers Section (to ensure schema and function alignment).
1. **Fix Table Name Consistencies**: This will directly resolve numerous failures and prevent potential confusion.
2. **Add `pub` Keyword Requirement**: Ensuring access to fields would significantly improve usability and reduce errors in testing.
3. **Document API Stability**: Prevent future issues arising from unexpected API changes.

---

## C# Failures

### 1. Root Causes
- **Table Naming Issues (19 failures)**:
- The primary issue causing the failures is the inconsistency in the use of table names (e.g., `entities` vs. `Entity`). Lack of clear guidelines on naming conventions has led to widespread discrepancies.

- **Timeout Issues (3 failures)**:
- Additionally, the timeout failures indicate that certain processes aren’t being documented well in terms of what expectations exist for execution time and potential pitfalls leading to these issues.
- **Table Name Consistency**: Like Rust, several tests in C# fail due to improper table names, particularly for `users`, `results`, and `accounts`.
- **Lack of `public` Modifiers**: Many structs and fields are missing the `public` modifier, which can restrict access from the context in which they are used.
- **API Instability Documentation**: References to unstable API methods are rampant, causing uncertainty in method usage and expectations.

### 2. Recommendations
- **Documentation Updates**:
- **Table Naming Conventions**: Introduce a comprehensive section specifying the naming conventions for tables. Example for addition:
```markdown
### Table Naming Conventions
- Table names should be singular and PascalCase (e.g., `User` instead of `users`).
- Ensure that when creating and querying tables, the names are consistently used to avoid schema parity issues.
```

- **Timeout Handling Guidance**: Provide clearer information on how to handle potential timeout issues within operations:
```markdown
### Handling Timeouts
- If encountering timeout errors during transactions, consider optimizing the initial data load or query processes.
- Implement logging to help identify which part of your transaction is leading to the timeout.
```
- **Align Table Names**:
- Replace all instances of `Users` with `User`, both in definitions and references.
- Standardize other table names like `Results` and `Accounts` similarly.
- **Include `public` Modifiers**:
- The documentation should specify the need for `public` qualifiers on structs and fields explicitly, with examples demonstrating proper usage.
- **Mark Unstable APIs**:
- Document all APIs that are unstable and document their expected changes. Provide clear notes in the sections where these APIs are discussed, especially around construction methods and reducer configurations.

### 3. Priority
- **High Impact Fixes**:
1. Table Naming Conventions (most immediate to fix widespread errors).
2. Timeout Handling Guidance (to improve performance and reliability in operations).
1. **Align Table Names**: This will immediately tackle a significant number of test failures.
2. **Public Modifier Guidelines**: Enhancing understanding of access levels will greatly improve code quality and tests.
3. **Unstable API Documentation**: This is a long-term fix but is critical for preventing confusion around API usage in future versions.

---
---

This structured approach will help improve the accessibility and clarity of the SpacetimeDB documentation, directly addressing the root causes of current benchmark test failures.
These actionable insights, with specific attention to documentation updates, can help mitigate the current benchmark failures and improve user experience in both Rust and C# implementations of the SpacetimeDB.
14 changes: 7 additions & 7 deletions docs/llms/docs-benchmark-comment.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

| Language | Mode | Category | Tests Passed | Task Pass % |
|----------|------|----------|--------------|-------------|
| Rust | rustdoc_json | basics | 25/27 | 83.3% ⬇️ -8.3% |
| Rust | rustdoc_json | schema | 29/34 | 80.0% |
| Rust | rustdoc_json | **total** | 54/61 | **81.8%** ⬇️ -4.5% |
| C# | docs | basics | 7/27 | 16.0% ⬇️ -84.0% |
| C# | docs | schema | 18/34 | 48.5% ⬇️ -31.5% |
| C# | docs | **total** | 25/61 | **30.8%** ⬇️ -60.2% |
| Rust | rustdoc_json | basics | 9/27 | 36.1% ⬇️ -47.2% |
| Rust | rustdoc_json | schema | 9/34 | 26.5% ⬇️ -53.5% |
| Rust | rustdoc_json | **total** | 18/61 | **31.7%** ⬇️ -50.1% |
| C# | docs | basics | 12/27 | 61.1% |
| C# | docs | schema | 10/34 | 31.5% |
| C# | docs | **total** | 22/61 | **47.7%** |

_Compared against master branch baseline_

<sub>Generated at: 2026-01-16T19:34:55.608Z</sub>
<sub>Generated at: 2026-01-20T01:46:16.880Z</sub>
Loading
Loading