Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,73 @@ idle for a configurable duration.
- **HTTP middleware example** — with accurate `Retry-After` and `RateLimit-*`
response headers.

## Architecture

`BucketLimiter` is a thin manager: it maps each key to its own `Limiter`,
builds new ones on demand through a factory, persists them in a pluggable
`Storage`, and runs a single background goroutine that evicts idle keys.

```mermaid
flowchart TD
subgraph caller["Your code"]
C["GetOrAdd(key)"]
end

subgraph manager["BucketLimiter[K]"]
direction TB
F["newLimiter func() Limiter<br/>(factory)"]
A["access map<br/>K → last-use time"]
S["sweepLoop goroutine<br/>evicts idle keys"]
end

subgraph store["Storage[K, Limiter]"]
direction LR
K1["user-123 → bucket"]
K2["user-456 → bucket"]
K3["10.0.0.7 → bucket"]
end

C -->|"1. Load / LoadOrStore"| store
C -.->|"2. build on miss"| F
F -.->|"fresh *rate.Limiter"| store
C -->|"3. touch"| A
S -->|"Delete idle"| store
S -->|"Delete idle"| A

K1 & K2 & K3 -->|"independent<br/>token buckets"| RL["golang.org/x/time/rate"]
```

Each key owns an **independent** token bucket, so one client draining its
budget has no effect on any other. A typical `GetOrAdd(key).Allow()` call:

```mermaid
sequenceDiagram
autonumber
participant App as Your code
participant BL as BucketLimiter
participant St as Storage
participant Lim as Limiter (bucket)

App->>BL: GetOrAdd(key)
BL->>St: Load(key)
alt key exists
St-->>BL: existing Limiter
else first use of key
BL->>BL: newLimiter()
BL->>St: LoadOrStore(key, fresh)
Note over BL,St: atomic — racing callers<br/>share one instance
St-->>BL: stored Limiter
end
BL->>BL: touch(key) — refresh idle timer
BL-->>App: Limiter
App->>Lim: Allow()
alt token available
Lim-->>App: true (consume 1 token)
else bucket empty
Lim-->>App: false (rate limited → 429)
end
```

## Installation

```bash
Expand Down
59 changes: 59 additions & 0 deletions docs/TOKEN_BUCKET.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,19 @@ Picture a bucket that holds tokens:
request limited (bucket empty)
```

The same idea as a flow diagram:

```mermaid
flowchart TD
R["Refill: +r tokens / second"] -->|"drip"| B
B{"Bucket<br/>capacity = burst b"}
B -->|"overflow above b"| X["discarded"]
Req(["Incoming request<br/>needs 1 token"]) --> Q{"tokens ≥ 1 ?"}
Q -->|"yes"| Take["consume 1 token"] --> Allowed(["✅ allowed"])
Q -->|"no"| Limited(["⛔ limited — reject or wait"])
B -.->|"current level"| Q
```

Rules:

1. Tokens are added to the bucket at a steady **rate** `r` (tokens per second).
Expand Down Expand Up @@ -208,6 +221,24 @@ full bucket. Choose `deleteAfter` comfortably longer than the window over which
you want the limit to hold (e.g. minutes, not milliseconds) so a client cannot
reset its own bucket by pausing briefly.

The lifecycle of a single key:

```mermaid
stateDiagram-v2
[*] --> Absent
Absent --> Active: GetOrAdd(key)<br/>newLimiter() builds a fresh bucket
Active --> Active: GetOrAdd(key)<br/>refreshes idle timer
Active --> Idle: no access for deleteAfter
Idle --> Active: GetOrAdd(key)<br/>before the sweep runs
Idle --> Absent: sweepLoop evicts<br/>(state discarded)
Active --> Absent: Remove(key)
Absent --> [*]
```

The sweep is not instantaneous, so a key idle past `deleteAfter` lingers until
the next tick — worst case ~1.5·`deleteAfter` after its last use with the
default interval.

## HTTP response headers

Well-behaved HTTP clients can self-throttle if you tell them the state of their
Expand All @@ -227,6 +258,34 @@ the example also emits the legacy `X-RateLimit-*` variants for older clients.
[RFC 9110 §10.2.3](https://www.rfc-editor.org/rfc/rfc9110#section-10.2.3) and is
computed from the reservation delay so it is accurate rather than a guess.

End to end, a request through the middleware:

```mermaid
sequenceDiagram
autonumber
participant Client
participant MW as Middleware
participant BL as BucketLimiter
participant Lim as Limiter (per-IP bucket)
participant H as Handler

Client->>MW: HTTP request
MW->>MW: extract client IP<br/>(net.SplitHostPort)
MW->>BL: GetOrAdd(ip)
BL-->>MW: Limiter
MW->>Lim: ReserveN(now, 1)
alt token available now (delay == 0)
Lim-->>MW: reservation, DelayFrom(now) = 0
MW->>H: serve
H-->>MW: response
MW-->>Client: 200 OK<br/>RateLimit-Limit / Remaining / Reset
else bucket empty (delay > 0)
Lim-->>MW: reservation, DelayFrom(now) = d
MW->>Lim: reservation.Cancel()<br/>(return the token)
MW-->>Client: 429 Too Many Requests<br/>Retry-After: ⌈d⌉
end
```

## Comparison with other algorithms

| Algorithm | Bursts | Memory/key | Notes |
Expand Down
Loading