Skip to content

Commit a9042ea

Browse files
authored
docs(examples): Adds an example of configuration via environment variables (#246)
Adds a section on configuring Druid using environment variables, recommended for sensitive data. Includes an example of secure injection of Druid secrets and properties via Kubernetes Secrets. Removes sensitive settings from `runtime.properties` and replaces them with environment variables in the cluster examples. This improves security and flexibility, allowing configurations to be easily changed between environments without modifying configuration files.
1 parent 0bb0ac5 commit a9042ea

3 files changed

Lines changed: 114 additions & 21 deletions

File tree

docs/examples.md

Lines changed: 82 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@
3636
requests:
3737
cpu: 1
3838
memory: 1Gi
39-
runtime.properties:
39+
runtime.properties:
4040
druid.plaintextPort=8083
4141
druid.service=druid/historical/hot
4242
cold:
@@ -73,7 +73,7 @@
7373
requests:
7474
cpu: 1
7575
memory: 1Gi
76-
runtime.properties:
76+
runtime.properties:
7777
druid.plaintextPort=8083
7878
druid.service=druid/historical/cold
7979
...
@@ -264,3 +264,83 @@ spec:
264264
druid.metadata.storage.connector.password={ "type": "environment", "variable": "METADATA_STORAGE_PASSWORD" }
265265
...
266266
```
267+
268+
## Configuration via Environment Variables (Recommended for sensitive data)
269+
270+
Standard Druid Docker images allow you to convert any environment variable starting with `druid_` into a configuration property (replacing `_` with `.`). This is often more reliable than the JSON replacement method found in `runtime.properties` and allows you to securely inject any configuration using Kubernetes Secrets.
271+
272+
**Note:** If you define a configuration here via `env`, ensure you remove it from your `runtime.properties` or `common.runtime.properties` to avoid conflicts.
273+
274+
### Example: Securely Injecting Secrets and Druid Properties
275+
276+
```yaml
277+
apiVersion: v1
278+
kind: Secret
279+
metadata:
280+
name: prod-druid
281+
namespace: druid
282+
type: Opaque
283+
stringData:
284+
# Sensitive values
285+
AWS_ACCESS_KEY_ID: "AKIA..."
286+
AWS_SECRET_ACCESS_KEY: "SECRET..."
287+
# You can map full property values here
288+
druid.metadata.storage.connector.password: "db-password"
289+
druid.metadata.storage.connector.connectURI: "jdbc:postgresql://..."
290+
---
291+
apiVersion: druid.apache.org/v1alpha1
292+
kind: Druid
293+
metadata:
294+
name: druid
295+
spec:
296+
env:
297+
# 1. Standard Env Vars
298+
- name: AWS_REGION
299+
value: "nyc3"
300+
301+
# 2. Loading Secrets
302+
- name: AWS_ACCESS_KEY_ID
303+
valueFrom:
304+
secretKeyRef:
305+
name: prod-druid
306+
key: AWS_ACCESS_KEY_ID
307+
- name: AWS_SECRET_ACCESS_KEY
308+
valueFrom:
309+
secretKeyRef:
310+
name: prod-druid
311+
key: AWS_SECRET_ACCESS_KEY
312+
313+
# 3. Mapping Secrets directly to Druid Properties
314+
# The Docker entrypoint converts 'druid_x_y' -> 'druid.x.y'
315+
316+
# Maps to: druid.metadata.storage.connector.password
317+
- name: druid_metadata_storage_connector_password
318+
valueFrom:
319+
secretKeyRef:
320+
name: prod-druid
321+
key: druid.metadata.storage.connector.password
322+
323+
# Maps to: druid.metadata.storage.connector.connectURI
324+
- name: druid_metadata_storage_connector_connectURI
325+
valueFrom:
326+
secretKeyRef:
327+
name: prod-druid
328+
key: druid.metadata.storage.connector.connectURI
329+
330+
nodes:
331+
coordinators:
332+
nodeType: "coordinator"
333+
# ...
334+
runtime.properties: |
335+
druid.service=druid/coordinator
336+
# Note: AWS Keys and Metadata configs are NOT listed here
337+
# because they are injected via the 'env' section above.
338+
druid.metadata.storage.type=postgresql
339+
druid.metadata.storage.connector.user=druid
340+
```
341+
342+
### Notes
343+
344+
- **Environment variable expansion** (using `env` and `envFrom`) can be used for any Druid configuration property, not just passwords. This is the recommended approach for sensitive data and for properties that may change between environments.
345+
- The JSON replacement method (`{ "type": "environment", "variable": "METADATA_STORAGE_PASSWORD" }`) is only supported for certain password fields in `runtime.properties` and is not as flexible as environment variable expansion.
346+
- If you use both methods for the same property, the environment variable will take precedence, but it's best to avoid duplication to prevent confusion.

examples/tiny-cluster-hpa.yaml

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -60,10 +60,8 @@ spec:
6060
6161
# Metadata Store
6262
druid.metadata.storage.type=derby
63-
druid.metadata.storage.connector.connectURI=jdbc:derby://localhost:1527/druid/data/derbydb/metadata.db;create=true
64-
druid.metadata.storage.connector.host=localhost
65-
druid.metadata.storage.connector.port=1527
6663
druid.metadata.storage.connector.createTables=true
64+
# Other settings are set via environment variables
6765
6866
# Deep Storage
6967
druid.storage.type=local
@@ -96,6 +94,15 @@ spec:
9694
path: /tmp/druid/deepstorage
9795
type: DirectoryOrCreate
9896
env:
97+
- name: druid_metadata_storage_type
98+
value: derby
99+
- name: druid_metadata_storage_connector_connectURI
100+
value: jdbc:derby://localhost:1527/druid/data/derbydb/metadata.db;create=true
101+
- name: druid_metadata_storage_connector_host
102+
value: localhost
103+
- name: druid_metadata_storage_connector_port
104+
value: "1527"
105+
# Other env vars
99106
- name: POD_NAME
100107
valueFrom:
101108
fieldRef:
@@ -186,7 +193,7 @@ spec:
186193
extra.jvm.options: |-
187194
-Xmx512M
188195
-Xms512M
189-
196+
190197
routers:
191198
nodeType: "router"
192199
druid.port: 8088
@@ -206,7 +213,7 @@ spec:
206213
druid.router.coordinatorServiceName=druid/coordinator
207214
208215
# Management proxy to coordinator / overlord: required for unified web console.
209-
druid.router.managementProxy.enabled=true
216+
druid.router.managementProxy.enabled=true
210217
extra.jvm.options: |-
211218
-Xmx512M
212219
-Xms512M

examples/tiny-cluster.yaml

Lines changed: 20 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -60,10 +60,8 @@ spec:
6060
6161
# Metadata Store
6262
druid.metadata.storage.type=derby
63-
druid.metadata.storage.connector.connectURI=jdbc:derby://localhost:1527/druid/data/derbydb/metadata.db;create=true
64-
druid.metadata.storage.connector.host=localhost
65-
druid.metadata.storage.connector.port=1527
6663
druid.metadata.storage.connector.createTables=true
64+
# Other settings are set via environment variables
6765
6866
# Deep Storage
6967
druid.storage.type=local
@@ -259,6 +257,14 @@ spec:
259257
path: /tmp/druid/deepstorage
260258
type: DirectoryOrCreate
261259
env:
260+
- name: druid_metadata_storage_type
261+
value: derby
262+
- name: druid_metadata_storage_connector_connectURI
263+
value: jdbc:derby://localhost:1527/druid/data/derbydb/metadata.db;create=true
264+
- name: druid_metadata_storage_connector_host
265+
value: localhost
266+
- name: druid_metadata_storage_connector_port
267+
value: "1527"
262268
- name: POD_NAME
263269
valueFrom:
264270
fieldRef:
@@ -280,15 +286,15 @@ spec:
280286
nodeConfigMountPath: "/opt/druid/conf/druid/cluster/query/broker"
281287
replicas: 1
282288
volumeClaimTemplates:
283-
- metadata:
284-
name: data-volume
285-
spec:
286-
accessModes:
287-
- ReadWriteOnce
288-
resources:
289-
requests:
290-
storage: 2Gi
291-
storageClassName: standard
289+
- metadata:
290+
name: data-volume
291+
spec:
292+
accessModes:
293+
- ReadWriteOnce
294+
resources:
295+
requests:
296+
storage: 2Gi
297+
storageClassName: standard
292298
runtime.properties: |
293299
druid.service=druid/broker
294300
# HTTP server threads
@@ -344,7 +350,7 @@ spec:
344350
extra.jvm.options: |-
345351
-Xmx512M
346352
-Xms512M
347-
353+
348354
routers:
349355
nodeType: "router"
350356
druid.port: 8088
@@ -364,7 +370,7 @@ spec:
364370
druid.router.coordinatorServiceName=druid/coordinator
365371
366372
# Management proxy to coordinator / overlord: required for unified web console.
367-
druid.router.managementProxy.enabled=true
373+
druid.router.managementProxy.enabled=true
368374
extra.jvm.options: |-
369375
-Xmx512M
370376
-Xms512M

0 commit comments

Comments
 (0)