Skip to content

Expand Structured Streaming SKILL.md from table of contents to actionable hub#212

Open
CheeYuTan wants to merge 1 commit intodatabricks-solutions:mainfrom
CheeYuTan:feat/structured-streaming-skill-hub
Open

Expand Structured Streaming SKILL.md from table of contents to actionable hub#212
CheeYuTan wants to merge 1 commit intodatabricks-solutions:mainfrom
CheeYuTan:feat/structured-streaming-skill-hub

Conversation

@CheeYuTan
Copy link
Contributor

Summary

The Structured Streaming SKILL.md was only 66 lines — just a table of contents despite having 8 good reference files. Expanded to 247 lines with actionable content:

  • Quick starts: Kafka-to-Delta, foreachBatch MERGE, availableNow scheduled streaming
  • Trigger selection guide: realTime (DBR 16.4+), processingTime, availableNow, once
  • Watermark essentials with duration selection table
  • Stream join patterns: stream-stream with time bounds, stream-static with broadcast hints
  • Checkpoint best practices summary
  • Production checklist and common issues troubleshooting table

No new files — just an expanded SKILL.md that makes the existing reference files more discoverable and useful.

Test plan

  • All trigger syntax verified against PySpark docs
  • No install_skills.sh changes needed (no new files)
  • Cross-referenced with existing reference files for consistency

…able hub

The SKILL.md was 66 lines — just a table of contents despite having 8
good reference files. Expanded to 247 lines with:

- Actionable quick starts: Kafka-to-Delta, foreachBatch MERGE, availableNow
- Trigger selection guide (realTime, processingTime, availableNow)
- Watermark essentials with duration selection table
- Stream join patterns (stream-stream with time bounds, stream-static with broadcast)
- Checkpoint best practices summary
- Production checklist
- Common issues troubleshooting table
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant