Skip to content

Dataflow v1.0.8 Release Note

Choose a tag to compare

View all tags
@SunnyHaze SunnyHaze released this 19 Dec 08:08
· 113 commits to main since this release
v1.0.8
df4f689

🚀 DataFlow v1.0.8 更新日志（v1.0.7 → v1.0.8）

🔑 主要功能更新

⚙️ Pipeline Compile 体系增强

  • 修复 PromptedTemplatedGenerator 在 compile 与 forward 之间的不一致问题，解决 #363。
  • 新增 resume_step 功能，支持从任意 Operator 恢复执行，大幅提升长链路 Pipeline 的可控性。
    感谢 @SunnyHaze。

🤖 Serving 能力扩展：本地多模态 VLM Serving

  • 新增 本地模型 VLM Serving 与更多特性支持，使本地推理体验更完整。
    感谢 @fatty-belly。

🧪 Eval Pipeline 升级

  • 新增 EvalPipeline，实现更完善的评测能力与执行链路。
    感谢 @YalinFeng01。

🌐 Google VertexAI Serving 调试与优化

  • 完善 Google VertexAI Serving 的调试逻辑，增强稳定性与可用性。
    感谢 @wongzhenhao。

🈶 中文能力增强：N-gram 推理过滤器全面支持中文

  • Reasoning N-gram Filter 与 N-gram Filter 均新增中文支持，显著提升在中文大模型任务中的评测一致性。
    感谢 @zzy1127、@scuuy。

🧩 其他重要改进

🔧 多项 Bug 修复与行为一致性提升

  • 修复 Linux 下 text2vecsql 的兼容性问题。感谢 @yaodongwen。
  • 修复 GetFilterFinalScorerPrompt 调用错误。感谢 @DKAMX。
  • 删除冗余算子、修复部分 Descriptor 逻辑。感谢 @SunnyHaze。
  • 修复 pipeline.compile() 在无 llm_serving 时错误跳过算子的行为。感谢 @fatty-belly。

📚 Prompt Registry 新增

  • 在 core_text 系列算子中加入 Prompt Registry，使 prompt 调用更规范化。
    感谢 @scuuy。

🔍 代码结构优化

  • 对 Serving、Google vertex、编译流程等进行小规模整理与调试修复。

🌟 功能亮点摘要

  • 本地 VLM Serving 正式可用
  • Pipeline 支持 resume from operator
  • 中文 N-gram Filter 支持
  • Compile 行为更加稳定和一致
  • Google VertexAI Serving 更易调试
  • Eval Pipeline 全面升级

👨‍💻 新的贡献者

🚀 DataFlow v1.0.8 Release Notes（v1.0.7 → v1.0.8）

🔑 Major Feature Updates

⚙️ Enhanced Pipeline Compile System

  • Fixed inconsistency between compile and forward for PromptedTemplatedGenerator (issue #363).
  • Added resume_step, enabling pipelines to resume execution from any operator — a major improvement for long pipelines.
    Thanks to @SunnyHaze.

🤖 Expanded Serving Capabilities: Local VLM Serving

  • Introduced local VLM model serving with several new features, enabling more complete local inference workflows.
    Thanks to @fatty-belly.

🧪 Eval Pipeline Upgrade

  • Added a new EvalPipeline with more robust and flexible evaluation support.
    Thanks to @YalinFeng01.

🌐 Google VertexAI Serving Debug & Improvements

  • Improved debugging logic and reliability for Google VertexAI model serving.
    Thanks to @wongzhenhao.

🈶 Chinese Language Support for N-gram Filters

  • Both the reasoning N-gram Filter and general N-gram Filter now support Chinese, improving evaluation accuracy for Chinese LLM tasks.
    Thanks to @zzy1127 and @scuuy.

🧩 Additional Improvements

🔧 Key Bug Fixes & Behavior Improvements

  • Fixed Linux compatibility issues for text2vecsql. Thanks to @yaodongwen.
  • Fixed incorrect invocation of GetFilterFinalScorerPrompt. Thanks to @DKAMX.
  • Removed redundant operators and improved description logic. Thanks to @SunnyHaze.
  • Fixed pipeline.compile() incorrectly skipping operators without llm_serving. Thanks to @fatty-belly.

📚 Prompt Registry Added

  • Added a prompt registry mechanism across core_text operators, making prompt management more systematic.
    Thanks to @scuuy.

🔍 Structural and Codebase Improvements

  • Minor fixes across Serving, VertexAI integration, and compile routines.

🌟 Highlights Recap

  • Local VLM Serving is now supported
  • Pipelines can resume from any operator
  • Chinese support for N-gram Filters
  • More stable compile behavior
  • Improved Google VertexAI Serving debugging
  • Upgraded Eval Pipeline

👨‍💻 New Contributors

Contributors

DKAMX, SunnyHaze, and 6 other contributors
Assets 2
Loading