Skip to content

Commit 9338cf1

Browse files
committed
Add version 1.2.x documentation #140
1 parent fdfbde6 commit 9338cf1

53 files changed

Lines changed: 6060 additions & 1 deletion

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

docusaurus.config.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,9 +42,15 @@ const config: Config = {
4242
showLastUpdateTime: true,
4343
showLastUpdateAuthor: false,
4444
includeCurrentVersion: true,
45+
lastVersion: 'current',
4546
versions: {
47+
'1.2.x': {
48+
label: '1.2.x',
49+
banner: 'none',
50+
},
4651
current: {
47-
label: 'Version: 1.2.x',
52+
label: '1.3.x (Latest)',
53+
path: '/',
4854
},
4955
},
5056
},
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"label": "Concepts",
3+
"position": 2,
4+
"link": {
5+
"type": "generated-index",
6+
"slug": "/concepts",
7+
"description": "Learn the fundamental concepts behind FlowSynx, including architecture, data flow, and best practices."
8+
}
9+
}
Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
---
2+
sidebar_position: 3
3+
description: Understanding how FlowSynx manages task dependencies and execution order using DAG-based scheduling to ensure reliable, predictable, and optimized workflow execution.
4+
---
5+
6+
# Dependencies & Execution Order
7+
8+
Workflow execution is not just about running tasks—it's about running them **in the correct order**, **with the correct prerequisites**, and **at the correct moment**. In FlowSynx, this process is governed by a powerful dependency model rooted in Directed Acyclic Graphs (DAGs). Dependencies define the logical and temporal relationships between tasks, ensuring that each step executes exactly when it should and that the workflow behaves consistently, even as it grows in complexity.
9+
10+
## The Role of Dependencies in Workflow Design
11+
12+
Dependencies specify which tasks must complete before another task can begin. In DAG terminology, these are represented as **directed edges** between nodes:
13+
14+
- **Parent / Upstream Task** — A prerequisite
15+
- **Child / Downstream Task** — A task that depends on one or more parents
16+
17+
A task *cannot* start until all upstream dependencies have completed successfully.
18+
19+
### Basic Dependency Example
20+
21+
<img src="/img/Basic-Dependency-Example.jpg" />
22+
23+
## Validating Dependencies & Detecting Structural Problems
24+
25+
To guarantee correctness before execution begins, FlowSynx automatically runs a series of structural validation checks. These checks ensure that the workflow is valid, executable, and free of ambiguous or inconsistent definitions.
26+
27+
FlowSynx uses a combination of simple hash-based scans and a classic cycle-detection algorithm.
28+
29+
30+
### 1. Duplicate Task Detection (O(n))
31+
32+
FlowSynx uses a **single-pass HashSet membership test** to ensure task names are unique:
33+
34+
```
35+
for each task:
36+
if task.Name in hashset: DUPLICATE
37+
else add to hashset
38+
```
39+
40+
#### Diagram
41+
<img src="/img/Duplicate-Task-Detection.jpg" />
42+
43+
44+
### 2. Missing Dependencies / Missing Branch Targets (O(V + E))
45+
46+
FlowSynx builds a HashSet of all defined task names and checks whether dependencies reference anything that doesn't exist.
47+
48+
<img src="/img/Missing-Dependencies.jpg" />
49+
50+
### 3. Cyclic Dependency Detection (Kahn’s Algorithm) (O(V + E))
51+
52+
FlowSynx uses **Kahn’s Algorithm**, a topological sort strategy, to detect cycles. Kahn’s Algorithm is a classic method for detecting cycles in a directed graph (like a workflow DAG).
53+
If the workflow has a cycle, you cannot topologically sort it, and that’s exactly how the algorithm detects the problem.
54+
55+
#### Algorithm Steps
56+
- Each task is a node (V).
57+
- Each dependency is a directed edge (E): **A → B** means A must run before B.
58+
- We compute in-degree for each node:
59+
- How many tasks depend on this node?
60+
- Repeatedly remove nodes with in-degree = 0 (tasks with no unmet prerequisites).
61+
- Removing a node also removes its outgoing edges, lowering in-degrees of its neighbors.
62+
- If there is a cycle, some nodes will never reach in-degree 0.
63+
64+
#### Diagram — No Cycle
65+
66+
<img src="/img/DAG-NoCycle-Exists.jpg" />
67+
```
68+
in-degree:
69+
A:0, B:1, C:1
70+
71+
Queue: [A]
72+
Pop A → decrement B → B:0 → enqueue B
73+
Pop B → decrement C → C:0 → enqueue C
74+
Pop C → done
75+
76+
Processed = 3, Total = 3 → no cycle
77+
```
78+
79+
#### Diagram — Cycle Exists
80+
<img src="/img/DAG-Cycle-Exists.jpg" />
81+
82+
```
83+
in-degree:
84+
A:1, B:1, C:1
85+
86+
Queue: (empty)
87+
88+
Processed = 0 < 3 → cycle detected
89+
```
90+
91+
## Dynamic Execution Planning in FlowSynx
92+
93+
FlowSynx analyzes task dependencies and computes the execution plan dynamically.
94+
95+
### Execution Flow Diagram
96+
<img src="/img/Execution-Flow-Diagram.jpg" />
97+
98+
## Concurrency and Parallel Execution
99+
100+
DAGs allow tasks with no dependency relationship to run concurrently.
101+
102+
### Parallelism Example
103+
<img src="/img/Parallelism-Example.jpg" />
104+
105+
A and B run in parallel; C waits for both.
106+
107+
## Dependency-Driven Reliability
108+
109+
### Error Isolation Diagram
110+
<img src="/img/Error-Isolation-Diagram.jpg" />
111+
112+
```
113+
If [B] fails:
114+
[C] blocked
115+
[D] blocked
116+
[E] blocked
117+
```
118+
119+
FlowSynx can isolate the branch containing failures without affecting unrelated parts of the workflow.
120+
121+
## Conditional Branching and Advanced Flows
122+
<img src="/img/Conditional-Branching-Flow.jpg" />
123+
124+
## Scaling to Large Pipelines
125+
126+
### Large Pipeline DAG
127+
<img src="/img/Large-Pipeline-DAG.jpg" />
128+
129+
FlowSynx’s dependency model enables:
130+
131+
- Automated ordering
132+
- Fast topological sorting
133+
- Predictable state transitions
134+
- High concurrency
135+
- Fine-grained error localization
136+
- Improved debuggability through clear DAG visualization
137+
138+
---
139+
140+
Proper dependency management is not just a feature of FlowSynx—it is one of the core principles enabling powerful,
141+
reliable, and scalable workflow automation. By combining **hash-based validations**, **graph integrity checks**, and
142+
**Kahn’s Algorithm**, FlowSynx ensures that every workflow is structurally sound *before a single task executes*. And
143+
through DAG-driven scheduling, FlowSynx guarantees that each workflow runs exactly as intended, regardless of complexity or scale.
Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
---
2+
sidebar_position: 2
3+
description: An introduction to Directed Acyclic Graphs (DAGs) and how they power deterministic, reliable, and parallelizable workflow orchestration in FlowSynx.
4+
---
5+
6+
# Directed Acyclic Graphs (DAG)
7+
8+
Directed Acyclic Graphs (DAGs) form the mathematical and architectural backbone of modern workflow orchestration systems.
9+
In FlowSynx, DAGs provide the structure through which complex processes are broken down into manageable, deterministic
10+
sequences of tasks. By enforcing directionality and prohibiting cycles, a DAG makes it possible to define workflows that
11+
are predictable, parallelizable, and inherently safe from infinite loops.
12+
13+
## What Is a DAG?
14+
15+
A Directed Acyclic Graph is composed of **nodes** (representing tasks or operations) connected by **directed edges**
16+
(representing dependencies or ordering constraints). Each edge points from one task to another, indicating that the
17+
destination task cannot begin until the source task has completed. The term *acyclic* means that no circular paths
18+
exist—following the arrows will never bring you back to a previously visited node.
19+
20+
This property is essential in workflow systems because it ensures:
21+
22+
- **Deterministic Execution**: The system can always compute a valid order of execution.
23+
- **No Circular Dependencies**: Workflows cannot deadlock or recurse indefinitely.
24+
- **Safe Parallelism**: Independent tasks can run simultaneously without risk of violating dependencies.
25+
- **Clear Visualization**: Engineers can easily understand and communicate workflow logic.
26+
27+
## Why DAGs Matter in Workflow Orchestration
28+
29+
Modern orchestration engines such as FlowSynx rely on DAGs to ensure reliability, scalability, and fault isolation
30+
within complex processes. By having explicit dependencies, the workflow engine can decide *when* to execute a task,
31+
*whether* it can be retried, and *how* failures should propagate.
32+
33+
### Key Advantages
34+
35+
### 1. Deterministic Ordering
36+
FlowSynx uses DAG analysis to compute the task execution order at runtime. Because no cycles exist, the engine can
37+
always derive a topological sort—ensuring a clear, predictable sequence.
38+
39+
### 2. Parallelization of Independent Tasks
40+
If multiple nodes have no dependencies on each other, FlowSynx can execute them concurrently. This dramatically improves
41+
workflow throughput, especially in data-intensive or distributed environments.
42+
43+
### 3. Fine-Grained Error Handling
44+
Each node runs independently, so failures can be isolated to specific branches. FlowSynx supports:
45+
46+
- Dependency-aware retries
47+
- Graceful degradation
48+
- Fail-fast or fail-soft strategies
49+
- Human-in-the-loop interventions (optional)
50+
51+
### 4. Complex Logic Modeling
52+
DAGs are flexible enough to represent:
53+
54+
- Conditional paths
55+
- Branching and merging
56+
- Loops expressed as *repeated tasks* rather than true cycles
57+
- Multi-stage transformations
58+
- Integration pipelines and event-driven operations
59+
60+
This makes DAGs suitable not only for data pipelines, but also CI/CD automation, ETL operations, distributed job coordination,
61+
and business logic orchestration.
62+
63+
## Real-World DAG Structure Example
64+
65+
Below is an example visualization demonstrating how tasks can flow through multiple stages before converging. Notice that all
66+
edges point forward, and no cycles occur. This ensures the workflow progresses cleanly from the starting node (“Task 1”) toward
67+
the final output (“Task 8”).
68+
69+
<img src="/img/DAG-example.jpg" />
70+
71+
In this diagram:
72+
73+
- **Task 1** initiates the workflow.
74+
- Subsequent tasks branch out into parallel sub-processes.
75+
- Independent tasks run concurrently, improving performance.
76+
- All paths eventually merge at **Task 8**, forming a unified result.
77+
78+
This pattern is common in real orchestration scenarios, such as data preparation pipelines or multi-step file processing workflows.
79+
80+
## DAGs in FlowSynx
81+
82+
FlowSynx takes advantage of DAGs not only for execution sequencing, but also for:
83+
84+
- Workflow validation and structural integrity checks
85+
- Automatic dependency resolution
86+
- Optimized scheduling
87+
- Cross-node context propagation
88+
- Plugin-based task execution
89+
- Visualization through the FlowSynx UI
90+
91+
By representing workflows as DAGs, FlowSynx ensures that every automation—whether simple or highly complex—remains maintainable, scalable,
92+
and predictable.
Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
---
2+
sidebar_position: 8
3+
description: FlowSynx is fully container-ready, making it easy to deploy in isolated, reproducible environments using Docker. This option is highly recommended for users looking to minimize conflicts, simplify upgrades, and ensure consistent behavior across development, staging, and production environments.
4+
---
5+
6+
# Human-in-the-Loop Tasks
7+
8+
Automated workflows are powerful, but there are scenarios where **human judgment, approval, or input** is essential. FlowSynx supports this through **human-in-the-loop (HITL) tasks**, which pause workflow execution and require manual intervention before the workflow can continue. This capability combines automation with human oversight, enabling reliable, compliant, and flexible business processes.
9+
10+
Human-in-the-loop tasks are particularly valuable for:
11+
12+
- **Compliance and regulatory workflows**: Ensuring that sensitive operations meet legal or organizational requirements.
13+
- **Quality control**: Verifying outputs before committing changes or publishing results.
14+
- **Exception handling**: Allowing human operators to resolve errors or unexpected conditions.
15+
- **Complex decision-making**: Where automation alone cannot determine the correct next step.
16+
17+
18+
## How Human-in-the-Loop Works in FlowSynx
19+
20+
In FlowSynx, human-in-the-loop tasks are **explicitly modeled as nodes within a DAG**. When the workflow reaches a HITL node:
21+
22+
1. **Execution pauses** at the task.
23+
2. The system **notifies the assigned user(s)** via the configured channels (email, web UI, or integrations with collaboration tools).
24+
3. The user completes the required action—such as approving a request, entering data, or making a decision.
25+
4. Once the task is completed, **workflow execution resumes**, following the downstream dependencies defined in the DAG.
26+
27+
This ensures that workflows maintain **deterministic execution order** even when human interaction is required.
28+
29+
30+
## Benefits of Human-in-the-Loop Tasks
31+
32+
### 1. Compliance and Auditability
33+
HITL tasks leave a clear **audit trail**, recording who performed the action, when, and what decision was made. This is critical for industries such as finance, healthcare, or legal operations.
34+
35+
### 2. Flexible Workflow Design
36+
Designers can incorporate human judgment only where necessary, while keeping other parts of the workflow fully automated.
37+
38+
### 3. Error Mitigation
39+
By inserting human checkpoints, organizations reduce the risk of incorrect automated decisions or accidental data corruption.
40+
41+
### 4. Seamless Integration with Automation
42+
HITL tasks coexist with automated retries, error handling, and data validation mechanisms, ensuring workflows remain resilient and robust even with manual interventions.
43+
44+
### 5. Multi-User Collaboration
45+
Tasks can be assigned to individual users or groups, supporting parallel reviews, approvals, or joint decision-making.
46+
47+
48+
## Example Use Cases
49+
50+
- **Invoice processing:** Automatically extract invoice data, but require human approval for amounts above a certain threshold.
51+
- **Content publishing:** Automated content pipelines with human review before publishing articles or marketing material.
52+
- **Medical imaging workflows:** Automatically process scans but require a radiologist to verify critical findings.
53+
- **Financial transaction workflows:** Automated fraud detection triggers a human review for flagged transactions.
54+
- **Customer support automation:** Automatically route tickets but require a human agent to confirm sensitive responses.
55+
56+
57+
## Implementation Considerations
58+
59+
When designing HITL tasks in FlowSynx:
60+
61+
- Clearly **define the task owner or group** responsible for action.
62+
- Configure **notifications and reminders** to prevent workflow stalling.
63+
- Ensure **data visibility and security** so the assigned user has access to relevant information.
64+
- Set **timeouts or escalation paths** for critical workflows to maintain continuity.
65+
- Integrate **role-based permissions** to ensure only authorized users can complete specific tasks.
66+
67+
68+
## Combining HITL with DAGs
69+
70+
Human-in-the-loop tasks seamlessly integrate with FlowSynx’s **DAG-based workflow engine**, maintaining:
71+
72+
- Clear **execution order** with upstream and downstream dependencies.
73+
- **Retry and error-handling logic** around manual tasks.
74+
- Consistency in **data propagation** and **state management** across automated and manual steps.
75+
76+
This makes it possible to build **hybrid workflows** that combine the efficiency of automation with the intelligence of human decision-making.
77+
78+
---
79+
80+
Human-in-the-loop tasks are a cornerstone for building **robust, compliant, and intelligent workflows** in FlowSynx.
81+
By explicitly modeling points of human intervention within a DAG, organizations can balance automation speed with oversight,
82+
ensuring workflows remain both efficient and reliable.

0 commit comments

Comments
 (0)