fixed warning/errors when running .\scripts\build-documentation.ps1 by bmerkle · Pull Request #18 · AgentEvalHQ/AgentEval

bmerkle · 2026-03-05T17:34:54Z

.\scripts\build-documentation.ps1 now runs without warnings/errors
updated docfx.json because some files need a resouce mapping one level up
Strategy doc links removed (ADR 014, 015, 016, extensibility.md), strategy/ directory doesn't exist in the repo
ci-cd-integration.md links removed (redteam.md, workflows.md) (redteam.md, workflows.md) File doesn't exist in the repo

Description

Motivation and Context

current build process results in warnings/errors:

Building 46 file(s) in ResourceDocumentProcessor(ValidateResourceMetadata)...
C:\work\AgentEval\AgentEval\docs\adr\014-dataset-pipeline-two-model-architecture.md(169,1): warning InvalidFileLink: Invalid file link:(~/../strategy/AgentEval-dataloader-Implementation-Review-and-Refinement.md).
C:\work\AgentEval\AgentEval\docs\adr\015-extension-registration-manual-vs-auto-discovery.md(6,1): warning InvalidFileLink: Invalid file link:(~/../strategy/AgentEval-Extensibility-Implementation-Review-and-Refinement.md).
C:\work\AgentEval\AgentEval\docs\adr\016-monolith-modularization.md(108,1): warning InvalidFileLink: Invalid file link:(~/../strategy/impl/Phase-4-Validate-Document.md).
C:\work\AgentEval\AgentEval\docs\adr\016-monolith-modularization.md(106,1): warning InvalidFileLink: Invalid file link:(~/../strategy/impl/Phase-2-Extract-Core-DataLoaders.md).
C:\work\AgentEval\AgentEval\docs\adr\016-monolith-modularization.md(104,1): warning InvalidFileLink: Invalid file link:(~/../strategy/impl/Phase-0-Fix-Coupling-Anomalies.md).
C:\work\AgentEval\AgentEval\docs\adr\016-monolith-modularization.md(105,1): warning InvalidFileLink: Invalid file link:(~/../strategy/impl/Phase-1-Extract-Abstractions.md).
C:\work\AgentEval\AgentEval\docs\adr\016-monolith-modularization.md(103,1): warning InvalidFileLink: Invalid file link:(~/../strategy/AgentEval-Monolith-Modularization-ArchitectureRefactor.md).
C:\work\AgentEval\AgentEval\docs\adr\016-monolith-modularization.md(107,1): warning InvalidFileLink: Invalid file link:(~/../strategy/impl/Phase-3-Extract-MAF-RedTeam.md).
C:\work\AgentEval\AgentEval\docs\benchmarks.md(299,1): warning InvalidFileLink: Invalid file link:(~/../samples/AgentEval.Samples).
C:\work\AgentEval\AgentEval\docs\benchmarks.md(300,1): warning InvalidFileLink: Invalid file link:(~/../samples/AgentEval.Samples).
C:\work\AgentEval\AgentEval\docs\benchmarks.md(495,1): warning InvalidFileLink: Invalid file link:(~/../samples/AgentEval.Samples).
C:\work\AgentEval\AgentEval\docs\benchmarks.md(496,1): warning InvalidFileLink: Invalid file link:(~/../samples/AgentEval.Samples).
C:\work\AgentEval\AgentEval\docs\extensibility.md(759,1): warning InvalidFileLink: Invalid file link:(~/../strategy/AgentEval-Extensibility-Implementation-Review-and-Refinement.md).
C:\work\AgentEval\AgentEval\docs\redteam.md(1091,1): warning InvalidFileLink: Invalid file link:(~/ci-cd-integration.md).
C:\work\AgentEval\AgentEval\docs\workflows.md(1043,1): warning InvalidFileLink: Invalid file link:(~/ci-cd-integration.md).
Applying templates to 581 model(s)...

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update
Refactoring (no functional changes)
Performance improvement
Test coverage improvement

How Has This Been Tested?

Existing unit tests pass
Added new unit tests
Tested manually (describe below)

Test Configuration:

.NET version(s):
OS:

Test Details:

Checklist

My code follows the code style of this project
I have added XML documentation for new public APIs
I have updated the documentation (if applicable)
I have added tests to cover my changes
All new and existing tests pass
My changes generate no new compiler warnings
I have checked for breaking changes
I agree that my contributions are licensed under the MIT License and I have the right to submit this work (see CONTRIBUTING.md)

Screenshots (if applicable)

Additional Notes

Breaking Changes

- updated docfx.json because some files need a resouce mapping one level up - Strategy doc links removed (ADR 014, 015, 016, extensibility.md), strategy/ directory doesn't exist in the repo - ci-cd-integration.md links removed (redteam.md, workflows.md) (redteam.md, workflows.md) File doesn't exist in the repo

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Copilot · 2026-03-06T08:59:06Z

@@ -100,10 +100,4 @@ See `strategy/impl/` for detailed phase plans.

 ## References


Even though the broken links were removed from the References section, the ADR still points readers to strategy/impl/ earlier in the document. Since strategy/ isn’t in this repo, please remove/update that reference to avoid sending readers to a non-existent path.

Copilot · 2026-03-06T08:59:06Z

 - **M3** â€” `DatasetTestCase` extended model + `ToTestCase()` adapter
 - **M4** â€” `RunBatchAsync` on `MAFEvaluationHarness` accepting `DatasetTestCase`


These bullets contain mojibake (â€”) where an em dash (—) is likely intended. Please replace the garbled sequence so the ADR renders correctly.

@copilot apply changes based on this feedback

joslat · 2026-03-31T00:44:08Z

@copilot apply changes based on the comments in this thread

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 9 comments.

Copilot · 2026-03-31T00:44:28Z

 **Date:** 2026-02-25  
 **Decision Makers:** AgentEval Contributors  
-**Related:** [ADR-006](006-service-based-architecture-di.md), [Extensibility Strategy](../../strategy/AgentEval-Extensibility-Implementation-Review-and-Refinement.md), [Extensibility Guide](../extensibility.md)
+**Related:** [ADR-006](006-service-based-architecture-di.md), [Extensibility Guide](../extensibility.md)


After removing the Strategy doc link from the Related section, the body still says "AgentEval's extensibility strategy (see strategy doc)". Please update that wording to reference an in-repo document (e.g., the Extensibility Guide) so readers don't look for a missing strategy doc.

Copilot · 2026-03-31T00:44:29Z

+The full implementation plan covers specifically:

 - **M3** â€” `DatasetTestCase` extended model + `ToTestCase()` adapter
 - **M4** â€” `RunBatchAsync` on `MAFEvaluationHarness` accepting `DatasetTestCase`


The em dash appears to be mojibake (â€”), which renders incorrectly and suggests the file encoding is off. Replace â€” with a proper em dash (—) (and consider normalizing the file encoding to UTF-8) so this section renders correctly.

Copilot · 2026-03-31T00:44:29Z


 ## References

- [Strategy Document](../../strategy/AgentEval-Monolith-Modularization-ArchitectureRefactor.md)
- [Phase 0: Fix Coupling Anomalies](../../strategy/impl/Phase-0-Fix-Coupling-Anomalies.md)
- [Phase 1: Extract Abstractions](../../strategy/impl/Phase-1-Extract-Abstractions.md)
- [Phase 2: Extract Core + DataLoaders](../../strategy/impl/Phase-2-Extract-Core-DataLoaders.md)
- [Phase 3: Extract MAF + RedTeam](../../strategy/impl/Phase-3-Extract-MAF-RedTeam.md)
- [Phase 4: Validate, Document, CI](../../strategy/impl/Phase-4-Validate-Document.md)
 - [ADR-006: Service-Based Architecture with DI](006-service-based-architecture-di.md)


This ADR removes links to the strategy/ docs, but the document still tells readers to "See strategy/impl/ for detailed phase plans" earlier in the file. Please update/remove that reference so the ADR doesn't point to a directory that isn't in the repo.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

docs/adr/014-dataset-pipeline-two-model-architecture.md:173

The bullet labels contain a mojibake sequence (â€”) where an em dash is intended, which renders incorrectly in the docs. Replace â€” with — (or a plain -) to avoid garbled text.

- **M3** â€” `DatasetTestCase` extended model + `ToTestCase()` adapter
- **M4** â€” `RunBatchAsync` on `MAFEvaluationHarness` accepting `DatasetTestCase`
- **M1** â€” Documentation corrections

Copilot · 2026-03-31T06:23:30Z

+- [Sample 09: Real MAF Workflow Evaluation](samples/AgentEval.Samples/Sample09_WorkflowEvaluationReal.cs) - Sequential content pipeline (Planner → Researcher → Writer → Editor)
+- [Sample 10: Workflow with Tools](samples/AgentEval.Samples/Sample10_WorkflowWithTools.cs) - Tool-enabled trip planning workflow with function calling


The sample links use ../samples/..., which will resolve outside the generated site when DocFX outputs workflows.html at the site root. Since docfx.json maps samples into the docset (dest samples), these links should point to ~/samples/... (or samples/...) so they work in the built docs.

Copilot · 2026-03-31T06:23:31Z

+- [Sample 06](samples/AgentEval.Samples/Sample06_PerformanceProfiling.cs) - Performance profiling with MAFEvaluationHarness
+- [Sample 23](samples/AgentEval.Samples/Sample23_BenchmarkSystem.cs) - Benchmark system with JSONL data loading


These sample links use ../samples/..., which will likely break in the rendered DocFX output. Since samples are copied into the docset under samples/, link to ~/samples/... (or samples/...) instead.

bmerkle requested a review from joslat as a code owner March 5, 2026 17:34

Copilot AI review requested due to automatic review settings March 5, 2026 17:34

bmerkle mentioned this pull request Mar 5, 2026

[BUG] .\scripts\build-documentation.ps1 yields warnings and errors (non existing files, wrong docfx config) #19

Open

Copilot AI reviewed Mar 5, 2026

View reviewed changes

bmerkle requested a review from Copilot March 6, 2026 08:55

Copilot started reviewing on behalf of bmerkle March 6, 2026 08:55 View session

Copilot AI reviewed Mar 6, 2026

View reviewed changes

joslat requested a review from Copilot March 31, 2026 00:39

Copilot started reviewing on behalf of joslat March 31, 2026 00:40 View session

Copilot AI reviewed Mar 31, 2026

View reviewed changes

Update docs/docfx.json

1cfd784

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 31, 2026 06:19

Copilot started reviewing on behalf of bmerkle March 31, 2026 06:20 View session

bmerkle and others added 6 commits March 31, 2026 08:21

Update docs/workflows.md

bf7cd45

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/redteam.md

5b355c9

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/extensibility.md

b8edff3

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/benchmarks.md

30c06da

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/benchmarks.md

b46fe1d

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/docfx.json

8e705d1

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI reviewed Mar 31, 2026

View reviewed changes

		@@ -100,10 +100,4 @@ See `strategy/impl/` for detailed phase plans.

		## References

		- M3 â€” `DatasetTestCase` extended model + `ToTestCase()` adapter
		- M4 â€” `RunBatchAsync` on `MAFEvaluationHarness` accepting `DatasetTestCase`

		- [Sample 09: Real MAF Workflow Evaluation](samples/AgentEval.Samples/Sample09_WorkflowEvaluationReal.cs) - Sequential content pipeline (Planner → Researcher → Writer → Editor)
		- [Sample 10: Workflow with Tools](samples/AgentEval.Samples/Sample10_WorkflowWithTools.cs) - Tool-enabled trip planning workflow with function calling

		- [Sample 06](samples/AgentEval.Samples/Sample06_PerformanceProfiling.cs) - Performance profiling with MAFEvaluationHarness
		- [Sample 23](samples/AgentEval.Samples/Sample23_BenchmarkSystem.cs) - Benchmark system with JSONL data loading

Conversation

bmerkle commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Type of Change

How Has This Been Tested?

Checklist

Screenshots (if applicable)

Additional Notes

Breaking Changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

bmerkle Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

joslat commented Mar 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bmerkle commented Mar 5, 2026 •

edited

Loading