Reorganizing models and enabling "other" models#12
Merged
jrfaeder merged 64 commits intoRuleWorld:masterfrom May 4, 2026
Merged
Reorganizing models and enabling "other" models#12jrfaeder merged 64 commits intoRuleWorld:masterfrom
jrfaeder merged 64 commits intoRuleWorld:masterfrom
Conversation
- Create \`Examples\`, \`Published\`, and \`Tutorials\` root directories. - Move \`Contributed/BNGPlayground_Examples\` to \`Examples\`. - Move published model subdirectories (e.g., PyBioNetGen models, Contributed/Hlavacek2018, Contributed/Mitra2019, Validation models like Korwek2023, MyrtleBeach) to \`Published\`. - Move remaining items in \`Contributed/BNGPlayground_Validation\` and \`BNGPlayground_PublicRuntime\` to \`Tutorials\`. - Update \`metadata-schema.yaml\` to add explicit domains (\`compartments\`, \`energy\`, \`wacky\`, \`mechanistic-modeling\`, etc.). - Update \`metadata.yaml\` files previously labeled as \`other\` with the new, specific categories. - Update \`scripts/generate-manifest.js\` and \`scripts/validate-metadata.js\` with new \`SEARCH_ROOTS\` and categories. - Regenerate \`manifest.json\`. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
…969453 chore: Restructure RuleHub directories and improve metadata categories
Fixes a prototype pollution vulnerability in the `setNested` utility function within `scripts/generate-manifest.js`. The fix explicitly skips and filters out dangerous property names (`__proto__`, `constructor`, `prototype`) during both object traversal and final assignment. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
* Refactor `scripts/validate-metadata.js` to export `parseScalar` and conditionally execute `main()`. * Add `scripts/tests/validate-metadata.test.js` covering all behavior. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
Removed the `# FIX: Add Autoinducer dynamics` comment from `Examples/biology/quorumsensingcircuit/quorum-sensing-circuit.bngl` as the implementation for production and degradation is already present below it. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
Removed the '# FIX: Add resource dynamics' comment as the resource regeneration and decay rules are already implemented correctly in the model. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
Updated `setNested` to accept an array of parts instead of a dotted path string, avoiding redundant string splitting. Tested with a 100,000 iteration benchmark on `parseMetadataYaml`, measuring ~20% improvement (from ~1.25s to ~0.98s). Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
- Refactor `scripts/validate-metadata.js` to only run `main()` if executed directly. - Export `validateMetadataFile` to allow testing it in isolation. - Add comprehensive test suite in `tests/validate-metadata.test.js` covering valid files, missing files, and invalid configuration scenarios using temporary directory fixtures. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
Refactored generate-manifest.js to allow importing internal functions and added comprehensive unit tests for buildEntry using node:test. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
Updated `ds3()` in `sp_fourier_synthesizer.bngl` to use the exact derivative of the 3rd harmonic trigonometric identity instead of an approximation. Cleaned up corresponding comments. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
Extracted the duplicated listModelFiles function from scripts/generate-manifest.js and scripts/validate-metadata.js into a shared utility module (scripts/utils.js). This improves code maintainability and adheres to DRY principles. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
…-9077399113245613909 Fix: Remove Autoinducer dynamics FIX comment
…2033588725295054439 Remove resolved FIX comment in eco_food_web_chaos_3sp.bngl
…438250706711219829 🧪 Add generate-manifest.js testing coverage
…3267491605790 ⚡ perf(manifest): optimize setNested path handling
…-16219869687540900783 🧪 [testing] add tests for parseScalar in validate-metadata.js
…6954674524457256 🧪 testing: add tests for validateMetadataFile
…451205665425511 🧹 Refactor: Extract listModelFiles function into a shared utility module
…67357569153273 Implement 3rd harmonic using trigonometric identity
…-yaml-7716385411393383498
…411393383498 🧪 test: add unit tests for custom parseMetadataYaml function
…-15072088604429859966 🔒 Fix Prototype Pollution in setNested
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
…n generate-manifest.js - Add `listModelFilesAsync` utility leveraging `fs.promises.readdir` - Refactor `main` in `generate-manifest.js` to process directories concurrently via `Promise.all` - Update error handling in main module execution block Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
…7290809949050255506 ⚡ Optimize synchronous directory read in generate-manifest.js
…060263818870501 🧪 Test parseMetadataYaml in generate-manifest.js
…e-9942811136342454879 🧪 Add test for listMetadataFiles edge case
…0297 🧪 Add test file for scripts/utils.js
…947067788 Fix hack in Disturb_rate function by using exact step function
…55140467607 🧪 test: add exhaustive edge case tests for expectEnum validation
…24971767703189 ⚡ Optimize nested path construction in YAML parser
* perf: optimize generate-manifest.js by deferring file read Avoid unnecessarily reading and parsing metadata.yaml when a directory contains no .bngl model files by moving the `listModelFiles` check before the file read. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * Merge master into PR branch and resolve conflicts Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
… reads (#47) * ⚡ Refactor manifest generation to use concurrent async file reads Replaced the synchronous `fs.readFileSync` loop in `scripts/generate-manifest.js` with `Promise.all()` and `fs.promises.readFile()`. This allows all metadata files to be read concurrently rather than sequentially. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * Acknowledge obsolete PR and resolve conflicts Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* test: comprehensive test suite for parseMetadataYaml Add detailed test cases for `parseMetadataYaml` and `parseScalar` to `scripts/validate-metadata.test.js`, covering complex YAML parsing logic and prototype pollution defense. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * test: comprehensive test suite for parseMetadataYaml Add detailed test cases for `parseMetadataYaml` and `parseScalar` to `scripts/validate-metadata.test.js`, covering complex YAML parsing logic and prototype pollution defense. Resolved merge conflicts. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* 🧪 Test setNested in validate-metadata.js Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * 🧪 Test setNested in validate-metadata.js Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* 🧪 test: add coverage for listMetadataFiles in generate-manifest.js Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * 🧪 test: add coverage for listMetadataFiles in generate-manifest.js Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * 🧪 test: resolve merge conflicts after pull from master Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* 🧪 Add tests for setNested function Export setNested from validate-metadata.js and add a comprehensive test suite for it in validate-metadata.test.js. Tests cover basic creation, nested property assignment, and prototype pollution prevention mechanisms. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * 🧪 Add tests for setNested function Export setNested from validate-metadata.js and add a comprehensive test suite for it in validate-metadata.test.js. Tests cover basic creation, nested property assignment, and prototype pollution prevention mechanisms. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
- Exported `normalizeModelKey` from `scripts/validate-metadata.js`. - Added a full test suite for `normalizeModelKey` in `scripts/tests/validate-metadata.test.js` covering case conversion, extension stripping, sanitization of special characters, and edge cases. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* Refactor: Centralize YAML parsing and data structure functions in scripts/utils.js This commit moves the duplicated functions `parseMetadataYaml`, `parseScalar`, and `setNested` from `scripts/validate-metadata.js` and `scripts/generate-manifest.js` to a central utility file `scripts/utils.js`. It also updates tests and script imports to reflect this change, improving code maintainability and removing duplicated logic. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * Merge master and resolve conflicts Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
) * refactor(scripts): deduplicate setNested function\n\nExtracted the setNested function from scripts/generate-manifest.js and scripts/validate-metadata.js into the shared scripts/utils.js module to improve consistency and maintainability. Updated validate-metadata.js to pass the array of path parts instead of a joined string, aligning with the generalized signature. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * refactor(scripts): deduplicate setNested function\n\nExtracted the setNested function from scripts/generate-manifest.js and scripts/validate-metadata.js into the shared scripts/utils.js module to improve consistency and maintainability. Updated validate-metadata.js to pass the array of path parts instead of a joined string, aligning with the generalized signature. Resolved merge conflicts. Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* test: add test suite for parseArgs in generate-manifest.js Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * test: merge origin/master and resolve conflicts in generate-manifest.test.js Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Replaced synchronous `fs.readFileSync` with `await fs.promises.readFile` in `scripts/validate-metadata.js`. Modified `validateMetadataFile` and `main` to be `async` and used `Promise.all` to allow concurrent file I/O operations instead of blocking synchronous reads. Updated the corresponding tests to handle the asynchronous implementation. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* test: add coverage for setNested in generate-manifest.js Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * test: add coverage for setNested in generate-manifest.js Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* test: add tests for parseArgs in generate-manifest.js Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> * test: resolve merge conflicts and add tests for parseArgs in generate-manifest.js Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com> --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
- Add gallery-categories.yaml with 16 category definitions - Rewrite generate-manifest.js with collection collapsing, --slim flag, compatibility object - Add generate-gallery.js to build gallery.json from metadata - Add backfill-metadata.js to generate missing metadata.yaml from .bngl - Add apply-gallery-assignments.js to patch metadata with gallery categories - Add gallery-assignments.json with correct category mappings (264 models) - Update metadata-schema.yaml with new fields (gallery_categories, sort_priority, etc.) - Update CI workflow to validate manifest.json, manifest-slim.json, gallery.json - Add pybnf_files to DEFAULT_IGNORE_DIRS - Move broken extract-gallery-from-constants.js to scripts/migration/ Result: 482 manifest entries (down from 755), 367 gallery assignments with proper categories (cancer: 20, immunology: 30, neuroscience: 14, etc.)
The last commit changed the metadata schema from gallery_category (singular string) to gallery_categories (plural array), but the validation script was not updated to match. Also applied gallery-assignments.json to populate gallery_categories for models that had assignments defined.
- Make gallery_categories optional (new field, not all models have it) - Make source.original_repository optional - Make compatibility.simulation_methods optional - Remove strict check for multiple .bngl files without collection - Allow missing README.md for generated subdirectories
- Sort model IDs alphabetically in assignments for deterministic output - Sort category tags within each model for consistent ordering - Regenerate gallery.json with new deterministic format
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR refactors the RuleHub directory structure and updates metadata categories according to the latest requirements.
Key Changes
Published,Examples, andTutorials.Examplesdirectory.PyBioNetGenpublished models toPublished/PyBioNetGen.AuthorYearformat fromContributed/toPublished/.BNGPlayground_Validationmodels based on their origin (published vs. tutorials)."category: other".mechanistic-modeling,feature-demos,compartments,wacky,energy, etc.) to themetadata-schema.yaml.metadata.yamlfiles.generate-manifest.jsandvalidate-metadata.jsmodified to scan the new root directories.