Add reproducibility examples and inverted double pendulum example.

Author: CodeReclaimers

CHANGELOG.md

@@ -21,6 +21,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - Checkpoint system already preserved random state (unchanged)
   - All changes are fully backward compatible
 
+- **Reproducibility Examples**: Demonstration scripts for reproducible NEAT evolution
+  - Serial reproducibility example: `examples/xor/evolve-feedforward-reproducible.py`
+    - Tests reproducibility (same seed → identical results)
+    - Tests seed effect (different seeds → different evolution)
+    - Tests backward compatibility (no seed parameter works)
+  - Parallel reproducibility example: `examples/parallel-reproducible/`
+    - `evolve-parallel.py` - Demonstrates parallel evaluation with reproducibility
+    - `config-parallel` - Configuration file with seed parameter
+    - `README.md` - Comprehensive documentation with best practices and troubleshooting
+    - Tests parallel reproducibility (same seed + multiple workers → identical results)
+    - Tests worker count independence (results consistent across worker counts)
+
+- **New Example**: Inverted Double Pendulum example using Gymnasium
+  - Complete example in `examples/inverted-double-pendulum/`
+  - Uses `InvertedDoublePendulum-v5` environment (MuJoCo-based)
+  - Demonstrates continuous control with 9-dimensional observation space
+  - Includes evolution script with parallel evaluation
+  - Includes test/visualization script for trained controllers
+  - Full documentation in example README with usage tips
+
+### Fixed
+- **Orphaned Nodes Bug**: Fixed silent failure when nodes have no incoming connections after deletion mutations
+  - `feed_forward_layers()` now correctly handles orphaned nodes (nodes with no incoming connections)
+  - Orphaned nodes are treated as "bias neurons" that output `activation(bias)` independent of inputs
+  - Placed in first evaluation layer as they are always ready
+  - `required_for_output()` now includes orphaned nodes that feed into outputs
+  - Aggregation functions (`max`, `min`, `maxabs`, `mean`, `median`) now handle empty inputs (return 0.0)
+  - Comprehensive test coverage in `tests/test_graphs.py` (5 new tests) and `tests/test_nn.py` (integration test)
+
 ## [1.0.0] - 2025-01-09
 
 ### Added

docs/reproducibility.rst

@@ -527,6 +527,52 @@ Multi-Seed Statistical Analysis
        results = run_experiment(num_runs=10)
        analyze_results(results)
 
+Example Scripts
+---------------
+
+The NEAT-Python repository includes comprehensive example scripts demonstrating reproducibility:
+
+**Serial Reproducibility Example: XOR Problem**
+
+Location: ``examples/xor/evolve-feedforward-reproducible.py``
+
+This script demonstrates:
+- Reproducibility verification (same seed produces identical results)
+- Different seed comparison (different seeds produce different evolution paths)
+- Backward compatibility (evolution works without seed parameter)
+
+Run with:
+
+.. code-block:: bash
+
+   cd examples/xor
+   python evolve-feedforward-reproducible.py
+
+Expected output shows all three tests passing with clear verification of reproducibility.
+
+**Parallel Reproducibility Example**
+
+Location: ``examples/parallel-reproducible/``
+
+This example demonstrates parallel evaluation with reproducibility:
+- ``evolve-parallel.py`` - Main script with 3 reproducibility tests
+- ``config-parallel`` - Configuration file with seed parameter
+- ``README.md`` - Detailed documentation with best practices
+
+The script tests:
+1. Parallel reproducibility (same seed + multiple workers → identical results)
+2. Seed effects (different seeds → different evolution)
+3. Worker count independence (consistent results with different worker counts)
+
+Run with:
+
+.. code-block:: bash
+
+   cd examples/parallel-reproducible
+   python evolve-parallel.py
+
+Expected output shows all reproducibility tests passing across different worker configurations.
+
 See Also
 --------
 

examples/inverted-double-pendulum/README.md

@@ -0,0 +1,137 @@
+# Inverted Double Pendulum Example
+
+This example demonstrates using NEAT to evolve a neural network controller for the Gymnasium `InvertedDoublePendulum-v5` environment.
+
+## Problem Description
+
+The inverted double pendulum consists of two poles connected serially and mounted on a cart that moves along a frictionless track. The objective is to balance both poles in an upright position by applying horizontal forces to the cart.
+
+### Environment Details
+
+- **Observation Space**: 9-dimensional continuous vector containing:
+  - Position of the cart (x, y, z)
+  - Angular positions of both poles (θ1, θ2)
+  - Velocities of the cart (ẋ, ẏ, ż)
+  - Velocity magnitude at the tip of the second pole
+  
+- **Action Space**: 1-dimensional continuous action representing the force applied to the cart (ranging from -1 to 1)
+
+- **Reward**: The agent receives a reward for each timestep the poles remain balanced. The episode terminates when the poles fall beyond a certain angle or the maximum number of steps (1000) is reached.
+
+- **Success Criterion**: A fitness of 9000+ (staying balanced for most/all of the maximum 1000 steps over multiple episodes)
+
+## Files
+
+- `evolve-feedforward.py` - Main evolution script using feedforward networks
+- `config-feedforward` - NEAT configuration file
+- `test-feedforward.py` - Script to test and visualize trained controllers
+- `README.md` - This file
+
+## Requirements
+
+```bash
+pip install neat-python gymnasium
+```
+
+For visualization of the environment, you may also need:
+```bash
+pip install pygame
+```
+
+## Usage
+
+### Training a Controller
+
+To evolve a controller from scratch:
+
+```bash
+python evolve-feedforward.py
+```
+
+This will:
+1. Create a population of 150 random neural networks
+2. Evolve them for up to 300 generations
+3. Use parallel evaluation across all CPU cores
+4. Save checkpoints every 10 generations
+5. Save the best genome as `winner-feedforward.pickle`
+6. Generate visualization files (fitness plots, network diagrams)
+
+The evolution will stop when a genome achieves the fitness threshold (9000.0) or after 300 generations.
+
+### Testing a Trained Controller
+
+To test a trained controller with visualization:
+
+```bash
+python test-feedforward.py
+```
+
+Or to test a specific genome file:
+
+```bash
+python test-feedforward.py path/to/genome.pickle
+```
+
+This will run the controller for 5 episodes and display:
+- Real-time visualization of the pendulum
+- Step count and fitness for each episode
+- Average, max, and min fitness across episodes
+
+### Resuming from a Checkpoint
+
+If training is interrupted, you can resume from the most recent checkpoint:
+
+```python
+import neat
+import os
+
+local_dir = os.path.dirname(__file__)
+config_path = os.path.join(local_dir, 'config-feedforward')
+config = neat.Config(neat.DefaultGenome, neat.DefaultReproduction,
+                    neat.DefaultSpeciesSet, neat.DefaultStagnation,
+                    config_path)
+
+# Find the most recent checkpoint
+pop = neat.Checkpointer.restore_checkpoint('neat-checkpoint-XX')
+# Continue evolution
+# ... (same as in evolve-feedforward.py)
+```
+
+## Configuration Notes
+
+The `config-feedforward` file contains several important parameters:
+
+- **Population size**: 150 individuals
+- **Network structure**: Starts with no hidden nodes, allowing NEAT to evolve the topology
+- **Activation functions**: tanh (default), with sigmoid and relu available through mutation
+- **Mutation rates**: Configured to allow both structural and weight mutations
+- **Speciation threshold**: 3.0 for maintaining diversity
+
+These parameters can be tuned based on your computational resources and desired performance.
+
+## Tips for Success
+
+1. **Patience**: This is a challenging control problem. Evolution may take many generations to find good solutions.
+
+2. **Parallel evaluation**: The script uses all available CPU cores by default. Adjust the number of workers in `evolve-feedforward.py` if needed.
+
+3. **Hyperparameter tuning**: If evolution stagnates, try:
+   - Increasing population size
+   - Adjusting mutation rates
+   - Changing the activation functions
+   - Modifying the compatibility threshold
+
+4. **Multiple runs**: Due to the stochastic nature of evolution, running multiple independent trials may help find better solutions.
+
+## Expected Results
+
+A successfully evolved controller should:
+- Balance both poles for the full 1000 steps
+- Achieve fitness scores consistently above 9000
+- Use relatively small networks (often fewer than 10 nodes)
+
+## References
+
+- [NEAT Paper](http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf)
+- [Gymnasium Documentation](https://gymnasium.farama.org/environments/mujoco/inverted_double_pendulum/)
+- [neat-python Documentation](https://neat-python.readthedocs.io/)

examples/inverted-double-pendulum/check_dependencies.py

@@ -0,0 +1,76 @@
+#!/usr/bin/env python3
+"""
+Check if all required dependencies are installed for the inverted double pendulum example.
+"""
+
+import sys
+
+def check_dependencies():
+    """Check if all required packages are available."""
+    missing = []
+    
+    # Check core dependencies
+    try:
+        import neat
+        print("✓ neat-python is installed")
+    except ImportError:
+        print("✗ neat-python is NOT installed")
+        missing.append("neat-python")
+    
+    try:
+        import gymnasium
+        print("✓ gymnasium is installed")
+    except ImportError:
+        print("✗ gymnasium is NOT installed")
+        missing.append("gymnasium")
+    
+    # Check optional dependencies
+    try:
+        import pygame
+        print("✓ pygame is installed (for rendering)")
+    except ImportError:
+        print("⚠ pygame is NOT installed (optional, needed for visualization)")
+    
+    try:
+        import graphviz
+        print("✓ graphviz is installed (for network visualization)")
+    except ImportError:
+        print("⚠ graphviz is NOT installed (optional, needed for network diagrams)")
+    
+    try:
+        import matplotlib
+        print("✓ matplotlib is installed (for plotting)")
+    except ImportError:
+        print("⚠ matplotlib is NOT installed (optional, needed for fitness plots)")
+    
+    # Check if InvertedDoublePendulum environment is available
+    if not missing or "gymnasium" not in missing:
+        try:
+            import gymnasium as gym
+            env = gym.make('InvertedDoublePendulum-v5')
+            obs_space = env.observation_space
+            action_space = env.action_space
+            env.close()
+            print(f"✓ InvertedDoublePendulum-v5 environment is available")
+            print(f"  - Observation space: {obs_space}")
+            print(f"  - Action space: {action_space}")
+        except Exception as e:
+            print(f"✗ InvertedDoublePendulum-v5 environment error: {e}")
+            missing.append("mujoco (InvertedDoublePendulum requires MuJoCo)")
+    
+    print("\n" + "="*60)
+    if missing:
+        print("MISSING REQUIRED DEPENDENCIES:")
+        for dep in missing:
+            print(f"  - {dep}")
+        print("\nInstall them with:")
+        print(f"  pip install {' '.join(missing)}")
+        return False
+    else:
+        print("All required dependencies are installed!")
+        print("You can start training with: python evolve-feedforward.py")
+        return True
+
+if __name__ == "__main__":
+    success = check_dependencies()
+    sys.exit(0 if success else 1)

examples/inverted-double-pendulum/clean.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+# Clean up generated files
+
+rm -f winner-*.pickle
+rm -f winner-*.gv
+rm -f winner-*.gv.pdf
+rm -f neat-checkpoint-*
+rm -f *.svg
+rm -rf __pycache__
+rm -rf .pytest_cache
+
+echo "Cleanup complete!"

examples/inverted-double-pendulum/config-feedforward

@@ -0,0 +1,74 @@
+[NEAT]
+fitness_criterion     = max
+fitness_threshold     = 9000.0
+pop_size              = 150
+reset_on_extinction   = 0
+
+no_fitness_termination         = False
+
+[DefaultGenome]
+# InvertedDoublePendulum-v5 has 9 observation dimensions and 1 action dimension
+num_inputs              = 9
+num_hidden              = 0
+num_outputs             = 1
+initial_connection      = partial_direct 0.5
+feed_forward            = True
+compatibility_disjoint_coefficient    = 1.0
+compatibility_weight_coefficient      = 0.6
+conn_add_prob           = 0.3
+conn_delete_prob        = 0.2
+node_add_prob           = 0.3
+node_delete_prob        = 0.2
+activation_default      = tanh
+activation_options      = tanh sigmoid relu
+activation_mutate_rate  = 0.1
+aggregation_default     = sum
+aggregation_options     = sum
+aggregation_mutate_rate = 0.0
+bias_init_mean          = 0.0
+bias_init_stdev         = 1.0
+bias_replace_rate       = 0.1
+bias_mutate_rate        = 0.7
+bias_mutate_power       = 0.5
+bias_max_value          = 30.0
+bias_min_value          = -30.0
+response_init_mean      = 1.0
+response_init_stdev     = 0.1
+response_replace_rate   = 0.1
+response_mutate_rate    = 0.2
+response_mutate_power   = 0.1
+response_max_value      = 30.0
+response_min_value      = -30.0
+
+weight_max_value        = 30
+weight_min_value        = -30
+weight_init_mean        = 0.0
+weight_init_stdev       = 1.0
+weight_mutate_rate      = 0.8
+weight_replace_rate     = 0.1
+weight_mutate_power     = 0.5
+enabled_default         = True
+enabled_mutate_rate     = 0.01
+
+single_structural_mutation     = false
+structural_mutation_surer      = default
+bias_init_type                 = gaussian
+response_init_type             = gaussian
+weight_init_type               = gaussian
+enabled_rate_to_true_add       = 0.0
+enabled_rate_to_false_add      = 0.0
+
+[DefaultSpeciesSet]
+compatibility_threshold = 3.0
+
+[DefaultStagnation]
+species_fitness_func = max
+max_stagnation  = 20
+
+species_elitism                = 2
+
+[DefaultReproduction]
+elitism            = 3
+survival_threshold = 0.2
+
+min_species_size               = 2