Skip to content

refactor: generalize entropy calculation and improve semantic clarity #14202

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
definus6-dev wants to merge 2 commits into
base: master
Choose a base branch
from
Open

refactor: generalize entropy calculation and improve semantic clarity #14202

definus6-dev wants to merge 2 commits into from
+101 −116

Conversation

@definus6-dev
Copy link

@definus6-dev definus6-dev commented Jan 23, 2026

Describe your change:

  • Add an algorithm?
  • Fix a bug or typo in an existing algorithm?
  • Add or change doctests?
  • Documentation change?

Checklist:

  • I have read CONTRIBUTING.md.
  • This pull request is all my own work -- I have not plagiarized.
  • I know that pull requests will not be merged if they fail the automated tests.
  • This PR only changes one algorithm file.
  • All new Python files are placed inside an existing directory.
  • All filenames are in all lowercase characters with no spaces or dashes.
  • All functions and variable names follow Python naming conventions.
  • All function parameters and return values are annotated with Python type hints.
  • All functions have doctests that pass the automated testing.
  • All new algorithms include at least one URL that points to Wikipedia or another similar explanation.
  • If this pull request resolves one or more open issues then the description above includes the issue number(s) with a closing keyword: "Fixes #ISSUE-NUMBER".

Additional Context

This PR refactors the entropy.py module to improve its modularity, efficiency, and theoretical clarity.

  • Generalization: Extracted the Shannon Entropy formula into a standalone function shannon_entropy that accepts any probability distribution, making it reusable beyond text analysis.
  • Modularity: Separated the N-gram text analysis logic from the core computation.
  • Optimization: Replaced nested loops with a single-pass frequency analysis using Counter, reducing complexity from O(Alphabet^2) to O(Input Length).
  • Theoretical Essence: Updated documentation to reflect the relationship between information theory and system disorder (entropy).
  • Safety: Added validation for input probabilities (non-negativity and sum-to-one check).

@definus6-dev
refactor: generalize entropy calculation and improve semantic clarity
f7b8c07 
- Extracted core Shannon entropy calculation into a reusable pure function
- Separated text analysis logic from computation for better modularity
- Improved variable naming to reflect information theory concepts
- Optimized computational complexity from O(A^2) to O(N)
- Added physical and mathematical context to documentation
@algorithms-keeper algorithms-keeper bot added enhancement This PR modified some existing files awaiting reviews This PR is ready to be reviewed labels Jan 23, 2026
@algorithms-keeper algorithms-keeper bot added tests are failing Do not merge until tests pass and removed tests are failing Do not merge until tests pass labels Jan 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

awaiting reviews This PR is ready to be reviewed enhancement This PR modified some existing files

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@definus6-dev