fix: do not add a line continuation to a file without a trailing newline - #369
Open
Eljees wants to merge 1 commit into
Open
fix: do not add a line continuation to a file without a trailing newline#369Eljees wants to merge 1 commit into
Eljees wants to merge 1 commit into
Conversation
The synthetic NEWLINE token that tokenize emits for a source without a trailing newline has line='', so the strip() guard added in PyCQA#360 stopped counting it as a continuation and untokenize bridged the row gap with a backslash, corrupting the file. Exclude only blank-but-non-empty lines instead. Signed-off-by: Eljees <3.14hell@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
docformatter --in-placeappends a backslash to the last line of any file thatdoes not end with a newline, which leaves the file unparseable:
This is a regression from #360 (4849fce), found by bisecting the test suite. That
commit changed
is_newline_continuation()inclassify.pyfromto
The intent was to stop treating a blank line with trailing whitespace as a
continuation (#355).
token.line.strip()also drops the synthetic NEWLINE tokenthat
tokenizeemits for a source without a trailing newline -- that token hasline=''. It stops counting as a continuation,_get_unmatched_start_end_indicesmoves it onto the next row, and
untokenizebridges the gap with a backslash.The fix adds a small
_is_blank_line()helper -- a line that is non-empty and allwhitespace -- so
''passes again while" \n"is still excluded. #355 staysfixed; I kept its reproducer as a control and it is still green.
Only 3.10 and 3.11 are affected. Under PEP 701 the 3.12+ tokenizer reports the real
line for the synthetic NEWLINE, so the old branch keeps working:
tokenizeforx = 1without\ntest_do_format_code.pyNEWLINE '' line=''NEWLINE '' line='x = 1'pyproject.tomldeclarespython = "^3.10", so this is a supported version.Why CI is green
The version matrix does not run the versions it names. From the log of job
ubuntu-latest Python: 3.10, run 30781552231:toxis installed undersetup-python 3.13, andtox -e py --skip-pkg-installtakes basepython from tox itself, so all six matrix jobs run 3.13.14. That is why
this survived on master for three weeks. Happy to send a separate PR for the
workflow if you want it -- I left it out of this one.
Verification
On
e154acb, Python 3.10.12: master is10 failed, 536 passed; with this change546 passed, 4 skipped, plus the test added here. Reverting the code and keepingthe test turns exactly
issue_360_no_trailing_newlinered, with the diffx = 1->x = 1\, whileissue_355stays green.