Docs intermittently mix translations of Sphinx labels

[medmunds]

(I'm assuming this is related to the docs.djangoproject.com build and not the Django docs source. Apologies if this ends up belonging in code.djangoproject.com instead.)

Some—but not all—of the versionchanged (and maybe versionadded?) labels in the English docs are showing up in Swedish. Example from https://docs.djangoproject.com/en/6.0/topics/email/:

Changed in Django 6.0:
Support for MIMEPart attachments was added.

Ersatt sedan version 6.0:
Support for email.mime.base.MIMEBase attachments is deprecated. Use MIMEPart instead.

Here's a screenshot:

(Search for "Ersatt" in the page to find several other examples.)

[DhanashreePetare]

Greetings @medmunds ..I am interested to work on the issue. Can you please assign this to me. ThankYou

[adamzap]

@DhanashreePetare As long as there is no pull request for this issue, you are free to attempt a fix. No need to wait for the issue to be assigned to you.

[sarahboyce]

I confirmed this issue when this was reported. This often goes away on it's own with a docs rebuild (I believe this is better now).
That we have sparidic mixed translations is still an issue. This was also reported in #1973 which was not really fixed but rather "improved"

[adamzap]

@sarahboyce Hmm, you're right. The example posted above is now correct in production.

[medmunds]

What's odd is that the exact same Sphinx directive (versionchanged) used twice in a row is ending up with different translations. That feels like some sort of incremental parsing/caching problem.

The _clean_up_global_state() workaround should probably be moved to a finally clause. But I'm not sure that would explain this.

If the note about needing to build versions in reverse order is still accurate, that suggests Sphinx's parsing/metadata caches (e.g., _build/_html/.doctrees) are somehow not being purged between builds, despite code that attempts to remove them. Is builder the correct subdirectory for the path to purge?

Is there any way multiple instances of update_docs could be running simultaneously with the same working dir?

[sarahboyce]

Is there any way multiple instances of update_docs could be running simultaneously with the same working dir?

@bmispelon might be able to share running logs and answer that

[JorgeOteiza]

Thanks everyone for the context, this is very interesting.

I wasn’t able to reproduce the issue directly in the rendered docs at the moment, but based on the behavior described (same Sphinx directive producing different translations within the same page), this does seem consistent with some kind of incremental build / caching or shared global state issue rather than a content problem.

The hypothesis about doctrees or gettext-related caches not being fully cleared between builds sounds plausible, especially if multiple update_docs runs can share a working directory or overlap in time.

I’m going to take a closer look at how update_docs handles workspace isolation and cache cleanup, but at first glance this feels more like a build pipeline concern than something that can be fixed at the documentation source level.

[kehach07]

This is my PR for the issue:- #2437
Sphinx maintains global translation and doctree state which was leaking
between documentation builds, causing intermittent mixed-language labels
(e.g. Swedish labels appearing in English docs).

This change:

• Isolates doctrees per language and builder
• Forces fresh Sphinx build environments
• Ensures global state cleanup always runs

Fixes #2416

[medmunds]

@kehach07 that looks promising.

I suspect the underlying issue is that Sphinx just isn't intended to be invoked multiple times in the same Python process.

sphinx-multiversion forks a subprocess for each build, which seems like a more robust way to ensure Sphinx and docutils don't share global state between builds. (And even that approach must use caution to avoid accidental doctree sharing, similar to what @kehach07's PR does. See also sphinx-doc/sphinx#2946.)

sphinx-polyversion goes a step further, running each build in a separate venv. (Though it's also trying to account for multiple versions of the source.)

[kehach07]

@medmunds, Thanks for the feedback! I agree — the root issue is Sphinx being invoked multiple times in the same Python process.
PR:- #2441
This PR moves Sphinx execution into isolated subprocesses (similar to sphinx-multiversion), ensuring no global translation or doctree state is shared between builds, while still keeping doctrees isolated per language and builder.

[kehach07]

@ulgens, this is the existing issue for which I had created the earlier pull request. Based on the suggestions from @medmunds, I have created another pull request with the requested changes.