Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions Doc/library/contextlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ Functions and classes provided:
.. class:: AbstractContextManager

An :term:`abstract base class` for classes that implement
:meth:`object.__enter__` and :meth:`object.__exit__`. A default
implementation for :meth:`object.__enter__` is provided which returns
``self`` while :meth:`object.__exit__` is an abstract method which by default
:meth:`~object.__enter__` and :meth:`~object.__exit__`. A default
implementation for :meth:`~object.__enter__` is provided which returns
``self`` while :meth:`~object.__exit__` is an abstract method which by default
returns ``None``. See also the definition of :ref:`typecontextmanager`.

.. versionadded:: 3.6
Expand All @@ -32,9 +32,9 @@ Functions and classes provided:
.. class:: AbstractAsyncContextManager

An :term:`abstract base class` for classes that implement
:meth:`object.__aenter__` and :meth:`object.__aexit__`. A default
implementation for :meth:`object.__aenter__` is provided which returns
``self`` while :meth:`object.__aexit__` is an abstract method which by default
:meth:`~object.__aenter__` and :meth:`~object.__aexit__`. A default
implementation for :meth:`~object.__aenter__` is provided which returns
``self`` while :meth:`~object.__aexit__` is an abstract method which by default
returns ``None``. See also the definition of
:ref:`async-context-managers`.

Expand Down Expand Up @@ -228,7 +228,7 @@ Functions and classes provided:

.. function:: nullcontext(enter_result=None)

Return a context manager that returns *enter_result* from ``__enter__``, but
Return a context manager that returns *enter_result* from :meth:`~object.__enter__`, but
otherwise does nothing. It is intended to be used as a stand-in for an
optional context manager, for example::

Expand Down Expand Up @@ -335,7 +335,7 @@ Functions and classes provided:
For example, the output of :func:`help` normally is sent to *sys.stdout*.
You can capture that output in a string by redirecting the output to an
:class:`io.StringIO` object. The replacement stream is returned from the
``__enter__`` method and so is available as the target of the
:meth:`~object.__enter__` method and so is available as the target of the
:keyword:`with` statement::

with redirect_stdout(io.StringIO()) as f:
Expand Down Expand Up @@ -396,7 +396,8 @@ Functions and classes provided:
A base class that enables a context manager to also be used as a decorator.

Context managers inheriting from ``ContextDecorator`` have to implement
``__enter__`` and ``__exit__`` as normal. ``__exit__`` retains its optional
:meth:`~object.__enter__` and :meth:`~object.__exit__` as normal.
``__exit__`` retains its optional
exception handling even when used as a decorator.

``ContextDecorator`` is used by :func:`contextmanager`, so you get this
Expand Down Expand Up @@ -710,9 +711,9 @@ context management protocol.
Catching exceptions from ``__enter__`` methods
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It is occasionally desirable to catch exceptions from an ``__enter__``
It is occasionally desirable to catch exceptions from an :meth:`~object.__enter__`
method implementation, *without* inadvertently catching exceptions from
the :keyword:`with` statement body or the context manager's ``__exit__``
the :keyword:`with` statement body or the context manager's :meth:`~object.__exit__`
method. By using :class:`ExitStack` the steps in the context management
protocol can be separated slightly in order to allow this::

Expand Down
15 changes: 15 additions & 0 deletions Doc/using/cmdline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -687,6 +687,13 @@ Miscellaneous options

.. versionadded:: 3.14

* :samp:`-X pathconfig_warnings={0,1}` if true (``1``) then
:ref:`sys-path-init` is allowed to log warnings into stderr.
If false (``0``) suppress these warnings. Set to true by default.
See also :envvar:`PYTHON_PATHCONFIG_WARNINGS`.

.. versionadded:: next

* :samp:`-X tlbc={0,1}` enables (1, the default) or disables (0) thread-local
bytecode in builds configured with :option:`--disable-gil`. When disabled,
this also disables the specializing interpreter. See also
Expand Down Expand Up @@ -1354,6 +1361,14 @@ conflict.

.. versionadded:: 3.14

.. envvar:: PYTHON_PATHCONFIG_WARNINGS

If true (``1``) then :ref:`sys-path-init` is allowed to log warnings into
stderr. If false (``0``) suppress these warnings. Set to true by default.
See also :option:`-X pathconfig_warnings<-X>`.

.. versionadded:: next

.. envvar:: PYTHON_JIT

On builds where experimental just-in-time compilation is available, this
Expand Down
8 changes: 4 additions & 4 deletions Lib/inspect.py
Original file line number Diff line number Diff line change
Expand Up @@ -153,19 +153,19 @@
import itertools
import linecache
import os
import re
lazy import re
import sys
import tokenize
lazy import tokenize
import token
import types
import functools
import builtins
from keyword import iskeyword
from operator import attrgetter
from collections import namedtuple, OrderedDict
from weakref import ref as make_weakref
from _weakref import ref as make_weakref

# Create constants for the compiler flags in Include/code.h
# Create constants for the compiler flags in Include/cpython/code.h
# We try to get them from dis to avoid duplication
mod_dict = globals()
for k, v in dis.COMPILER_FLAG_NAMES.items():
Expand Down
1 change: 1 addition & 0 deletions Lib/test/pythoninfo.py
Original file line number Diff line number Diff line change
Expand Up @@ -539,6 +539,7 @@ def collect_sysconfig(info_add):
'SHELL',
'SOABI',
'TEST_MODULES',
'VAPTH',
'abs_builddir',
'abs_srcdir',
'prefix',
Expand Down
5 changes: 3 additions & 2 deletions Lib/test/support/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -1716,9 +1716,10 @@ def _platform_specific(self):
))

self._env = {k.upper(): os.getenv(k) for k in os.environ}
self._env["PYTHONHOME"] = os.path.dirname(self.real)
home = os.path.dirname(self.real)
if sysconfig.is_python_build():
self._env["PYTHONPATH"] = STDLIB_DIR
home = os.path.join(home, sysconfig.get_config_var('VPATH'))
self._env["PYTHONHOME"] = home
else:
def _platform_specific(self):
pass
Expand Down
6 changes: 6 additions & 0 deletions Lib/test/test_builtin.py
Original file line number Diff line number Diff line change
Expand Up @@ -2993,6 +2993,12 @@ def test_type_doc(self):
A.__doc__ = doc
self.assertEqual(A.__doc__, doc)

def test_type_frozendict(self):
A = type('A', (), frozendict({'x': 4, 'y': 2}))
self.assertEqual(A.x, 4)
self.assertEqual(A.y, 2)
self.assertEqual(A.__name__, 'A')

def test_bad_args(self):
with self.assertRaises(TypeError):
type()
Expand Down
20 changes: 20 additions & 0 deletions Lib/test/test_dict.py
Original file line number Diff line number Diff line change
Expand Up @@ -1569,6 +1569,26 @@ def make_pairs():
self.assertEqual(d.get(key3_3), 44)
self.assertGreaterEqual(eq_count, 1)

def test_overwrite_managed_dict(self):
# GH-130327: Overwriting an object's managed dictionary with another object's
# skipped traversal in favor of inline values, causing the GC to believe that
# the __dict__ wasn't reachable.
import gc

class Shenanigans:
pass

to_be_deleted = Shenanigans()
to_be_deleted.attr = "whatever"
holds_reference = Shenanigans()
holds_reference.__dict__ = to_be_deleted.__dict__
holds_reference.ref = {"circular": to_be_deleted, "data": 42}

del to_be_deleted
gc.collect()
self.assertEqual(holds_reference.ref['data'], 42)
self.assertEqual(holds_reference.attr, "whatever")

def test_unhashable_key(self):
d = {'a': 1}
key = [1, 2, 3]
Expand Down
30 changes: 26 additions & 4 deletions Lib/test/test_embed.py
Original file line number Diff line number Diff line change
Expand Up @@ -1491,8 +1491,12 @@ def test_init_setpythonhome(self):
}
self.default_program_name(config)
env = {'TESTHOME': home, 'PYTHONPATH': paths_str}
# When running from source, TESTHOME will be the build directory, which
# isn't a valid home unless _is_python_build is set. getpath will then
# fail to find the standard library and show a warning, so we need to
# ignore stderr.
self.check_all_configs("test_init_setpythonhome", config,
api=API_COMPAT, env=env)
api=API_COMPAT, env=env, ignore_stderr=True)

def test_init_is_python_build_with_home(self):
# Test _Py_path_config._is_python_build configuration (gh-91985)
Expand Down Expand Up @@ -1528,15 +1532,26 @@ def test_init_is_python_build_with_home(self):
'exec_prefix': exec_prefix,
'base_exec_prefix': exec_prefix,
'pythonpath_env': paths_str,
'stdlib_dir': stdlib,
'stdlib_dir': stdlib, # Only correct on _is_python_build==0!
}
# The code above is taken from test_init_setpythonhome()
env = {'TESTHOME': home, 'PYTHONPATH': paths_str}

env['NEGATIVE_ISPYTHONBUILD'] = '1'
config['_is_python_build'] = 0
# This configuration doesn't set a valid stdlibdir/plststdlibdir because
# with _is_python_build=0 getpath doesn't check for the build directory
# landmarks in PYTHONHOME/Py_SetPythonHome.
# getpath correctly shows a warning, which messes up check_all_configs,
# so we need to ignore stderr.
self.check_all_configs("test_init_is_python_build", config,
api=API_COMPAT, env=env)
api=API_COMPAT, env=env, ignore_stderr=True)

# config['stdlib_dir'] = os.path.join(home, 'Lib')
# FIXME: This test does not check if stdlib_dir is calculated correctly.
# test_init_is_python_build runs the initialization twice,
# setting stdlib_dir in _Py_path_config on the first run, which
# then overrides the stdlib_dir calculation (as of GH-108730).

env['NEGATIVE_ISPYTHONBUILD'] = '0'
config['_is_python_build'] = 1
Expand All @@ -1551,8 +1566,14 @@ def test_init_is_python_build_with_home(self):
expected_paths[0] = self.module_search_paths(prefix=prefix)[0]
config.update(prefix=prefix, base_prefix=prefix,
exec_prefix=exec_prefix, base_exec_prefix=exec_prefix)
# This also shows the bad stdlib warning, getpath is run twice. The
# first time with _is_python_build=0, which results in the warning just
# as explained above. However, the second time a valid standard library
# should be found, but the stdlib_dir is cached in _Py_path_config from
# the first run, which ovewrites it, so it also shows the warning.
# Also ignore stderr.
self.check_all_configs("test_init_is_python_build", config,
api=API_COMPAT, env=env)
api=API_COMPAT, env=env, ignore_stderr=True)

def copy_paths_by_env(self, config):
all_configs = self._get_expected_config()
Expand Down Expand Up @@ -1612,6 +1633,7 @@ def test_init_pybuilddir_win32(self):
prefix = os.path.normpath(os.path.join(tmpdir, vpath))
# The stdlib dir is dirname(executable) + VPATH + 'Lib'
stdlibdir = os.path.normpath(os.path.join(tmpdir, vpath, 'Lib'))
os.mkdir(stdlibdir)

filename = os.path.join(tmpdir, 'pybuilddir.txt')
with open(filename, "w", encoding="utf8") as fp:
Expand Down
9 changes: 9 additions & 0 deletions Lib/test/test_inspect/test_inspect.py
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,15 @@ def __get__(self, instance, owner):
return self.func.__get__(instance, owner)


class TestImportTime(unittest.TestCase):

@cpython_only
def test_lazy_import(self):
import_helper.ensure_lazy_imports(
"inspect", {"re", "tokenize"}
)


class TestPredicates(IsTestBase):

def test_excluding_predicates(self):
Expand Down
10 changes: 10 additions & 0 deletions Lib/test/test_os/test_os.py
Original file line number Diff line number Diff line change
Expand Up @@ -2778,6 +2778,16 @@ def test_fpathconf_bad_fd(self):
self.check(os.pathconf, "PC_NAME_MAX")
self.check(os.fpathconf, "PC_NAME_MAX")

@unittest.skipUnless(hasattr(os, 'pathconf'), 'test needs os.pathconf()')
@unittest.skipIf(
support.linked_to_musl(),
'musl fpathconf ignores the file descriptor and returns a constant',
)
def test_pathconf_negative_fd_uses_fd_semantics(self):
with self.assertRaises(OSError) as ctx:
os.pathconf(-1, 1)
self.assertEqual(ctx.exception.errno, errno.EBADF)

@unittest.skipUnless(hasattr(os, 'ftruncate'), 'test needs os.ftruncate()')
def test_ftruncate(self):
self.check(os.truncate, 0)
Expand Down
2 changes: 1 addition & 1 deletion Makefile.pre.in
Original file line number Diff line number Diff line change
Expand Up @@ -1004,7 +1004,7 @@ platform: $(PYTHON_FOR_BUILD_DEPS) pybuilddir.txt
# or removed in case of failure.
pybuilddir.txt: $(PYTHON_FOR_BUILD_DEPS)
@echo "none" > ./pybuilddir.txt
$(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars ;\
$(RUNSHARED) $(PYTHON_FOR_BUILD) -S -X pathconfig_warnings=0 -m sysconfig --generate-posix-vars ;\
if test $$? -ne 0 ; then \
echo "generate-posix-vars failed" ; \
rm -f ./pybuilddir.txt ; \
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix erroneous clearing of an object's :attr:`~object.__dict__` if
overwritten at runtime.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
:func:`type` now accepts :class:`frozendict` as an argument.
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
A warning is now shown during :ref:`sys-path-init` if it can't find a valid
standard library.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Added the :option:`-X pathconfig_warnings<-X>` and
:envvar:`PYTHON_PATHCONFIG_WARNINGS` options, allowing to disable warnings
from :ref:`sys-path-init`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix a crash in :func:`os.pathconf` when called with ``-1`` as the path
argument.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Reduce the import time of :mod:`inspect` module by ~20%.
25 changes: 22 additions & 3 deletions Modules/getpath.py
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,7 @@ def search_up(prefix, *landmarks, test=isfile):

real_executable_dir = None
platstdlib_dir = None
stdlib_zip = None

# ******************************************************************************
# CALCULATE program_name
Expand Down Expand Up @@ -697,12 +698,13 @@ def search_up(prefix, *landmarks, test=isfile):
library_dir = dirname(library)
else:
library_dir = executable_dir
pythonpath.append(joinpath(library_dir, ZIP_LANDMARK))
stdlib_zip = joinpath(library_dir, ZIP_LANDMARK)
elif build_prefix:
# QUIRK: POSIX uses the default prefix when in the build directory
pythonpath.append(joinpath(PREFIX, ZIP_LANDMARK))
stdlib_zip = joinpath(PREFIX, ZIP_LANDMARK)
else:
pythonpath.append(joinpath(base_prefix, ZIP_LANDMARK))
stdlib_zip = joinpath(base_prefix, ZIP_LANDMARK)
pythonpath.append(stdlib_zip)

if os_name == 'nt' and use_environment and winreg:
# QUIRK: Windows also lists paths in the registry. Paths are stored
Expand Down Expand Up @@ -767,6 +769,23 @@ def search_up(prefix, *landmarks, test=isfile):
config['module_search_paths_set'] = 1


# ******************************************************************************
# SANITY CHECKS
# ******************************************************************************

# Warn if the standard library is missing, unless pythonpath_was_set was set, as
# that skips parts of the stdlib directories calculation — assume the provided
# pythonpath is correct. This is how subinterpreters initialize the path for eg.
if not py_setpath and not pythonpath_was_set and (not stdlib_zip or not isfile(stdlib_zip)):
home_hint = f"The Python 'home' directory was set to {home!r}, is this correct?"
if not stdlib_dir or not isdir(stdlib_dir):
hint = home_hint if home else f'sys.prefix is set to {prefix}, is this correct?'
warn('WARN: Could not find the standard library directory! ' + hint)
elif not platstdlib_dir or not isdir(platstdlib_dir):
hint = home_hint if home else f'sys.exec_prefix is set to {exec_prefix}, is this correct?'
warn('WARN: Could not find the platform standard library directory! ' + hint)


# ******************************************************************************
# POSIX prefix/exec_prefix QUIRKS
# ******************************************************************************
Expand Down
9 changes: 7 additions & 2 deletions Modules/posixmodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -1280,6 +1280,8 @@ get_posix_state(PyObject *module)
* Contains a file descriptor if path.accept_fd was true
* and the caller provided a signed integer instead of any
* sort of string.
* path.is_fd
* True if path was provided as a file descriptor.
*
* WARNING: if your "path" parameter is optional, and is
* unspecified, path_converter will never get called.
Expand Down Expand Up @@ -1332,6 +1334,7 @@ typedef struct {
const wchar_t *wide;
const char *narrow;
int fd;
bool is_fd;
int value_error;
Py_ssize_t length;
PyObject *object;
Expand All @@ -1341,7 +1344,7 @@ typedef struct {
#define PATH_T_INITIALIZE(function_name, argument_name, nullable, nonstrict, \
make_wide, suppress_value_error, allow_fd) \
{function_name, argument_name, nullable, nonstrict, make_wide, \
suppress_value_error, allow_fd, NULL, NULL, -1, 0, 0, NULL, NULL}
suppress_value_error, allow_fd, NULL, NULL, -1, false, 0, 0, NULL, NULL}
#ifdef MS_WINDOWS
#define PATH_T_INITIALIZE_P(function_name, argument_name, nullable, \
nonstrict, suppress_value_error, allow_fd) \
Expand Down Expand Up @@ -1475,6 +1478,7 @@ path_converter(PyObject *o, void *p)
}
path->wide = NULL;
path->narrow = NULL;
path->is_fd = true;
goto success_exit;
}
else {
Expand Down Expand Up @@ -14328,8 +14332,9 @@ os_pathconf_impl(PyObject *module, path_t *path, int name)

errno = 0;
#ifdef HAVE_FPATHCONF
if (path->fd != -1)
if (path->is_fd) {
limit = fpathconf(path->fd, name);
}
else
#endif
limit = pathconf(path->narrow, name);
Expand Down
Loading
Loading