Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Doc/library/site.rst
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ When running under a :ref:`virtual environment <sys-path-init-virtual-environmen
the ``pyvenv.cfg`` file in :data:`sys.prefix` is checked for site-specific
configurations. If the ``include-system-site-packages`` key exists and is set to
``true`` (case-insensitive), the system-level prefixes will be searched for
site-packages, otherwise they won't.
site-packages, otherwise they won't. If the system-level prefixes are not searched then
the user site prefixes are also implicitly not searched for site-packages.

.. index::
single: # (hash); comment
Expand Down
25 changes: 14 additions & 11 deletions Doc/library/sys_path_init.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,15 +57,19 @@ otherwise they are set to the same value as :data:`sys.base_prefix` and
:data:`sys.base_exec_prefix`, respectively.
This is used by :ref:`sys-path-init-virtual-environments`.

Finally, the :mod:`site` module is processed and :file:`site-packages` directories
are added to the module search path. A common way to customize the search path is
to create :mod:`sitecustomize` or :mod:`usercustomize` modules as described in
the :mod:`site` module documentation.
Finally, the :mod:`site` module is processed and :file:`site-packages`
directories are added to the module search path. The :envvar:`PYTHONUSERBASE`
environment variable controls where is searched for user site-packages and the
:envvar:`PYTHONNOUSERSITE` environment variable prevents searching for user
site-packages all together. A common way to customize the search path is to
create :mod:`sitecustomize` or :mod:`usercustomize` modules as described in the
:mod:`site` module documentation.

.. note::

Certain command line options may further affect path calculations.
See :option:`-E`, :option:`-I`, :option:`-s` and :option:`-S` for further details.
The command line options :option:`-E`, :option:`-P`, :option:`-I`,
:option:`-S` and :option:`-s` further affect path calculations, see their
documentation for details.

.. versionchanged:: 3.14

Expand Down Expand Up @@ -96,11 +100,10 @@ Please refer to :mod:`site`'s

.. note::

There are other ways how "virtual environments" could be implemented, this
documentation refers implementations based on the ``pyvenv.cfg`` mechanism,
such as :mod:`venv`. Most virtual environment implementations follow the
model set by :mod:`venv`, but there may be exotic implementations that
diverge from it.
There are other ways "virtual environments" could be implemented.
This documentation refers to implementations based on the ``pyvenv.cfg``
mechanism, such as :mod:`venv`, that many virtual environment implementations
follow.

_pth files
----------
Expand Down
8 changes: 6 additions & 2 deletions Doc/using/cmdline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -949,8 +949,9 @@ conflict.

.. envvar:: PYTHONNOUSERSITE

If this is set, Python won't add the :data:`user site-packages directory
<site.USER_SITE>` to :data:`sys.path`.
This is equivalent to the :option:`-s` option. If this is set, Python won't
add the :data:`user site-packages directory <site.USER_SITE>` to
:data:`sys.path`.

.. seealso::

Expand All @@ -964,6 +965,9 @@ conflict.
and :ref:`installation paths <sysconfig-user-scheme>` for
``python -m pip install --user``.

To disable the user site-packages, see :envvar:`PYTHONNOUSERSITE` or the :option:`-s`
option.

.. seealso::

:pep:`370` -- Per user site-packages directory
Expand Down
3 changes: 3 additions & 0 deletions Include/cpython/ceval.h
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ _PyEval_RequestCodeExtraIndex(freefunc f) {

PyAPI_FUNC(int) _PyEval_SliceIndex(PyObject *, Py_ssize_t *);
PyAPI_FUNC(int) _PyEval_SliceIndexNotNone(PyObject *, Py_ssize_t *);
PyAPI_FUNC(int) _PyEval_UnpackIndices(PyObject *, PyObject *,
Py_ssize_t,
Py_ssize_t *, Py_ssize_t *);


// Trampoline API
Expand Down
1 change: 1 addition & 0 deletions Include/internal/pycore_list.h
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ extern "C" {

PyAPI_FUNC(PyObject*) _PyList_Extend(PyListObject *, PyObject *);
PyAPI_FUNC(PyObject) *_PyList_SliceSubscript(PyObject*, PyObject*);
PyAPI_FUNC(PyObject *) _PyList_BinarySlice(PyObject *, PyObject *, PyObject *);
extern void _PyList_DebugMallocStats(FILE *out);
// _PyList_GetItemRef should be used only when the object is known as a list
// because it doesn't raise TypeError when the object is not a list, whereas PyList_GetItemRef does.
Expand Down
1 change: 1 addition & 0 deletions Include/internal/pycore_tuple.h
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ extern PyStatus _PyTuple_InitGlobalObjects(PyInterpreterState *);

PyAPI_FUNC(PyObject *)_PyTuple_FromStackRefStealOnSuccess(const union _PyStackRef *, Py_ssize_t);
PyAPI_FUNC(PyObject *)_PyTuple_FromArraySteal(PyObject *const *, Py_ssize_t);
PyAPI_FUNC(PyObject *) _PyTuple_BinarySlice(PyObject *, PyObject *, PyObject *);

typedef struct {
PyObject_HEAD
Expand Down
1 change: 1 addition & 0 deletions Include/internal/pycore_unicodeobject.h
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ extern PyObject* _PyUnicode_ResizeCompact(
PyObject *unicode,
Py_ssize_t length);
extern PyObject* _PyUnicode_GetEmpty(void);
PyAPI_FUNC(PyObject*) _PyUnicode_BinarySlice(PyObject *, PyObject *, PyObject *);


/* Generic helper macro to convert characters of different types.
Expand Down
26 changes: 16 additions & 10 deletions Lib/ctypes/util.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,24 +85,30 @@ def find_library(name):
wintypes.DWORD,
)

_psapi = ctypes.WinDLL('psapi', use_last_error=True)
_enum_process_modules = _psapi["EnumProcessModules"]
_enum_process_modules.restype = wintypes.BOOL
_enum_process_modules.argtypes = (
wintypes.HANDLE,
ctypes.POINTER(wintypes.HMODULE),
wintypes.DWORD,
wintypes.LPDWORD,
)
# gh-145307: We defer loading psapi.dll until _get_module_handles is called.
# Loading additional DLLs at startup for functionality that may never be
# used is wasteful.
_enum_process_modules = None

def _get_module_filename(module: wintypes.HMODULE):
name = (wintypes.WCHAR * 32767)() # UNICODE_STRING_MAX_CHARS
if _k32_get_module_file_name(module, name, len(name)):
return name.value
return None


def _get_module_handles():
global _enum_process_modules
if _enum_process_modules is None:
_psapi = ctypes.WinDLL('psapi', use_last_error=True)
_enum_process_modules = _psapi["EnumProcessModules"]
_enum_process_modules.restype = wintypes.BOOL
_enum_process_modules.argtypes = (
wintypes.HANDLE,
ctypes.POINTER(wintypes.HMODULE),
wintypes.DWORD,
wintypes.LPDWORD,
)

process = _get_current_process()
space_needed = wintypes.DWORD()
n = 1024
Expand Down
27 changes: 24 additions & 3 deletions Lib/glob.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

def glob(pathname, *, root_dir=None, dir_fd=None, recursive=False,
include_hidden=False):
"""Return a list of paths matching a pathname pattern.
"""Return a list of paths matching a `pathname` pattern.

The pattern may contain simple shell-style wildcards a la
fnmatch. Unlike fnmatch, filenames starting with a
Expand All @@ -25,6 +25,15 @@ def glob(pathname, *, root_dir=None, dir_fd=None, recursive=False,
The order of the returned list is undefined. Sort it if you need a
particular order.

If `root_dir` is not None, it should be a path-like object specifying the
root directory for searching. It has the same effect as changing the
current directory before calling it (without actually
changing it). If pathname is relative, the result will contain
paths relative to `root_dir`.

If `dir_fd` is not None, it should be a file descriptor referring to a
directory, and paths will then be relative to that directory.

If `include_hidden` is true, the patterns '*', '?', '**' will match hidden
directories.

Expand All @@ -36,7 +45,7 @@ def glob(pathname, *, root_dir=None, dir_fd=None, recursive=False,

def iglob(pathname, *, root_dir=None, dir_fd=None, recursive=False,
include_hidden=False):
"""Return an iterator which yields the paths matching a pathname pattern.
"""Return an iterator which yields the paths matching a `pathname` pattern.

The pattern may contain simple shell-style wildcards a la
fnmatch. However, unlike fnmatch, filenames starting with a
Expand All @@ -46,7 +55,19 @@ def iglob(pathname, *, root_dir=None, dir_fd=None, recursive=False,
The order of the returned paths is undefined. Sort them if you need a
particular order.

If recursive is true, the pattern '**' will match any files and
If `root_dir` is not None, it should be a path-like object specifying
the root directory for searching. It has the same effect as changing
the current directory before calling it (without actually
changing it). If pathname is relative, the result will contain
paths relative to `root_dir`.

If `dir_fd` is not None, it should be a file descriptor referring to a
directory, and paths will then be relative to that directory.

If `include_hidden` is true, the patterns '*', '?', '**' will match hidden
directories.

If `recursive` is true, the pattern '**' will match any files and
zero or more directories and subdirectories.
"""
sys.audit("glob.glob", pathname, recursive)
Expand Down
6 changes: 4 additions & 2 deletions Lib/site.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,11 @@
it is also checked for site-packages (sys.base_prefix and
sys.base_exec_prefix will always be the "real" prefixes of the Python
installation). If "pyvenv.cfg" (a bootstrap configuration file) contains
the key "include-system-site-packages" set to anything other than "false"
the key "include-system-site-packages" is set to "true"
(case-insensitive), the system-level prefixes will still also be
searched for site-packages; otherwise they won't.
searched for site-packages; otherwise they won't. If the system-level
prefixes are not included then the user site prefixes are also implicitly
not searched for site-packages.
All of the resulting site-specific directories, if they exist, are
appended to sys.path, and also inspected for path configuration
Expand Down
63 changes: 63 additions & 0 deletions Lib/test/test_dict.py
Original file line number Diff line number Diff line change
Expand Up @@ -1680,6 +1680,69 @@ def test_hash_collision_remove_add(self):
self.assertEqual(len(d), len(items), d)
self.assertEqual(d, dict(items))

def test_clear_reentrant_embedded(self):
# gh-130555: dict.clear() must be safe when values are embedded
# in an object and a destructor mutates the dict.
class MyObj: pass
class ClearOnDelete:
def __del__(self):
nonlocal x
del x

x = MyObj()
x.a = ClearOnDelete()

d = x.__dict__
d.clear()

def test_clear_reentrant_cycle(self):
# gh-130555: dict.clear() must be safe for embedded dicts when the
# object is part of a reference cycle and the last reference to the
# dict is via the cycle.
class MyObj: pass
obj = MyObj()
obj.f = obj
obj.attr = "attr"

d = obj.__dict__
del obj

d.clear()

def test_clear_reentrant_force_combined(self):
# gh-130555: dict.clear() must be safe when a destructor forces the
# dict from embedded/split to combined (setting ma_values to NULL).
class MyObj: pass
class ForceConvert:
def __del__(self):
d[1] = "trigger"

x = MyObj()
x.a = ForceConvert()
x.b = "other"

d = x.__dict__
d.clear()

def test_clear_reentrant_delete(self):
# gh-130555: dict.clear() must be safe when a destructor deletes
# a key from the same embedded dict.
class MyObj: pass
class DelKey:
def __del__(self):
try:
del d['b']
except KeyError:
pass

x = MyObj()
x.a = DelKey()
x.b = "value_b"
x.c = "value_c"

d = x.__dict__
d.clear()


class CAPITest(unittest.TestCase):

Expand Down
38 changes: 38 additions & 0 deletions Lib/test/test_zoneinfo/test_zoneinfo.py
Original file line number Diff line number Diff line change
Expand Up @@ -1577,6 +1577,44 @@ class EvilZoneInfo(self.klass):
class CZoneInfoCacheTest(ZoneInfoCacheTest):
module = c_zoneinfo

def test_inconsistent_weak_cache_get(self):
class Cache:
def get(self, key, default=None):
return 1337

class ZI(self.klass):
pass
# Class attribute must be set after class creation
# to override zoneinfo.ZoneInfo.__init_subclass__.
ZI._weak_cache = Cache()

with self.assertRaises(RuntimeError) as te:
ZI("America/Los_Angeles")
self.assertEqual(
str(te.exception),
"Unexpected instance of int in ZI weak cache for key 'America/Los_Angeles'"
)

def test_inconsistent_weak_cache_setdefault(self):
class Cache:
def get(self, key, default=None):
return default
def setdefault(self, key, value):
return 1337

class ZI(self.klass):
pass
# Class attribute must be set after class creation
# to override zoneinfo.ZoneInfo.__init_subclass__.
ZI._weak_cache = Cache()

with self.assertRaises(RuntimeError) as te:
ZI("America/Los_Angeles")
self.assertEqual(
str(te.exception),
"Unexpected instance of int in ZI weak cache for key 'America/Los_Angeles'"
)


class ZoneInfoPickleTest(TzPathUserMixin, ZoneInfoTestBase):
module = py_zoneinfo
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Optimize ``BINARY_SLICE`` for list, tuple, and unicode by avoiding temporary ``slice`` object creation.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Fix use-after-free in :meth:`dict.clear` when the dictionary values are
embedded in an object and a destructor causes re-entrant mutation of the
dictionary.
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
:mod:`zoneinfo`: fix a crash when instantiating :class:`~zoneinfo.ZoneInfo`
objects for which the internal class-level cache is inconsistent.
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Added missing explanations for some parameters in :func:`glob.glob` and
:func:`glob.iglob`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Defers loading of the ``psapi.dll`` module until it is used by
:func:`ctypes.util.dllist`.
Loading
Loading