Skip to content

(google-cloud-spanner) offset-naive and offset-aware datetime substraction with (Async)FixedSizePool #18093

Description

@mfragaFP

Determine this is the right repository

  • I determined this is the correct repository in which to report this bug.

Summary of the issue

Context
I'm pretty new with spanner and the client libraries. I was testing the async client for spanner and bumped into this issue as I tried to disabling multiplexed sessions. I was also using a fixed size pool I could create by hand instead the one created by default by the Client.

Expected Behavior:
Bumped into a random exception because of a substraction between a non naive and a naive datetime

Actual Behavior:
I'm never setting any internal datetimes by hand so I would not expect this to ever happen.

API client name and version

google-cloud-spanner v3.69.1

Reproduction steps: code

from google.cloud.spanner_v1 import AsyncClient, AsyncFixedSizePool 
import os
os.environ.setdefault("SPANNER_EMULATOR_HOST", "localhost:9010")
os.environ.setdefault("GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS", "false")
os.environ.setdefault("GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS_PARTITIONED_OPS", "false")
os.environ.setdefault("GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS_FOR_RW", "false")
os.environ.setdefault("SPANNER_DISABLE_BUILTIN_METRICS", "true")
os.environ.setdefault("OTEL_SDK_DISABLED", "true")

async def reproduce():
    client = AsyncClient(project=PROJECT_ID)
    instance = client.instance(INSTANCE_ID)
    # size=1 so every statement checks out the same session and hits the age math.
    pool = AsyncFixedSizePool(size=1)
    database = await instance.database(DATABASE_ID, pool=pool)
    try:
        for attempt in (1, 2):
            async with database.snapshot() as snapshot:
                result_set = await snapshot.execute_sql("SELECT 1")
                async for _ in result_set:
                    pass
            session = pool._sessions._queue[0]
            print(f"statement {attempt} ok, session last_use_time tzinfo = {session.last_use_time.tzinfo}")
    finally:
        await database.close()

same thing happened for the sync client:

from google.cloud.spanner_v1 import Client, FixedSizePool
import os
os.environ.setdefault("SPANNER_EMULATOR_HOST", "localhost:9010")
os.environ.setdefault("GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS", "false")
os.environ.setdefault("GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS_PARTITIONED_OPS", "false")
os.environ.setdefault("GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS_FOR_RW", "false")
os.environ.setdefault("SPANNER_DISABLE_BUILTIN_METRICS", "true")
os.environ.setdefault("OTEL_SDK_DISABLED", "true")

def reproduce():
    client = Client(project=PROJECT_ID)
    instance = client.instance(INSTANCE_ID)
    # size=1 so every statement checks out the same session and hits the age math.
    pool = FixedSizePool(size=1)
    database = instance.database(DATABASE_ID, pool=pool)
    try:
        for attempt in (1, 2):
            with database.snapshot() as snapshot:
                list(snapshot.execute_sql("SELECT 1"))
                
            session = pool._sessions.queue[0]
            print(f"statement {attempt} ok, session last_use_time tzinfo = {session.last_use_time.tzinfo}")
    finally:
        database.close()

Reproduction steps: supporting files

Reproduction steps: actual results

  File "/Users/mfraga/Documents/.venv/lib/python3.11/site-packages/google/cloud/spanner_v1/_async/pool.py", line 444, in get
    age = _NOW() - session.last_use_time
          ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
TypeError: can't subtract offset-naive and offset-aware datetimes

Reproduction steps: expected results

OS & version + platform

OSX Tahoe Version 26.6.1

Python environment

Python 3.12.13

Python dependencies

No response

Additional context

Looking at the code, both the sync and async versions of FixedSizePool do a substraction between a non naive datetime (datetime.now(UTC)) and session.last_time_use.

I'm not sure why but trace_call is setting it to a naive datetime.

This didn't happen with other pool types either

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage meI really want to be triaged.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions