Fix broken refactor of awaitable generator patching

[ilevkivskyi]

Fixes #21426

Fix is straightforward: make the new logic more 1:1 with the old one (which was not the case for untyped functions).

[JukkaL]

It seems to regress unannotated functions -- their bodies may now be checked (but this wasn't the case in 1.20):

import types

@types.coroutine
def f(x):
    1 + "x"  # Error (but shouldn't be, since no annotation)
    yield

If I omit @types.coroutine, there is no error reported.

[ilevkivskyi]

Hm, this is kind of a pre-existing problem, I can reproduce this on older versions with e.g. a deferral (or otherwise force re-visiting the function). But now the issue is more prominent, because with two-phase checking every function is in some sense deferred.

Patching the defn.type in-place is a sketchy thing (and also all the code around awaitable generators is sketchy, but I knew about this when I started the whole two-phase checking thing).

After some thinking, we probably shouldn't do the wrapping at all for dynamic functions for consistency with regular async def:

async def f(): ...
reveal_type(f)  # "def () -> Any", not "def () -> Coroutine[Any, Any, Any]"

Or vice-versa, we should wrap both. So what would it be? @JukkaL

(to be clear both ways have relatively simple fixes)

[hauntsaninja]

Let's try wrapping both!

[ilevkivskyi]

Let's try wrapping both!

OK, let's see how it works. Btw there is another quirk that needed fixing, the

Use "-> None" if function does not return a value

note is added based on some random heuristics. I made some changes to preserve the current behavior (just to minimize number of ~unrelated changes in this PR)

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

psycopg (https://github.com/psycopg/psycopg)
+ tests/test_notify_async.py:223: error: Untyped decorator makes function "test_generator_and_handler" untyped  [untyped-decorator]
+ tests/pool/test_pool_async.py:543: error: All conditional function variants must have identical signatures  [misc]
+ tests/pool/test_pool_async.py:543: note: Original:
+ tests/pool/test_pool_async.py:543: note:     def failed(pool: Any) -> Coroutine[Any, Any, Any]
+ tests/pool/test_pool_async.py:543: note: Redefinition:
+ tests/pool/test_pool_async.py:543: note:     def failed(pool: Any) -> Any
+ tests/pool/test_pool_async.py:868: error: Untyped decorator makes function "test_stats_connect" untyped  [untyped-decorator]

aiohttp-devtools (https://github.com/aio-libs/aiohttp-devtools)
+ tests/test_runserver_config.py:34: error: Untyped decorator makes function "test_create_app_wrong_name" untyped  [untyped-decorator]
+ tests/test_runserver_config.py:44: error: Untyped decorator makes function "test_no_loop_coroutine" untyped  [untyped-decorator]
+ tests/test_runserver_config.py:65: error: Untyped decorator makes function "test_not_app" untyped  [untyped-decorator]
+ tests/test_runserver_config.py:80: error: Untyped decorator makes function "test_wrong_function_signature" untyped  [untyped-decorator]
+ tests/test_runserver_config.py:95: error: Untyped decorator makes function "test_no_ssl_context_factory" untyped  [untyped-decorator]
+ tests/test_runserver_config.py:110: error: Untyped decorator makes function "test_invalid_ssl_context" untyped  [untyped-decorator]
+ tests/test_runserver_main.py:123: error: Untyped decorator makes function "test_start_runserver_with_multi_app_modules" untyped  [untyped-decorator]
+ tests/test_runserver_main.py:163: error: Untyped decorator makes function "test_run_app_aiohttp_client" untyped  [untyped-decorator]
+ tests/test_runserver_main.py:180: error: Untyped decorator makes function "test_run_app_browser_cache" untyped  [untyped-decorator]
+ tests/test_runserver_main.py:207: error: Untyped decorator makes function "test_serve_main_app" untyped  [untyped-decorator]
+ tests/test_runserver_main.py:224: error: Untyped decorator makes function "test_start_main_app_app_instance" untyped  [untyped-decorator]

aioredis (https://github.com/aio-libs/aioredis)
+ aioredis/client.py:1697: error: Value of type "Coroutine[Any, Any, Any]" must be used  [unused-coroutine]
+ aioredis/client.py:1697: note: Are you missing an await?
+ aioredis/client.py:4426: error: Return type "Pipeline | Awaitable[Pipeline]" of "execute_command" incompatible with return type "Coroutine[Any, Any, Any]" in supertype "Redis"  [override]

scrapy (https://github.com/scrapy/scrapy)
+ tests/test_utils_reactor.py:20: error: Untyped decorator makes function "test_set_asyncio_event_loop" untyped  [untyped-decorator]
+ tests/test_utils_reactor.py:21: error: Untyped decorator makes function "test_set_asyncio_event_loop" untyped  [untyped-decorator]
+ tests/test_utils_defer.py:124: error: Untyped decorator makes function "test_coroutine_test_xfail" untyped  [untyped-decorator]
+ tests/test_spider_start.py:84: error: Untyped decorator makes function "test_asyncio_delayed" untyped  [untyped-decorator]
+ tests/test_spider_start.py:93: error: Untyped decorator makes function "test_twisted_delayed" untyped  [untyped-decorator]
+ tests/test_downloadermiddleware.py:272: error: Untyped decorator makes function "test_asyncdef_asyncio" untyped  [untyped-decorator]
+ tests/test_crawler.py:117: error: Untyped decorator makes function "test_crawler_crawl_async_twice_parallel_unsupported" untyped  [untyped-decorator]
+ tests/test_crawler.py:509: error: Argument 1 to "coroutine_test" has incompatible type "Callable[[TestCrawlerLogging, Any], Coroutine[Any, Any, Coroutine[Any, Any, Any]]]"; expected "Callable[[TestCrawlerLogging, Any], Awaitable[None]]"  [arg-type]
+ tests/test_crawler.py:510: error: Missing return statement  [return]
+ tests/test_spidermiddleware_process_start.py:101: error: Untyped decorator makes function "test_asyncio_sleep_single" untyped  [untyped-decorator]
+ tests/test_spidermiddleware_process_start.py:106: error: Untyped decorator makes function "test_asyncio_sleep_multiple" untyped  [untyped-decorator]
+ tests/test_spidermiddleware_process_start.py:113: error: Untyped decorator makes function "test_twisted_sleep_single" untyped  [untyped-decorator]
+ tests/test_spidermiddleware_process_start.py:118: error: Untyped decorator makes function "test_twisted_sleep_multiple" untyped  [untyped-decorator]
+ tests/test_spider_sitemap.py:424: error: Argument 1 to "coroutine_test" has incompatible type "Callable[[TestSitemapSpider], Coroutine[Any, Any, Coroutine[Any, Any, Any]]]"; expected "Callable[[TestSitemapSpider], Awaitable[None]]"  [arg-type]
+ tests/test_spider_sitemap.py:425: error: Missing return statement  [return]
+ tests/test_pipeline_media.py:335: error: Argument 1 to "coroutine_test" has incompatible type "Callable[[TestMediaPipeline], Coroutine[Any, Any, Coroutine[Any, Any, Any]]]"; expected "Callable[[TestMediaPipeline], Awaitable[None]]"  [arg-type]
+ tests/test_pipeline_media.py:336: error: Missing return statement  [return]
+ tests/test_engine.py:454: error: Untyped decorator makes function "test_start_already_running_exception_asyncio" untyped  [untyped-decorator]
+ tests/test_crawl.py:546: error: Untyped decorator makes function "test_async_def_asyncio_parse_items_list" untyped  [untyped-decorator]
+ tests/test_crawl.py:571: error: Untyped decorator makes function "test_async_def_asyncgen_parse" untyped  [untyped-decorator]
+ tests/test_crawl.py:579: error: Untyped decorator makes function "test_async_def_asyncgen_parse_loop" untyped  [untyped-decorator]
+ tests/test_crawl.py:589: error: Untyped decorator makes function "test_async_def_asyncgen_parse_exc" untyped  [untyped-decorator]
+ tests/test_crawl.py:601: error: Untyped decorator makes function "test_async_def_asyncgen_parse_complex" untyped  [untyped-decorator]
+ tests/test_crawl.py:613: error: Untyped decorator makes function "test_async_def_asyncio_parse_reqs_list" untyped  [untyped-decorator]
+ tests/test_crawl.py:620: error: Untyped decorator makes function "test_async_def_deferred_direct" untyped  [untyped-decorator]
+ tests/test_crawl.py:626: error: Untyped decorator makes function "test_async_def_deferred_wrapped" untyped  [untyped-decorator]

discord.py (https://github.com/Rapptz/discord.py)
+ discord/member.py:199: error: All conditional function variants must have identical signatures  [misc]
+ discord/member.py:199: note: Original:
+ discord/member.py:199: note:     def general(self: Any, *args: Any, **kwargs: Any) -> Coroutine[Any, Any, Any]
+ discord/member.py:199: note: Redefinition:
+ discord/member.py:199: note:     def general(self: Any, *args: Any, **kwargs: Any) -> Any

werkzeug (https://github.com/pallets/werkzeug)
- tests/test_local.py:62: error: Function is missing a type annotation  [no-untyped-def]
+ tests/test_local.py:62: error: Function is missing a return type annotation  [no-untyped-def]
+ tests/test_local.py:62: error: Function is missing a type annotation for one or more parameters  [no-untyped-def]
- tests/test_local.py:603: error: Function is missing a type annotation  [no-untyped-def]
+ tests/test_local.py:603: error: Function is missing a return type annotation  [no-untyped-def]
+ tests/test_local.py:603: error: Function is missing a type annotation for one or more parameters  [no-untyped-def]

tornado (https://github.com/tornadoweb/tornado)
+ tornado/test/asyncio_test.py:90: error: Argument 1 to "to_asyncio_future" has incompatible type "Coroutine[Any, Any, Any]"; expected "Future[Any]"  [arg-type]

graphql-core (https://github.com/graphql-python/graphql-core)
+ tests/utils/assert_equal_awaitables_or_values.py:26: error: Incompatible return value type (got "Coroutine[Any, Any, Any]", expected "T")  [return-value]
+ tests/execution/test_stream.py:1565: error: Incompatible types in assignment (expression has type "str", variable has type "Coroutine[Any, Any, Any]")  [assignment]
+ tests/execution/test_stream.py:1631: error: Incompatible types in assignment (expression has type "str", variable has type "Coroutine[Any, Any, Any]")  [assignment]
+ tests/execution/test_executor.py:442: error: Incompatible return value type (got "str", expected "Coroutine[Any, Any, Any]")  [return-value]
+ tests/execution/test_executor.py:451: error: Incompatible return value type (got "GraphQLError", expected "Coroutine[Any, Any, Any]")  [return-value]
+ tests/execution/test_executor.py:454: error: Incompatible return value type (got "GraphQLError", expected "Coroutine[Any, Any, Any]")  [return-value]
+ tests/execution/test_executor.py:938: error: Incompatible return value type (got "str", expected "Coroutine[Any, Any, Any]")  [return-value]
+ tests/execution/test_executor.py:944: error: Incompatible return value type (got "str", expected "Coroutine[Any, Any, Any]")  [return-value]
+ tests/execution/test_abstract.py:61: error: All conditional function variants must have identical signatures  [misc]
+ tests/execution/test_abstract.py:61: note: Original:
+ tests/execution/test_abstract.py:61: note:     def is_type_of(obj: Any, _info: Any) -> Any
+ tests/execution/test_abstract.py:61: note: Redefinition:
+ tests/execution/test_abstract.py:61: note:     def is_type_of(obj: Any, _info: Any) -> Coroutine[Any, Any, Any]
+ tests/execution/test_abstract.py:77: error: All conditional function variants must have identical signatures  [misc]
+ tests/execution/test_abstract.py:77: note: Original:
+ tests/execution/test_abstract.py:77: note:     def type_error(*_args: Any) -> Any
+ tests/execution/test_abstract.py:77: note: Redefinition:
+ tests/execution/test_abstract.py:77: note:     def type_error(*_args: Any) -> Coroutine[Any, Any, Any]