Any advice for tracking down the source of the unclosed event loop warning?

[alcarney]

I'm struggling to figure out why I am getting this warning

DeprecationWarning: pytest-asyncio detected an unclosed event loop when tearing down the event_loop
  fixture: <_UnixSelectorEventLoop running=False closed=False debug=False>
  pytest-asyncio will close the event loop for you, but future versions of the
  library will no longer do so. In order to ensure compatibility with future
  versions, please make sure that:
      1. Any custom "event_loop" fixture properly closes the loop after yielding it
      2. Your code does not modify the event loop in async fixtures or tests
  
    warnings.warn(

Originally, I thought it was due to me using a session scoped event_loop fixture.
After seeing #531 and doing some print() debugging I was only able to conclude that pytest-asyncio must be finding a different event loop to the one I was creating - where that loop is coming from I have no idea...

============================================================================== test session starts ===============================================================================
platform linux -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0
rootdir: /var/home/alex/Projects/esbonio/lib/esbonio
configfile: pyproject.toml
plugins: typeguard-2.13.3, asyncio-0.21.0, timeout-2.1.0, lsp-0.2.1
asyncio: mode=Mode.AUTO
collected 1 item                                                                                                                                                                 

tests/sphinx-default/test_sd_directives.py 
SETUP    S event_loop
SETUP    S client (fixtures used: event_loop)
        SETUP    F uri['definitions.rst']
        SETUP    F line[25]
        SETUP    F character[18]
        SETUP    F expected[None]
        tests/sphinx-default/test_sd_directives.py::test_directive_implementation[definitions.rst-25-18-None] (fixtures used: character, client, event_loop, expected, line, request, uri).
        TEARDOWN F expected[None]
        TEARDOWN F character[18]
        TEARDOWN F line[25]
        TEARDOWN F uri['definitions.rst']
TEARDOWN S client
Loop: <_UnixSelectorEventLoop running=False closed=False debug=False>     <--- Before calling loop.close()
Loop: <_UnixSelectorEventLoop running=False closed=True debug=False>      <--- After calling loop.close()

TEARDOWN S event_loop

================================================================================ warnings summary ================================================================================
tests/sphinx-default/test_sd_directives.py::test_directive_implementation[definitions.rst-25-18-None]
  /var/home/alex/Projects/esbonio/.env/lib64/python3.11/site-packages/pytest_asyncio/plugin.py:444: DeprecationWarning: pytest-asyncio detected an unclosed event loop when tearing down the event_loop
  fixture: <_UnixSelectorEventLoop running=False closed=False debug=False>
  pytest-asyncio will close the event loop for you, but future versions of the
  library will no longer do so. In order to ensure compatibility with future
  versions, please make sure that:
      1. Any custom "event_loop" fixture properly closes the loop after yielding it
      2. Your code does not modify the event loop in async fixtures or tests
  
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================== 1 passed, 1 warning in 2.23s ==========================================================================

But now I'm not so sure, since I can also trigger the warning using the standard event_loop that comes with pytest-asyncio

Contents of test_server.py (though I doubt it will be that useful)

import sys

from lsprotocol.types import ClientCapabilities
from lsprotocol.types import CompletionList
from lsprotocol.types import CompletionParams
from lsprotocol.types import InitializeParams
from lsprotocol.types import Position
from lsprotocol.types import TextDocumentIdentifier

import pytest_lsp
from pytest_lsp import ClientServerConfig
from pytest_lsp import LanguageClient


@pytest_lsp.fixture(
    config=ClientServerConfig(server_command=[sys.executable, "server.py"]),
)
async def client(lsp_client: LanguageClient):
    # Setup
    params = InitializeParams(capabilities=ClientCapabilities())
    await lsp_client.initialize_session(params)

    yield

    # Teardown
    await lsp_client.shutdown_session()


async def test_completions(client: LanguageClient):
    test_uri = "file:///path/to/file.txt"
    results = await client.text_document_completion_async(
        params=CompletionParams(
            position=Position(line=1, character=0),
            text_document=TextDocumentIdentifier(uri=test_uri),
        )
    )

    assert results is not None

    if isinstance(results, CompletionList):
        items = results.items
    else:
        items = results

    labels = [item.label for item in items]
    assert labels == [f"item-{i}" for i in range(10)]

    assert client.shown_documents[0].uri == test_uri

============================================================================== test session starts ===============================================================================
platform linux -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0
rootdir: /tmp/pytest-of-alex/pytest-1/test_documentation_examples4
configfile: tox.ini
plugins: typeguard-2.13.3, asyncio-0.21.0, timeout-2.1.0, lsp-0.2.1
asyncio: mode=Mode.AUTO
collected 1 item                                                                                                                                                                 

test_server.py 
        SETUP    F event_loop
        SETUP    F client (fixtures used: event_loop)
        test_server.py::test_completions (fixtures used: client, event_loop, request).
        TEARDOWN F client
        TEARDOWN F event_loop

================================================================================ warnings summary ================================================================================
test_server.py::test_completions
  /var/home/alex/Projects/esbonio/.env/lib64/python3.11/site-packages/pytest_asyncio/plugin.py:444: DeprecationWarning: pytest-asyncio detected an unclosed event loop when tearing down the event_loop
  fixture: <_UnixSelectorEventLoop running=False closed=False debug=False>
  pytest-asyncio will close the event loop for you, but future versions of the
  library will no longer do so. In order to ensure compatibility with future
  versions, please make sure that:
      1. Any custom "event_loop" fixture properly closes the loop after yielding it
      2. Your code does not modify the event loop in async fixtures or tests
  
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================== 1 passed, 1 warning in 1.10s ==========================================================================

So perhaps my issue isn't that I'm not closing the loops correctly... but I'm instead running into 2. Your code does not modify the event loop in async fixtures or tests? What would count as modification?

Within pytest-lsp I do create some tasks on the current event loop - would that be enough to upset things?

Any suggestions on how I might figure out what I'm doing wrong would be much appreciated 🙂

[seifertm]

So perhaps my issue isn't that I'm not closing the loops correctly... but I'm instead running into 2. Your code does not modify the event loop in async fixtures or tests? What would count as modification?

Sorry for the bad wording. Creating tasks should not cause any issues. Modification here essentially means replacing the current event loop within an async test or an async fixture.

I had a brief look at the pytest plugins you're using and none of them seem to do anything like this.

If you're willing to hack a bit, you can try replacing the DeprecationWarning with a ResourceWarning. Originally, the warning was intended as a ResourceWarning, because it allows users to track down the source of the unclosed event loop. ResourceWarnings are not shown by default, which is why it was changed to a DeprecationWarning (see commit 947e31a).

If you modify the _close_event_loop function in pytest-asyncio code from this:

            warnings.warn(
                _UNCLOSED_EVENT_LOOP_WARNING % loop,
                DeprecationWarning,
            )

to this:

            warnings.warn(
                _UNCLOSED_EVENT_LOOP_WARNING % loop,
                ResourceWarning,
                source=loop,
            )

and run your tests with -W default, you should be pointed to the line of code which creates the offending loop.

From the top of my head, this is the best I've got, sorry :/

[alcarney]

I made the change you suggested and running pytest with -W default resulted in an additional message being included with the warning Enable tracemalloc to get traceback where the object was allocated. However, I couldn't figure out how to get Python/pytest to reveal any more information, despite playing around with the PYTHONTRACEMALLOC environment variable as the documentation suggested... 🤔

But it did give me an idea. By setting a breakpoint in the constructor of asyncio.BaseEventLoop I was able to catch when each event loop was created! - It turns out that pygls (used by pytest-lsp) creates an event loop even when it's given one to use.

Thanks for the help! 🙂