Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

F822 Undefined name in __all__ does not work in __init__.py #10095

Closed
ilius opened this issue Feb 23, 2024 · 5 comments · Fixed by #11370
Closed

F822 Undefined name in __all__ does not work in __init__.py #10095

ilius opened this issue Feb 23, 2024 · 5 comments · Fixed by #11370

Comments

@ilius
Copy link

ilius commented Feb 23, 2024

If you have __all__ in a module, and it contains one or more names that are undefined in that module, ruff will tell you about it:

F822 Undefined name `abcd` in `__all__`

Unless that module file is named __init__.py.

To reproduce, simply create a file __init__.py in a directory / sub-directory, and add:

__all__ = ["abcd"]

and nothing else.
Run ruff and it gives no error!

Copy that file to another file like test.py and it will give error.

I have tried this in an empty project with no pyproject.toml file, so that doesn't seem to be related.

ruff --version
ruff 0.2.2
@ilius ilius changed the title F822 Undefined name in __all__ does not work in __init__.py F822 Undefined name in __all__ does not work in __init__.py Feb 23, 2024
@ilius
Copy link
Author

ilius commented Feb 23, 2024

Although an unknown name might be the name a module inside that directory!
So maybe ruff also has to check if abcd.py file or abcd directory exists in that directory.

@MichaReiser
Copy link
Member

Hmm, I'm not sure why this is the case. I see a few checks where we exclude __init__.py files but @charliermarsh has rewritten the logic a couple times which makes it rather challenging to find the origin PR. @charliermarsh do you remember why and when we exclude __init__.py files. We do have a setting related to init modules but I don't think that one is relevant here.

@AlexWaygood
Copy link
Member

AlexWaygood commented Feb 23, 2024

As @ilius mentioned, you can have __init__.py files that look like this: https://github.com/python/cpython/blob/main/Lib/xml/__init__.py. None of the names in __all__ there are defined in that file; __all__ is being used there to declare submodules of the xml package as public.

If you think this is an antipattern, so do I! At runtime, it has this (bizarre) effect, where * imports work differently (for some reason) to regular imports. I'm not sure why you'd ever actually want this:

% python                                                                                                                                       ~/dev
Python 3.12.2 (v3.12.2:6abddd9f6a, Feb  6 2024, 17:02:06) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml
>>> xml.dom
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'xml' has no attribute 'dom'
>>> xml.parsers
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'xml' has no attribute 'parsers'
>>> xml.etree
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'xml' has no attribute 'etree'
>>> xml.sax
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'xml' has no attribute 'sax'
>>> from xml import *
>>> xml.dom
<module 'xml.dom' from '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/xml/dom/__init__.py'>
>>> xml.parsers
<module 'xml.parsers' from '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/xml/parsers/__init__.py'>
>>> xml.etree
<module 'xml.etree' from '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/xml/etree/__init__.py'>
>>> xml.sax
<module 'xml.sax' from '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/xml/sax/__init__.py'>

It is (sadly) valid Python, though.

@hassec
Copy link
Contributor

hassec commented Feb 28, 2024

I'd argue that the more consistent behaviour would be to have ruff raise a F822 warning for all files, and then one can add an ignore comment for __init__.py files like the one linked @AlexWaygood

That IMHO is the better default as it would create less surprises.

Note that e.g. Pyright follows that approach.

@charliermarsh
Copy link
Member

I kind of agree with @hassec. I'm curious how disruptive this would be to change.

charliermarsh pushed a commit that referenced this issue May 30, 2024
## Summary

This PR aims to close #10095 by adding an option
`init-allow-undef-export` to the `pyflakes` settings. This option is
currently set to `true` such that behavior is kept identical.
But setting this option to `false` will lead to `F822` warnings to be
shown in all files, **including** `__init__.py` files.

As I've mentioned on #10095, I think `init-allow-undef-export=false`
would be the more user-friendly default option, as it creates fewer
surprises. @charliermarsh what do you think about making that the
default?

With this option in place, it's a single line fix for people that rely
on the old behavior.

And thinking longer term, for future major releases, one could probably
consider deprecating the option and eventually having people just `noqa`
these warnings if they are not wanted.


## Test Plan

I've added a `test_init_f822_enabled` test which repeats the test that
is done in the `init` test but this time with
`init-allow-undef-export=false` and the snap file correctly shows that
ruff will then trigger the otherwise suppressed F822 warning.


closes #10095
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants