Local-scoped annotations for "global" functions & objects

[castarco]

This comes from a previous issue I opened in MyPy's repository ( python/mypy#6947 ).

It started with the quite common topic of immutability and the usual stoppers (having to annotate very big libraries & stubs to make the mutability/immutability annotations useful at all).

The basic idea is to introduce local-scoped annotations that refer to objects & functions defined outside the particular scope where the annotation is introduced, in order to make the type checker happy (of course, it would be a "good faith" annotation).

The expected benefit is to allow programmers to define more strongly typed interfaces without having to wait for the whole Python ecosystem to annotate core & basic libraries.

As an example:

def bar(x):
    # No annotations at all here
    return x.to_str()

def foo(x: Immutable[C]) -> str:
    # We attach a local annotation to a function/callable, so the type checker
    # won't complain for calling a function that does not provide the guarantees
    # that `foo`'s signature is promising.
    #
    # annotate(bar): Callable[[Immutable[C]], str]
    return bar(x)

def moo(x: Immutable[C]) -> str:
    # This should generate a warning, because the annotation is redundant
    # annotate(foo): Callable[[Immutable[C]], str]
    return foo(x)

def func(x: Immutable[C]) -> int:
    # This should generate an error, because the annotation is not compatible
    # annotate(foo): Callable[[Immutable[C]], int]
    return foo(x)

Does it make sense to implement something on these lines? I could try to formalize this idea further if enough people find it interesting.

P.D.: This is currently possible by performing an assignment operation, and probably it's optimized out when Python is translated to its bytecode, although I'm not sure:

def foo(x: Immutable[C]) -> str:
    # Instead of using a new syntax inside comments, we can rely on
    # mechanisms that already exist.
    bar: Callable[[Immutable[C]], str] = bar
    return bar(x)

[gvanrossum]

Introducing a new notation like that would be a lot of effort, esp. if you want all type checkers to honor it. So I prefer the "typed alias" approach you found (although I think you can't reuse the name of the function for the name of the alias). It's not optimized out of the bytecode but the cost is just a local variable store+lookup, which is very fast.

[castarco]

although I think you can't reuse the name of the function for the name of the alias

True, it fails. I thought it worked because I tried it in ipython before without problems, but there's some magic going on in ipython.

from typing import Callable

def f(x):
    return x * 2

def g(x: str) -> str:
    f: Callable[[str], str] = f  # works in ipython, not in cpython
    return f(x)

g('hello')  # return 'hellohello'

[ilevkivskyi]

@castarco Regarding the syntax, note that there is https://www.python.org/dev/peps/pep-0593/ that allows arbitrary experimental extensions to the type system to play with. You can define:

from typing_extensions import Annotated
from typing import TypeVar

T = TypeVar('T')
Immutable = Annotated[T, 'Immutable']

x: Immutable[int]

The point is that type checkers that don't support this feature will just ignore it. So the actual syntax is not a problem. The problem is to find a type-checker whose maintainers will be willing to play with this proposed concept.

[castarco]

@ilevkivskyi Not sure about whether this (using the type system extensions) helps in case we try to apply something like the second approach I proposed (creating an alias with same name for external functions).

As far as I can see, no extra capabilities should be implemented in the current type checkers, but cpython complains at the time of creating the alias because we're referring to a variable that still does not exist (which is not entirely true, but I guess it's related to some implementation case I'm not familiar with).

This problem forces us (for now) to rewrite the previous example as:

from typing import Callable

def f(x):
    return x * 2

def g(x: str) -> str:
    # We can't reuse f, which adds some maintenance burden because
    # once `f` becomes properly annotated, we'll have to touch multiple
    # lines instead of just removing the following assignment.
    _f: Callable[[str], str] = f
    return _f(x)

g('hello')  # returns 'hellohello'

[gvanrossum]

@castarco This is just not going to happen. We've tried to explain why: complicated implementation, little benefit, easy workaround. We understand your idea, we're just not that interested in it, and it's not easy to implement. Please understand.