Iterating over ChainMap.values() and ChainMap.items() updates underlying defaultdicts

[Alexamk]

Bug report

Hello,
I encountered some confusing behavior when combining ChainMap with defaultdicts. Might be related to #23534
Specifically, iterating over the values or items of the ChainMap updates one of the underlying dictionaries, and returns the wrong result.
Just iterating over they keys or the dictionary directly produces the right results still and doesn't update the underlying dictionaries.

from collections import ChainMap, defaultdict

d1 = defaultdict(int, {'a': 1, 'b': 2})
d2 = defaultdict(int, {'c': 3, 'd': 4})
d3 = ChainMap(d1, d2)

print('Prior to iter')
print(d1)
print(d2)
print('After iter')
a = list(d3.values()) # Same result when calling d3.items
print(a) # Should return: [1, 2, 3, 4]. Returns: [0, 0, 1, 2]
print(d1) # Now: {'a': 1, 'b': 2, 'c': 0, 'd': 0}
print(d2) # Unchanged

Your environment

Tested on Python version 3.10.4.
Linux 5.13.0-41-generic #46~20.04.1-Ubuntu SMP
x86_64 x86_64 x86_64 GNU/Linux

[corona10]

cc @serhiy-storchaka

[corona10]

The following patch will solve this issue, but I am waiting for @rhettinger and @serhiy-storchaka

(Sorry for editing the previous comment: it will break other behavior.)

Lookups search the underlying mappings successively until a key is found. ¹

This behavior looks like it is intended because this is a matter of look-up order.
IMO, under the script, d1 did what it should do as defaultdict when meeting the non-existence key.
but I am waiting for both of the members whom I mentioned.

Footnotes

1. https://docs.python.org/3/library/collections.html?#collections.ChainMap ↩

[serhiy-storchaka]

Yes, it looks like an intentional behavior. There is even a comment in the code about this:

cpython/Lib/collections/__init__.py

Line 1001 in 8150b8c

return mapping[key] # can't use 'key in mapping' with defaultdict

[rhettinger]

To the extent that this is a problem, it lies with the defaultdict updating as a side-effect of lookups. If that isn't desired, it isn't hard to define a dict subclass with __missing__ that doesn't have the updating behavior (see collections.Counter for an example of how to do this).

[rhettinger]

FWIW, you can turn-off defaultdict's updating behavior prior to iteration:

d1.default_factory = None
d2.default_factory = None
a = list(d3.values()) # This no longer mutates d1 and d2