Floating point comparison not used for dict and list of tuples which mix strings and numbers

[lesteve]

With the following file, I would have expected the doctest to pass thanks to floating point comparison but it doesn't.

def test():
    """
    >>> 1/3
    0.33
    
    >>> ['a', 1/3, 'c']
    ['a', 0.33, 'c']
    
    >>> ('a', 1/3, 'c')
    ('a', 0.33, 'c')
   
    >>> [('a', 1/3), ('b', 1)] # fails
    [('a', 0.33), ('b', 1)]

    >>> {'a': 1/3} # fails
    {'a': 0.33}
    """

Output is:

❯ python -m scipy_doctest /tmp/test-doctest.py 

 test-doctest.test
 -----------------

File "/tmp/test-doctest.py", line 12, in test-doctest.test
Failed example:
    [('a', 1/3), ('b', 1)] # fails
Expected:
    [('a', 0.33), ('b', 1)]
Got:
    [('a', 0.3333333333333333), ('b', 1)]

 test-doctest.test
 -----------------

File "/tmp/test-doctest.py", line 15, in test-doctest.test
Failed example:
    {'a': 1/3} # fails
Expected:
    {'a': 0.33}
Got:
    {'a': 0.3333333333333333}


1 item had failures:
   2 of   5 in test-doctest.test
***Test Failed*** 2 failures.

This was seen noticed in scikit-learn. For now the easy work-around is to use ... i.e. something like:

>>> {'a': 1/3}
{'a': 0.33...}

[ev-br]

Thanks for the report!
It gets pretty obvious that scikit-learn usage is way more varied than what we saw in numpy or scipy, and we just never saw these kinds of uses before. Will fix.

[ev-br]

So the problem is (on numpy 2.x at least)

(Pdb) p want
[('a', 0.33), ('b', 1)]
(Pdb) p got
[('a', 0.3333333333333333), ('b', 1)]
(Pdb) p np.allclose(want, got)
*** numpy.exceptions.DTypePromotionError: The DType <class 'numpy.dtypes.StrDType'> could not be promoted by <class 'numpy.dtypes._PyFloatDType'>. This means that no common DType exists for the given inputs. For example they cannot be stored in a single array unless the dtype is `object`. The full list of DTypes is: (<class 'numpy.dtypes.StrDType'>, <class 'numpy.dtypes._PyFloatDType'>)

We do have some gymnastics to handle heterogenous tuples, but the code to handle them was written in numpy 1 time, and it's possible that we did not track a numpy 2 change.

[lesteve]

Thanks! For completeness, we only have a single instance of a doctest with a dict in scikit-learn doc. We don't have doctests with list of tuples.

I thought I tested that floating point comparison was used with dict in the context of #195 but apparently not 😅 ...

[ev-br]

Thanks for the context! Could you check that gh-203 fixes the scikit-learn use case now?

I thought I tested that floating point comparison was used with dict in the context of #195 but apparently not 😅 ...

Same here :-).

After spending a bit of time in the debugger and a bit of time doing repo archeology, I think the issue here is somewhat tricky. And it is not a numpy 2 thing, the root problem has been there since day one. The root cause is that the recursion into nested data structures is fiddly and is limited to a single level. Rewrting it in a more general case is not entirely straightforward.
That np.allclose('a', 'a') fails with a DTypePromotionError and that np.asarray(['a', 0.33]) produces an array of strings, array(['a', '0.33'], dtype='<U32'), makes it even more 'interesting'.

Overall, I now think that

• Dict comparison only checks the keys and not the values? #195 was a real bug, fixed by ENH: handle dicts #196;
• enabling floating-point comparisons of dicts with string keys, {"a": 1/3} is an enhancement, which ENH: make dicts with string keys use numeric comparisons #203 implements;
• enabling floating-point comparisons of lists of tuples with mixed string and numeric values would be a further enhancement;
• enabling floating-point comparisons for nested dictionaries with string keys would be an further enhancement, too.

The last two items would require quite a bit of surgery. Given that neither of them is found in either numpy, scipy or scikit-learn codebases suggests a pragmatic choice of leaving the last two enhancements for some other day.

[lesteve]

Thanks I can confirm #203 fixes the issue seen in scikit-learn with dict comparison.

I agree that more advanced comparison can be left for later if/when a strong need arises.