Skip to content

Strange test_linspace assertion errors in jax #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
simonetgordon opened this issue Oct 25, 2022 · 3 comments
Closed

Strange test_linspace assertion errors in jax #153

simonetgordon opened this issue Oct 25, 2022 · 3 comments

Comments

@simonetgordon
Copy link
Contributor

simonetgordon commented Oct 25, 2022

Hi there,

test_linspace is failing for the jax backend for ivy, and hypothesis picks up two distinct failures:

  1. AssertionError: out[0]=ivy.array(nan), but should be 9.9792015476736e+291 [linspace(9.9792015476736e+291, -1.7976931348623157e+308, 2)]
  • raised on line
  • Falsifying example: linspace(9.9792015476736e+291, -1.7976931348623157e+308, num =2, dtype= 'float64', endpoint=False)
  • The output for the same inputs in numpy is: ivy.array([[ nan, -inf]]) (i.e. out[0] = nan, the same as in jax)
  1. AssertionError: out[0]=ivy.array(9.97920155e+291), should be ivy.array(nan) [linspace()]
  • raised on line: ph.assert_array_elements("linspace", out, expected)
  • Falsifying example: linspace(9.9792015476736e+291, -1.7976931348623157e+308, num =1, dtype= 'float64', endpoint=False)
  • The output for the same inputs in numpy is: ivy.array([[ nan]]) (i.e. out[0] = nan, different from the jax output)

I went about fixing the second failure as jax was diverging from numpy. However, I then (locally) got:
AssertionError: out[0]=ivy.array(nan), but should be 9.9792015476736e+291 [linspace(9.9792015476736e+291, -1.7976931348623157e+308, 1)].

Summary: the expected value for linspace seems to vary from the numpy value when start=9.9792015476736e+291 and stop= -1.7976931348623157e+308 (close to float64 min value) with num=1/2, causing the test to fail for jax.

I can't reproduce these inputs for numpy test_linspace, so it is unclear if it would also fail for numpy or any other backend - jax is the only one that has raised this issue.

Reference stack trace from ivy CI (click Run Array API Tests'

@asmeurer
Copy link
Member

This looks related to #141. Can you confirm whether you're using a version of the suite that includes that change? If so, perhaps the restriction there needs to be tightened.

@simonetgordon
Copy link
Contributor Author

@asmeurer in fact I'm not, I will update the submodule and get back to you. Thanks!

@simonetgordon
Copy link
Contributor Author

@asmeurer the update seems to have fixed the linspace issue - no failures in the first few commits. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants