From 88150283bf41843e9a59376ab769cf2facb4ba19 Mon Sep 17 00:00:00 2001 From: Stephannie Jimenez Date: Thu, 22 Apr 2021 01:30:44 -0500 Subject: [PATCH 01/40] Add note for functions that doesn't support boolean array indexing --- spec/API_specification/searching_functions.md | 10 ++++++++++ spec/API_specification/set_functions.md | 5 +++++ 2 files changed, 15 insertions(+) diff --git a/spec/API_specification/searching_functions.md b/spec/API_specification/searching_functions.md index d3d06d7a6..0330e5f6e 100644 --- a/spec/API_specification/searching_functions.md +++ b/spec/API_specification/searching_functions.md @@ -69,6 +69,11 @@ Returns the indices of the minimum values along a specified axis. When the minim (function-nonzero)= ### nonzero(x, /) +```{note} + +This function does not support boolean array indexing. +``` + Returns the indices of the array elements which are non-zero. #### Parameters @@ -86,6 +91,11 @@ Returns the indices of the array elements which are non-zero. (function-where)= ### where(condition, x1, x2, /) +```{note} + +This function does not support boolean array indexing. +``` + Returns elements chosen from `x1` or `x2` depending on `condition`. #### Parameters diff --git a/spec/API_specification/set_functions.md b/spec/API_specification/set_functions.md index 73d8338cc..8a94817e5 100644 --- a/spec/API_specification/set_functions.md +++ b/spec/API_specification/set_functions.md @@ -15,6 +15,11 @@ A conforming implementation of the array API standard must provide and support t (function-unique)= ### unique(x, /, *, return_counts=False, return_index=False, return_inverse=False) +```{note} + +This function does not support boolean array indexing. +``` + Returns the unique elements of an input array `x`. #### Parameters From 2757e73cc413d89b6f898831e8f00a7ccbf4a5d6 Mon Sep 17 00:00:00 2001 From: Stephannie Jimenez Date: Mon, 26 Apr 2021 12:23:10 -0500 Subject: [PATCH 02/40] clarify the notes --- spec/API_specification/searching_functions.md | 4 ++-- spec/API_specification/set_functions.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/searching_functions.md b/spec/API_specification/searching_functions.md index 0330e5f6e..90ddc6e83 100644 --- a/spec/API_specification/searching_functions.md +++ b/spec/API_specification/searching_functions.md @@ -71,7 +71,7 @@ Returns the indices of the minimum values along a specified axis. When the minim ```{note} -This function does not support boolean array indexing. +This function is not compatible with static memory allocation and, thus, just-in-time compilation (JIT) as the shape of each output array is data-dependent (i.e., dependent on the contents of `x`). ``` Returns the indices of the array elements which are non-zero. @@ -93,7 +93,7 @@ Returns the indices of the array elements which are non-zero. ```{note} -This function does not support boolean array indexing. +This function is not compatible with static memory allocation and, thus, just-in-time compilation (JIT) as the shape of each output array is data-dependent (i.e., dependent on the contents of `x`). ``` Returns elements chosen from `x1` or `x2` depending on `condition`. diff --git a/spec/API_specification/set_functions.md b/spec/API_specification/set_functions.md index 8a94817e5..eb952b355 100644 --- a/spec/API_specification/set_functions.md +++ b/spec/API_specification/set_functions.md @@ -17,7 +17,7 @@ A conforming implementation of the array API standard must provide and support t ```{note} -This function does not support boolean array indexing. +This function is not compatible with static memory allocation and, thus, just-in-time compilation (JIT) as the shape of each output array is data-dependent (i.e., dependent on the contents of `x`). ``` Returns the unique elements of an input array `x`. From b98c9083a38fd80c03838e3a087f9de20d1ba0be Mon Sep 17 00:00:00 2001 From: Stephannie Jimenez Date: Mon, 26 Apr 2021 18:05:54 -0500 Subject: [PATCH 03/40] remove note from where function --- spec/API_specification/searching_functions.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/spec/API_specification/searching_functions.md b/spec/API_specification/searching_functions.md index 90ddc6e83..8662a12e2 100644 --- a/spec/API_specification/searching_functions.md +++ b/spec/API_specification/searching_functions.md @@ -91,11 +91,6 @@ Returns the indices of the array elements which are non-zero. (function-where)= ### where(condition, x1, x2, /) -```{note} - -This function is not compatible with static memory allocation and, thus, just-in-time compilation (JIT) as the shape of each output array is data-dependent (i.e., dependent on the contents of `x`). -``` - Returns elements chosen from `x1` or `x2` depending on `condition`. #### Parameters From ad3156d6c3b1f9abeb2de59be95b69b210f3730d Mon Sep 17 00:00:00 2001 From: Aaron Meurer Date: Tue, 27 Apr 2021 05:15:24 -0500 Subject: [PATCH 04/40] Some small fixes to function signatures to make them valid Python (#174) * Some small fixes to function signatures to make them valid Python * Fix an issue that was breaking the spec parsing in the test suite * Call the argument to broadcast_arrays 'arrays' This is consistent with meshgrid(). --- spec/API_specification/array_object.md | 2 +- spec/API_specification/creation_functions.md | 2 +- spec/API_specification/data_type_functions.md | 14 +++++++------- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/spec/API_specification/array_object.md b/spec/API_specification/array_object.md index c5ea23491..f0d695b8d 100644 --- a/spec/API_specification/array_object.md +++ b/spec/API_specification/array_object.md @@ -598,7 +598,7 @@ Returns `self[key]`. #### Parameters -- **self**: _<array;>_ +- **self**: _<array>_ - array instance. diff --git a/spec/API_specification/creation_functions.md b/spec/API_specification/creation_functions.md index 7ce12f5d9..90fe0dfda 100644 --- a/spec/API_specification/creation_functions.md +++ b/spec/API_specification/creation_functions.md @@ -290,7 +290,7 @@ Returns evenly spaced numbers over a specified interval. - a one-dimensional array containing evenly spaced values. (function-meshgrid)= -### meshgrid(*arrays, /, *, indexing='xy') +### meshgrid(*arrays, indexing='xy') Returns coordinate matrices from coordinate vectors. diff --git a/spec/API_specification/data_type_functions.md b/spec/API_specification/data_type_functions.md index 8df12d409..9f517dcf5 100644 --- a/spec/API_specification/data_type_functions.md +++ b/spec/API_specification/data_type_functions.md @@ -7,14 +7,14 @@ A conforming implementation of the array API standard must provide and support t ## Objects in API -(function-broadcast-arrays)= -### broadcast_arrays(\*args, /) +(function-broadcast_arrays)= +### broadcast_arrays(*arrays) Broadcasts one or more arrays against one another. #### Parameters -- **\*args**: _Sequence\[ <array> ]_ +- **arrays**: _Sequence\[ <array> ]_ - arrays to broadcast. @@ -24,7 +24,7 @@ Broadcasts one or more arrays against one another. - a list of broadcasted arrays. Each array must have the same shape. Each array must have the same dtype as its corresponding input array. -(function-broadcast-to)= +(function-broadcast_to)= ### broadcast_to(x, shape, /) Broadcasts an array to a specified shape. @@ -49,14 +49,14 @@ Broadcasts an array to a specified shape. - if the array is incompatible with the specified shape (see {ref}`broadcasting`). -(function-can-cast)= -### can_cast(from, to, /) +(function-can_cast)= +### can_cast(from_, to, /) Determines if one data type can be cast to another data type according {ref}`type-promotion` rules. #### Parameters -- **from**: _Union\[ <dtype>, <array>]_ +- **from_**: _Union\[ <dtype>, <array>]_ - input data type or array from which to cast. From e6d827afdbeb8db0f18288526231aa1904e63a5c Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Tue, 27 Apr 2021 20:40:33 +0200 Subject: [PATCH 05/40] Update API specification for creation and manipulation functions (#167) * Update specification for arange Addresses comments in gh-85 and gh-107 * Update the specification for `full` and `full_like` Addresses comments in gh-85 and gh-107 * Update specification for `linspace` Addresses comments in gh-85 and gh-107 * Update specification for `empty`, `ones`, `zeros` Addresses comments in gh-85 and gh-107 * Update specification for `eye` This is useful/needed because `M` is not a descriptive name and that name does not match between different array libraries. * Update specification for `expand_dims`, `roll` and `reshape` Address comment in gh-85 * One more change to `eye`, more descriptive positional arguments * Address the default integer dtype issue for 32/64-bit Python Closes gh-151 * Update signature of `broadcast_to` Address a review comment; makes it consistent with other functions using `shape`. --- spec/API_specification/creation_functions.md | 34 +++++++++---------- spec/API_specification/data_type_functions.md | 2 +- spec/API_specification/data_types.md | 3 +- .../manipulation_functions.md | 6 ++-- spec/API_specification/set_functions.md | 4 +-- 5 files changed, 24 insertions(+), 25 deletions(-) diff --git a/spec/API_specification/creation_functions.md b/spec/API_specification/creation_functions.md index 90fe0dfda..71491fe24 100644 --- a/spec/API_specification/creation_functions.md +++ b/spec/API_specification/creation_functions.md @@ -12,7 +12,7 @@ A conforming implementation of the array API standard must provide and support t (function-arange)= -### arange(start, /, *, stop=None, step=1, dtype=None, device=None) +### arange(start, /, stop=None, step=1, *, dtype=None, device=None) Returns evenly spaced values within the half-open interval `[start, stop)` as a one-dimensional array. @@ -33,11 +33,11 @@ This function cannot guarantee that the interval does not include the `stop` val - **step**: _Union\[ int, float ]_ - - the distance between two adjacent elements (`out[i+1] - out[i]`). Default: `1`. + - the distance between two adjacent elements (`out[i+1] - out[i]`). Must not be `0`; may be negative, this results in an empty array if `stop >= start`. Default: `1`. - **dtype**: _Optional\[ <dtype> ]_ - - output array data type. If `dtype` is `None`, the output array data type must be the default floating-point data type. Default: `None`. + - output array data type. If `dtype` is `None`, the output array data type must be inferred from `start`, `stop` and `step`. If those are all integers, the output array dtype must be the default integer dtype; if one or more have type `float`, then the output array dtype must be the default floating-point data type. Default: `None`. - **device**: _Optional\[ <device> ]_ @@ -47,7 +47,7 @@ This function cannot guarantee that the interval does not include the `stop` val - **out**: _<array>_ - - a one-dimensional array containing evenly spaced values. The length of the output array must be `ceil((stop-start)/step)`. + - a one-dimensional array containing evenly spaced values. The length of the output array must be `ceil((stop-start)/step)` if `stop - start` and `step` have the same sign, and length 0 otherwise. (function-asarray)= @@ -68,7 +68,7 @@ Convert the input to an array. - **dtype**: _Optional\[ <dtype> ]_ - - output array data type. If `dtype` is `None`, the output array data type must be inferred from the data type(s) in `obj`. Default: `None`. + - output array data type. If `dtype` is `None`, the output array data type must be inferred from the data type(s) in `obj`. If all input values are Python scalars, then if they're all `bool` the output dtype will be `bool`; if they're a mix of `bool`s and `int` the output dtype will be the default integer data type; if they contain `float`s the output dtype will be the default floating-point data type. Default: `None`. - **device**: _Optional\[ <device> ]_ @@ -86,7 +86,7 @@ Convert the input to an array. (function-empty)= -### empty(shape, /, *, dtype=None, device=None) +### empty(shape, *, dtype=None, device=None) Returns an uninitialized array having a specified `shape`. @@ -136,19 +136,19 @@ Returns an uninitialized array with the same `shape` as an input array `x`. - an array having the same shape as `x` and containing uninitialized data. (function-eye)= -### eye(N, /, *, M=None, k=0, dtype=None, device=None) +### eye(n_rows, n_cols=None, /, *, k=0, dtype=None, device=None) Returns a two-dimensional array with ones on the `k`th diagonal and zeros elsewhere. #### Parameters -- **N**: _int_ +- **n_rows**: _int_ - number of rows in the output array. -- **M**: _Optional\[ int ]_ +- **n_cols**: _Optional\[ int ]_ - - number of columns in the output array. If `None`, the default number of columns in the output array is `N`. Default: `None`. + - number of columns in the output array. If `None`, the default number of columns in the output array is equal to `n_rows`. Default: `None`. - **k**: _Optional\[ int ]_ @@ -190,7 +190,7 @@ Returns a new array containing the data from another (array) object with a `__dl ``` (function-full)= -### full(shape, fill_value, /, *, dtype=None, device=None) +### full(shape, fill_value, *, dtype=None, device=None) Returns a new array having a specified `shape` and filled with `fill_value`. @@ -206,7 +206,7 @@ Returns a new array having a specified `shape` and filled with `fill_value`. - **dtype**: _Optional\[ <dtype> ]_ - - output array data type. If `dtype` is `None`, the output array data type must be the default floating-point data type. Default: `None`. + - output array data type. If `dtype` is `None`, the output array data type must be inferred from `fill_value`. If it's an `int`, the output array dtype must be the default integer dtype; if it's a `float`, then the output array dtype must be the default floating-point data type; if it's a `bool` then the output array must have boolean dtype. Default: `None`. - **device**: _Optional\[ <device> ]_ @@ -219,7 +219,7 @@ Returns a new array having a specified `shape` and filled with `fill_value`. - an array where every element is equal to `fill_value`. (function-full_like)= -### full_like(x, fill_value, /, *, dtype=None, device=None) +### full_like(x, /, fill_value, *, dtype=None, device=None) Returns a new array filled with `fill_value` and having the same `shape` as an input array `x`. @@ -235,7 +235,7 @@ Returns a new array filled with `fill_value` and having the same `shape` as an i - **dtype**: _Optional\[ <dtype> ]_ - - output array data type. If `dtype` is `None`, the output array data type must be inferred from `x`. Default: `None`. + - output array data type. If `dtype` is `None`, the output array data type must be inferred from `fill_value` (see {ref}`function-full`). Default: `None`. - **device**: _Optional\[ <device> ]_ @@ -248,7 +248,7 @@ Returns a new array filled with `fill_value` and having the same `shape` as an i - an array having the same shape as `x` and where every element is equal to `fill_value`. (function-linspace)= -### linspace(start, stop, num, /, *, dtype=None, device=None, endpoint=True) +### linspace(start, stop, /, num, *, dtype=None, device=None, endpoint=True) Returns evenly spaced numbers over a specified interval. @@ -321,7 +321,7 @@ Returns coordinate matrices from coordinate vectors. The returned arrays must have a numeric data type determined by {ref}`type-promotion`. (function-ones)= -### ones(shape, /, *, dtype=None, device=None) +### ones(shape, *, dtype=None, device=None) Returns a new array having a specified `shape` and filled with ones. @@ -371,7 +371,7 @@ Returns a new array filled with ones and having the same `shape` as an input arr - an array having the same shape as `x` and filled with ones. (function-zeros)= -### zeros(shape, /, *, dtype=None, device=None) +### zeros(shape, *, dtype=None, device=None) Returns a new array having a specified `shape` and filled with zeros. diff --git a/spec/API_specification/data_type_functions.md b/spec/API_specification/data_type_functions.md index 9f517dcf5..0b76b386b 100644 --- a/spec/API_specification/data_type_functions.md +++ b/spec/API_specification/data_type_functions.md @@ -25,7 +25,7 @@ Broadcasts one or more arrays against one another. - a list of broadcasted arrays. Each array must have the same shape. Each array must have the same dtype as its corresponding input array. (function-broadcast_to)= -### broadcast_to(x, shape, /) +### broadcast_to(x, /, shape) Broadcasts an array to a specified shape. diff --git a/spec/API_specification/data_types.md b/spec/API_specification/data_types.md index b1c83b4d5..3788d8494 100644 --- a/spec/API_specification/data_types.md +++ b/spec/API_specification/data_types.md @@ -6,10 +6,9 @@ A conforming implementation of the array API standard must provide and support the following data types. -A conforming implementation of the array API standard must define a default floating-point data type (either `float32` or `float64`), as well as a default data type for an array index (either `int32` or `int64`). +A conforming implementation of the array API standard must define a default floating-point data type (either `float32` or `float64`), as well as a default integer data type (`int32` or `int64`). These default data types must be the same across platforms. The default integer data type may vary depending on whether Python is 32-bit or 64-bit. ```{note} - The default floating-point and array index integer data types should be clearly defined in a conforming library's documentation. ``` diff --git a/spec/API_specification/manipulation_functions.md b/spec/API_specification/manipulation_functions.md index 29fca5a29..edb116139 100644 --- a/spec/API_specification/manipulation_functions.md +++ b/spec/API_specification/manipulation_functions.md @@ -39,7 +39,7 @@ Joins a sequence of arrays along an existing axis. ``` (function-expand_dims)= -### expand_dims(x, axis, /) +### expand_dims(x, /, *, axis) Expands the shape of an array by inserting a new axis (dimension) of size one at the position specified by `axis`. @@ -81,7 +81,7 @@ Reverses the order of elements in an array along the given axis. The shape of th - an output array having the same data type and shape as `x` and whose elements, relative to `x`, are reordered. (function-reshape)= -### reshape(x, shape, /) +### reshape(x, /, shape) Reshapes an array without changing its data. @@ -102,7 +102,7 @@ Reshapes an array without changing its data. - an output array having the same data type, elements, and underlying element order as `x`. (function-roll)= -### roll(x, shift, /, *, axis=None) +### roll(x, /, shift, *, axis=None) Rolls array elements along a specified axis. Array elements that roll beyond the last position are re-introduced at the first position. Array elements that roll beyond the first position are re-introduced at the last position. diff --git a/spec/API_specification/set_functions.md b/spec/API_specification/set_functions.md index 73d8338cc..c8d04cffa 100644 --- a/spec/API_specification/set_functions.md +++ b/spec/API_specification/set_functions.md @@ -51,11 +51,11 @@ Returns the unique elements of an input array `x`. - **indices**: _<array>_ - - an array containing the indices (first occurrences) of `x` that result in `unique`. The returned array must have the default array index data type. + - an array containing the indices (first occurrences) of `x` that result in `unique`. The returned array must have the default integer data type. - **inverse**: _<array>_ - - an array containing the indices of `unique` that reconstruct `x`. The returned array must have the default array index data type. + - an array containing the indices of `unique` that reconstruct `x`. The returned array must have the default integer data type. - **counts**: _<array>_ From b0f4ca054c60858ada57ade15bc2f4d429c1d267 Mon Sep 17 00:00:00 2001 From: Stephannie Jimenez Gacha Date: Mon, 3 May 2021 13:55:28 -0500 Subject: [PATCH 06/40] Change deployment to github actions for github pages (#176) --- .github/workflows/pages.yml | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index 94419103c..77357707a 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -1,9 +1,9 @@ -on: +on: push: branches: - main pull_request: - branches: + branches: - "**" jobs: run: @@ -33,9 +33,7 @@ jobs: if-no-files-found: error - name: Deploy if: ${{ github.ref == 'refs/heads/main'}} - uses: JamesIves/github-pages-deploy-action@4.1.1 - env: - ACCESS_TOKEN: ${{ secrets.GITHUB_TOKEN }} - BASE_BRANCH: main # The branch the action should deploy from. - BRANCH: gh-pages # The branch the action should deploy to. - FOLDER: build + uses: peaceiris/actions-gh-pages@v3 + with: + github_token: ${{ secrets.GITHUB_TOKEN }} + publish_dir: ./build From 0c5d9134f4314bed0d54489d5d6b35be0923f599 Mon Sep 17 00:00:00 2001 From: Aaron Meurer Date: Mon, 3 May 2021 13:38:58 -0600 Subject: [PATCH 07/40] Specify the input types for logaddexp (#175) --- spec/API_specification/elementwise_functions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/API_specification/elementwise_functions.md b/spec/API_specification/elementwise_functions.md index 3dcf0dcf4..c1d14d159 100644 --- a/spec/API_specification/elementwise_functions.md +++ b/spec/API_specification/elementwise_functions.md @@ -946,11 +946,11 @@ For floating-point operands, - **x1**: _<array>_ - - first input array. + - first input array. Should have a floating-point data type. - **x2**: _<array>_ - - second input array. Must be compatible with `x1` (see {ref}`broadcasting`). + - second input array. Must be compatible with `x1` (see {ref}`broadcasting`). Should have a floating-point data type. #### Returns From ffc4d0ce2358898b5cb38168b65d5941512f0ade Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Mon, 3 May 2021 22:02:01 +0200 Subject: [PATCH 08/40] Create a unique admonition for data-dependent shapes --- spec/API_specification/searching_functions.md | 5 +++-- spec/API_specification/set_functions.md | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/spec/API_specification/searching_functions.md b/spec/API_specification/searching_functions.md index 8662a12e2..af85ee2d9 100644 --- a/spec/API_specification/searching_functions.md +++ b/spec/API_specification/searching_functions.md @@ -69,10 +69,11 @@ Returns the indices of the minimum values along a specified axis. When the minim (function-nonzero)= ### nonzero(x, /) -```{note} +:::{admonition} Data-dependent output shape +:class: important This function is not compatible with static memory allocation and, thus, just-in-time compilation (JIT) as the shape of each output array is data-dependent (i.e., dependent on the contents of `x`). -``` +::: Returns the indices of the array elements which are non-zero. diff --git a/spec/API_specification/set_functions.md b/spec/API_specification/set_functions.md index eb952b355..2104ef5bc 100644 --- a/spec/API_specification/set_functions.md +++ b/spec/API_specification/set_functions.md @@ -15,10 +15,11 @@ A conforming implementation of the array API standard must provide and support t (function-unique)= ### unique(x, /, *, return_counts=False, return_index=False, return_inverse=False) -```{note} +:::{admonition} Data-dependent output shape +:class: important This function is not compatible with static memory allocation and, thus, just-in-time compilation (JIT) as the shape of each output array is data-dependent (i.e., dependent on the contents of `x`). -``` +::: Returns the unique elements of an input array `x`. From 2e759ef79fe52542044bf075e29a31f4eadbd8a5 Mon Sep 17 00:00:00 2001 From: Athan Date: Mon, 3 May 2021 13:18:58 -0700 Subject: [PATCH 09/40] Add data type guidance (#173) --- spec/API_specification/array_object.md | 116 +++++++++--------- .../elementwise_functions.md | 2 +- 2 files changed, 59 insertions(+), 59 deletions(-) diff --git a/spec/API_specification/array_object.md b/spec/API_specification/array_object.md index f0d695b8d..ca29db0a2 100644 --- a/spec/API_specification/array_object.md +++ b/spec/API_specification/array_object.md @@ -288,7 +288,7 @@ For floating-point operands, let `self` equal `x`. - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. #### Returns @@ -337,11 +337,11 @@ Floating-point addition is a commutative operation, but not always associative. - **self**: _<array>_ - - array instance (augend array). + - array instance (augend array). Should have a numeric data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, <array> ]_ - - addend array. Must be compatible with `self` (see {ref}`broadcasting`). + - addend array. Must be compatible with `self` (see {ref}`broadcasting`). Should have a numeric data type. #### Returns @@ -363,11 +363,11 @@ Evaluates `self_i & other_i` for each element of an array instance with the resp - **self**: _<array>_ - - array instance. Must have an integer or boolean data type. + - array instance. Should have an integer or boolean data type. -- **other**: _<array>_ +- **other**: _Union\[ int, bool, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). Must have an integer or boolean data type. + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have an integer or boolean data type. #### Returns @@ -505,11 +505,11 @@ Computes the truth value of `self_i == other_i` for each element of an array ins - **self**: _<array>_ - - array instance. + - array instance. May have any data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, bool, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). + - other array. Must be compatible with `self` (see {ref}`broadcasting`). May have any data type. #### Returns @@ -548,11 +548,11 @@ Evaluates `self_i // other_i` for each element of an array instance with the res - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have a numeric data type. #### Returns @@ -574,11 +574,11 @@ Computes the truth value of `self_i >= other_i` for each element of an array ins - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have a numeric data type. #### Returns @@ -621,11 +621,11 @@ Computes the truth value of `self_i > other_i` for each element of an array inst - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have a numeric data type. #### Returns @@ -664,7 +664,7 @@ Evaluates `~self_i` for each element of an array instance. - **self**: _<array>_ - - array instance. Must have an integer or boolean data type. + - array instance. Should have an integer or boolean data type. #### Returns @@ -686,11 +686,11 @@ Computes the truth value of `self_i <= other_i` for each element of an array ins - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have a numeric data type. #### Returns @@ -717,11 +717,11 @@ Evaluates `self_i << other_i` for each element of an array instance with the res - **self**: _<array>_ - - array instance. Must have an integer data type. + - array instance. Should have an integer data type. -- **other**: _<array>_ +- **other**: _Union\[ int, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). Must have an integer data type. Each element must be greater than or equal to `0`. + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have an integer data type. Each element must be greater than or equal to `0`. #### Returns @@ -743,11 +743,11 @@ Computes the truth value of `self_i < other_i` for each element of an array inst - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have a numeric data type. #### Returns @@ -790,11 +790,11 @@ Evaluates `self_i % other_i` for each element of an array instance with the resp - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have a numeric data type. #### Returns @@ -835,11 +835,11 @@ Floating-point multiplication is not always associative due to finite precision. - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have a numeric data type. #### Returns @@ -861,11 +861,11 @@ Computes the truth value of `self_i != other_i` for each element of an array ins - **self**: _<array>_ - - array instance. + - array instance. May have any data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, bool, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). + - other array. Must be compatible with `self` (see {ref}`broadcasting`). May have any data type. #### Returns @@ -887,7 +887,7 @@ Evaluates `-self_i` for each element of an array instance. - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. #### Returns @@ -909,11 +909,11 @@ Evaluates `self_i | other_i` for each element of an array instance with the resp - **self**: _<array>_ - - array instance. Must have an integer or boolean data type. + - array instance. Should have an integer or boolean data type. -- **other**: _<array>_ +- **other**: _Union\[ int, bool, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). Must have an integer or boolean data type. + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have an integer or boolean data type. #### Returns @@ -935,7 +935,7 @@ Evaluates `+self_i` for each element of an array instance. - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. #### Returns @@ -986,11 +986,11 @@ For floating-point operands, let `self` equal `x1` and `other` equal `x2`. - **self**: _<array>_ - - array instance whose elements correspond to the exponentiation base. + - array instance whose elements correspond to the exponentiation base. Should have a numeric data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, <array> ]_ - - other array whose elements correspond to the exponentiation exponent. Must be compatible with `self` (see {ref}`broadcasting`). + - other array whose elements correspond to the exponentiation exponent. Must be compatible with `self` (see {ref}`broadcasting`). Should have a numeric data type. #### Returns @@ -1012,11 +1012,11 @@ Evaluates `self_i >> other_i` for each element of an array instance with the res - **self**: _<array>_ - - array instance. Must have an integer data type. + - array instance. Should have an integer data type. -- **other**: _<array>_ +- **other**: _Union\[ int, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). Must have an integer data type. Each element must be greater than or equal to `0`. + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have an integer data type. Each element must be greater than or equal to `0`. #### Returns @@ -1043,11 +1043,11 @@ Calculates the difference for each element of an array instance with the respect - **self**: _<array>_ - - array instance (minuend array). + - array instance (minuend array). Should have a numeric data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, <array> ]_ - - subtrahend array. Must be compatible with `self` (see {ref}`broadcasting`). + - subtrahend array. Must be compatible with `self` (see {ref}`broadcasting`). Should have a numeric data type. #### Returns @@ -1096,11 +1096,11 @@ For floating-point operands, let `self` equal `x1` and `other` equal `x2`. - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. -- **other**: _<array>_ +- **other**: _Union\[ int, float, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have a numeric data type. #### Returns @@ -1122,11 +1122,11 @@ Evaluates `self_i ^ other_i` for each element of an array instance with the resp - **self**: _<array>_ - - array instance. Must have an integer or boolean data type. + - array instance. Should have an integer or boolean data type. -- **other**: _<array>_ +- **other**: _Union\[ int, bool, <array> ]_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). Must have an integer or boolean data type. + - other array. Must be compatible with `self` (see {ref}`broadcasting`). Should have an integer or boolean data type. #### Returns diff --git a/spec/API_specification/elementwise_functions.md b/spec/API_specification/elementwise_functions.md index c1d14d159..aa69c9a6a 100644 --- a/spec/API_specification/elementwise_functions.md +++ b/spec/API_specification/elementwise_functions.md @@ -570,7 +570,7 @@ Computes the truth value of `x1_i == x2_i` for each element `x1_i` of the input - **x2**: _<array>_ - - second input array. Must be compatible with `x1` (see {ref}`broadcasting`). + - second input array. Must be compatible with `x1` (see {ref}`broadcasting`). May have any data type. #### Returns From 204e283e5dc458d64d0acd802b949e6f3ab1365e Mon Sep 17 00:00:00 2001 From: Stephannie Jimenez Date: Thu, 6 May 2021 14:54:23 -0500 Subject: [PATCH 10/40] Rewrite note with the given suggestions --- spec/API_specification/searching_functions.md | 2 +- spec/API_specification/set_functions.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/API_specification/searching_functions.md b/spec/API_specification/searching_functions.md index af85ee2d9..b028ee5f8 100644 --- a/spec/API_specification/searching_functions.md +++ b/spec/API_specification/searching_functions.md @@ -72,7 +72,7 @@ Returns the indices of the minimum values along a specified axis. When the minim :::{admonition} Data-dependent output shape :class: important -This function is not compatible with static memory allocation and, thus, just-in-time compilation (JIT) as the shape of each output array is data-dependent (i.e., dependent on the contents of `x`). +The shape of the output array for this function depends on the data values in the input array, hence it can be difficult to implement for libraries that build computation graphs for arrays without knowing their values. See {ref}`indexing` section for more details. ::: Returns the indices of the array elements which are non-zero. diff --git a/spec/API_specification/set_functions.md b/spec/API_specification/set_functions.md index 2104ef5bc..e6e0ca34c 100644 --- a/spec/API_specification/set_functions.md +++ b/spec/API_specification/set_functions.md @@ -18,7 +18,7 @@ A conforming implementation of the array API standard must provide and support t :::{admonition} Data-dependent output shape :class: important -This function is not compatible with static memory allocation and, thus, just-in-time compilation (JIT) as the shape of each output array is data-dependent (i.e., dependent on the contents of `x`). +The shape of the output array for this function depends on the data values in the input array, hence it can be difficult to implement for libraries that build computation graphs for arrays without knowing their values. See {ref}`indexing` section for more details. ::: Returns the unique elements of an input array `x`. From 4d74293b97870ce27387d29d7d0322fd71303ad7 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Tue, 11 May 2021 21:49:49 +0200 Subject: [PATCH 11/40] Make axis keyword to squeeze() positional (#100) As suggested by @shoyer in https://github.com/data-apis/array-api/issues/97#issuecomment-747116655 This makes it possible to predict resulting rank of output array, which is otherwise undetermined (see discussion in gh-97). Using squeeze without specifying the axis in library code often results in unintuitive behaviour. For the common use case of extracting a scalar from a size-1 2-D array, this gets a little more verbose (e.g. `np.squeeze(np.array([[1]]), axis=(0, 1))`), but that's probably a price worth paying for the extra clarity. Also changes specified behaviour for a given axis not having size 1 to raising a `ValueError`, which is what NumPy does. This wasn't considered before, and the current description seems simply incorrect. Finally, this makes `squeeze` the exact inverse of `expand_dims`, which is probably a good thing. --- spec/API_specification/manipulation_functions.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/manipulation_functions.md b/spec/API_specification/manipulation_functions.md index edb116139..ae8499370 100644 --- a/spec/API_specification/manipulation_functions.md +++ b/spec/API_specification/manipulation_functions.md @@ -127,7 +127,7 @@ Rolls array elements along a specified axis. Array elements that roll beyond the - an output array having the same data type as `x` and whose elements, relative to `x`, are shifted. (function-squeeze)= -### squeeze(x, /, *, axis=None) +### squeeze(x, /, axis) Removes singleton dimensions (axes) from `x`. @@ -137,9 +137,9 @@ Removes singleton dimensions (axes) from `x`. - input array. -- **axis**: _Optional\[ Union\[ int, Tuple\[ int, ... ] ] ]_ +- **axis**: _Union\[ int, Tuple\[ int, ... ] ]_ - - axis (or axes) to squeeze. If provided, only the specified axes must be squeezed. If `axis` is `None`, all singleton dimensions (axes) must be removed. If a specified axis has a size greater than one, the specified axis must be left unchanged. Default: `None`. + - axis (or axes) to squeeze. If a specified axis has a size greater than one, a `ValueError` must be raised. #### Returns From 76730f15370a1620a78c46799184195834324f2d Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 19:56:00 -0700 Subject: [PATCH 12/40] Add Cholesky function specification (#110) * Add Cholesky spec * Update description * Return an array rather than a tuple * Update dtype requirements * Update dtype requirements * Move API to submodule --- .../linear_algebra_functions.md | 24 ++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index 100efccf2..bc4b8ee6a 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -15,10 +15,28 @@ A conforming implementation of the array API standard must provide and support t -(function-cholesky)= -### cholesky() +(function-linalg-cholesky)= +### linalg.cholesky(x, /, *, upper=False) -TODO +Returns the Cholesky decomposition of a symmetric positive-definite matrix (or a stack of symmetric positive-definite matrices) `x`. + + + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. + +- **upper**: _bool_ + + - If `True`, the result must be the upper-triangular Cholesky factor. If `False`, the result must be the lower-triangular Cholesky factor. Default: `False`. + +#### Returns + +- **out**: _<array>_ + + - an array containing the Cholesky factors for each square matrix. The returned array must have a floating-point data type determined by {ref}`type-promotion` and shape as `x`. (function-cross)= ### cross(x1, x2, /, *, axis=-1) From 059e4fe27f7aa8d68d61a1032ba3dc515a82be87 Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 19:57:00 -0700 Subject: [PATCH 13/40] Add slogdet specification (#111) * Add slogdet spec * Add note * Update description * Reformat description and update return value type * Add article * Simplify description * Update copy * Update copy * Update dtype requirements * Update dtype requirement for returned arrays * Update copy * Update copy * Move API to submodule --- .../linear_algebra_functions.md | 30 +++++++++++++++++-- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index bc4b8ee6a..04924b93d 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -282,10 +282,34 @@ TODO TODO -(function-slogdet)= -### slogdet() +(function-linalg-slogdet)= +### linalg.slogdet(x, /) -TODO +Returns the sign and the natural logarithm of the absolute value of the determinant of a square matrix (or a stack of square matrices) `x`. + +```{note} + +The purpose of this function is to calculate the determinant more accurately when the determinant is either very small or very large, as calling `det` may overflow or underflow. +``` + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. + +#### Returns + +- **out**: _Tuple\[ <array>, <array> ]_ + + - a namedtuple (`sign`, `logabsdet`) whose + + - first element must be an array containing a number representing the sign of the determinant for each square matrix. + - second element must be an array containing the determinant for each square matrix. + + For a real matrix, the sign of the determinant must be either `1`, `0`, or `-1`. If a determinant is zero, then the corresponding `sign` must be `0` and `logabsdet` must be `-infinity`. In all cases, the determinant must be equal to `sign * exp(logsabsdet)`. + + Each returned array must have shape `shape(x)[:-2]` and a floating-point data type determined by {ref}`type-promotion`. (function-solve)= ### solve() From 9893373ecad034fe4c92b7877b4aded9edf97c49 Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 21:52:23 -0700 Subject: [PATCH 14/40] Add specification for computing the pseudo-inverse (linalg: pinv) (#118) * Add pinv specification * Fix statement on broadcasting * Fix type annotation * Rename keyword argument and drop scaling factor * Update copy * Rename keyword argument * Fix argument name * Update dtype requirements * Fix missing period * Update copy * Update copy * Update dtype requirements * Update copy * Move API to submodule --- .../linear_algebra_functions.md | 22 ++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index 04924b93d..9b041b366 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -272,10 +272,26 @@ Computes the outer product of two vectors `x1` and `x2`. - a two-dimensional array containing the outer product and whose shape is `(N, M)`. The returned array must have a data type determined by {ref}`type-promotion`. -(function-pinv)= -### pinv() +(function-linalg-pinv)= +### linalg.pinv(x, /, *, rtol=None) -TODO +Computes the (Moore-Penrose) pseudo-inverse of a matrix (or a stack of square matrices) `x`. + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. + +- **rtol**: _Optional\[ Union\[ float, <array> ] ]_ + + - relative tolerance for small singular values. Singular values less than or equal to `rtol * largest_singular_value` are set to zero. If a `float`, the value is equivalent to a zero-dimensional array having a floating-point data type determined by {ref}`type-promotion` (as applied to `x`) and must be broadcast against each matrix. If an `array`, must have a floating-point data type and must be compatible with `shape(x)[:-2]` (see {ref}`broadcasting`). If `None`, the default value is `max(M, N) * eps`, where `eps` must be the machine epsilon associated with the floating-point data type determined by {ref}`type-promotion` (as applied to `x`). Default: `None`. + +#### Returns + +- **out**: _<array>_ + + - an array containing the pseudo-inverses. The returned array must have a floating-point data type determined by {ref}`type-promotion` and must have shape `(..., N, M)` (i.e., must have the same shape as `x`, except the innermost two dimensions must be transposed). (function-qr)= ### qr() From 36fd440f4121fc98a6f45cf9f53837cba08476b2 Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 21:54:19 -0700 Subject: [PATCH 15/40] Add specification for computing the dot product (linalg: vecdot) (#137) * Add dot product specification * Update specification * Fix output shape * Update description * Update copy * Update copy * Rename to inner_dot * Update dtype requirements * Rename to vecdot --- .../linear_algebra_functions.md | 35 ++++++++++++++++--- 1 file changed, 30 insertions(+), 5 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index 9b041b366..aba0a6e93 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -115,11 +115,6 @@ Returns the specified diagonals. If `x` has more than two dimensions, then the a - if `x` is a two-dimensional array, a one-dimensional array containing the diagonal; otherwise, a multi-dimensional array containing the diagonals and whose shape is determined by removing `axis1` and `axis2` and appending a dimension equal to the size of the resulting diagonals. The returned array must have the same data type as `x`. -(function-dot)= -### dot() - -TODO - (function-eig)= ### eig() @@ -400,3 +395,33 @@ Transposes (or permutes the axes (dimensions)) of an array `x`. - **out**: _<array>_ - an array containing the transpose. The returned array must have the same data type as `x`. + +(function-vecdot)= +### vecdot(x1, x2, /, *, axis=None) + +Computes the (vector) dot product of two arrays. + +#### Parameters + +- **x1**: _<array>_ + + - first input array. Should have a numeric data type. + +- **x2**: _<array>_ + + - second input array. Must be compatible with `x1` (see {ref}`broadcasting`). Should have a numeric data type. + +- **axis**: _Optional\[ int ]_ + + - axis over which to compute the dot product. Must be an integer on the interval `[-N, N)`, where `N` is the rank (number of dimensions) of the shape determined according to {ref}`broadcasting`. If specified as a negative integer, the function must determine the axis along which to compute the dot product by counting backward from the last dimension (where `-1` refers to the last dimension). If `None`, the function must compute the dot product over the last axis. Default: `None`. + +#### Returns + +- **out**: _<array;>_ + + - if `x1` and `x2` are both one-dimensional arrays, a zero-dimensional containing the dot product; otherwise, a non-zero-dimensional array containing the dot products and having rank `N-1`, where `N` is the rank (number of dimensions) of the shape determined according to {ref}`broadcasting`. The returned array must have a data type determined by {ref}`type-promotion`. + +#### Raises + +- if provided an invalid `axis`. +- if the size of the axis over which to compute the dot product is not the same for both `x1` and `x2`. From 0768fc8a550e09d27ff4a0265253330b1f565136 Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 21:55:38 -0700 Subject: [PATCH 16/40] Add specification for computing a tensor contraction (linalg: tensordot) (#136) * Add tensordot specification * Update data type requirements * Update dtype requirements * Fix missing header --- .../linear_algebra_functions.md | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index aba0a6e93..d868103f0 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -332,6 +332,39 @@ TODO TODO +(function-tensordot)= +### tensordot(x1, x2, /, *, axes=2) + +Returns a tensor contraction of `x1` and `x2` over specific axes. + +#### Parameters + +- **x1**: _<array>_ + + - first input array. Should have a numeric data type. + +- **x2**: _<array>_ + + - second input array. Must be compatible with `x1` (see {ref}`broadcasting`). Should have a numeric data type. + +- **axes**: _Union\[ int, Tuple\[ Sequence\[ int ], Sequence\[ int ] ] ]_ + + - number of axes (dimensions) to contract or explicit sequences of axes (dimensions) for `x1` and `x2`, respectively. + + If `axes` is an `int` equal to `N`, then contraction must be performed over the last `N` axes of `x1` and the first `N` axes of `x2` in order. The size of each corresponding axis (dimension) must match. Must be nonnegative. + + - If `N` equals `0`, the result is the tensor (outer) product. + - If `N` equals `1`, the result is the tensor dot product. + - If `N` equals `2`, the result is the tensor double contraction (default). + + If `axes` is a tuple of two sequences `(x1_axes, x2_axes)`, the first sequence must apply to `x` and the second sequence to `x2`. Both sequences must have the same length. Each axis (dimension) `x1_axes[i]` for `x1` must have the same size as the respective axis (dimension) `x2_axes[i]` for `x2`. Each sequence must consist of unique (nonnegative) integers that specify valid axes for each respective array. + +#### Returns + +- **out**: _<array>_ + + - an array containing the tensor contraction whose shape consists of the non-contracted axes (dimensions) of the first array `x1`, followed by the non-contracted axes (dimensions) of the second array `x2`. The returned array must have a data type determined by {ref}`type-promotion`. + (function-trace)= ### trace(x, /, *, axis1=0, axis2=1, offset=0) From 5677c24d8fbe33523c4a9c9b3aa92457df28292d Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 21:56:36 -0700 Subject: [PATCH 17/40] Add linalg solve function specification (#115) * Add solve specification * Remove statement * Update to return an array, rather than a single-element tuple * Update language * Update copy * Update description * Add note regarding broadcast compatibility * Add support for providing an ordinate vector * Update dtype requirements * Update copy * Move API to submodule --- .../linear_algebra_functions.md | 22 ++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index d868103f0..bbe6e54b1 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -322,10 +322,26 @@ The purpose of this function is to calculate the determinant more accurately whe Each returned array must have shape `shape(x)[:-2]` and a floating-point data type determined by {ref}`type-promotion`. -(function-solve)= -### solve() +(function-linalg-solve)= +### linalg.solve(x1, x2, /) -TODO +Returns the solution to the system of linear equations represented by the well-determined (i.e., full rank) linear matrix equation `AX = B`. + +#### Parameters + +- **x1**: _<array>_ + + - coefficient array `A` having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Must be of full rank (i.e., all rows or, equivalently, columns must be linearly independent). Should have a floating-point data type. + +- **x2**: _<array>_ + + - ordinate (or "dependent variable") array `B`. If `x2` has shape `(..., M)`, `x2` is equivalent to an array having shape `(..., M, 1)`, and `shape(x2)` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). If `x2` has shape `(..., M, K)`, each column `k` defines a set of ordinate values for which to compute a solution, and `shape(x2)[:-1]` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). Should have a floating-point data type. + +#### Returns + +- **out**: _<array>_ + + - an array containing the solution to the system `AX = B` for each square matrix. The returned array must have the same shape as `x2` (i.e., the array corresponding to `B`) and must have a floating-point data type determined by {ref}`type-promotion`. (function-svd)= ### svd() From 56ce7847b4eb4312db54bac893ee1e648a0a382a Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 21:57:41 -0700 Subject: [PATCH 18/40] Add SVD function specification (#114) * Add SVD spec * Update spec * Update to follow NumPy * Fix annotation * Update type * Update descriptions * Update annotation * Return a tuple only when `compute_uv` is `True` * Fix type annotation * Update copy Co-authored-by: Leo Fang * Update copy Co-authored-by: Leo Fang * Update copy Co-authored-by: Leo Fang * Update copy Co-authored-by: Leo Fang * Fix docs * Update dtype requirements * Update copy * Always return u and v singular vectors * Update return type * Add article * Move API to submodule Co-authored-by: Leo Fang --- .../linear_algebra_functions.md | 28 +++++++++++++++++-- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index bbe6e54b1..5d35d30c4 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -343,10 +343,32 @@ Returns the solution to the system of linear equations represented by the well-d - an array containing the solution to the system `AX = B` for each square matrix. The returned array must have the same shape as `x2` (i.e., the array corresponding to `B`) and must have a floating-point data type determined by {ref}`type-promotion`. -(function-svd)= -### svd() +(function-linalg-svd)= +### linalg.svd(x, /, *, full_matrices=True) -TODO +Computes the singular value decomposition `A = USV` of a matrix (or a stack of matrices) `x`. + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, N)` and whose innermost two dimensions form matrices on which to perform singular value decomposition. Should have a floating-point data type. + +- **full_matrices**: _bool_ + + - If `True`, compute full-sized `u` and `v`, such that `u` has shape `(..., M, M)` and `v` has shape `(..., N, N)`. If `False`, compute on the leading `K` singular vectors, such that `u` has shape `(..., M, K)` and `v` has shape `(..., K, N)` and where `K = min(M, N)`. Default: `True`. + +#### Returns + +- **out**: _Union\[ <array>, Tuple\[ <array>, ... ] ]_ + + - a namedtuple `(u, s, v)` whose + + - first element must be an array whose shape depends on the value of `full_matrices` and contain unitary array(s) (i.e., the left singular vectors). The left singular vectors must be stored as columns. If `full_matrices` is `True`, the array must have shape `(..., M, M)`. If `full_matrices` is `False`, the array must have shape `(..., M, K)`, where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. + - second element must be an array with shape `(..., K)` that contains the vector(s) of singular values of length `K`. For each vector, the singular values must be sorted in descending order by magnitude, such that `s[..., 0]` is the largest value, `s[..., 1]` is the second largest value, et cetera. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. + - third element must be an array whose shape depends on the value of `full_matrices` and contain unitary array(s) (i.e., the right singular vectors). The right singular vectors must be stored as rows (i.e., the array is the adjoint). If `full_matrices` is `True`, the array must have shape `(..., N, N)`. If `full_matrices` is `False`, the array must have shape `(..., K, N)` where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. + + Each returned array must have the same floating-point data type as `x`. (function-tensordot)= ### tensordot(x1, x2, /, *, axes=2) From e32a6a8d53c64849591c205fa1dc78a7e160efc8 Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 21:58:29 -0700 Subject: [PATCH 19/40] Add specification for returning the least-squares solution to a linear matrix equation (linalg: lstsq) (#119) * Stub spec * Document keyword * Update spec * Add missing parenthesis * Remove duplicate words * Rename keyword argument and add support for setting tolerance to a float * Update copy * Reorder sentences * Rename keyword argument * Fix name * Update copy * Add support for providing an ordinate vector * Update dtype requirements * Update dtype requirements * Update type annotation * Update copy * Move API to submodule --- .../linear_algebra_functions.md | 31 +++++++++++++++++-- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index 5d35d30c4..c2cecf67e 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -147,10 +147,35 @@ Computes the multiplicative inverse of a square matrix (or a stack of square mat - an array containing the multiplicative inverses. The returned array must have a floating-point data type determined by {ref}`type-promotion` and must have the same shape as `x`. -(function-lstsq)= -### lstsq() +(function-linalg-lstsq)= +### linalg.lstsq(x1, x2, /, *, rtol=None) -TODO +Returns the least-squares solution to a linear matrix equation `Ax = b`. + +#### Parameters + +- **x1**: _<array>_ + + - coefficient array `A` having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. + +- **x2**: _<array>_ + + - ordinate (or "dependent variable") array `b`. If `x2` has shape `(..., M)`, `x2` is equivalent to an array having shape `(..., M, 1)`, and `shape(x2)` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). If `x2` has shape `(..., M, K)`, each column `k` defines a set of ordinate values for which to compute a solution, and `shape(x2)[:-1]` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). Should have a floating-point data type. + +- **rtol**: _Optional\[ Union\[ float, <array> ] ]_ + + - relative tolerance for small singular values. Singular values less than or equal to `rtol * largest_singular_value` are set to zero. If a `float`, the value is equivalent to a zero-dimensional array having a data type determined by {ref}`type-promotion` (as applied to `x1` and `x2`) and must be broadcast against each matrix. If an `array`, must have a floating-point data type and must be compatible with `shape(x1)[:-2]` (see {ref}`broadcasting`). If `None`, the default value is `max(M, N) * eps`, where `eps` must be the machine epsilon associated with the floating-point data type determined by {ref}`type-promotion` (as applied to `x1` and `x2`). Default: `None`. + +#### Returns + +- **out**: _Tuple\[ <array>, <array>, <array>, <array> ]_ + + - a namedtuple `(x, residuals, rank, s)` whose + + - first element must have the field name `x` and must be an array containing the least-squares solution for each `MxN` matrix in `x1`. The array containing the solutions must have shape `(N, K)` and must have a floating-point data type determined by {ref}`type-promotion`. + - second element must have the field name `residuals` and must be an array containing the sum of squares residuals (i.e., the squared Euclidean 2-norm for each column in `b - Ax`). The array containing the residuals must have shape `(K,)` and must have a floating-point data type determined by {ref}`type-promotion`. + - third element must have the field name `rank` and must be an array containing the effective rank of each `MxN` matrix. The array containing the ranks must have shape `shape(x1)[:-2]` and must have an integer data type. + - fourth element must have the field name `s` and must be an array containing the singular values for each `MxN` matrix in `x1`. The array containing the singular values must have shape `(..., min(M, N))` and must have a floating-point data type determined by {ref}`type-promotion`. (function-matmul)= ### matmul() From a33a710aa60566d16c9fb966bc59f88dbaf220f6 Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 21:59:06 -0700 Subject: [PATCH 20/40] Add specification for computing the qr factorization (#126) * Add qr specification * Update copy * Add dtype requirements * Update copy * Move API to submodule --- .../linear_algebra_functions.md | 32 +++++++++++++++++-- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index c2cecf67e..16090b054 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -313,10 +313,36 @@ Computes the (Moore-Penrose) pseudo-inverse of a matrix (or a stack of square ma - an array containing the pseudo-inverses. The returned array must have a floating-point data type determined by {ref}`type-promotion` and must have shape `(..., N, M)` (i.e., must have the same shape as `x`, except the innermost two dimensions must be transposed). -(function-qr)= -### qr() +(function-linalg-qr)= +### linalg.qr(x, /, *, mode='reduced') -TODO +Computes the qr factorization of a matrix (or a stack of matrices), where `q` is an orthonormal matrix (or a stack of matrices) and `r` is an upper-triangular matrix (or a stack of matrices). + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. + +- **mode**: _str_ + + - factorization mode. Should be one of the following modes: + + - `'reduced'`: compute only the leading `K` columns of `q`, such that `q` and `r` have dimensions `(..., M, K)` and `(..., K, N)`, respectively, and where `K = min(M, N)`. + - `'complete'`: compute `q` and `r` with dimensions `(..., M, M)` and `(..., M, N)`, respectively. + + Default: `'reduced'`. + +#### Returns + +- **out**: _Tuple\[ <array>, <array> ]_ + + - a namedtuple `(q, r)` whose + + - first element must be an array whose shape depends on the value of `mode` and contain orthonormal matrices. If `mode` is `'complete'`, the array must have shape `(..., M, M)`. If `mode` is `'reduced'`, the array must have shape `(..., M, K)`, where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same size as those of the input `x`. + - second element must be an array whose shape depends on the value of `mode` and contain upper-triangular matrices. If `mode` is `'complete'`, the array must have shape `(..., M, M)`. If `mode` is `'reduced'`, the array must have shape `(..., K, N)`, where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same size as those of the input `x`. + + Each returned array must have a floating-point data type determined by {ref}`type-promotion`. (function-linalg-slogdet)= ### linalg.slogdet(x, /) From 3add230eabef6f4eb2e1215b4811ffeeaf33a439 Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 22:01:22 -0700 Subject: [PATCH 21/40] Add specification for computing singular values using singular value decomposition (linalg: svdvals) (#160) * Add svdvals specification * Add article * Move API to submodule --- .../linear_algebra_functions.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index 16090b054..d4eb468df 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -454,6 +454,23 @@ Returns a tensor contraction of `x1` and `x2` over specific axes. - an array containing the tensor contraction whose shape consists of the non-contracted axes (dimensions) of the first array `x1`, followed by the non-contracted axes (dimensions) of the second array `x2`. The returned array must have a data type determined by {ref}`type-promotion`. +(function-linalg-svdvals)= +### linalg.svdvals(x, /) + +Computes the singular values of a matrix (or a stack of matrices) `x`. + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, N)` and whose innermost two dimensions form matrices on which to perform singular value decomposition. Should have a floating-point data type. + +#### Returns + +- **out**: _Union\[ <array>, Tuple\[ <array>, ... ] ]_ + + - an array with shape `(..., K)` that contains the vector(s) of singular values of length `K`. For each vector, the singular values must be sorted in descending order by magnitude, such that `s[..., 0]` is the largest value, `s[..., 1]` is the second largest value, et cetera. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. The returned array must have the same floating-point data type as `x`. + (function-trace)= ### trace(x, /, *, axis1=0, axis2=1, offset=0) From fe3b410feb297b3c4f254c108325d92ff94d62f3 Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 22:02:08 -0700 Subject: [PATCH 22/40] Add specification for computing the eigenvalues and eigenvectors of a real symmetric matrix (linalg: eigh) (#161) * Add specification for eigh * Move API to submodule --- .../linear_algebra_functions.md | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index d4eb468df..e43ea6efe 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -120,6 +120,39 @@ Returns the specified diagonals. If `x` has more than two dimensions, then the a TODO +(function-linalg-eigh)= +### linalg.eigh(x, /, *, upper=False) + +Returns the eigenvalues and eigenvectors of a symmetric matrix (or a stack of symmetric matrices) `x`. + + + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Must have a floating-point data type. + +- **upper**: _bool_ + + - If `True`, use the upper-triangular part to compute the eigenvalues and eigenvectors. If `False`, use the lower-triangular part to compute the eigenvalues and eigenvectors. Default: `False`. + +#### Returns + +- **out**: _Tuple\[ <array> ]_ + + - a namedtuple (`e`, `v`) whose + + - first element must have shape `(..., M)` and consist of computed eigenvalues. + - second element must have shape `(..., M, M)`and have the columns of the inner most matrices contain the computed eigenvectors. + + Each returned array must have the same floating-point data type as `x`. + +```{note} + +Eigenvalue sort order is left unspecified. +``` + (function-eigvalsh)= ### eigvalsh() From a1d00d550f52964bf61ffa4c12121b10e33a6bfc Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 22:06:15 -0700 Subject: [PATCH 23/40] Add specification for computing the eigenvalues of a symmetric matrix (linalg: eigvalsh) (#162) * Add eigvalsh * Fix duplicate target * Move API to submodule --- .../linear_algebra_functions.md | 29 +++++++++++++++++-- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index e43ea6efe..3ad62279d 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -153,10 +153,33 @@ Returns the eigenvalues and eigenvectors of a symmetric matrix (or a stack of sy Eigenvalue sort order is left unspecified. ``` -(function-eigvalsh)= -### eigvalsh() +(function-linalg-eigvalsh)= +### linalg.eigvalsh(x, /, *, upper=False) -TODO +Computes the eigenvalues of a symmetric matrix (or a stack of symmetric matrices) `x`. + + + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Must have a floating-point data type. + +- **upper**: _bool_ + + - If `True`, use the upper-triangular part to compute the eigenvalues. If `False`, use the lower-triangular part to compute the eigenvalues. Default: `False`. + +#### Returns + +- **out**: _<array>_ + + - an array containing the computed eigenvalues. The returned array must have shape `(..., M)` and have the same data type as `x`. + +```{note} + +Eigenvalue sort order is left unspecified. +``` (function-einsum)= ### einsum() From 7557e6929993204b36ccb3a05b28fbe4388c27fa Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 22:08:12 -0700 Subject: [PATCH 24/40] Add specification for computing the matrix product (linalg: matmul) (#134) * Add matmul specification * Document return value dtype * Document input array dtypes * Add note * Require at least one dimension * Document exceptions * Update array object signatures and fix typos * Fix merge --- spec/API_specification/array_object.md | 32 +++++++++++++-- .../linear_algebra_functions.md | 39 ++++++++++++++++++- 2 files changed, 65 insertions(+), 6 deletions(-) diff --git a/spec/API_specification/array_object.md b/spec/API_specification/array_object.md index ca29db0a2..b44ad2dbb 100644 --- a/spec/API_specification/array_object.md +++ b/spec/API_specification/array_object.md @@ -763,23 +763,47 @@ Element-wise results must equal the results returned by the equivalent element-w (method-__matmul__)= ### \_\_matmul\_\_(self, other, /) -_TODO: awaiting `matmul` functional equivalent._ +Computes the matrix product. + +```{note} + +The `matmul` function must implement the same semantics as the built-in `@` operator (see [PEP 465](https://www.python.org/dev/peps/pep-0465)). +``` #### Parameters - **self**: _<array>_ - - array instance. + - array instance. Should have a numeric data type. Must have at least one dimension. If `self` is one-dimensional having shape `(M)` and `other` has more than one dimension, `self` must be promoted to a two-dimensional array by prepending `1` to its dimensions (i.e., must have shape `(1, M)`). After matrix multiplication, the prepended dimensions in the returned array must be removed. If `self` has more than one dimension (including after vector-to-matrix promotion), `self` must be compatible with `other` (see {ref}`broadcasting`). If `self` has shape `(..., M, K)`, the innermost two dimensions form matrices on which to perform matrix multiplication. - **other**: _<array>_ - - other array. Must be compatible with `self` (see {ref}`broadcasting`). + - other array. Should have a numeric data type. Must have at least one dimension. If `other` is one-dimensional having shape `(N)` and `self` has more than one dimension, `other` must be promoted to a two-dimensional array by appending `1` to its dimensions (i.e., must have shape `(N, 1)`). After matrix multiplication, the appended dimensions in the returned array must be removed. If `other` has more than one dimension (including after vector-to-matrix promotion), `other` must be compatible with `self` (see {ref}`broadcasting`). If `other` has shape `(..., K, N)`, the innermost two dimensions form matrices on which to perform matrix multiplication. #### Returns - **out**: _<array>_ - - _TODO_ + - if both `self` and `other` are one-dimensional arrays having shape `(N)`, a zero-dimensional array containing the inner product as its only element. + - if `self` is a two-dimensional array having shape `(M, K)` and `other` is a two-dimensional array having shape `(K, N)`, a two-dimensional array containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) and having shape `(M, N)`. + - if `self` is a one-dimensional array having shape `(K)` and `other` is an array having shape `(..., K, N)`, an array having shape `(..., N)` (i.e., prepended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). + - if `self` is an array having shape `(..., M, K)` and `other` is a one-dimensional array having shape `(K)`, an array having shape `(..., M)` (i.e., appended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). + - if `self` is a two-dimensional array having shape `(M, K)` and `other` is an array having shape `(..., K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + - if `self` is an array having shape `(..., M, K)` and `other` is a two-dimensional array having shape `(K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + - if either `self` or `other` has more than two dimensions, an array having a shape determined by {ref}`broadcasting` `self` against `other` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + + The returned array must have a data type determined by {ref}`type-promotion`. + + ```{note} + + Results must equal the results returned by the equivalent function [`matmul(x1, x2)`](linear_algebra_functions.md#matmulx1-x2-). + ``` + +#### Raises + +- if either `self` or `other` is a zero-dimensional array. +- if `self` is a one-dimensional array having shape `(N)`, `other` is a one-dimensional array having shape `(M)`, and `N != M`. +- if `self` is an array having shape `(..., M, K)`, `other` is an array having shape `(..., L, N)`, and `K != L`. (method-__mod__)= ### \_\_mod\_\_(self, other, /) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index 3ad62279d..63a569983 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -234,9 +234,44 @@ Returns the least-squares solution to a linear matrix equation `Ax = b`. - fourth element must have the field name `s` and must be an array containing the singular values for each `MxN` matrix in `x1`. The array containing the singular values must have shape `(..., min(M, N))` and must have a floating-point data type determined by {ref}`type-promotion`. (function-matmul)= -### matmul() +### matmul(x1, x2, /) -TODO +Computes the matrix product. + +```{note} + +The `matmul` function must implement the same semantics as the built-in `@` operator (see [PEP 465](https://www.python.org/dev/peps/pep-0465)). +``` + +#### Parameters + +- **x1**: _<array>_ + + - first input array. Should have a numeric data type. Must have at least one dimension. If `x1` is one-dimensional having shape `(M)` and `x2` has more than one dimension, `x1` must be promoted to a two-dimensional array by prepending `1` to its dimensions (i.e., must have shape `(1, M)`). After matrix multiplication, the prepended dimensions in the returned array must be removed. If `x1` has more than one dimension (including after vector-to-matrix promotion), `x1` must be compatible with `x2` (see {ref}`broadcasting`). If `x1` has shape `(..., M, K)`, the innermost two dimensions form matrices on which to perform matrix multiplication. + +- **x2**: _<array>_ + + - second input array. Should have a numeric data type. Must have at least one dimension. If `x2` is one-dimensional having shape `(N)` and `x1` has more than one dimension, `x2` must be promoted to a two-dimensional array by appending `1` to its dimensions (i.e., must have shape `(N, 1)`). After matrix multiplication, the appended dimensions in the returned array must be removed. If `x2` has more than one dimension (including after vector-to-matrix promotion), `x2` must be compatible with `x1` (see {ref}`broadcasting`). If `x2` has shape `(..., K, N)`, the innermost two dimensions form matrices on which to perform matrix multiplication. + +#### Returns + +- **out**: _<array>_ + + - if both `x1` and `x2` are one-dimensional arrays having shape `(N)`, a zero-dimensional array containing the inner product as its only element. + - if `x1` is a two-dimensional array having shape `(M, K)` and `x2` is a two-dimensional array having shape `(K, N)`, a two-dimensional array containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) and having shape `(M, N)`. + - if `x1` is a one-dimensional array having shape `(K)` and `x2` is an array having shape `(..., K, N)`, an array having shape `(..., N)` (i.e., prepended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). + - if `x1` is an array having shape `(..., M, K)` and `x2` is a one-dimensional array having shape `(K)`, an array having shape `(..., M)` (i.e., appended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). + - if `x1` is a two-dimensional array having shape `(M, K)` and `x2` is an array having shape `(..., K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + - if `x1` is an array having shape `(..., M, K)` and `x2` is a two-dimensional array having shape `(K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + - if either `x1` or `x2` has more than two dimensions, an array having a shape determined by {ref}`broadcasting` `x1` against `x2` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + + The returned array must have a data type determined by {ref}`type-promotion`. + +#### Raises + +- if either `x1` or `x2` is a zero-dimensional array. +- if `x1` is a one-dimensional array having shape `(N)`, `x2` is a one-dimensional array having shape `(M)`, and `N != M`. +- if `x1` is an array having shape `(..., M, K)`, `x2` is an array having shape `(..., L, N)`, and `K != L`. (function-matrix_power)= ### matrix_power() From e0c0c871ffe5f15fb7a696f7c138cb7b2cde203d Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 22:10:20 -0700 Subject: [PATCH 25/40] Add specification for computing the number of non-zero singular values of a matrix (linalg: matrix_rank) (#128) * Add matrix_rank specification * Update copy * Reorder sentences * Fix missing clause * Rename keyword argument * Update copy * Update dtype requirements * Update wording * Move API to submodule --- .../linear_algebra_functions.md | 22 ++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index 63a569983..08d259e4d 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -278,10 +278,26 @@ The `matmul` function must implement the same semantics as the built-in `@` oper TODO -(function-matrix_rank)= -### matrix_rank() +(function-linalg-matrix_rank)= +### linalg.matrix_rank(x, /, *, rtol=None) -TODO +Computes the rank (i.e., number of non-zero singular values) of a matrix (or a stack of matrices). + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. + +- **rtol**: _Optional\[ Union\[ float, <array> ] ]_ + + - relative tolerance for small singular values. Singular values less than or equal to `rtol * largest_singular_value` are set to zero. If a `float`, the value is equivalent to a zero-dimensional array having a floating-point data type determined by {ref}`type-promotion` (as applied to `x`) and must be broadcast against each matrix. If an `array`, must have a floating-point data type and must be compatible with `shape(x)[:-2]` (see {ref}`broadcasting`). If `None`, the default value is `max(M, N) * eps`, where `eps` must be the machine epsilon associated with the floating-point data type determined by {ref}`type-promotion` (as applied to `x`). Default: `None`. + +#### Returns + +- **out**: _<array>_ + + - an array containing the ranks. The returned array must have a floating-point data type determined by {ref}`type-promotion` and must have shape `(...)` (i.e., must have a shape equal to `shape(x)[:-2]`). (function-norm)= ### norm(x, /, *, axis=None, keepdims=False, ord=None) From 30b039151c8b3097ca90458e33570eceff9635ab Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 22:14:07 -0700 Subject: [PATCH 26/40] Add matrix_power specification (#112) * Add matrix_power spec * Update description * Add article * Add note on possible exceptions * Add exception * Update dtype requirements * Update copy * Move API to submodule --- .../linear_algebra_functions.md | 27 ++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index 08d259e4d..129861b2c 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -273,10 +273,31 @@ The `matmul` function must implement the same semantics as the built-in `@` oper - if `x1` is a one-dimensional array having shape `(N)`, `x2` is a one-dimensional array having shape `(M)`, and `N != M`. - if `x1` is an array having shape `(..., M, K)`, `x2` is an array having shape `(..., L, N)`, and `K != L`. -(function-matrix_power)= -### matrix_power() +(function-linalg-matrix_power)= +### linalg.matrix_power(x, n, /) -TODO +Raises a square matrix (or a stack of square matrices) `x` to an integer power `n`. + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. + +- **n**: _int_ + + - integer exponent. + +#### Returns + +- **out**: _<array>_ + + - if `n` is equal to zero, an array containing the identity matrix for each square matrix. If `n` is less than zero, an array containing the inverse of each square matrix raised to the absolute value of `n`, provided that each square matrix is invertible. If `n` is greater than zero, an array containing the result of raising each square matrix to the power `n`. The returned array must have the same shape as `x` and a floating-point data type determined by {ref}`type-promotion`. + +#### Raises + +- if the innermost two dimensions of `x` are not the same size (i.e., form square matrices). +- if `n` is less than zero and a square matrix is not invertible. (function-linalg-matrix_rank)= ### linalg.matrix_rank(x, /, *, rtol=None) From d00d255089e5099bcba9a89e7c6cf7144aa2462a Mon Sep 17 00:00:00 2001 From: Leo Fang Date: Wed, 12 May 2021 01:29:39 -0400 Subject: [PATCH 27/40] Fix meshgrid formatting (#178) --- spec/API_specification/creation_functions.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/spec/API_specification/creation_functions.md b/spec/API_specification/creation_functions.md index 71491fe24..8b1bec16e 100644 --- a/spec/API_specification/creation_functions.md +++ b/spec/API_specification/creation_functions.md @@ -302,23 +302,23 @@ Returns coordinate matrices from coordinate vectors. - **indexing**: _str_ - - Cartesian 'xy' or matrix 'ij' indexing of output. If provided zero or one one-dimensional vector(s) (i.e., the zero- and one-dimensional cases, respectively), the `indexing` keyword has no effect and should be ignored. Default: `'xy'`. + - Cartesian 'xy' or matrix 'ij' indexing of output. If provided zero or one one-dimensional vector(s) (i.e., the zero- and one-dimensional cases, respectively), the `indexing` keyword has no effect and should be ignored. Default: `'xy'`. #### Returns - **out**: _List\[ <array>, ... ]_ - - list of N arrays, where `N` is the number of provided one-dimensional input arrays. Each returned array must have rank `N`. For `N` one-dimensional arrays having lengths `Ni = len(xi)`, + - list of N arrays, where `N` is the number of provided one-dimensional input arrays. Each returned array must have rank `N`. For `N` one-dimensional arrays having lengths `Ni = len(xi)`, - - if matrix indexing `ij`, then each returned array must have the shape `(N1, N2, N3, ..., Nn)`. + - if matrix indexing `ij`, then each returned array must have the shape `(N1, N2, N3, ..., Nn)`. - - if Cartesian indexing `xy`, then each returned array must have shape `(N2, N1, N3, ..., Nn)`. + - if Cartesian indexing `xy`, then each returned array must have shape `(N2, N1, N3, ..., Nn)`. - Accordingly, for the two-dimensional case with input one-dimensional arrays of length `M` and `N`, if matrix indexing `ij`, then each returned array must have shape `(M, N)`, and, if Cartesian indexing `xy`, then each returned array must have shape `(N, M)`. + Accordingly, for the two-dimensional case with input one-dimensional arrays of length `M` and `N`, if matrix indexing `ij`, then each returned array must have shape `(M, N)`, and, if Cartesian indexing `xy`, then each returned array must have shape `(N, M)`. - Similarly, for the three-dimensional case with input one-dimensional arrays of length `M`, `N`, and `P`, if matrix indexing `ij`, then each returned array must have shape `(M, N, P)`, and, if Cartesian indexing `xy`, then each returned array must have shape `(N, M, P)`. + Similarly, for the three-dimensional case with input one-dimensional arrays of length `M`, `N`, and `P`, if matrix indexing `ij`, then each returned array must have shape `(M, N, P)`, and, if Cartesian indexing `xy`, then each returned array must have shape `(N, M, P)`. - The returned arrays must have a numeric data type determined by {ref}`type-promotion`. + The returned arrays must have a numeric data type determined by {ref}`type-promotion`. (function-ones)= ### ones(shape, *, dtype=None, device=None) From 83c6be38dcaf494897e91f47f62ba2a528c0b6a3 Mon Sep 17 00:00:00 2001 From: Athan Date: Tue, 11 May 2021 23:24:44 -0700 Subject: [PATCH 28/40] Add linear algebra design principles (#149) * Add design principles * Fix grammar * Add additional principle and note regarding namespaces * Remove note --- .../linear_algebra_functions.md | 59 +++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index 129861b2c..fd92422ab 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -11,6 +11,65 @@ A conforming implementation of the array API standard must provide and support t - Unless stated otherwise, functions must adhere to the type promotion rules defined in {ref}`type-promotion`. - Unless stated otherwise, floating-point operations must adhere to IEEE 754-2019. +## Design Principles + +A principal goal of this specification is to standardize commonly implemented interfaces among array libraries. While this specification endeavors to avoid straying too far from common practice, this specification does, with due restraint, seek to address design decisions arising more from historical accident than first principles. This is especially true for linear algebra APIs, which have arisen and evolved organically over time and have often been tied to particular underlying implementations (e.g., to BLAS and LAPACK). + +Accordingly, the standardization process affords the opportunity to reduce interface complexity among linear algebra APIs by inferring and subsequently codifying common design themes, thus allowing more consistent APIs. What follows is the set of design principles governing the APIs which follow: + +1. **Batching**: if an operation is explicitly defined in terms of matrices (i.e., two-dimensional arrays), then the associated interface should support "batching" (i.e., the ability to perform the operation over a "stack" of matrices). Example operations include: + + - `inv`: computing the multiplicative inverse of a square matrix. + - `cholesky`: performing Cholesky decomposition. + - `matmul`: performing matrix multiplication. + +2. **Data types**: if an operation requires decimal operations and {ref}`type-promotion` semantics are undefined (e.g., as is the case for mixed-kind promotions), then the associated interface should be specified as being restricted to floating-point data types. While the specification uses the term "SHOULD" rather than "MUST", a conforming implementation of the array API standard should only ignore the restriction provided overly compelling reasons for doing so. Example operations which should be limited to floating-point data types include: + + - `inv`: computing the multiplicative inverse. + - `slogdet`: computing the natural logarithm of the absolute value of the determinant. + - `norm`: computing the matrix or vector norm. + + Certain operations are solely comprised of multiplications and additions. Accordingly, associated interfaces need not be restricted to floating-point data types. However, careful consideration should be given to overflow, and use of floating-point data types may be more prudent in practice. Example operations include: + + - `matmul`: performing matrix multiplication. + - `trace`: computing the sum along the diagonal. + - `cross`: computing the vector cross product. + + Lastly, certain operations may be performed independent of data type, and, thus, the associated interfaces should support all data types specified in this standard. Example operations include: + + - `transpose`: computing the transpose. + - `diagonal`: returning the diagonal. + +3. **Return values**: if an interface has more than one return value, the interface should return a namedtuple consisting of each value. + + In general, interfaces should avoid polymorphic return values (e.g., returning an array **or** a namedtuple, dependent on, e.g., an optional keyword argument). Dedicated interfaces for each return value type are preferred, as dedicated interfaces are easier to reason about at both the implementation level and user level. Example interfaces which could be combined into a single overloaded interface, but are not, include: + + - `eig`: computing both eigenvalues and eignvectors. + - `eigvals`: computing only eigenvalues. + +4. **Implementation agnosticism**: a standardized interface should eschew parameterization (including keyword arguments) biased toward particular implementations. + + Historically, at a time when all array computing happened on CPUs, BLAS and LAPACK underpinned most numerical computing libraries and environments. Naturally, language and library abstractions catered to the parameterization of those libraries, often exposing low-level implementation details verbatim in their higher-level interfaces, even if such choices would be considered poor or ill-advised by today's standards (e.g., NumPy's use of `UPLO` in `eigh`). However, the present day is considerably different. While still important, BLAS and LAPACK no longer hold a monopoly over linear algebra operations, especially given the proliferation of devices and hardware on which such operations must be performed. Accordingly, interfaces must be conservative in the parameterization they support in order to best ensure universality. Such conservatism applies even to performance optimization parameters afforded by certain hardware. + +5. **Orthogonality**: an interface should have clearly defined and delineated functionality which, ideally, has no overlap with the functionality of other interfaces in the specification. Providing multiple interfaces which can all perform the same operation creates unnecessary confusion regarding interface applicability (i.e., which interface is best at which time) and decreases readability of both library and user code. Where overlap is possible, the specification must be parsimonious in the number of interfaces, ensuring that each interface provides a unique and compelling abstraction. Examples of related interfaces which provide distinct levels of abstraction (and generality) include: + + - `vecdot`: computing the dot product of two vectors. + - `matmul`: performing matrix multiplication (including between two vectors and thus the dot product). + - `tensordot`: computing tensor contractions (generalized sum-products). + - `einsum`: expressing operations in terms of Einstein summation convention, including dot products and tensor contractions. + + The above can be contrasted with, e.g., NumPy, which provides the following interfaces for computing the dot product or related operations: + + - `dot`: dot product, matrix multiplication, and tensor contraction. + - `inner`: dot product. + - `vdot`: dot product with flattening and complex conjugation. + - `multi_dot`: chained dot product. + - `tensordot`: tensor contraction. + - `matmul`: matrix multiplication (dot product for two vectors). + - `einsum`: Einstein summation convention. + + where `dot` is overloaded based on input array dimensionality and `vdot` and `inner` exhibit a high degree of overlap with other interfaces. By consolidating interfaces and more clearly delineating behavior, this specification aims to ensure that each interface has a unique purpose and defined use case. + ## Objects in API From c2ecccc1eb49f16698d99908d5938b7abef97fad Mon Sep 17 00:00:00 2001 From: Leo Fang Date: Wed, 12 May 2021 02:27:24 -0400 Subject: [PATCH 29/40] Fix matmul formatting (#179) --- .../linear_algebra_functions.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index fd92422ab..f51843794 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -316,15 +316,15 @@ The `matmul` function must implement the same semantics as the built-in `@` oper - **out**: _<array>_ - - if both `x1` and `x2` are one-dimensional arrays having shape `(N)`, a zero-dimensional array containing the inner product as its only element. - - if `x1` is a two-dimensional array having shape `(M, K)` and `x2` is a two-dimensional array having shape `(K, N)`, a two-dimensional array containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) and having shape `(M, N)`. - - if `x1` is a one-dimensional array having shape `(K)` and `x2` is an array having shape `(..., K, N)`, an array having shape `(..., N)` (i.e., prepended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). - - if `x1` is an array having shape `(..., M, K)` and `x2` is a one-dimensional array having shape `(K)`, an array having shape `(..., M)` (i.e., appended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). - - if `x1` is a two-dimensional array having shape `(M, K)` and `x2` is an array having shape `(..., K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. - - if `x1` is an array having shape `(..., M, K)` and `x2` is a two-dimensional array having shape `(K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. - - if either `x1` or `x2` has more than two dimensions, an array having a shape determined by {ref}`broadcasting` `x1` against `x2` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. - - The returned array must have a data type determined by {ref}`type-promotion`. + - if both `x1` and `x2` are one-dimensional arrays having shape `(N)`, a zero-dimensional array containing the inner product as its only element. + - if `x1` is a two-dimensional array having shape `(M, K)` and `x2` is a two-dimensional array having shape `(K, N)`, a two-dimensional array containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) and having shape `(M, N)`. + - if `x1` is a one-dimensional array having shape `(K)` and `x2` is an array having shape `(..., K, N)`, an array having shape `(..., N)` (i.e., prepended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). + - if `x1` is an array having shape `(..., M, K)` and `x2` is a one-dimensional array having shape `(K)`, an array having shape `(..., M)` (i.e., appended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). + - if `x1` is a two-dimensional array having shape `(M, K)` and `x2` is an array having shape `(..., K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + - if `x1` is an array having shape `(..., M, K)` and `x2` is a two-dimensional array having shape `(K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + - if either `x1` or `x2` has more than two dimensions, an array having a shape determined by {ref}`broadcasting` `x1` against `x2` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + + The returned array must have a data type determined by {ref}`type-promotion`. #### Raises From 9efd2844ad6d78fe15f1a0c791a1ecdf625b9201 Mon Sep 17 00:00:00 2001 From: Leo Fang Date: Wed, 12 May 2021 15:40:32 -0400 Subject: [PATCH 30/40] Fix norm formatting (#181) --- spec/API_specification/linear_algebra_functions.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index f51843794..65b0a715a 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -411,7 +411,6 @@ Computes the matrix or vector norm of `x`. - **ord**: _Optional\[ Union\[ int, float, Literal\[ inf, -inf, 'fro', 'nuc' ] ] ]_ - order of the norm. The following mathematical norms must be supported: - | ord | matrix | vector | | ---------------- | ------------------------------- | -------------------------- | | 'fro' | 'fro' | - | @@ -422,7 +421,6 @@ Computes the matrix or vector norm of `x`. | (int,float >= 1) | - | p-norm | The following non-mathematical "norms" must be supported: - | ord | matrix | vector | | ---------------- | ------------------------------- | ------------------------------ | | 0 | - | sum(a != 0) | @@ -432,7 +430,6 @@ Computes the matrix or vector norm of `x`. | (int,float < 1) | - | sum(abs(a)\*\*ord)\*\*(1./ord) | When `ord` is `None`, the following norms must be the default norms: - | ord | matrix | vector | | ---------------- | ------------------------------- | -------------------------- | | None | 'fro' | L2-norm (Euclidean) | From 03a2b41c43149c908c45db2e6df3863270bb9fac Mon Sep 17 00:00:00 2001 From: Athan Date: Thu, 20 May 2021 05:53:51 -0700 Subject: [PATCH 31/40] Move linear algebra APIs to an extension (#182) --- .../linear_algebra_functions.md | 623 +---------------- spec/extensions/index.rst | 10 + spec/extensions/linear_algebra_functions.md | 651 ++++++++++++++++++ spec/index.rst | 1 + 4 files changed, 671 insertions(+), 614 deletions(-) create mode 100644 spec/extensions/index.rst create mode 100644 spec/extensions/linear_algebra_functions.md diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index 65b0a715a..97da8f110 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -11,287 +11,15 @@ A conforming implementation of the array API standard must provide and support t - Unless stated otherwise, functions must adhere to the type promotion rules defined in {ref}`type-promotion`. - Unless stated otherwise, floating-point operations must adhere to IEEE 754-2019. -## Design Principles - -A principal goal of this specification is to standardize commonly implemented interfaces among array libraries. While this specification endeavors to avoid straying too far from common practice, this specification does, with due restraint, seek to address design decisions arising more from historical accident than first principles. This is especially true for linear algebra APIs, which have arisen and evolved organically over time and have often been tied to particular underlying implementations (e.g., to BLAS and LAPACK). - -Accordingly, the standardization process affords the opportunity to reduce interface complexity among linear algebra APIs by inferring and subsequently codifying common design themes, thus allowing more consistent APIs. What follows is the set of design principles governing the APIs which follow: - -1. **Batching**: if an operation is explicitly defined in terms of matrices (i.e., two-dimensional arrays), then the associated interface should support "batching" (i.e., the ability to perform the operation over a "stack" of matrices). Example operations include: - - - `inv`: computing the multiplicative inverse of a square matrix. - - `cholesky`: performing Cholesky decomposition. - - `matmul`: performing matrix multiplication. - -2. **Data types**: if an operation requires decimal operations and {ref}`type-promotion` semantics are undefined (e.g., as is the case for mixed-kind promotions), then the associated interface should be specified as being restricted to floating-point data types. While the specification uses the term "SHOULD" rather than "MUST", a conforming implementation of the array API standard should only ignore the restriction provided overly compelling reasons for doing so. Example operations which should be limited to floating-point data types include: - - - `inv`: computing the multiplicative inverse. - - `slogdet`: computing the natural logarithm of the absolute value of the determinant. - - `norm`: computing the matrix or vector norm. - - Certain operations are solely comprised of multiplications and additions. Accordingly, associated interfaces need not be restricted to floating-point data types. However, careful consideration should be given to overflow, and use of floating-point data types may be more prudent in practice. Example operations include: - - - `matmul`: performing matrix multiplication. - - `trace`: computing the sum along the diagonal. - - `cross`: computing the vector cross product. - - Lastly, certain operations may be performed independent of data type, and, thus, the associated interfaces should support all data types specified in this standard. Example operations include: - - - `transpose`: computing the transpose. - - `diagonal`: returning the diagonal. - -3. **Return values**: if an interface has more than one return value, the interface should return a namedtuple consisting of each value. - - In general, interfaces should avoid polymorphic return values (e.g., returning an array **or** a namedtuple, dependent on, e.g., an optional keyword argument). Dedicated interfaces for each return value type are preferred, as dedicated interfaces are easier to reason about at both the implementation level and user level. Example interfaces which could be combined into a single overloaded interface, but are not, include: - - - `eig`: computing both eigenvalues and eignvectors. - - `eigvals`: computing only eigenvalues. - -4. **Implementation agnosticism**: a standardized interface should eschew parameterization (including keyword arguments) biased toward particular implementations. - - Historically, at a time when all array computing happened on CPUs, BLAS and LAPACK underpinned most numerical computing libraries and environments. Naturally, language and library abstractions catered to the parameterization of those libraries, often exposing low-level implementation details verbatim in their higher-level interfaces, even if such choices would be considered poor or ill-advised by today's standards (e.g., NumPy's use of `UPLO` in `eigh`). However, the present day is considerably different. While still important, BLAS and LAPACK no longer hold a monopoly over linear algebra operations, especially given the proliferation of devices and hardware on which such operations must be performed. Accordingly, interfaces must be conservative in the parameterization they support in order to best ensure universality. Such conservatism applies even to performance optimization parameters afforded by certain hardware. - -5. **Orthogonality**: an interface should have clearly defined and delineated functionality which, ideally, has no overlap with the functionality of other interfaces in the specification. Providing multiple interfaces which can all perform the same operation creates unnecessary confusion regarding interface applicability (i.e., which interface is best at which time) and decreases readability of both library and user code. Where overlap is possible, the specification must be parsimonious in the number of interfaces, ensuring that each interface provides a unique and compelling abstraction. Examples of related interfaces which provide distinct levels of abstraction (and generality) include: - - - `vecdot`: computing the dot product of two vectors. - - `matmul`: performing matrix multiplication (including between two vectors and thus the dot product). - - `tensordot`: computing tensor contractions (generalized sum-products). - - `einsum`: expressing operations in terms of Einstein summation convention, including dot products and tensor contractions. - - The above can be contrasted with, e.g., NumPy, which provides the following interfaces for computing the dot product or related operations: - - - `dot`: dot product, matrix multiplication, and tensor contraction. - - `inner`: dot product. - - `vdot`: dot product with flattening and complex conjugation. - - `multi_dot`: chained dot product. - - `tensordot`: tensor contraction. - - `matmul`: matrix multiplication (dot product for two vectors). - - `einsum`: Einstein summation convention. - - where `dot` is overloaded based on input array dimensionality and `vdot` and `inner` exhibit a high degree of overlap with other interfaces. By consolidating interfaces and more clearly delineating behavior, this specification aims to ensure that each interface has a unique purpose and defined use case. - ## Objects in API -(function-linalg-cholesky)= -### linalg.cholesky(x, /, *, upper=False) - -Returns the Cholesky decomposition of a symmetric positive-definite matrix (or a stack of symmetric positive-definite matrices) `x`. - - - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. - -- **upper**: _bool_ - - - If `True`, the result must be the upper-triangular Cholesky factor. If `False`, the result must be the lower-triangular Cholesky factor. Default: `False`. - -#### Returns - -- **out**: _<array>_ - - - an array containing the Cholesky factors for each square matrix. The returned array must have a floating-point data type determined by {ref}`type-promotion` and shape as `x`. - -(function-cross)= -### cross(x1, x2, /, *, axis=-1) - -Returns the cross product of 3-element vectors. If `x1` and `x2` are multi-dimensional arrays (i.e., both have a rank greater than `1`), then the cross-product of each pair of corresponding 3-element vectors is independently computed. - -#### Parameters - -- **x1**: _<array>_ - - - first input array. Should have a numeric data type. - -- **x2**: _<array>_ - - - second input array. Must have the same shape as `x1`. Should have a numeric data type. - -- **axis**: _int_ - - - the axis (dimension) of `x1` and `x2` containing the vectors for which to compute the cross product. If set to `-1`, the function computes the cross product for vectors defined by the last axis (dimension). Default: `-1`. - -#### Returns - -- **out**: _<array>_ - - - an array containing the cross products. The returned array must have a data type determined by {ref}`type-promotion`. - -(function-det)= -### det(x, /) - -Returns the determinant of a square matrix (or stack of square matrices) `x`. - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. - -#### Returns - -- **out**: _<array>_ - - - if `x` is a two-dimensional array, a zero-dimensional array containing the determinant; otherwise, a non-zero dimensional array containing the determinant for each square matrix. The returned array must have the same data type as `x`. - -(function-diagonal)= -### diagonal(x, /, *, axis1=0, axis2=1, offset=0) - -Returns the specified diagonals. If `x` has more than two dimensions, then the axes (dimensions) specified by `axis1` and `axis2` are used to determine the two-dimensional sub-arrays from which to return diagonals. - -#### Parameters - -- **x**: _<array>_ - - - input array. Must have at least `2` dimensions. - -- **axis1**: _int_ - - - first axis (dimension) with respect to which to take diagonal. Default: `0`. - -- **axis2**: _int_ - - - second axis (dimension) with respect to which to take diagonal. Default: `1`. - -- **offset**: _int_ - - - offset specifying the off-diagonal relative to the main diagonal. - - - `offset = 0`: the main diagonal. - - `offset > 0`: off-diagonal above the main diagonal. - - `offset < 0`: off-diagonal below the main diagonal. - - Default: `0`. - -#### Returns - -- **out**: _<array>_ - - - if `x` is a two-dimensional array, a one-dimensional array containing the diagonal; otherwise, a multi-dimensional array containing the diagonals and whose shape is determined by removing `axis1` and `axis2` and appending a dimension equal to the size of the resulting diagonals. The returned array must have the same data type as `x`. - -(function-eig)= -### eig() - -TODO - -(function-linalg-eigh)= -### linalg.eigh(x, /, *, upper=False) - -Returns the eigenvalues and eigenvectors of a symmetric matrix (or a stack of symmetric matrices) `x`. - - - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Must have a floating-point data type. - -- **upper**: _bool_ - - - If `True`, use the upper-triangular part to compute the eigenvalues and eigenvectors. If `False`, use the lower-triangular part to compute the eigenvalues and eigenvectors. Default: `False`. - -#### Returns - -- **out**: _Tuple\[ <array> ]_ - - - a namedtuple (`e`, `v`) whose - - - first element must have shape `(..., M)` and consist of computed eigenvalues. - - second element must have shape `(..., M, M)`and have the columns of the inner most matrices contain the computed eigenvectors. - - Each returned array must have the same floating-point data type as `x`. - -```{note} - -Eigenvalue sort order is left unspecified. -``` - -(function-linalg-eigvalsh)= -### linalg.eigvalsh(x, /, *, upper=False) - -Computes the eigenvalues of a symmetric matrix (or a stack of symmetric matrices) `x`. - - - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Must have a floating-point data type. - -- **upper**: _bool_ - - - If `True`, use the upper-triangular part to compute the eigenvalues. If `False`, use the lower-triangular part to compute the eigenvalues. Default: `False`. - -#### Returns - -- **out**: _<array>_ - - - an array containing the computed eigenvalues. The returned array must have shape `(..., M)` and have the same data type as `x`. - -```{note} - -Eigenvalue sort order is left unspecified. -``` - (function-einsum)= ### einsum() TODO -(function-inv)= -### inv(x, /) - -Computes the multiplicative inverse of a square matrix (or a stack of square matrices) `x`. - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. - -#### Returns - -- **out**: _<array>_ - - - an array containing the multiplicative inverses. The returned array must have a floating-point data type determined by {ref}`type-promotion` and must have the same shape as `x`. - -(function-linalg-lstsq)= -### linalg.lstsq(x1, x2, /, *, rtol=None) - -Returns the least-squares solution to a linear matrix equation `Ax = b`. - -#### Parameters - -- **x1**: _<array>_ - - - coefficient array `A` having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. - -- **x2**: _<array>_ - - - ordinate (or "dependent variable") array `b`. If `x2` has shape `(..., M)`, `x2` is equivalent to an array having shape `(..., M, 1)`, and `shape(x2)` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). If `x2` has shape `(..., M, K)`, each column `k` defines a set of ordinate values for which to compute a solution, and `shape(x2)[:-1]` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). Should have a floating-point data type. - -- **rtol**: _Optional\[ Union\[ float, <array> ] ]_ - - - relative tolerance for small singular values. Singular values less than or equal to `rtol * largest_singular_value` are set to zero. If a `float`, the value is equivalent to a zero-dimensional array having a data type determined by {ref}`type-promotion` (as applied to `x1` and `x2`) and must be broadcast against each matrix. If an `array`, must have a floating-point data type and must be compatible with `shape(x1)[:-2]` (see {ref}`broadcasting`). If `None`, the default value is `max(M, N) * eps`, where `eps` must be the machine epsilon associated with the floating-point data type determined by {ref}`type-promotion` (as applied to `x1` and `x2`). Default: `None`. - -#### Returns - -- **out**: _Tuple\[ <array>, <array>, <array>, <array> ]_ - - - a namedtuple `(x, residuals, rank, s)` whose - - - first element must have the field name `x` and must be an array containing the least-squares solution for each `MxN` matrix in `x1`. The array containing the solutions must have shape `(N, K)` and must have a floating-point data type determined by {ref}`type-promotion`. - - second element must have the field name `residuals` and must be an array containing the sum of squares residuals (i.e., the squared Euclidean 2-norm for each column in `b - Ax`). The array containing the residuals must have shape `(K,)` and must have a floating-point data type determined by {ref}`type-promotion`. - - third element must have the field name `rank` and must be an array containing the effective rank of each `MxN` matrix. The array containing the ranks must have shape `shape(x1)[:-2]` and must have an integer data type. - - fourth element must have the field name `s` and must be an array containing the singular values for each `MxN` matrix in `x1`. The array containing the singular values must have shape `(..., min(M, N))` and must have a floating-point data type determined by {ref}`type-promotion`. - (function-matmul)= ### matmul(x1, x2, /) @@ -314,17 +42,17 @@ The `matmul` function must implement the same semantics as the built-in `@` oper #### Returns -- **out**: _<array>_ +- **out**: _<array>_ - - if both `x1` and `x2` are one-dimensional arrays having shape `(N)`, a zero-dimensional array containing the inner product as its only element. - - if `x1` is a two-dimensional array having shape `(M, K)` and `x2` is a two-dimensional array having shape `(K, N)`, a two-dimensional array containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) and having shape `(M, N)`. - - if `x1` is a one-dimensional array having shape `(K)` and `x2` is an array having shape `(..., K, N)`, an array having shape `(..., N)` (i.e., prepended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). - - if `x1` is an array having shape `(..., M, K)` and `x2` is a one-dimensional array having shape `(K)`, an array having shape `(..., M)` (i.e., appended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). - - if `x1` is a two-dimensional array having shape `(M, K)` and `x2` is an array having shape `(..., K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. - - if `x1` is an array having shape `(..., M, K)` and `x2` is a two-dimensional array having shape `(K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. - - if either `x1` or `x2` has more than two dimensions, an array having a shape determined by {ref}`broadcasting` `x1` against `x2` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + - if both `x1` and `x2` are one-dimensional arrays having shape `(N)`, a zero-dimensional array containing the inner product as its only element. + - if `x1` is a two-dimensional array having shape `(M, K)` and `x2` is a two-dimensional array having shape `(K, N)`, a two-dimensional array containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) and having shape `(M, N)`. + - if `x1` is a one-dimensional array having shape `(K)` and `x2` is an array having shape `(..., K, N)`, an array having shape `(..., N)` (i.e., prepended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). + - if `x1` is an array having shape `(..., M, K)` and `x2` is a one-dimensional array having shape `(K)`, an array having shape `(..., M)` (i.e., appended dimensions during vector-to-matrix promotion must be removed) and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication). + - if `x1` is a two-dimensional array having shape `(M, K)` and `x2` is an array having shape `(..., K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + - if `x1` is an array having shape `(..., M, K)` and `x2` is a two-dimensional array having shape `(K, N)`, an array having shape `(..., M, N)` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. + - if either `x1` or `x2` has more than two dimensions, an array having a shape determined by {ref}`broadcasting` `x1` against `x2` and containing the [conventional matrix product](https://en.wikipedia.org/wiki/Matrix_multiplication) for each stacked matrix. - The returned array must have a data type determined by {ref}`type-promotion`. + The returned array must have a data type determined by {ref}`type-promotion`. #### Raises @@ -332,279 +60,6 @@ The `matmul` function must implement the same semantics as the built-in `@` oper - if `x1` is a one-dimensional array having shape `(N)`, `x2` is a one-dimensional array having shape `(M)`, and `N != M`. - if `x1` is an array having shape `(..., M, K)`, `x2` is an array having shape `(..., L, N)`, and `K != L`. -(function-linalg-matrix_power)= -### linalg.matrix_power(x, n, /) - -Raises a square matrix (or a stack of square matrices) `x` to an integer power `n`. - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. - -- **n**: _int_ - - - integer exponent. - -#### Returns - -- **out**: _<array>_ - - - if `n` is equal to zero, an array containing the identity matrix for each square matrix. If `n` is less than zero, an array containing the inverse of each square matrix raised to the absolute value of `n`, provided that each square matrix is invertible. If `n` is greater than zero, an array containing the result of raising each square matrix to the power `n`. The returned array must have the same shape as `x` and a floating-point data type determined by {ref}`type-promotion`. - -#### Raises - -- if the innermost two dimensions of `x` are not the same size (i.e., form square matrices). -- if `n` is less than zero and a square matrix is not invertible. - -(function-linalg-matrix_rank)= -### linalg.matrix_rank(x, /, *, rtol=None) - -Computes the rank (i.e., number of non-zero singular values) of a matrix (or a stack of matrices). - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. - -- **rtol**: _Optional\[ Union\[ float, <array> ] ]_ - - - relative tolerance for small singular values. Singular values less than or equal to `rtol * largest_singular_value` are set to zero. If a `float`, the value is equivalent to a zero-dimensional array having a floating-point data type determined by {ref}`type-promotion` (as applied to `x`) and must be broadcast against each matrix. If an `array`, must have a floating-point data type and must be compatible with `shape(x)[:-2]` (see {ref}`broadcasting`). If `None`, the default value is `max(M, N) * eps`, where `eps` must be the machine epsilon associated with the floating-point data type determined by {ref}`type-promotion` (as applied to `x`). Default: `None`. - -#### Returns - -- **out**: _<array>_ - - - an array containing the ranks. The returned array must have a floating-point data type determined by {ref}`type-promotion` and must have shape `(...)` (i.e., must have a shape equal to `shape(x)[:-2]`). - -(function-norm)= -### norm(x, /, *, axis=None, keepdims=False, ord=None) - -Computes the matrix or vector norm of `x`. - -#### Parameters - -- **x**: _<array>_ - - - input array. Should have a floating-point data type. - -- **axis**: _Optional\[ Union\[ int, Tuple\[ int, int ] ] ]_ - - - If an integer, `axis` specifies the axis (dimension) along which to compute vector norms. - - If a 2-tuple, `axis` specifies the axes (dimensions) defining two-dimensional matrices for which to compute matrix norms. - - If `None`, - - - if `x` is one-dimensional, the function must compute the vector norm. - - if `x` is two-dimensional, the function must compute the matrix norm. - - if `x` has more than two dimensions, the function must compute the vector norm over all array values (i.e., equivalent to computing the vector norm of a flattened array). - - Negative indices must be supported. Default: `None`. - -- **keepdims**: _bool_ - - - If `True`, the axes (dimensions) specified by `axis` must be included in the result as singleton dimensions, and, accordingly, the result must be compatible with the input array (see {ref}`broadcasting`). Otherwise, if `False`, the axes (dimensions) specified by `axis` must not be included in the result. Default: `False`. - -- **ord**: _Optional\[ Union\[ int, float, Literal\[ inf, -inf, 'fro', 'nuc' ] ] ]_ - - - order of the norm. The following mathematical norms must be supported: - | ord | matrix | vector | - | ---------------- | ------------------------------- | -------------------------- | - | 'fro' | 'fro' | - | - | 'nuc' | 'nuc' | - | - | 1 | max(sum(abs(x), axis=0)) | L1-norm (Manhattan) | - | 2 | largest singular value | L2-norm (Euclidean) | - | inf | max(sum(abs(x), axis=1)) | infinity norm | - | (int,float >= 1) | - | p-norm | - - The following non-mathematical "norms" must be supported: - | ord | matrix | vector | - | ---------------- | ------------------------------- | ------------------------------ | - | 0 | - | sum(a != 0) | - | -1 | min(sum(abs(x), axis=0)) | 1./sum(1./abs(a)) | - | -2 | smallest singular value | 1./sqrt(sum(1./abs(a)\*\*2)) | - | -inf | min(sum(abs(x), axis=1)) | min(abs(a)) | - | (int,float < 1) | - | sum(abs(a)\*\*ord)\*\*(1./ord) | - - When `ord` is `None`, the following norms must be the default norms: - | ord | matrix | vector | - | ---------------- | ------------------------------- | -------------------------- | - | None | 'fro' | L2-norm (Euclidean) | - - where `fro` corresponds to the **Frobenius norm**, `nuc` corresponds to the **nuclear norm**, and `-` indicates that the norm is **not** supported. - - For matrices, - - - if `ord=1`, the norm corresponds to the induced matrix norm where `p=1` (i.e., the maximum absolute value column sum). - - if `ord=2`, the norm corresponds to the induced matrix norm where `p=inf` (i.e., the maximum absolute value row sum). - - if `ord=inf`, the norm corresponds to the induced matrix norm where `p=2` (i.e., the largest singular value). - - If `None`, - - - if matrix (or matrices), the function must compute the Frobenius norm. - - if vector (or vectors), the function must compute the L2-norm (Euclidean norm). - - Default: `None`. - -#### Returns - -- **out**: _<array>_ - - - an array containing the norms. If `axis` is `None`, the returned array must be a zero-dimensional array containing a vector norm. If `axis` is a scalar value (`int` or `float`), the returned array must have a rank which is one less than the rank of `x`. If `axis` is a 2-tuple, the returned array must have a rank which is two less than the rank of `x`. The returned array must have a floating-point data type determined by {ref}`type-promotion`. - -(function-outer)= -### outer(x1, x2, /) - -Computes the outer product of two vectors `x1` and `x2`. - -#### Parameters - -- **x1**: _<array>_ - - - first one-dimensional input array of size `N`. Should have a numeric data type. - -- **x2**: _<array>_ - - - second one-dimensional input array of size `M`. Should have a numeric data type. - -#### Returns - -- **out**: _<array>_ - - - a two-dimensional array containing the outer product and whose shape is `(N, M)`. The returned array must have a data type determined by {ref}`type-promotion`. - -(function-linalg-pinv)= -### linalg.pinv(x, /, *, rtol=None) - -Computes the (Moore-Penrose) pseudo-inverse of a matrix (or a stack of square matrices) `x`. - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. - -- **rtol**: _Optional\[ Union\[ float, <array> ] ]_ - - - relative tolerance for small singular values. Singular values less than or equal to `rtol * largest_singular_value` are set to zero. If a `float`, the value is equivalent to a zero-dimensional array having a floating-point data type determined by {ref}`type-promotion` (as applied to `x`) and must be broadcast against each matrix. If an `array`, must have a floating-point data type and must be compatible with `shape(x)[:-2]` (see {ref}`broadcasting`). If `None`, the default value is `max(M, N) * eps`, where `eps` must be the machine epsilon associated with the floating-point data type determined by {ref}`type-promotion` (as applied to `x`). Default: `None`. - -#### Returns - -- **out**: _<array>_ - - - an array containing the pseudo-inverses. The returned array must have a floating-point data type determined by {ref}`type-promotion` and must have shape `(..., N, M)` (i.e., must have the same shape as `x`, except the innermost two dimensions must be transposed). - -(function-linalg-qr)= -### linalg.qr(x, /, *, mode='reduced') - -Computes the qr factorization of a matrix (or a stack of matrices), where `q` is an orthonormal matrix (or a stack of matrices) and `r` is an upper-triangular matrix (or a stack of matrices). - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. - -- **mode**: _str_ - - - factorization mode. Should be one of the following modes: - - - `'reduced'`: compute only the leading `K` columns of `q`, such that `q` and `r` have dimensions `(..., M, K)` and `(..., K, N)`, respectively, and where `K = min(M, N)`. - - `'complete'`: compute `q` and `r` with dimensions `(..., M, M)` and `(..., M, N)`, respectively. - - Default: `'reduced'`. - -#### Returns - -- **out**: _Tuple\[ <array>, <array> ]_ - - - a namedtuple `(q, r)` whose - - - first element must be an array whose shape depends on the value of `mode` and contain orthonormal matrices. If `mode` is `'complete'`, the array must have shape `(..., M, M)`. If `mode` is `'reduced'`, the array must have shape `(..., M, K)`, where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same size as those of the input `x`. - - second element must be an array whose shape depends on the value of `mode` and contain upper-triangular matrices. If `mode` is `'complete'`, the array must have shape `(..., M, M)`. If `mode` is `'reduced'`, the array must have shape `(..., K, N)`, where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same size as those of the input `x`. - - Each returned array must have a floating-point data type determined by {ref}`type-promotion`. - -(function-linalg-slogdet)= -### linalg.slogdet(x, /) - -Returns the sign and the natural logarithm of the absolute value of the determinant of a square matrix (or a stack of square matrices) `x`. - -```{note} - -The purpose of this function is to calculate the determinant more accurately when the determinant is either very small or very large, as calling `det` may overflow or underflow. -``` - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. - -#### Returns - -- **out**: _Tuple\[ <array>, <array> ]_ - - - a namedtuple (`sign`, `logabsdet`) whose - - - first element must be an array containing a number representing the sign of the determinant for each square matrix. - - second element must be an array containing the determinant for each square matrix. - - For a real matrix, the sign of the determinant must be either `1`, `0`, or `-1`. If a determinant is zero, then the corresponding `sign` must be `0` and `logabsdet` must be `-infinity`. In all cases, the determinant must be equal to `sign * exp(logsabsdet)`. - - Each returned array must have shape `shape(x)[:-2]` and a floating-point data type determined by {ref}`type-promotion`. - -(function-linalg-solve)= -### linalg.solve(x1, x2, /) - -Returns the solution to the system of linear equations represented by the well-determined (i.e., full rank) linear matrix equation `AX = B`. - -#### Parameters - -- **x1**: _<array>_ - - - coefficient array `A` having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Must be of full rank (i.e., all rows or, equivalently, columns must be linearly independent). Should have a floating-point data type. - -- **x2**: _<array>_ - - - ordinate (or "dependent variable") array `B`. If `x2` has shape `(..., M)`, `x2` is equivalent to an array having shape `(..., M, 1)`, and `shape(x2)` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). If `x2` has shape `(..., M, K)`, each column `k` defines a set of ordinate values for which to compute a solution, and `shape(x2)[:-1]` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). Should have a floating-point data type. - -#### Returns - -- **out**: _<array>_ - - - an array containing the solution to the system `AX = B` for each square matrix. The returned array must have the same shape as `x2` (i.e., the array corresponding to `B`) and must have a floating-point data type determined by {ref}`type-promotion`. - -(function-linalg-svd)= -### linalg.svd(x, /, *, full_matrices=True) - -Computes the singular value decomposition `A = USV` of a matrix (or a stack of matrices) `x`. - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, N)` and whose innermost two dimensions form matrices on which to perform singular value decomposition. Should have a floating-point data type. - -- **full_matrices**: _bool_ - - - If `True`, compute full-sized `u` and `v`, such that `u` has shape `(..., M, M)` and `v` has shape `(..., N, N)`. If `False`, compute on the leading `K` singular vectors, such that `u` has shape `(..., M, K)` and `v` has shape `(..., K, N)` and where `K = min(M, N)`. Default: `True`. - -#### Returns - -- **out**: _Union\[ <array>, Tuple\[ <array>, ... ] ]_ - - - a namedtuple `(u, s, v)` whose - - - first element must be an array whose shape depends on the value of `full_matrices` and contain unitary array(s) (i.e., the left singular vectors). The left singular vectors must be stored as columns. If `full_matrices` is `True`, the array must have shape `(..., M, M)`. If `full_matrices` is `False`, the array must have shape `(..., M, K)`, where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. - - second element must be an array with shape `(..., K)` that contains the vector(s) of singular values of length `K`. For each vector, the singular values must be sorted in descending order by magnitude, such that `s[..., 0]` is the largest value, `s[..., 1]` is the second largest value, et cetera. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. - - third element must be an array whose shape depends on the value of `full_matrices` and contain unitary array(s) (i.e., the right singular vectors). The right singular vectors must be stored as rows (i.e., the array is the adjoint). If `full_matrices` is `True`, the array must have shape `(..., N, N)`. If `full_matrices` is `False`, the array must have shape `(..., K, N)` where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. - - Each returned array must have the same floating-point data type as `x`. - (function-tensordot)= ### tensordot(x1, x2, /, *, axes=2) @@ -638,66 +93,6 @@ Returns a tensor contraction of `x1` and `x2` over specific axes. - an array containing the tensor contraction whose shape consists of the non-contracted axes (dimensions) of the first array `x1`, followed by the non-contracted axes (dimensions) of the second array `x2`. The returned array must have a data type determined by {ref}`type-promotion`. -(function-linalg-svdvals)= -### linalg.svdvals(x, /) - -Computes the singular values of a matrix (or a stack of matrices) `x`. - -#### Parameters - -- **x**: _<array>_ - - - input array having shape `(..., M, N)` and whose innermost two dimensions form matrices on which to perform singular value decomposition. Should have a floating-point data type. - -#### Returns - -- **out**: _Union\[ <array>, Tuple\[ <array>, ... ] ]_ - - - an array with shape `(..., K)` that contains the vector(s) of singular values of length `K`. For each vector, the singular values must be sorted in descending order by magnitude, such that `s[..., 0]` is the largest value, `s[..., 1]` is the second largest value, et cetera. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. The returned array must have the same floating-point data type as `x`. - -(function-trace)= -### trace(x, /, *, axis1=0, axis2=1, offset=0) - -Returns the sum along the specified diagonals. If `x` has more than two dimensions, then the axes (dimensions) specified by `axis1` and `axis2` are used to determine the two-dimensional sub-arrays for which to compute the trace. - -#### Parameters - -- **x**: _<array>_ - - - input array. Must have at least `2` dimensions. Should have a numeric data type. - -- **axis1**: _int_ - - - first axis (dimension) with respect to which to compute the trace. Default: `0`. - -- **axis2**: _int_ - - - second axis (dimension) with respect to which to compute the trace. Default: `1`. - -- **offset**: _int_ - - - offset specifying the off-diagonal relative to the main diagonal. - - - `offset = 0`: the main diagonal. - - `offset > 0`: off-diagonal above the main diagonal. - - `offset < 0`: off-diagonal below the main diagonal. - - Default: `0`. - -#### Returns - -- **out**: _<array>_ - - - if `x` is a two-dimensional array, the returned array must be a zero-dimensional array containing the trace; otherwise, the returned array must be a multi-dimensional array containing the traces. - - The shape of a multi-dimensional output array is determined by removing `axis1` and `axis2` and storing the traces in the last array dimension. For example, if `x` has rank `k` and shape `(I, J, K, ..., L, M, N)` and `axis1=-2` and `axis1=-1`, then a multi-dimensional output array has rank `k-2` and shape `(I, J, K, ..., L)` where - - ```text - out[i, j, k, ..., l] = trace(a[i, j, k, ..., l, :, :]) - ``` - - The returned array must have the same data type as `x`. - (function-transpose)= ### transpose(x, /, *, axes=None) diff --git a/spec/extensions/index.rst b/spec/extensions/index.rst new file mode 100644 index 000000000..1b3b7470f --- /dev/null +++ b/spec/extensions/index.rst @@ -0,0 +1,10 @@ +.. _extensions: + +Extensions +========== + +.. toctree:: + :caption: Extensions + :maxdepth: 3 + + linear_algebra_functions diff --git a/spec/extensions/linear_algebra_functions.md b/spec/extensions/linear_algebra_functions.md new file mode 100644 index 000000000..5f9eaf0a7 --- /dev/null +++ b/spec/extensions/linear_algebra_functions.md @@ -0,0 +1,651 @@ +# Linear Algebra Functions + +> Array API specification for linear algebra functions. + +A conforming implementation of the array API standard must provide and support the following functions adhering to the following conventions. + +- Positional parameters must be [positional-only](https://www.python.org/dev/peps/pep-0570/) parameters. Positional-only parameters have no externally-usable name. When a function accepting positional-only parameters is called, positional arguments are mapped to these parameters based solely on their order. +- Optional parameters must be [keyword-only](https://www.python.org/dev/peps/pep-3102/) arguments. +- Broadcasting semantics must follow the semantics defined in {ref}`broadcasting`. +- Unless stated otherwise, functions must support the data types defined in {ref}`data-types`. +- Unless stated otherwise, functions must adhere to the type promotion rules defined in {ref}`type-promotion`. +- Unless stated otherwise, floating-point operations must adhere to IEEE 754-2019. + +## Design Principles + +A principal goal of this specification is to standardize commonly implemented interfaces among array libraries. While this specification endeavors to avoid straying too far from common practice, this specification does, with due restraint, seek to address design decisions arising more from historical accident than first principles. This is especially true for linear algebra APIs, which have arisen and evolved organically over time and have often been tied to particular underlying implementations (e.g., to BLAS and LAPACK). + +Accordingly, the standardization process affords the opportunity to reduce interface complexity among linear algebra APIs by inferring and subsequently codifying common design themes, thus allowing more consistent APIs. What follows is the set of design principles governing the APIs which follow: + +1. **Batching**: if an operation is explicitly defined in terms of matrices (i.e., two-dimensional arrays), then the associated interface should support "batching" (i.e., the ability to perform the operation over a "stack" of matrices). Example operations include: + + - `inv`: computing the multiplicative inverse of a square matrix. + - `cholesky`: performing Cholesky decomposition. + - `matmul`: performing matrix multiplication. + +2. **Data types**: if an operation requires decimal operations and {ref}`type-promotion` semantics are undefined (e.g., as is the case for mixed-kind promotions), then the associated interface should be specified as being restricted to floating-point data types. While the specification uses the term "SHOULD" rather than "MUST", a conforming implementation of the array API standard should only ignore the restriction provided overly compelling reasons for doing so. Example operations which should be limited to floating-point data types include: + + - `inv`: computing the multiplicative inverse. + - `slogdet`: computing the natural logarithm of the absolute value of the determinant. + - `norm`: computing the matrix or vector norm. + + Certain operations are solely comprised of multiplications and additions. Accordingly, associated interfaces need not be restricted to floating-point data types. However, careful consideration should be given to overflow, and use of floating-point data types may be more prudent in practice. Example operations include: + + - `matmul`: performing matrix multiplication. + - `trace`: computing the sum along the diagonal. + - `cross`: computing the vector cross product. + + Lastly, certain operations may be performed independent of data type, and, thus, the associated interfaces should support all data types specified in this standard. Example operations include: + + - `transpose`: computing the transpose. + - `diagonal`: returning the diagonal. + +3. **Return values**: if an interface has more than one return value, the interface should return a namedtuple consisting of each value. + + In general, interfaces should avoid polymorphic return values (e.g., returning an array **or** a namedtuple, dependent on, e.g., an optional keyword argument). Dedicated interfaces for each return value type are preferred, as dedicated interfaces are easier to reason about at both the implementation level and user level. Example interfaces which could be combined into a single overloaded interface, but are not, include: + + - `eig`: computing both eigenvalues and eignvectors. + - `eigvals`: computing only eigenvalues. + +4. **Implementation agnosticism**: a standardized interface should eschew parameterization (including keyword arguments) biased toward particular implementations. + + Historically, at a time when all array computing happened on CPUs, BLAS and LAPACK underpinned most numerical computing libraries and environments. Naturally, language and library abstractions catered to the parameterization of those libraries, often exposing low-level implementation details verbatim in their higher-level interfaces, even if such choices would be considered poor or ill-advised by today's standards (e.g., NumPy's use of `UPLO` in `eigh`). However, the present day is considerably different. While still important, BLAS and LAPACK no longer hold a monopoly over linear algebra operations, especially given the proliferation of devices and hardware on which such operations must be performed. Accordingly, interfaces must be conservative in the parameterization they support in order to best ensure universality. Such conservatism applies even to performance optimization parameters afforded by certain hardware. + +5. **Orthogonality**: an interface should have clearly defined and delineated functionality which, ideally, has no overlap with the functionality of other interfaces in the specification. Providing multiple interfaces which can all perform the same operation creates unnecessary confusion regarding interface applicability (i.e., which interface is best at which time) and decreases readability of both library and user code. Where overlap is possible, the specification must be parsimonious in the number of interfaces, ensuring that each interface provides a unique and compelling abstraction. Examples of related interfaces which provide distinct levels of abstraction (and generality) include: + + - `vecdot`: computing the dot product of two vectors. + - `matmul`: performing matrix multiplication (including between two vectors and thus the dot product). + - `tensordot`: computing tensor contractions (generalized sum-products). + - `einsum`: expressing operations in terms of Einstein summation convention, including dot products and tensor contractions. + + The above can be contrasted with, e.g., NumPy, which provides the following interfaces for computing the dot product or related operations: + + - `dot`: dot product, matrix multiplication, and tensor contraction. + - `inner`: dot product. + - `vdot`: dot product with flattening and complex conjugation. + - `multi_dot`: chained dot product. + - `tensordot`: tensor contraction. + - `matmul`: matrix multiplication (dot product for two vectors). + - `einsum`: Einstein summation convention. + + where `dot` is overloaded based on input array dimensionality and `vdot` and `inner` exhibit a high degree of overlap with other interfaces. By consolidating interfaces and more clearly delineating behavior, this specification aims to ensure that each interface has a unique purpose and defined use case. + +## Objects in API + + + +(function-linalg-cholesky)= +### linalg.cholesky(x, /, *, upper=False) + +Returns the Cholesky decomposition of a symmetric positive-definite matrix (or a stack of symmetric positive-definite matrices) `x`. + + + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. + +- **upper**: _bool_ + + - If `True`, the result must be the upper-triangular Cholesky factor. If `False`, the result must be the lower-triangular Cholesky factor. Default: `False`. + +#### Returns + +- **out**: _<array>_ + + - an array containing the Cholesky factors for each square matrix. The returned array must have a floating-point data type determined by {ref}`type-promotion` and shape as `x`. + +(function-linalg-cross)= +### linalg.cross(x1, x2, /, *, axis=-1) + +Returns the cross product of 3-element vectors. If `x1` and `x2` are multi-dimensional arrays (i.e., both have a rank greater than `1`), then the cross-product of each pair of corresponding 3-element vectors is independently computed. + +#### Parameters + +- **x1**: _<array>_ + + - first input array. Should have a numeric data type. + +- **x2**: _<array>_ + + - second input array. Must have the same shape as `x1`. Should have a numeric data type. + +- **axis**: _int_ + + - the axis (dimension) of `x1` and `x2` containing the vectors for which to compute the cross product. If set to `-1`, the function computes the cross product for vectors defined by the last axis (dimension). Default: `-1`. + +#### Returns + +- **out**: _<array>_ + + - an array containing the cross products. The returned array must have a data type determined by {ref}`type-promotion`. + +(function-linalg-det)= +### linalg.det(x, /) + +Returns the determinant of a square matrix (or stack of square matrices) `x`. + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. + +#### Returns + +- **out**: _<array>_ + + - if `x` is a two-dimensional array, a zero-dimensional array containing the determinant; otherwise, a non-zero dimensional array containing the determinant for each square matrix. The returned array must have the same data type as `x`. + +(function-linalg-diagonal)= +### linalg.diagonal(x, /, *, axis1=0, axis2=1, offset=0) + +Returns the specified diagonals. If `x` has more than two dimensions, then the axes (dimensions) specified by `axis1` and `axis2` are used to determine the two-dimensional sub-arrays from which to return diagonals. + +#### Parameters + +- **x**: _<array>_ + + - input array. Must have at least `2` dimensions. + +- **axis1**: _int_ + + - first axis (dimension) with respect to which to take diagonal. Default: `0`. + +- **axis2**: _int_ + + - second axis (dimension) with respect to which to take diagonal. Default: `1`. + +- **offset**: _int_ + + - offset specifying the off-diagonal relative to the main diagonal. + + - `offset = 0`: the main diagonal. + - `offset > 0`: off-diagonal above the main diagonal. + - `offset < 0`: off-diagonal below the main diagonal. + + Default: `0`. + +#### Returns + +- **out**: _<array>_ + + - if `x` is a two-dimensional array, a one-dimensional array containing the diagonal; otherwise, a multi-dimensional array containing the diagonals and whose shape is determined by removing `axis1` and `axis2` and appending a dimension equal to the size of the resulting diagonals. The returned array must have the same data type as `x`. + +(function-linalg-eig)= +### linalg.eig() + +TODO + +(function-linalg-eigh)= +### linalg.eigh(x, /, *, upper=False) + +Returns the eigenvalues and eigenvectors of a symmetric matrix (or a stack of symmetric matrices) `x`. + + + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Must have a floating-point data type. + +- **upper**: _bool_ + + - If `True`, use the upper-triangular part to compute the eigenvalues and eigenvectors. If `False`, use the lower-triangular part to compute the eigenvalues and eigenvectors. Default: `False`. + +#### Returns + +- **out**: _Tuple\[ <array> ]_ + + - a namedtuple (`e`, `v`) whose + + - first element must have shape `(..., M)` and consist of computed eigenvalues. + - second element must have shape `(..., M, M)`and have the columns of the inner most matrices contain the computed eigenvectors. + + Each returned array must have the same floating-point data type as `x`. + +```{note} + +Eigenvalue sort order is left unspecified. +``` + +(function-linalg-eigvals)= +### linalg.eigvals() + +TODO + +(function-linalg-eigvalsh)= +### linalg.eigvalsh(x, /, *, upper=False) + +Computes the eigenvalues of a symmetric matrix (or a stack of symmetric matrices) `x`. + + + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Must have a floating-point data type. + +- **upper**: _bool_ + + - If `True`, use the upper-triangular part to compute the eigenvalues. If `False`, use the lower-triangular part to compute the eigenvalues. Default: `False`. + +#### Returns + +- **out**: _<array>_ + + - an array containing the computed eigenvalues. The returned array must have shape `(..., M)` and have the same data type as `x`. + +```{note} + +Eigenvalue sort order is left unspecified. +``` + +(function-linalg-einsum)= +### linalg.einsum() + +Alias for {ref}`function-einsum`. + +(function-linalg-inv)= +### linalg.inv(x, /) + +Computes the multiplicative inverse of a square matrix (or a stack of square matrices) `x`. + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. + +#### Returns + +- **out**: _<array>_ + + - an array containing the multiplicative inverses. The returned array must have a floating-point data type determined by {ref}`type-promotion` and must have the same shape as `x`. + +(function-linalg-lstsq)= +### linalg.lstsq(x1, x2, /, *, rtol=None) + +Returns the least-squares solution to a linear matrix equation `Ax = b`. + +#### Parameters + +- **x1**: _<array>_ + + - coefficient array `A` having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. + +- **x2**: _<array>_ + + - ordinate (or "dependent variable") array `b`. If `x2` has shape `(..., M)`, `x2` is equivalent to an array having shape `(..., M, 1)`, and `shape(x2)` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). If `x2` has shape `(..., M, K)`, each column `k` defines a set of ordinate values for which to compute a solution, and `shape(x2)[:-1]` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). Should have a floating-point data type. + +- **rtol**: _Optional\[ Union\[ float, <array> ] ]_ + + - relative tolerance for small singular values. Singular values less than or equal to `rtol * largest_singular_value` are set to zero. If a `float`, the value is equivalent to a zero-dimensional array having a data type determined by {ref}`type-promotion` (as applied to `x1` and `x2`) and must be broadcast against each matrix. If an `array`, must have a floating-point data type and must be compatible with `shape(x1)[:-2]` (see {ref}`broadcasting`). If `None`, the default value is `max(M, N) * eps`, where `eps` must be the machine epsilon associated with the floating-point data type determined by {ref}`type-promotion` (as applied to `x1` and `x2`). Default: `None`. + +#### Returns + +- **out**: _Tuple\[ <array>, <array>, <array>, <array> ]_ + + - a namedtuple `(x, residuals, rank, s)` whose + + - first element must have the field name `x` and must be an array containing the least-squares solution for each `MxN` matrix in `x1`. The array containing the solutions must have shape `(N, K)` and must have a floating-point data type determined by {ref}`type-promotion`. + - second element must have the field name `residuals` and must be an array containing the sum of squares residuals (i.e., the squared Euclidean 2-norm for each column in `b - Ax`). The array containing the residuals must have shape `(K,)` and must have a floating-point data type determined by {ref}`type-promotion`. + - third element must have the field name `rank` and must be an array containing the effective rank of each `MxN` matrix. The array containing the ranks must have shape `shape(x1)[:-2]` and must have an integer data type. + - fourth element must have the field name `s` and must be an array containing the singular values for each `MxN` matrix in `x1`. The array containing the singular values must have shape `(..., min(M, N))` and must have a floating-point data type determined by {ref}`type-promotion`. + +(function-linalg-matmul)= +### linalg.matmul(x1, x2, /) + +Alias for {ref}`function-matmul`. + +(function-linalg-matrix_power)= +### linalg.matrix_power(x, n, /) + +Raises a square matrix (or a stack of square matrices) `x` to an integer power `n`. + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. + +- **n**: _int_ + + - integer exponent. + +#### Returns + +- **out**: _<array>_ + + - if `n` is equal to zero, an array containing the identity matrix for each square matrix. If `n` is less than zero, an array containing the inverse of each square matrix raised to the absolute value of `n`, provided that each square matrix is invertible. If `n` is greater than zero, an array containing the result of raising each square matrix to the power `n`. The returned array must have the same shape as `x` and a floating-point data type determined by {ref}`type-promotion`. + +#### Raises + +- if the innermost two dimensions of `x` are not the same size (i.e., form square matrices). +- if `n` is less than zero and a square matrix is not invertible. + +(function-linalg-matrix_rank)= +### linalg.matrix_rank(x, /, *, rtol=None) + +Computes the rank (i.e., number of non-zero singular values) of a matrix (or a stack of matrices). + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. + +- **rtol**: _Optional\[ Union\[ float, <array> ] ]_ + + - relative tolerance for small singular values. Singular values less than or equal to `rtol * largest_singular_value` are set to zero. If a `float`, the value is equivalent to a zero-dimensional array having a floating-point data type determined by {ref}`type-promotion` (as applied to `x`) and must be broadcast against each matrix. If an `array`, must have a floating-point data type and must be compatible with `shape(x)[:-2]` (see {ref}`broadcasting`). If `None`, the default value is `max(M, N) * eps`, where `eps` must be the machine epsilon associated with the floating-point data type determined by {ref}`type-promotion` (as applied to `x`). Default: `None`. + +#### Returns + +- **out**: _<array>_ + + - an array containing the ranks. The returned array must have a floating-point data type determined by {ref}`type-promotion` and must have shape `(...)` (i.e., must have a shape equal to `shape(x)[:-2]`). + +(function-linalg-norm)= +### linalg.norm(x, /, *, axis=None, keepdims=False, ord=None) + +Computes the matrix or vector norm of `x`. + +#### Parameters + +- **x**: _<array>_ + + - input array. Should have a floating-point data type. + +- **axis**: _Optional\[ Union\[ int, Tuple\[ int, int ] ] ]_ + + - If an integer, `axis` specifies the axis (dimension) along which to compute vector norms. + + If a 2-tuple, `axis` specifies the axes (dimensions) defining two-dimensional matrices for which to compute matrix norms. + + If `None`, + + - if `x` is one-dimensional, the function must compute the vector norm. + - if `x` is two-dimensional, the function must compute the matrix norm. + - if `x` has more than two dimensions, the function must compute the vector norm over all array values (i.e., equivalent to computing the vector norm of a flattened array). + + Negative indices must be supported. Default: `None`. + +- **keepdims**: _bool_ + + - If `True`, the axes (dimensions) specified by `axis` must be included in the result as singleton dimensions, and, accordingly, the result must be compatible with the input array (see {ref}`broadcasting`). Otherwise, if `False`, the axes (dimensions) specified by `axis` must not be included in the result. Default: `False`. + +- **ord**: _Optional\[ Union\[ int, float, Literal\[ inf, -inf, 'fro', 'nuc' ] ] ]_ + + - order of the norm. The following mathematical norms must be supported: + | ord | matrix | vector | + | ---------------- | ------------------------------- | -------------------------- | + | 'fro' | 'fro' | - | + | 'nuc' | 'nuc' | - | + | 1 | max(sum(abs(x), axis=0)) | L1-norm (Manhattan) | + | 2 | largest singular value | L2-norm (Euclidean) | + | inf | max(sum(abs(x), axis=1)) | infinity norm | + | (int,float >= 1) | - | p-norm | + + The following non-mathematical "norms" must be supported: + | ord | matrix | vector | + | ---------------- | ------------------------------- | ------------------------------ | + | 0 | - | sum(a != 0) | + | -1 | min(sum(abs(x), axis=0)) | 1./sum(1./abs(a)) | + | -2 | smallest singular value | 1./sqrt(sum(1./abs(a)\*\*2)) | + | -inf | min(sum(abs(x), axis=1)) | min(abs(a)) | + | (int,float < 1) | - | sum(abs(a)\*\*ord)\*\*(1./ord) | + + When `ord` is `None`, the following norms must be the default norms: + | ord | matrix | vector | + | ---------------- | ------------------------------- | -------------------------- | + | None | 'fro' | L2-norm (Euclidean) | + + where `fro` corresponds to the **Frobenius norm**, `nuc` corresponds to the **nuclear norm**, and `-` indicates that the norm is **not** supported. + + For matrices, + + - if `ord=1`, the norm corresponds to the induced matrix norm where `p=1` (i.e., the maximum absolute value column sum). + - if `ord=2`, the norm corresponds to the induced matrix norm where `p=inf` (i.e., the maximum absolute value row sum). + - if `ord=inf`, the norm corresponds to the induced matrix norm where `p=2` (i.e., the largest singular value). + + If `None`, + + - if matrix (or matrices), the function must compute the Frobenius norm. + - if vector (or vectors), the function must compute the L2-norm (Euclidean norm). + + Default: `None`. + +#### Returns + +- **out**: _<array>_ + + - an array containing the norms. If `axis` is `None`, the returned array must be a zero-dimensional array containing a vector norm. If `axis` is a scalar value (`int` or `float`), the returned array must have a rank which is one less than the rank of `x`. If `axis` is a 2-tuple, the returned array must have a rank which is two less than the rank of `x`. The returned array must have a floating-point data type determined by {ref}`type-promotion`. + +(function-linalg-outer)= +### linalg.outer(x1, x2, /) + +Computes the outer product of two vectors `x1` and `x2`. + +#### Parameters + +- **x1**: _<array>_ + + - first one-dimensional input array of size `N`. Should have a numeric data type. + +- **x2**: _<array>_ + + - second one-dimensional input array of size `M`. Should have a numeric data type. + +#### Returns + +- **out**: _<array>_ + + - a two-dimensional array containing the outer product and whose shape is `(N, M)`. The returned array must have a data type determined by {ref}`type-promotion`. + +(function-linalg-pinv)= +### linalg.pinv(x, /, *, rtol=None) + +Computes the (Moore-Penrose) pseudo-inverse of a matrix (or a stack of square matrices) `x`. + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. + +- **rtol**: _Optional\[ Union\[ float, <array> ] ]_ + + - relative tolerance for small singular values. Singular values less than or equal to `rtol * largest_singular_value` are set to zero. If a `float`, the value is equivalent to a zero-dimensional array having a floating-point data type determined by {ref}`type-promotion` (as applied to `x`) and must be broadcast against each matrix. If an `array`, must have a floating-point data type and must be compatible with `shape(x)[:-2]` (see {ref}`broadcasting`). If `None`, the default value is `max(M, N) * eps`, where `eps` must be the machine epsilon associated with the floating-point data type determined by {ref}`type-promotion` (as applied to `x`). Default: `None`. + +#### Returns + +- **out**: _<array>_ + + - an array containing the pseudo-inverses. The returned array must have a floating-point data type determined by {ref}`type-promotion` and must have shape `(..., N, M)` (i.e., must have the same shape as `x`, except the innermost two dimensions must be transposed). + +(function-linalg-qr)= +### linalg.qr(x, /, *, mode='reduced') + +Computes the qr factorization of a matrix (or a stack of matrices), where `q` is an orthonormal matrix (or a stack of matrices) and `r` is an upper-triangular matrix (or a stack of matrices). + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, N)` and whose innermost two dimensions form `MxN` matrices. Should have a floating-point data type. + +- **mode**: _str_ + + - factorization mode. Should be one of the following modes: + + - `'reduced'`: compute only the leading `K` columns of `q`, such that `q` and `r` have dimensions `(..., M, K)` and `(..., K, N)`, respectively, and where `K = min(M, N)`. + - `'complete'`: compute `q` and `r` with dimensions `(..., M, M)` and `(..., M, N)`, respectively. + + Default: `'reduced'`. + +#### Returns + +- **out**: _Tuple\[ <array>, <array> ]_ + + - a namedtuple `(q, r)` whose + + - first element must be an array whose shape depends on the value of `mode` and contain orthonormal matrices. If `mode` is `'complete'`, the array must have shape `(..., M, M)`. If `mode` is `'reduced'`, the array must have shape `(..., M, K)`, where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same size as those of the input `x`. + - second element must be an array whose shape depends on the value of `mode` and contain upper-triangular matrices. If `mode` is `'complete'`, the array must have shape `(..., M, M)`. If `mode` is `'reduced'`, the array must have shape `(..., K, N)`, where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same size as those of the input `x`. + + Each returned array must have a floating-point data type determined by {ref}`type-promotion`. + +(function-linalg-slogdet)= +### linalg.slogdet(x, /) + +Returns the sign and the natural logarithm of the absolute value of the determinant of a square matrix (or a stack of square matrices) `x`. + +```{note} + +The purpose of this function is to calculate the determinant more accurately when the determinant is either very small or very large, as calling `det` may overflow or underflow. +``` + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Should have a floating-point data type. + +#### Returns + +- **out**: _Tuple\[ <array>, <array> ]_ + + - a namedtuple (`sign`, `logabsdet`) whose + + - first element must be an array containing a number representing the sign of the determinant for each square matrix. + - second element must be an array containing the determinant for each square matrix. + + For a real matrix, the sign of the determinant must be either `1`, `0`, or `-1`. If a determinant is zero, then the corresponding `sign` must be `0` and `logabsdet` must be `-infinity`. In all cases, the determinant must be equal to `sign * exp(logsabsdet)`. + + Each returned array must have shape `shape(x)[:-2]` and a floating-point data type determined by {ref}`type-promotion`. + +(function-linalg-solve)= +### linalg.solve(x1, x2, /) + +Returns the solution to the system of linear equations represented by the well-determined (i.e., full rank) linear matrix equation `AX = B`. + +#### Parameters + +- **x1**: _<array>_ + + - coefficient array `A` having shape `(..., M, M)` and whose innermost two dimensions form square matrices. Must be of full rank (i.e., all rows or, equivalently, columns must be linearly independent). Should have a floating-point data type. + +- **x2**: _<array>_ + + - ordinate (or "dependent variable") array `B`. If `x2` has shape `(..., M)`, `x2` is equivalent to an array having shape `(..., M, 1)`, and `shape(x2)` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). If `x2` has shape `(..., M, K)`, each column `k` defines a set of ordinate values for which to compute a solution, and `shape(x2)[:-1]` must be compatible with `shape(x1)[:-1]` (see {ref}`broadcasting`). Should have a floating-point data type. + +#### Returns + +- **out**: _<array>_ + + - an array containing the solution to the system `AX = B` for each square matrix. The returned array must have the same shape as `x2` (i.e., the array corresponding to `B`) and must have a floating-point data type determined by {ref}`type-promotion`. + +(function-linalg-svd)= +### linalg.svd(x, /, *, full_matrices=True) + +Computes the singular value decomposition `A = USV` of a matrix (or a stack of matrices) `x`. + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, N)` and whose innermost two dimensions form matrices on which to perform singular value decomposition. Should have a floating-point data type. + +- **full_matrices**: _bool_ + + - If `True`, compute full-sized `u` and `v`, such that `u` has shape `(..., M, M)` and `v` has shape `(..., N, N)`. If `False`, compute on the leading `K` singular vectors, such that `u` has shape `(..., M, K)` and `v` has shape `(..., K, N)` and where `K = min(M, N)`. Default: `True`. + +#### Returns + +- **out**: _Union\[ <array>, Tuple\[ <array>, ... ] ]_ + + - a namedtuple `(u, s, v)` whose + + - first element must be an array whose shape depends on the value of `full_matrices` and contain unitary array(s) (i.e., the left singular vectors). The left singular vectors must be stored as columns. If `full_matrices` is `True`, the array must have shape `(..., M, M)`. If `full_matrices` is `False`, the array must have shape `(..., M, K)`, where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. + - second element must be an array with shape `(..., K)` that contains the vector(s) of singular values of length `K`. For each vector, the singular values must be sorted in descending order by magnitude, such that `s[..., 0]` is the largest value, `s[..., 1]` is the second largest value, et cetera. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. + - third element must be an array whose shape depends on the value of `full_matrices` and contain unitary array(s) (i.e., the right singular vectors). The right singular vectors must be stored as rows (i.e., the array is the adjoint). If `full_matrices` is `True`, the array must have shape `(..., N, N)`. If `full_matrices` is `False`, the array must have shape `(..., K, N)` where `K = min(M, N)`. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. + + Each returned array must have the same floating-point data type as `x`. + +(function-linalg-tensordot)= +### linalg.tensordot(x1, x2, /, *, axes=2) + +Alias for {ref}`function-tensordot`. + +(function-linalg-svdvals)= +### linalg.svdvals(x, /) + +Computes the singular values of a matrix (or a stack of matrices) `x`. + +#### Parameters + +- **x**: _<array>_ + + - input array having shape `(..., M, N)` and whose innermost two dimensions form matrices on which to perform singular value decomposition. Should have a floating-point data type. + +#### Returns + +- **out**: _Union\[ <array>, Tuple\[ <array>, ... ] ]_ + + - an array with shape `(..., K)` that contains the vector(s) of singular values of length `K`. For each vector, the singular values must be sorted in descending order by magnitude, such that `s[..., 0]` is the largest value, `s[..., 1]` is the second largest value, et cetera. The first `x.ndim-2` dimensions must have the same shape as those of the input `x`. The returned array must have the same floating-point data type as `x`. + +(function-linalg-trace)= +### linalg.trace(x, /, *, axis1=0, axis2=1, offset=0) + +Returns the sum along the specified diagonals. If `x` has more than two dimensions, then the axes (dimensions) specified by `axis1` and `axis2` are used to determine the two-dimensional sub-arrays for which to compute the trace. + +#### Parameters + +- **x**: _<array>_ + + - input array. Must have at least `2` dimensions. Should have a numeric data type. + +- **axis1**: _int_ + + - first axis (dimension) with respect to which to compute the trace. Default: `0`. + +- **axis2**: _int_ + + - second axis (dimension) with respect to which to compute the trace. Default: `1`. + +- **offset**: _int_ + + - offset specifying the off-diagonal relative to the main diagonal. + + - `offset = 0`: the main diagonal. + - `offset > 0`: off-diagonal above the main diagonal. + - `offset < 0`: off-diagonal below the main diagonal. + + Default: `0`. + +#### Returns + +- **out**: _<array>_ + + - if `x` is a two-dimensional array, the returned array must be a zero-dimensional array containing the trace; otherwise, the returned array must be a multi-dimensional array containing the traces. + + The shape of a multi-dimensional output array is determined by removing `axis1` and `axis2` and storing the traces in the last array dimension. For example, if `x` has rank `k` and shape `(I, J, K, ..., L, M, N)` and `axis1=-2` and `axis1=-1`, then a multi-dimensional output array has rank `k-2` and shape `(I, J, K, ..., L)` where + + ```text + out[i, j, k, ..., l] = trace(a[i, j, k, ..., l, :, :]) + ``` + + The returned array must have the same data type as `x`. + +(function-linalg-transpose)= +### linalg.transpose(x, /, *, axes=None) + +Alias for {ref}`function-transpose`. + +(function-linalg-vecdot)= +### linalg.vecdot(x1, x2, /, *, axis=None) + +Alias for {ref}`function-vecdot`. \ No newline at end of file diff --git a/spec/index.rst b/spec/index.rst index e7a466413..706c2f5e6 100644 --- a/spec/index.rst +++ b/spec/index.rst @@ -19,6 +19,7 @@ Contents design_topics/index future_API_evolution API_specification/index + extensions/index .. toctree:: :caption: Methodology and Usage From 0429693ca26c55c9dd697279bc13cfa5169c0994 Mon Sep 17 00:00:00 2001 From: Aaron Meurer Date: Thu, 20 May 2021 18:58:14 -0500 Subject: [PATCH 32/40] Fix a typo in the spec (#184) --- spec/API_specification/linear_algebra_functions.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/spec/API_specification/linear_algebra_functions.md b/spec/API_specification/linear_algebra_functions.md index 97da8f110..3724c2c24 100644 --- a/spec/API_specification/linear_algebra_functions.md +++ b/spec/API_specification/linear_algebra_functions.md @@ -34,7 +34,7 @@ The `matmul` function must implement the same semantics as the built-in `@` oper - **x1**: _<array>_ - - first input array. Should have a numeric data type. Must have at least one dimension. If `x1` is one-dimensional having shape `(M)` and `x2` has more than one dimension, `x1` must be promoted to a two-dimensional array by prepending `1` to its dimensions (i.e., must have shape `(1, M)`). After matrix multiplication, the prepended dimensions in the returned array must be removed. If `x1` has more than one dimension (including after vector-to-matrix promotion), `x1` must be compatible with `x2` (see {ref}`broadcasting`). If `x1` has shape `(..., M, K)`, the innermost two dimensions form matrices on which to perform matrix multiplication. + - first input array. Should have a numeric data type. Must have at least one dimension. If `x1` is one-dimensional having shape `(M)` and `x2` has more than one dimension, `x1` must be promoted to a two-dimensional array by prepending `1` to its dimensions (i.e., must have shape `(1, M)`). After matrix multiplication, the prepended dimensions in the returned array must be removed. If `x1` has more than one dimension (including after vector-to-matrix promotion), `x1` must be compatible with `x2` (see {ref}`broadcasting`). If `x1` has shape `(..., M, K)`, the innermost two dimensions form matrices on which to perform matrix multiplication. - **x2**: _<array>_ @@ -58,7 +58,7 @@ The `matmul` function must implement the same semantics as the built-in `@` oper - if either `x1` or `x2` is a zero-dimensional array. - if `x1` is a one-dimensional array having shape `(N)`, `x2` is a one-dimensional array having shape `(M)`, and `N != M`. -- if `x1` is an array having shape `(..., M, K)`, `x2` is an array having shape `(..., L, N)`, and `K != L`. +- if `x1` is an array having shape `(..., M, K)`, `x2` is an array having shape `(..., L, N)`, and `K != L`. (function-tensordot)= ### tensordot(x1, x2, /, *, axes=2) @@ -78,13 +78,13 @@ Returns a tensor contraction of `x1` and `x2` over specific axes. - **axes**: _Union\[ int, Tuple\[ Sequence\[ int ], Sequence\[ int ] ] ]_ - number of axes (dimensions) to contract or explicit sequences of axes (dimensions) for `x1` and `x2`, respectively. - + If `axes` is an `int` equal to `N`, then contraction must be performed over the last `N` axes of `x1` and the first `N` axes of `x2` in order. The size of each corresponding axis (dimension) must match. Must be nonnegative. - If `N` equals `0`, the result is the tensor (outer) product. - If `N` equals `1`, the result is the tensor dot product. - If `N` equals `2`, the result is the tensor double contraction (default). - + If `axes` is a tuple of two sequences `(x1_axes, x2_axes)`, the first sequence must apply to `x` and the second sequence to `x2`. Both sequences must have the same length. Each axis (dimension) `x1_axes[i]` for `x1` must have the same size as the respective axis (dimension) `x2_axes[i]` for `x2`. Each sequence must consist of unique (nonnegative) integers that specify valid axes for each respective array. #### Returns @@ -135,7 +135,7 @@ Computes the (vector) dot product of two arrays. #### Returns -- **out**: _<array;>_ +- **out**: _<array>_ - if `x1` and `x2` are both one-dimensional arrays, a zero-dimensional containing the dot product; otherwise, a non-zero-dimensional array containing the dot products and having rank `N-1`, where `N` is the rank (number of dimensions) of the shape determined according to {ref}`broadcasting`. The returned array must have a data type determined by {ref}`type-promotion`. From 2472846f1287764dd6b54f0fae70435d960e4507 Mon Sep 17 00:00:00 2001 From: Leo Fang Date: Tue, 25 May 2021 07:41:14 -0400 Subject: [PATCH 33/40] Add a requirement for the producer of a DLPack capsule(#186) --- spec/design_topics/data_interchange.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/spec/design_topics/data_interchange.md b/spec/design_topics/data_interchange.md index 0cfa0c784..a834db3ae 100644 --- a/spec/design_topics/data_interchange.md +++ b/spec/design_topics/data_interchange.md @@ -111,6 +111,8 @@ The `__dlpack__` method will produce a `PyCapsule` containing a `from_dlpack` - therefore it is consumed exactly once, and it will not be visible to users of the Python API. +The producer must set the PyCapsule name to ``"dltensor"`` so that it can +be inspected by name. The consumer must set the PyCapsule name to `"used_dltensor"`, and call the `deleter` of the `DLPackManagedTensor` when it no longer needs the data. From e3927fd2134e24158076a8bfbde38ffea99a0a0f Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Wed, 26 May 2021 19:53:45 +0200 Subject: [PATCH 34/40] Improve documentation for `stream` argument to `__dlpack__` (#185) Closes gh-183 (SYCL related, don't require an integer for `stream`) The need to document the ownership of `stream` came up in https://github.com/pytorch/pytorch/pull/57781. --- spec/API_specification/array_object.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/spec/API_specification/array_object.md b/spec/API_specification/array_object.md index b44ad2dbb..0f2c3deca 100644 --- a/spec/API_specification/array_object.md +++ b/spec/API_specification/array_object.md @@ -432,9 +432,15 @@ Exports the array for consumption by {ref}`function-from_dlpack` as a DLPack cap - array instance. -- **stream**: _Optional\[ int ]_ +- **stream**: _Optional\[ Union\[ int, Any ]]_ - - a Python integer representing a pointer to a stream. `stream` is provided by the consumer to the producer to instruct the producer to ensure that operations can safely be performed on the array. The pointer must be a positive integer or `-1`. If `stream` is `-1`, the value may be used by the consumer to signal "producer must not perform any synchronization". Device-specific notes: + - for CUDA and ROCm, a Python integer representing a pointer to a stream, on devices that support streams. `stream` is provided by the consumer to the producer to instruct the producer to ensure that operations can safely be performed on the array (e.g., by inserting a dependency between streams via "wait for event"). The pointer must be a positive integer or `-1`. If `stream` is `-1`, the value may be used by the consumer to signal "producer must not perform any synchronization". The ownership of the stream stays with the consumer. + + On CPU and other device types without streams, only `None` is accepted. + + For other device types which do have a stream, queue or similar synchronization mechanism, the most appropriate type to use for `stream` is not yet determined. E.g., for SYCL one may want to use an object containing an in-order `cl::sycl::queue`. This is allowed when libraries agree on such a convention, and may be standardized in a future version of this API standard. + + Device-specific notes: :::{admonition} CUDA - `None`: producer must assume the legacy default stream (default). From 192b4d714ac7aa0e1f7311cef8e216e982e9b9a0 Mon Sep 17 00:00:00 2001 From: Athan Date: Mon, 31 May 2021 00:47:57 -0700 Subject: [PATCH 35/40] Clarify that constants are Python scalars (#169) --- spec/API_specification/constants.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/spec/API_specification/constants.md b/spec/API_specification/constants.md index 4a9cd040f..78cdc8de7 100644 --- a/spec/API_specification/constants.md +++ b/spec/API_specification/constants.md @@ -2,7 +2,9 @@ > Array API specification for constants. -A conforming implementation of the array API standard must provide and support the following constants. +A conforming implementation of the array API standard must provide and support the following constants adhering to the following conventions. + +- Each constant must have a Python floating-point data type (i.e., `float`) and be provided as a Python scalar value. @@ -11,7 +13,7 @@ A conforming implementation of the array API standard must provide and support t (constant-e)= ### e -Euler's constant. +IEEE 754 floating-point representation of Euler's constant. ```text e = 2.71828182845904523536028747135266249775724709369995... @@ -30,7 +32,7 @@ IEEE 754 floating-point representation of Not a Number (`NaN`). (constant-pi)= ### pi -The mathematical constant `π`. +IEEE 754 floating-point representation of the mathematical constant `π`. ```text pi = 3.1415926535897932384626433... From 63c592eadb63e2ca42f842fdd51f786ef1a1ff06 Mon Sep 17 00:00:00 2001 From: Tom White Date: Fri, 4 Jun 2021 18:59:36 +0100 Subject: [PATCH 36/40] Fix typo in definition of __lshift__ (#190) --- spec/API_specification/array_object.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/API_specification/array_object.md b/spec/API_specification/array_object.md index 0f2c3deca..fb711feda 100644 --- a/spec/API_specification/array_object.md +++ b/spec/API_specification/array_object.md @@ -737,7 +737,7 @@ Evaluates `self_i << other_i` for each element of an array instance with the res ```{note} -Element-wise results must equal the results returned by the equivalent element-wise function [`less_equal(x1, x2)`](elementwise_functions.md#bitwise_left_shiftx1-x2-). +Element-wise results must equal the results returned by the equivalent element-wise function [`bitwise_left_shift(x1, x2)`](elementwise_functions.md#bitwise_left_shiftx1-x2-). ``` (method-__lt__)= From 0a61b9437e5fba425b989ff94a848eaf78848691 Mon Sep 17 00:00:00 2001 From: Athan Reines Date: Mon, 7 Jun 2021 10:51:48 -0700 Subject: [PATCH 37/40] Update notes --- spec/API_specification/searching_functions.md | 2 +- spec/API_specification/set_functions.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/API_specification/searching_functions.md b/spec/API_specification/searching_functions.md index b028ee5f8..c6a1c981b 100644 --- a/spec/API_specification/searching_functions.md +++ b/spec/API_specification/searching_functions.md @@ -72,7 +72,7 @@ Returns the indices of the minimum values along a specified axis. When the minim :::{admonition} Data-dependent output shape :class: important -The shape of the output array for this function depends on the data values in the input array, hence it can be difficult to implement for libraries that build computation graphs for arrays without knowing their values. See {ref}`indexing` section for more details. +The shape of the output array for this function depends on the data values in the input array; hence, libraries that build array computation graphs (e.g., JAX, Dask, etc.) may find this function difficult to implement without knowing array values. Accordingly, such libraries may choose to omit this function. See {ref}`indexing` section for more details. ::: Returns the indices of the array elements which are non-zero. diff --git a/spec/API_specification/set_functions.md b/spec/API_specification/set_functions.md index c3d4aef07..635ceb123 100644 --- a/spec/API_specification/set_functions.md +++ b/spec/API_specification/set_functions.md @@ -18,7 +18,7 @@ A conforming implementation of the array API standard must provide and support t :::{admonition} Data-dependent output shape :class: important -The shape of the output array for this function depends on the data values in the input array, hence it can be difficult to implement for libraries that build computation graphs for arrays without knowing their values. See {ref}`indexing` section for more details. +The shapes of one or more of output arrays for this function depend on the data values in the input array; hence, libraries that build array computation graphs (e.g., JAX, Dask, etc.) may find this function difficult to implement without knowing array values. Accordingly, such libraries may choose to omit this function. See {ref}`indexing` section for more details. ::: Returns the unique elements of an input array `x`. From c9541aa73f028d36a5592a9288e5cd1ef63fdd61 Mon Sep 17 00:00:00 2001 From: Athan Reines Date: Mon, 7 Jun 2021 10:51:55 -0700 Subject: [PATCH 38/40] Add document stub --- spec/design_topics/data_dependent_output_shapes.md | 4 ++++ spec/design_topics/index.rst | 1 + 2 files changed, 5 insertions(+) create mode 100644 spec/design_topics/data_dependent_output_shapes.md diff --git a/spec/design_topics/data_dependent_output_shapes.md b/spec/design_topics/data_dependent_output_shapes.md new file mode 100644 index 000000000..11b6828e3 --- /dev/null +++ b/spec/design_topics/data_dependent_output_shapes.md @@ -0,0 +1,4 @@ +(data-dependent-output-shapes)= + +# Data-dependent output shapes + diff --git a/spec/design_topics/index.rst b/spec/design_topics/index.rst index 6e41899ca..2729cdbe4 100644 --- a/spec/design_topics/index.rst +++ b/spec/design_topics/index.rst @@ -6,6 +6,7 @@ Design topics & constraints :maxdepth: 1 copies_views_and_mutation + data_dependent_output_shapes data_interchange device_support static_typing From b1fd96ec73486c296f947e3adf93b21794d408e5 Mon Sep 17 00:00:00 2001 From: Athan Reines Date: Mon, 7 Jun 2021 11:03:14 -0700 Subject: [PATCH 39/40] Add admonition for boolean array indexing and update reference --- spec/API_specification/indexing.md | 6 ++++++ spec/API_specification/searching_functions.md | 2 +- spec/API_specification/set_functions.md | 2 +- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/spec/API_specification/indexing.md b/spec/API_specification/indexing.md index 57689d489..98db203a3 100644 --- a/spec/API_specification/indexing.md +++ b/spec/API_specification/indexing.md @@ -160,6 +160,12 @@ _Rationale: this is consistent with bounds-checking for single-axis indexing. An ## Boolean Array Indexing +:::{admonition} Data-dependent output shape +:class: important + +For common boolean array use cases (e.g., using a dynamically-sized boolean array mask to filter the values of another array), the shape of the output array is data-dependent; hence, libraries that build array computation graphs (e.g., JAX, Dask, etc.) may find boolean array indexing difficult to implement. Accordingly, such libraries may choose to omit boolean array indexing. See {ref}`data-dependent-output-shapes` section for more details. +::: + An array must support indexing where the **sole index** is an `M`-dimensional boolean array `B` with shape `S1 = (s1, ..., sM)` according to the following rules. Let `A` be an `N`-dimensional array with shape `S2 = (s1, ..., sM, ..., sN)`. - If `N >= M`, then `A[B]` must replace the first `M` dimensions of `A` with a single dimension having a size equal to the number of `True` elements in `B`. The values in the resulting array must be in row-major (C-style order); this is equivalent to `A[nonzero(B)]`. diff --git a/spec/API_specification/searching_functions.md b/spec/API_specification/searching_functions.md index c6a1c981b..a521f98aa 100644 --- a/spec/API_specification/searching_functions.md +++ b/spec/API_specification/searching_functions.md @@ -72,7 +72,7 @@ Returns the indices of the minimum values along a specified axis. When the minim :::{admonition} Data-dependent output shape :class: important -The shape of the output array for this function depends on the data values in the input array; hence, libraries that build array computation graphs (e.g., JAX, Dask, etc.) may find this function difficult to implement without knowing array values. Accordingly, such libraries may choose to omit this function. See {ref}`indexing` section for more details. +The shape of the output array for this function depends on the data values in the input array; hence, libraries that build array computation graphs (e.g., JAX, Dask, etc.) may find this function difficult to implement without knowing array values. Accordingly, such libraries may choose to omit this function. See {ref}`data-dependent-output-shapes` section for more details. ::: Returns the indices of the array elements which are non-zero. diff --git a/spec/API_specification/set_functions.md b/spec/API_specification/set_functions.md index 635ceb123..e6b949cb7 100644 --- a/spec/API_specification/set_functions.md +++ b/spec/API_specification/set_functions.md @@ -18,7 +18,7 @@ A conforming implementation of the array API standard must provide and support t :::{admonition} Data-dependent output shape :class: important -The shapes of one or more of output arrays for this function depend on the data values in the input array; hence, libraries that build array computation graphs (e.g., JAX, Dask, etc.) may find this function difficult to implement without knowing array values. Accordingly, such libraries may choose to omit this function. See {ref}`indexing` section for more details. +The shapes of one or more of output arrays for this function depend on the data values in the input array; hence, libraries that build array computation graphs (e.g., JAX, Dask, etc.) may find this function difficult to implement without knowing array values. Accordingly, such libraries may choose to omit this function. See {ref}`data-dependent-output-shapes` section for more details. ::: Returns the unique elements of an input array `x`. From 9f336ae3b559b2062850aaa688b13ad44709d26b Mon Sep 17 00:00:00 2001 From: Athan Reines Date: Mon, 7 Jun 2021 11:14:23 -0700 Subject: [PATCH 40/40] Add explainer on data-dependent output shapes --- spec/design_topics/data_dependent_output_shapes.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/spec/design_topics/data_dependent_output_shapes.md b/spec/design_topics/data_dependent_output_shapes.md index 11b6828e3..b110d7880 100644 --- a/spec/design_topics/data_dependent_output_shapes.md +++ b/spec/design_topics/data_dependent_output_shapes.md @@ -2,3 +2,14 @@ # Data-dependent output shapes +Array libraries which build array computation graphs commonly employ static memory allocation techniques for performance optimization. In order to efficiently perform static memory allocation, such libraries must be able to infer array sizes during ahead-of-time (AOT) compilation. Functions and operations which are value-dependent present difficulties for such libraries, as array sizes cannot be inferred AOT without also knowing the contents of the respective arrays. + +While value-dependent functions and operations are not impossible for array computation graph libraries to implement, this specification does not want to impose and undue burden on such libraries and permits omission of value-dependent operations. All other array libraries are expected, however, to implement the value-dependent operations included in this specification in order to be array specification compliant. + +Value-dependent operations are demarcated in this specification using an admonition similar to the following: + +:::{admonition} Data-dependent output shape +:class: important + +The shape of the output array for this function/operation depends on the data values in the input array; hence, libraries that build array computation graphs (e.g., JAX, Dask, etc.) may find this function/operation difficult to implement without knowing array values. Accordingly, such libraries may choose to omit this function. See {ref}`data-dependent-output-shapes` section for more details. +::: \ No newline at end of file