diff options
Diffstat (limited to 'doc/source/reference')
46 files changed, 765 insertions, 703 deletions
diff --git a/doc/source/reference/array_api.rst b/doc/source/reference/array_api.rst index 01c8787eb..6982ef310 100644 --- a/doc/source/reference/array_api.rst +++ b/doc/source/reference/array_api.rst @@ -152,8 +152,9 @@ Keyword Argument Renames The following functions have keyword arguments that have been renamed. The functionality of the keyword argument is identical unless otherwise stated. -Each new keyword argument is not already present on the given function in -``numpy``, so the changes are **compatible**. +Renamed keyword arguments with the same semantic definition may be considered +either **compatible** or **breaking**, depending on how the change is +implemented. Note, this page does not list function keyword arguments that are in the main ``numpy`` namespace but not in the array API. Such keyword arguments are @@ -189,7 +190,11 @@ functions to include additional keyword arguments from those required. - ``correction`` - ``ddof`` - - + * - ``reshape`` + - ``shape`` + - ``newshape`` + - The argument may be passed as a positional or keyword argument for both + NumPy and the array API. .. _array_api-type-promotion-differences: @@ -228,6 +233,12 @@ independently of values or shapes. promoted. - **Breaking** - Example: ``a = np.array(1, dtype=np.int8); a += np.array(1, dtype=np.int16)``. The spec explicitly disallows this. + * - In-place operators are disallowed when the right-hand side operand + cannot broadcast to the shape of the left-hand side operand. + - **Strictness** + - This so-called "reverse broadcasting" should not be allowed. Example: + ``a = np.empty((2, 3, 4)); a += np.empty((3, 4))`` should error. See + https://github.com/numpy/numpy/issues/10404. * - ``int`` promotion for operators is only specified for integers within the bounds of the dtype. - **Strictness** diff --git a/doc/source/reference/arrays.classes.rst b/doc/source/reference/arrays.classes.rst index 2f40423ee..34da83670 100644 --- a/doc/source/reference/arrays.classes.rst +++ b/doc/source/reference/arrays.classes.rst @@ -71,10 +71,11 @@ NumPy provides several hooks that classes can customize: The method should return either the result of the operation, or :obj:`NotImplemented` if the operation requested is not implemented. - If one of the input or output arguments has a :func:`__array_ufunc__` + If one of the input, output, or ``where`` arguments has a :func:`__array_ufunc__` method, it is executed *instead* of the ufunc. If more than one of the arguments implements :func:`__array_ufunc__`, they are tried in the - order: subclasses before superclasses, inputs before outputs, otherwise + order: subclasses before superclasses, inputs before outputs, + outputs before ``where``, otherwise left to right. The first routine returning something other than :obj:`NotImplemented` determines the result. If all of the :func:`__array_ufunc__` operations return :obj:`NotImplemented`, a @@ -162,15 +163,6 @@ NumPy provides several hooks that classes can customize: .. versionadded:: 1.16 - .. note:: - - - In NumPy 1.17, the protocol is enabled by default, but can be disabled - with ``NUMPY_EXPERIMENTAL_ARRAY_FUNCTION=0``. - - In NumPy 1.16, you need to set the environment variable - ``NUMPY_EXPERIMENTAL_ARRAY_FUNCTION=1`` before importing NumPy to use - NumPy function overrides. - - Eventually, expect to ``__array_function__`` to always be enabled. - - ``func`` is an arbitrary callable exposed by NumPy's public API, which was called in the form ``func(*args, **kwargs)``. - ``types`` is a collection :py:class:`collections.abc.Collection` diff --git a/doc/source/reference/arrays.dtypes.rst b/doc/source/reference/arrays.dtypes.rst index 8606bc8f1..f4218ce86 100644 --- a/doc/source/reference/arrays.dtypes.rst +++ b/doc/source/reference/arrays.dtypes.rst @@ -188,10 +188,10 @@ Built-in Python types (all others) :class:`object_` ================ =============== - Note that ``str`` refers to either null terminated bytes or unicode strings - depending on the Python version. In code targeting both Python 2 and 3 - ``np.unicode_`` should be used as a dtype for strings. - See :ref:`Note on string types<string-dtype-note>`. + Note that ``str`` corresponds to UCS4 encoded unicode strings, while + ``string`` is an alias to ``bytes_``. The name ``np.unicode_`` is also + available as an alias to ``np.str_``, see :ref:`Note on string + types<string-dtype-note>`. .. admonition:: Example @@ -263,11 +263,11 @@ Array-protocol type strings (see :ref:`arrays.interface`) .. admonition:: Note on string types - For backward compatibility with Python 2 the ``S`` and ``a`` typestrings - remain zero-terminated bytes and `numpy.string_` continues to alias - `numpy.bytes_`. To use actual strings in Python 3 use ``U`` or `numpy.str_`. - For signed bytes that do not need zero-termination ``b`` or ``i1`` can be - used. + For backward compatibility with existing code originally written to support + Python 2, ``S`` and ``a`` typestrings are zero-terminated bytes and + `numpy.string_` continues to alias `numpy.bytes_`. For unicode strings, + use ``U``, `numpy.str_`, or `numpy.unicode_`. For signed bytes that do not + need zero-termination ``b`` or ``i1`` can be used. String with comma-separated fields A short-hand notation for specifying the format of a structured data type is diff --git a/doc/source/reference/arrays.nditer.rst b/doc/source/reference/arrays.nditer.rst index 8cabc1a06..fb1c91a07 100644 --- a/doc/source/reference/arrays.nditer.rst +++ b/doc/source/reference/arrays.nditer.rst @@ -3,7 +3,7 @@ .. _arrays.nditer: ********************* -Iterating Over Arrays +Iterating over arrays ********************* .. note:: @@ -23,7 +23,7 @@ can accelerate the inner loop in Cython. Since the Python exposure of iterator API, these ideas will also provide help working with array iteration from C or C++. -Single Array Iteration +Single array iteration ====================== The most basic task that can be done with the :class:`nditer` is to @@ -64,7 +64,7 @@ namely the order they are stored in memory, whereas the elements of `a.T.copy(order='C')` get visited in a different order because they have been put into a different memory layout. -Controlling Iteration Order +Controlling iteration order --------------------------- There are times when it is important to visit the elements of an array @@ -88,7 +88,7 @@ order='C' for C order and order='F' for Fortran order. .. _nditer-context-manager: -Modifying Array Values +Modifying array values ---------------------- By default, the :class:`nditer` treats the input operand as a read-only @@ -128,7 +128,7 @@ note that prior to 1.15, :class:`nditer` was not a context manager and did not have a `close` method. Instead it relied on the destructor to initiate the writeback of the buffer. -Using an External Loop +Using an external loop ---------------------- In all the examples so far, the elements of `a` are provided by the @@ -161,7 +161,7 @@ elements each. ... [0 3] [1 4] [2 5] -Tracking an Index or Multi-Index +Tracking an index or multi-index -------------------------------- During iteration, you may want to use the index of the current @@ -210,7 +210,7 @@ raise an exception. File "<stdin>", line 1, in <module> ValueError: Iterator flag EXTERNAL_LOOP cannot be used if an index or multi-index is being tracked -Alternative Looping and Element Access +Alternative looping and element access -------------------------------------- To make its properties more readily accessible during iteration, @@ -246,7 +246,7 @@ produce identical results to the ones in the previous section. array([[ 0, 1, 2], [-1, 0, 1]]) -Buffering the Array Elements +Buffering the array elements ---------------------------- When forcing an iteration order, we observed that the external loop @@ -274,7 +274,7 @@ is enabled. ... [0 3 1 4 2 5] -Iterating as a Specific Data Type +Iterating as a specific data type --------------------------------- There are times when it is necessary to treat an array as a different @@ -382,7 +382,7 @@ would violate the casting rule. File "<stdin>", line 2, in <module> TypeError: Iterator requested dtype could not be cast from dtype('float64') to dtype('int64'), the operand 0 dtype, according to the rule 'same_kind' -Broadcasting Array Iteration +Broadcasting array iteration ============================ NumPy has a set of rules for dealing with arrays that have differing @@ -417,7 +417,7 @@ which includes the input shapes to help diagnose the problem. ... ValueError: operands could not be broadcast together with shapes (2,) (2,3) -Iterator-Allocated Output Arrays +Iterator-allocated output arrays -------------------------------- A common case in NumPy functions is to have outputs allocated based diff --git a/doc/source/reference/arrays.scalars.rst b/doc/source/reference/arrays.scalars.rst index 30d7be9f9..4dba54d6b 100644 --- a/doc/source/reference/arrays.scalars.rst +++ b/doc/source/reference/arrays.scalars.rst @@ -293,6 +293,10 @@ elements the data type consists of.) :members: __init__ :exclude-members: __init__ +.. autoclass:: numpy.character + :members: __init__ + :exclude-members: __init__ + .. autoclass:: numpy.bytes_ :members: __init__ :exclude-members: __init__ @@ -333,8 +337,6 @@ are also provided. .. note that these are documented with ..attribute because that is what autoclass does for aliases under the hood. -.. autoclass:: numpy.bool8 - .. attribute:: int8 int16 int32 diff --git a/doc/source/reference/c-api/array.rst b/doc/source/reference/c-api/array.rst index 8772b494c..baebfdb02 100644 --- a/doc/source/reference/c-api/array.rst +++ b/doc/source/reference/c-api/array.rst @@ -423,7 +423,7 @@ From other objects :c:data:`NPY_ARRAY_FORCECAST` is present in ``flags``, this call will generate an error if the data type cannot be safely obtained from the object. If you want to use - ``NULL`` for the *dtype* and ensure the array is notswapped then + ``NULL`` for the *dtype* and ensure the array is not swapped then use :c:func:`PyArray_CheckFromAny`. A value of 0 for either of the depth parameters causes the parameter to be ignored. Any of the following array flags can be added (*e.g.* using \|) to get the @@ -548,22 +548,6 @@ From other objects :c:data:`NPY_ARRAY_F_CONTIGUOUS` \| :c:data:`NPY_ARRAY_WRITEABLE` \| :c:data:`NPY_ARRAY_ALIGNED` \| :c:data:`NPY_ARRAY_WRITEBACKIFCOPY` -.. c:function:: int PyArray_GetArrayParamsFromObject( \ - PyObject* op, PyArray_Descr* requested_dtype, npy_bool writeable, \ - PyArray_Descr** out_dtype, int* out_ndim, npy_intp* out_dims, \ - PyArrayObject** out_arr, PyObject* context) - - .. deprecated:: NumPy 1.19 - - Unless NumPy is made aware of an issue with this, this function - is scheduled for rapid removal without replacement. - - .. versionchanged:: NumPy 1.19 - - `context` is never used. Its use results in an error. - - .. versionadded:: 1.6 - .. c:function:: PyObject* PyArray_CheckFromAny( \ PyObject* op, PyArray_Descr* dtype, int min_depth, int max_depth, \ int requirements, PyObject* context) @@ -1202,17 +1186,6 @@ Converting data types return value is the enumerated typenumber that represents the data-type that *op* should have. -.. c:function:: void PyArray_ArrayType( \ - PyObject* op, PyArray_Descr* mintype, PyArray_Descr* outtype) - - This function is superseded by :c:func:`PyArray_ResultType`. - - This function works similarly to :c:func:`PyArray_ObjectType` (...) - except it handles flexible arrays. The *mintype* argument can have - an itemsize member and the *outtype* argument will have an - itemsize member at least as big but perhaps bigger depending on - the object *op*. - .. c:function:: PyArrayObject** PyArray_ConvertToCommonType( \ PyObject* op, int* n) @@ -1490,7 +1463,7 @@ of the constant names is deprecated in 1.7. :c:func:`PyArray_FromAny` and a copy had to be made of some other array (and the user asked for this flag to be set in such a situation). The base attribute then points to the "misbehaved" - array (which is set read_only). :c:func`PyArray_ResolveWritebackIfCopy` + array (which is set read_only). :c:func:`PyArray_ResolveWritebackIfCopy` will copy its contents back to the "misbehaved" array (casting if necessary) and will reset the "misbehaved" array to :c:data:`NPY_ARRAY_WRITEABLE`. If the "misbehaved" array was not @@ -2276,7 +2249,7 @@ Array Functions output array must have the correct shape, type, and be C-contiguous, or an exception is raised. -.. c:function:: PyObject* PyArray_EinsteinSum( \ +.. c:function:: PyArrayObject* PyArray_EinsteinSum( \ char* subscripts, npy_intp nop, PyArrayObject** op_in, \ PyArray_Descr* dtype, NPY_ORDER order, NPY_CASTING casting, \ PyArrayObject* out) @@ -2475,9 +2448,9 @@ As of NumPy 1.6.0, these array iterators are superseded by the new array iterator, :c:type:`NpyIter`. An array iterator is a simple way to access the elements of an -N-dimensional array quickly and efficiently. Section `2 -<#sec-array-iterator>`__ provides more description and examples of -this useful approach to looping over an array. +N-dimensional array quickly and efficiently, as seen in :ref:`the +example <iteration-example>` which provides more description +of this useful approach to looping over an array from C. .. c:function:: PyObject* PyArray_IterNew(PyObject* arr) @@ -3378,7 +3351,7 @@ Memory management .. c:function:: int PyArray_ResolveWritebackIfCopy(PyArrayObject* obj) - If ``obj.flags`` has :c:data:`NPY_ARRAY_WRITEBACKIFCOPY`, this function + If ``obj->flags`` has :c:data:`NPY_ARRAY_WRITEBACKIFCOPY`, this function clears the flags, `DECREF` s `obj->base` and makes it writeable, and sets ``obj->base`` to NULL. It then copies ``obj->data`` to `obj->base->data`, and returns the error state of @@ -3608,29 +3581,17 @@ Miscellaneous Macros Returns the reference count of any Python object. -.. c:function:: void PyArray_DiscardWritebackIfCopy(PyObject* obj) +.. c:function:: void PyArray_DiscardWritebackIfCopy(PyArrayObject* obj) - If ``obj.flags`` has :c:data:`NPY_ARRAY_WRITEBACKIFCOPY`, this function + If ``obj->flags`` has :c:data:`NPY_ARRAY_WRITEBACKIFCOPY`, this function clears the flags, `DECREF` s `obj->base` and makes it writeable, and sets ``obj->base`` to NULL. In - contrast to :c:func:`PyArray_DiscardWritebackIfCopy` it makes no attempt - to copy the data from `obj->base` This undoes + contrast to :c:func:`PyArray_ResolveWritebackIfCopy` it makes no attempt + to copy the data from `obj->base`. This undoes :c:func:`PyArray_SetWritebackIfCopyBase`. Usually this is called after an error when you are finished with ``obj``, just before ``Py_DECREF(obj)``. It may be called multiple times, or with ``NULL`` input. -.. c:function:: void PyArray_XDECREF_ERR(PyObject* obj) - - Deprecated in 1.14, use :c:func:`PyArray_DiscardWritebackIfCopy` - followed by ``Py_XDECREF`` - - DECREF's an array object which may have the - :c:data:`NPY_ARRAY_WRITEBACKIFCOPY` - flag set without causing the contents to be copied back into the - original array. Resets the :c:data:`NPY_ARRAY_WRITEABLE` flag on the base - object. This is useful for recovering from an error condition when - writeback semantics are used, but will lead to wrong results. - Enumerated Types ~~~~~~~~~~~~~~~~ diff --git a/doc/source/reference/c-api/config.rst b/doc/source/reference/c-api/config.rst index 87130699b..963a106f8 100644 --- a/doc/source/reference/c-api/config.rst +++ b/doc/source/reference/c-api/config.rst @@ -118,14 +118,3 @@ Compiler directives .. c:macro:: NPY_LIKELY .. c:macro:: NPY_UNLIKELY .. c:macro:: NPY_UNUSED - - -Interrupt Handling ------------------- - -.. c:macro:: NPY_INTERRUPT_H -.. c:macro:: NPY_SIGSETJMP -.. c:macro:: NPY_SIGLONGJMP -.. c:macro:: NPY_SIGJMP_BUF -.. c:macro:: NPY_SIGINT_ON -.. c:macro:: NPY_SIGINT_OFF diff --git a/doc/source/reference/c-api/coremath.rst b/doc/source/reference/c-api/coremath.rst index eb0990edb..2293f0ae8 100644 --- a/doc/source/reference/c-api/coremath.rst +++ b/doc/source/reference/c-api/coremath.rst @@ -55,30 +55,30 @@ Floating point classification .. c:macro:: npy_isnan(x) - This is a macro, and is equivalent to C99 isnan: works for single, double + This is an alias for C99 isnan: works for single, double and extended precision, and return a non 0 value if x is a NaN. .. c:macro:: npy_isfinite(x) - This is a macro, and is equivalent to C99 isfinite: works for single, + This is an alias for C99 isfinite: works for single, double and extended precision, and return a non 0 value if x is neither a NaN nor an infinity. .. c:macro:: npy_isinf(x) - This is a macro, and is equivalent to C99 isinf: works for single, double + This is an alias for C99 isinf: works for single, double and extended precision, and return a non 0 value if x is infinite (positive and negative). .. c:macro:: npy_signbit(x) - This is a macro, and is equivalent to C99 signbit: works for single, double + This is an alias for C99 signbit: works for single, double and extended precision, and return a non 0 value if x has the signbit set (that is the number is negative). .. c:macro:: npy_copysign(x, y) - This is a function equivalent to C99 copysign: return x with the same sign + This is an alias for C99 copysign: return x with the same sign as y. Works for any value, including inf and nan. Single and extended precisions are available with suffix f and l. @@ -141,7 +141,7 @@ Those can be useful for precise floating point comparison. .. c:function:: double npy_nextafter(double x, double y) - This is a function equivalent to C99 nextafter: return next representable + This is an alias to C99 nextafter: return next representable floating point value from x in the direction of y. Single and extended precisions are available with suffix f and l. diff --git a/doc/source/reference/c-api/data_memory.rst b/doc/source/reference/c-api/data_memory.rst index 2084ab5d0..cab587efc 100644 --- a/doc/source/reference/c-api/data_memory.rst +++ b/doc/source/reference/c-api/data_memory.rst @@ -159,3 +159,87 @@ A better technique would be to use a ``PyCapsule`` as a base object: return NULL; } ... + +Example of memory tracing with ``np.lib.tracemalloc_domain`` +------------------------------------------------------------ + +Note that since Python 3.6 (or newer), the builtin ``tracemalloc`` module can be used to +track allocations inside NumPy. NumPy places its CPU memory allocations into the +``np.lib.tracemalloc_domain`` domain. +For additional information, check: `https://docs.python.org/3/library/tracemalloc.html`. + +Here is an example on how to use ``np.lib.tracemalloc_domain``: + +.. code-block:: python + + """ + The goal of this example is to show how to trace memory + from an application that has NumPy and non-NumPy sections. + We only select the sections using NumPy related calls. + """ + + import tracemalloc + import numpy as np + + # Flag to determine if we select NumPy domain + use_np_domain = True + + nx = 300 + ny = 500 + + # Start to trace memory + tracemalloc.start() + + # Section 1 + # --------- + + # NumPy related call + a = np.zeros((nx,ny)) + + # non-NumPy related call + b = [i**2 for i in range(nx*ny)] + + snapshot1 = tracemalloc.take_snapshot() + # We filter the snapshot to only select NumPy related calls + np_domain = np.lib.tracemalloc_domain + dom_filter = tracemalloc.DomainFilter(inclusive=use_np_domain, + domain=np_domain) + snapshot1 = snapshot1.filter_traces([dom_filter]) + top_stats1 = snapshot1.statistics('traceback') + + print("================ SNAPSHOT 1 =================") + for stat in top_stats1: + print(f"{stat.count} memory blocks: {stat.size / 1024:.1f} KiB") + print(stat.traceback.format()[-1]) + + # Clear traces of memory blocks allocated by Python + # before moving to the next section. + tracemalloc.clear_traces() + + # Section 2 + #---------- + + # We are only using NumPy + c = np.sum(a*a) + + snapshot2 = tracemalloc.take_snapshot() + top_stats2 = snapshot2.statistics('traceback') + + print() + print("================ SNAPSHOT 2 =================") + for stat in top_stats2: + print(f"{stat.count} memory blocks: {stat.size / 1024:.1f} KiB") + print(stat.traceback.format()[-1]) + + tracemalloc.stop() + + print() + print("============================================") + print("\nTracing Status : ", tracemalloc.is_tracing()) + + try: + print("\nTrying to Take Snapshot After Tracing is Stopped.") + snap = tracemalloc.take_snapshot() + except Exception as e: + print("Exception : ", e) + diff --git a/doc/source/reference/c-api/deprecations.rst b/doc/source/reference/c-api/deprecations.rst index 5b1abc6f2..d805421f2 100644 --- a/doc/source/reference/c-api/deprecations.rst +++ b/doc/source/reference/c-api/deprecations.rst @@ -58,3 +58,7 @@ On compilers which support a #warning mechanism, NumPy issues a compiler warning if you do not define the symbol NPY_NO_DEPRECATED_API. This way, the fact that there are deprecations will be flagged for third-party developers who may not have read the release notes closely. + +Note that defining NPY_NO_DEPRECATED_API is not sufficient to make your +extension ABI compatible with a given NumPy version. See +:ref:`for-downstream-package-authors`. diff --git a/doc/source/reference/c-api/dtype.rst b/doc/source/reference/c-api/dtype.rst index 642f62749..bdd5a6f55 100644 --- a/doc/source/reference/c-api/dtype.rst +++ b/doc/source/reference/c-api/dtype.rst @@ -25,161 +25,161 @@ select the precision desired. Enumerated Types ---------------- -.. c:enumerator:: NPY_TYPES +.. c:enum:: NPY_TYPES -There is a list of enumerated types defined providing the basic 24 -data types plus some useful generic names. Whenever the code requires -a type number, one of these enumerated types is requested. The types -are all called ``NPY_{NAME}``: + There is a list of enumerated types defined providing the basic 24 + data types plus some useful generic names. Whenever the code requires + a type number, one of these enumerated types is requested. The types + are all called ``NPY_{NAME}``: -.. c:enumerator:: NPY_BOOL + .. c:enumerator:: NPY_BOOL - The enumeration value for the boolean type, stored as one byte. - It may only be set to the values 0 and 1. + The enumeration value for the boolean type, stored as one byte. + It may only be set to the values 0 and 1. -.. c:enumerator:: NPY_BYTE -.. c:enumerator:: NPY_INT8 + .. c:enumerator:: NPY_BYTE + .. c:enumerator:: NPY_INT8 - The enumeration value for an 8-bit/1-byte signed integer. + The enumeration value for an 8-bit/1-byte signed integer. -.. c:enumerator:: NPY_SHORT -.. c:enumerator:: NPY_INT16 + .. c:enumerator:: NPY_SHORT + .. c:enumerator:: NPY_INT16 - The enumeration value for a 16-bit/2-byte signed integer. + The enumeration value for a 16-bit/2-byte signed integer. -.. c:enumerator:: NPY_INT -.. c:enumerator:: NPY_INT32 + .. c:enumerator:: NPY_INT + .. c:enumerator:: NPY_INT32 - The enumeration value for a 32-bit/4-byte signed integer. + The enumeration value for a 32-bit/4-byte signed integer. -.. c:enumerator:: NPY_LONG + .. c:enumerator:: NPY_LONG - Equivalent to either NPY_INT or NPY_LONGLONG, depending on the - platform. + Equivalent to either NPY_INT or NPY_LONGLONG, depending on the + platform. -.. c:enumerator:: NPY_LONGLONG -.. c:enumerator:: NPY_INT64 + .. c:enumerator:: NPY_LONGLONG + .. c:enumerator:: NPY_INT64 - The enumeration value for a 64-bit/8-byte signed integer. + The enumeration value for a 64-bit/8-byte signed integer. -.. c:enumerator:: NPY_UBYTE -.. c:enumerator:: NPY_UINT8 + .. c:enumerator:: NPY_UBYTE + .. c:enumerator:: NPY_UINT8 - The enumeration value for an 8-bit/1-byte unsigned integer. + The enumeration value for an 8-bit/1-byte unsigned integer. -.. c:enumerator:: NPY_USHORT -.. c:enumerator:: NPY_UINT16 + .. c:enumerator:: NPY_USHORT + .. c:enumerator:: NPY_UINT16 - The enumeration value for a 16-bit/2-byte unsigned integer. + The enumeration value for a 16-bit/2-byte unsigned integer. -.. c:enumerator:: NPY_UINT -.. c:enumerator:: NPY_UINT32 + .. c:enumerator:: NPY_UINT + .. c:enumerator:: NPY_UINT32 - The enumeration value for a 32-bit/4-byte unsigned integer. + The enumeration value for a 32-bit/4-byte unsigned integer. -.. c:enumerator:: NPY_ULONG + .. c:enumerator:: NPY_ULONG - Equivalent to either NPY_UINT or NPY_ULONGLONG, depending on the - platform. + Equivalent to either NPY_UINT or NPY_ULONGLONG, depending on the + platform. -.. c:enumerator:: NPY_ULONGLONG -.. c:enumerator:: NPY_UINT64 + .. c:enumerator:: NPY_ULONGLONG + .. c:enumerator:: NPY_UINT64 - The enumeration value for a 64-bit/8-byte unsigned integer. + The enumeration value for a 64-bit/8-byte unsigned integer. -.. c:enumerator:: NPY_HALF -.. c:enumerator:: NPY_FLOAT16 + .. c:enumerator:: NPY_HALF + .. c:enumerator:: NPY_FLOAT16 - The enumeration value for a 16-bit/2-byte IEEE 754-2008 compatible floating - point type. + The enumeration value for a 16-bit/2-byte IEEE 754-2008 compatible floating + point type. -.. c:enumerator:: NPY_FLOAT -.. c:enumerator:: NPY_FLOAT32 + .. c:enumerator:: NPY_FLOAT + .. c:enumerator:: NPY_FLOAT32 - The enumeration value for a 32-bit/4-byte IEEE 754 compatible floating - point type. + The enumeration value for a 32-bit/4-byte IEEE 754 compatible floating + point type. -.. c:enumerator:: NPY_DOUBLE -.. c:enumerator:: NPY_FLOAT64 + .. c:enumerator:: NPY_DOUBLE + .. c:enumerator:: NPY_FLOAT64 - The enumeration value for a 64-bit/8-byte IEEE 754 compatible floating - point type. + The enumeration value for a 64-bit/8-byte IEEE 754 compatible floating + point type. -.. c:enumerator:: NPY_LONGDOUBLE + .. c:enumerator:: NPY_LONGDOUBLE - The enumeration value for a platform-specific floating point type which is - at least as large as NPY_DOUBLE, but larger on many platforms. + The enumeration value for a platform-specific floating point type which is + at least as large as NPY_DOUBLE, but larger on many platforms. -.. c:enumerator:: NPY_CFLOAT -.. c:enumerator:: NPY_COMPLEX64 + .. c:enumerator:: NPY_CFLOAT + .. c:enumerator:: NPY_COMPLEX64 - The enumeration value for a 64-bit/8-byte complex type made up of - two NPY_FLOAT values. + The enumeration value for a 64-bit/8-byte complex type made up of + two NPY_FLOAT values. -.. c:enumerator:: NPY_CDOUBLE -.. c:enumerator:: NPY_COMPLEX128 + .. c:enumerator:: NPY_CDOUBLE + .. c:enumerator:: NPY_COMPLEX128 - The enumeration value for a 128-bit/16-byte complex type made up of - two NPY_DOUBLE values. + The enumeration value for a 128-bit/16-byte complex type made up of + two NPY_DOUBLE values. -.. c:enumerator:: NPY_CLONGDOUBLE + .. c:enumerator:: NPY_CLONGDOUBLE - The enumeration value for a platform-specific complex floating point - type which is made up of two NPY_LONGDOUBLE values. + The enumeration value for a platform-specific complex floating point + type which is made up of two NPY_LONGDOUBLE values. -.. c:enumerator:: NPY_DATETIME + .. c:enumerator:: NPY_DATETIME - The enumeration value for a data type which holds dates or datetimes with - a precision based on selectable date or time units. + The enumeration value for a data type which holds dates or datetimes with + a precision based on selectable date or time units. -.. c:enumerator:: NPY_TIMEDELTA + .. c:enumerator:: NPY_TIMEDELTA - The enumeration value for a data type which holds lengths of times in - integers of selectable date or time units. + The enumeration value for a data type which holds lengths of times in + integers of selectable date or time units. -.. c:enumerator:: NPY_STRING + .. c:enumerator:: NPY_STRING - The enumeration value for ASCII strings of a selectable size. The - strings have a fixed maximum size within a given array. + The enumeration value for ASCII strings of a selectable size. The + strings have a fixed maximum size within a given array. -.. c:enumerator:: NPY_UNICODE + .. c:enumerator:: NPY_UNICODE - The enumeration value for UCS4 strings of a selectable size. The - strings have a fixed maximum size within a given array. + The enumeration value for UCS4 strings of a selectable size. The + strings have a fixed maximum size within a given array. -.. c:enumerator:: NPY_OBJECT + .. c:enumerator:: NPY_OBJECT - The enumeration value for references to arbitrary Python objects. + The enumeration value for references to arbitrary Python objects. -.. c:enumerator:: NPY_VOID + .. c:enumerator:: NPY_VOID - Primarily used to hold struct dtypes, but can contain arbitrary - binary data. + Primarily used to hold struct dtypes, but can contain arbitrary + binary data. -Some useful aliases of the above types are + Some useful aliases of the above types are -.. c:enumerator:: NPY_INTP + .. c:enumerator:: NPY_INTP - The enumeration value for a signed integer type which is the same - size as a (void \*) pointer. This is the type used by all - arrays of indices. + The enumeration value for a signed integer type which is the same + size as a (void \*) pointer. This is the type used by all + arrays of indices. -.. c:enumerator:: NPY_UINTP + .. c:enumerator:: NPY_UINTP - The enumeration value for an unsigned integer type which is the - same size as a (void \*) pointer. + The enumeration value for an unsigned integer type which is the + same size as a (void \*) pointer. -.. c:enumerator:: NPY_MASK + .. c:enumerator:: NPY_MASK - The enumeration value of the type used for masks, such as with - the :c:data:`NPY_ITER_ARRAYMASK` iterator flag. This is equivalent - to :c:data:`NPY_UINT8`. + The enumeration value of the type used for masks, such as with + the :c:data:`NPY_ITER_ARRAYMASK` iterator flag. This is equivalent + to :c:data:`NPY_UINT8`. -.. c:enumerator:: NPY_DEFAULT_TYPE + .. c:enumerator:: NPY_DEFAULT_TYPE - The default type to use when no dtype is explicitly specified, for - example when calling np.zero(shape). This is equivalent to - :c:data:`NPY_DOUBLE`. + The default type to use when no dtype is explicitly specified, for + example when calling np.zero(shape). This is equivalent to + :c:data:`NPY_DOUBLE`. Other useful related constants are diff --git a/doc/source/reference/c-api/iterator.rst b/doc/source/reference/c-api/iterator.rst index 07187e7f1..92e3e435b 100644 --- a/doc/source/reference/c-api/iterator.rst +++ b/doc/source/reference/c-api/iterator.rst @@ -26,8 +26,10 @@ which may be of interest for those using this C API. In many instances, testing out ideas by creating the iterator in Python is a good idea before writing the C iteration code. -Simple Iteration Example ------------------------- +.. _iteration-example: + +Iteration Example +----------------- The best way to become familiar with the iterator is to look at its usage within the NumPy codebase itself. For example, here is a slightly @@ -115,10 +117,10 @@ number of non-zero elements in an array. return nonzero_count; } -Simple Multi-Iteration Example +Multi-Iteration Example ------------------------------ -Here is a simple copy function using the iterator. The ``order`` parameter +Here is a copy function using the iterator. The ``order`` parameter is used to control the memory layout of the allocated result, typically :c:data:`NPY_KEEPORDER` is desired. diff --git a/doc/source/reference/c-api/types-and-structures.rst b/doc/source/reference/c-api/types-and-structures.rst index 34437bd30..cff1b3d38 100644 --- a/doc/source/reference/c-api/types-and-structures.rst +++ b/doc/source/reference/c-api/types-and-structures.rst @@ -122,7 +122,7 @@ PyArray_Type and PyArrayObject ``ndarraytypes.h`` points to this data member. :c:data:`NPY_MAXDIMS` is the largest number of dimensions for any array. - .. c:member:: npy_intp dimensions + .. c:member:: npy_intp *dimensions An array of integers providing the shape in each dimension as long as nd :math:`\geq` 1. The integer is always large enough @@ -225,7 +225,7 @@ PyArrayDescr_Type and PyArray_Descr - Never declare a non-pointer instance of the struct - Never perform pointer arithmetic - - Never use ``sizof(PyArray_Descr)`` + - Never use ``sizeof(PyArray_Descr)`` It has the following structure: @@ -285,83 +285,16 @@ PyArrayDescr_Type and PyArray_Descr array like behavior. Each bit in this member is a flag which are named as: - .. c:member:: int alignment - - Non-NULL if this type is an array (C-contiguous) of some other type - - -.. - dedented to allow internal linking, pending a refactoring - -.. c:macro:: NPY_ITEM_REFCOUNT - - Indicates that items of this data-type must be reference - counted (using :c:func:`Py_INCREF` and :c:func:`Py_DECREF` ). - - .. c:macro:: NPY_ITEM_HASOBJECT - - Same as :c:data:`NPY_ITEM_REFCOUNT`. - -.. - dedented to allow internal linking, pending a refactoring - -.. c:macro:: NPY_LIST_PICKLE - - Indicates arrays of this data-type must be converted to a list - before pickling. - -.. c:macro:: NPY_ITEM_IS_POINTER - - Indicates the item is a pointer to some other data-type - -.. c:macro:: NPY_NEEDS_INIT - - Indicates memory for this data-type must be initialized (set - to 0) on creation. - -.. c:macro:: NPY_NEEDS_PYAPI - - Indicates this data-type requires the Python C-API during - access (so don't give up the GIL if array access is going to - be needed). - -.. c:macro:: NPY_USE_GETITEM - - On array access use the ``f->getitem`` function pointer - instead of the standard conversion to an array scalar. Must - use if you don't define an array scalar to go along with - the data-type. - -.. c:macro:: NPY_USE_SETITEM - - When creating a 0-d array from an array scalar use - ``f->setitem`` instead of the standard copy from an array - scalar. Must use if you don't define an array scalar to go - along with the data-type. - - .. c:macro:: NPY_FROM_FIELDS - - The bits that are inherited for the parent data-type if these - bits are set in any field of the data-type. Currently ( - :c:data:`NPY_NEEDS_INIT` \| :c:data:`NPY_LIST_PICKLE` \| - :c:data:`NPY_ITEM_REFCOUNT` \| :c:data:`NPY_NEEDS_PYAPI` ). - - .. c:macro:: NPY_OBJECT_DTYPE_FLAGS - - Bits set for the object data-type: ( :c:data:`NPY_LIST_PICKLE` - \| :c:data:`NPY_USE_GETITEM` \| :c:data:`NPY_ITEM_IS_POINTER` \| - :c:data:`NPY_ITEM_REFCOUNT` \| :c:data:`NPY_NEEDS_INIT` \| - :c:data:`NPY_NEEDS_PYAPI`). - - .. c:function:: int PyDataType_FLAGCHK(PyArray_Descr *dtype, int flags) - - Return true if all the given flags are set for the data-type - object. - - .. c:function:: int PyDataType_REFCHK(PyArray_Descr *dtype) - - Equivalent to :c:func:`PyDataType_FLAGCHK` (*dtype*, - :c:data:`NPY_ITEM_REFCOUNT`). + * :c:macro:`NPY_ITEM_REFCOUNT` + * :c:macro:`NPY_ITEM_HASOBJECT` + * :c:macro:`NPY_LIST_PICKLE` + * :c:macro:`NPY_ITEM_IS_POINTER` + * :c:macro:`NPY_NEEDS_INIT` + * :c:macro:`NPY_NEEDS_PYAPI` + * :c:macro:`NPY_USE_GETITEM` + * :c:macro:`NPY_USE_SETITEM` + * :c:macro:`NPY_FROM_FIELDS` + * :c:macro:`NPY_OBJECT_DTYPE_FLAGS` .. c:member:: int type_num @@ -452,6 +385,73 @@ PyArrayDescr_Type and PyArray_Descr Currently unused. Reserved for future use in caching hash values. +.. c:macro:: NPY_ITEM_REFCOUNT + + Indicates that items of this data-type must be reference + counted (using :c:func:`Py_INCREF` and :c:func:`Py_DECREF` ). + +.. c:macro:: NPY_ITEM_HASOBJECT + + Same as :c:data:`NPY_ITEM_REFCOUNT`. + +.. c:macro:: NPY_LIST_PICKLE + + Indicates arrays of this data-type must be converted to a list + before pickling. + +.. c:macro:: NPY_ITEM_IS_POINTER + + Indicates the item is a pointer to some other data-type + +.. c:macro:: NPY_NEEDS_INIT + + Indicates memory for this data-type must be initialized (set + to 0) on creation. + +.. c:macro:: NPY_NEEDS_PYAPI + + Indicates this data-type requires the Python C-API during + access (so don't give up the GIL if array access is going to + be needed). + +.. c:macro:: NPY_USE_GETITEM + + On array access use the ``f->getitem`` function pointer + instead of the standard conversion to an array scalar. Must + use if you don't define an array scalar to go along with + the data-type. + +.. c:macro:: NPY_USE_SETITEM + + When creating a 0-d array from an array scalar use + ``f->setitem`` instead of the standard copy from an array + scalar. Must use if you don't define an array scalar to go + along with the data-type. + +.. c:macro:: NPY_FROM_FIELDS + + The bits that are inherited for the parent data-type if these + bits are set in any field of the data-type. Currently ( + :c:data:`NPY_NEEDS_INIT` \| :c:data:`NPY_LIST_PICKLE` \| + :c:data:`NPY_ITEM_REFCOUNT` \| :c:data:`NPY_NEEDS_PYAPI` ). + +.. c:macro:: NPY_OBJECT_DTYPE_FLAGS + + Bits set for the object data-type: ( :c:data:`NPY_LIST_PICKLE` + \| :c:data:`NPY_USE_GETITEM` \| :c:data:`NPY_ITEM_IS_POINTER` \| + :c:data:`NPY_ITEM_REFCOUNT` \| :c:data:`NPY_NEEDS_INIT` \| + :c:data:`NPY_NEEDS_PYAPI`). + +.. c:function:: int PyDataType_FLAGCHK(PyArray_Descr *dtype, int flags) + + Return true if all the given flags are set for the data-type + object. + +.. c:function:: int PyDataType_REFCHK(PyArray_Descr *dtype) + + Equivalent to :c:func:`PyDataType_FLAGCHK` (*dtype*, + :c:data:`NPY_ITEM_REFCOUNT`). + .. c:type:: PyArray_ArrFuncs Functions implementing internal features. Not all of these @@ -973,7 +973,7 @@ PyUFunc_Type and PyUFuncObject Some fallback support for this slot exists, but will be removed eventually. A universal function that relied on this will have to be ported eventually. - See ref:`NEP 41 <NEP41>` and ref:`NEP 43 <NEP43>` + See :ref:`NEP 41 <NEP41>` and :ref:`NEP 43 <NEP43>` .. c:member:: void *reserved2 @@ -997,10 +997,14 @@ PyUFunc_Type and PyUFuncObject .. c:member:: npy_uint32 *core_dim_flags - For each distinct core dimension, a set of ``UFUNC_CORE_DIM*`` flags + For each distinct core dimension, a set of flags ( + :c:macro:`UFUNC_CORE_DIM_CAN_IGNORE` and + :c:macro:`UFUNC_CORE_DIM_SIZE_INFERRED`) + + .. c:member:: PyObject *identity_value -.. - dedented to allow internal linking, pending a refactoring + Identity for reduction, when :c:member:`PyUFuncObject.identity` + is equal to :c:data:`PyUFunc_IdentityValue`. .. c:macro:: UFUNC_CORE_DIM_CAN_IGNORE @@ -1011,11 +1015,6 @@ PyUFunc_Type and PyUFuncObject if the dim size will be determined from the operands and not from a :ref:`frozen <frozen>` signature - .. c:member:: PyObject *identity_value - - Identity for reduction, when :c:member:`PyUFuncObject.identity` - is equal to :c:data:`PyUFunc_IdentityValue`. - PyArrayIter_Type and PyArrayIterObject -------------------------------------- @@ -1253,7 +1252,7 @@ ScalarArrayTypes ---------------- There is a Python type for each of the different built-in data types -that can be present in the array Most of these are simple wrappers +that can be present in the array. Most of these are simple wrappers around the corresponding data type in C. The C-names for these types are ``Py{TYPE}ArrType_Type`` where ``{TYPE}`` can be @@ -1457,22 +1456,6 @@ memory management. These types are not accessible directly from Python, and are not exposed to the C-API. They are included here only for completeness and assistance in understanding the code. - -.. c:type:: PyUFuncLoopObject - - A loose wrapper for a C-structure that contains the information - needed for looping. This is useful if you are trying to understand - the ufunc looping code. The :c:type:`PyUFuncLoopObject` is the associated - C-structure. It is defined in the ``ufuncobject.h`` header. - -.. c:type:: PyUFuncReduceObject - - A loose wrapper for the C-structure that contains the information - needed for reduce-like methods of ufuncs. This is useful if you are - trying to understand the reduce, accumulate, and reduce-at - code. The :c:type:`PyUFuncReduceObject` is the associated C-structure. It - is defined in the ``ufuncobject.h`` header. - .. c:type:: PyUFunc_Loop1d A simple linked-list of C-structures containing the information needed @@ -1483,8 +1466,12 @@ for completeness and assistance in understanding the code. Advanced indexing is handled with this Python type. It is simply a loose wrapper around the C-structure containing the variables - needed for advanced array indexing. The associated C-structure, - ``PyArrayMapIterObject``, is useful if you are trying to + needed for advanced array indexing. + +.. c:type:: PyArrayMapIterObject + + The C-structure associated with :c:var:`PyArrayMapIter_Type`. + This structure is useful if you are trying to understand the advanced-index mapping code. It is defined in the ``arrayobject.h`` header. This type is not exposed to Python and could be replaced with a C-structure. As a Python type it takes diff --git a/doc/source/reference/distutils_guide.rst b/doc/source/reference/distutils_guide.rst index a72a9c4d6..6f927c231 100644 --- a/doc/source/reference/distutils_guide.rst +++ b/doc/source/reference/distutils_guide.rst @@ -1,6 +1,6 @@ .. _distutils-user-guide: -NumPy Distutils - Users Guide +NumPy distutils - users guide ============================= .. warning:: diff --git a/doc/source/reference/distutils_status_migration.rst b/doc/source/reference/distutils_status_migration.rst index eda4790b5..67695978c 100644 --- a/doc/source/reference/distutils_status_migration.rst +++ b/doc/source/reference/distutils_status_migration.rst @@ -83,7 +83,7 @@ present in ``setuptools``: - Support for a few other scientific libraries, like FFTW and UMFPACK - Better MinGW support - Per-compiler build flag customization (e.g. `-O3` and `SSE2` flags are default) -- a simple user build config system, see [site.cfg.example](https://github.com/numpy/numpy/blob/master/site.cfg.example) +- a simple user build config system, see `site.cfg.example <https://github.com/numpy/numpy/blob/master/site.cfg.example>`__ - SIMD intrinsics support The most widely used feature is nested ``setup.py`` files. This feature will diff --git a/doc/source/reference/figures/dtype-hierarchy.dia b/doc/source/reference/figures/dtype-hierarchy.dia Binary files differindex 62e925cfd..d5c01bf27 100644 --- a/doc/source/reference/figures/dtype-hierarchy.dia +++ b/doc/source/reference/figures/dtype-hierarchy.dia diff --git a/doc/source/reference/figures/dtype-hierarchy.pdf b/doc/source/reference/figures/dtype-hierarchy.pdf Binary files differindex 6ce496a3e..b2375e152 100644 --- a/doc/source/reference/figures/dtype-hierarchy.pdf +++ b/doc/source/reference/figures/dtype-hierarchy.pdf diff --git a/doc/source/reference/figures/dtype-hierarchy.png b/doc/source/reference/figures/dtype-hierarchy.png Binary files differindex 6c45758b1..891de096c 100644 --- a/doc/source/reference/figures/dtype-hierarchy.png +++ b/doc/source/reference/figures/dtype-hierarchy.png diff --git a/doc/source/reference/global_state.rst b/doc/source/reference/global_state.rst index 81685ec7d..a898450af 100644 --- a/doc/source/reference/global_state.rst +++ b/doc/source/reference/global_state.rst @@ -1,7 +1,7 @@ .. _global_state: ************ -Global State +Global state ************ NumPy has a few import-time, compile-time, or runtime options @@ -11,10 +11,10 @@ purposes and will not be interesting to the vast majority of users. -Performance-Related Options +Performance-related options =========================== -Number of Threads used for Linear Algebra +Number of threads used for Linear Algebra ----------------------------------------- NumPy itself is normally intentionally limited to a single thread @@ -49,25 +49,17 @@ is to use madvise on Kernels 4.6 and newer. These kernels presumably experience a large speedup with hugepage support. This flag is checked at import time. +SIMD feature selection +---------------------- -Interoperability-Related Options -================================ +Setting ``NPY_DISABLE_CPU_FEATURES`` will exclude simd features at runtime. +See :ref:`runtime-simd-dispatch`. -The array function protocol which allows array-like objects to -hook into the NumPy API is currently enabled by default. -This option exists since NumPy 1.16 and is enabled by default since -NumPy 1.17. It can be disabled using:: - NUMPY_EXPERIMENTAL_ARRAY_FUNCTION=0 - -See also :py:meth:`numpy.class.__array_function__` for more information. -This flag is checked at import time. - - -Debugging-Related Options +Debugging-related options ========================= -Relaxed Strides Checking +Relaxed strides checking ------------------------ The *compile-time* environment variable:: @@ -90,3 +82,17 @@ memory allocation policy, the default will be to call ``free``. If ``NUMPY_WARN_IF_NO_MEM_POLICY`` is set to ``"1"``, a ``RuntimeWarning`` will be emitted. A better alternative is to use a ``PyCapsule`` with a deallocator and set the ``ndarray.base``. + + +Testing planned future behavior +=============================== + +NumPy has some code paths which are planned to be activated in the future +but are not yet the default behavior. +You can try testing some of these which may be shipped with a new "major" +release (NumPy 2.0) by setting an environment before importing NumPy: + + NPY_NUMPY_2_BEHAVIOR=1 + +By default this will also activate the :ref:`NEP 50 <NEP50>` related setting +``NPY_PROMOTION_STATE`` (please see the NEP for details on this). diff --git a/doc/source/reference/index.rst b/doc/source/reference/index.rst index 8e4a81436..4756af5fa 100644 --- a/doc/source/reference/index.rst +++ b/doc/source/reference/index.rst @@ -3,7 +3,7 @@ .. _reference: ############### -NumPy Reference +NumPy reference ############### :Release: |version| diff --git a/doc/source/reference/internals.code-explanations.rst b/doc/source/reference/internals.code-explanations.rst index d34314610..db3b3b452 100644 --- a/doc/source/reference/internals.code-explanations.rst +++ b/doc/source/reference/internals.code-explanations.rst @@ -1,7 +1,7 @@ :orphan: ************************* -NumPy C Code Explanations +NumPy C code explanations ************************* .. This document has been moved to ../dev/internals.code-explanations.rst. diff --git a/doc/source/reference/maskedarray.baseclass.rst b/doc/source/reference/maskedarray.baseclass.rst index fcd310faa..7121914b9 100644 --- a/doc/source/reference/maskedarray.baseclass.rst +++ b/doc/source/reference/maskedarray.baseclass.rst @@ -72,19 +72,19 @@ Attributes and properties of masked arrays .. seealso:: :ref:`Array Attributes <arrays.ndarray.attributes>` -.. autoattribute:: MaskedArray.data +.. autoattribute:: numpy::ma.MaskedArray.data -.. autoattribute:: MaskedArray.mask +.. autoattribute:: numpy::ma.MaskedArray.mask -.. autoattribute:: MaskedArray.recordmask +.. autoattribute:: numpy::ma.MaskedArray.recordmask -.. autoattribute:: MaskedArray.fill_value +.. autoattribute:: numpy::ma.MaskedArray.fill_value -.. autoattribute:: MaskedArray.baseclass +.. autoattribute:: numpy::ma.MaskedArray.baseclass -.. autoattribute:: MaskedArray.sharedmask +.. autoattribute:: numpy::ma.MaskedArray.sharedmask -.. autoattribute:: MaskedArray.hardmask +.. autoattribute:: numpy::ma.MaskedArray.hardmask As :class:`MaskedArray` is a subclass of :class:`~numpy.ndarray`, a masked array also inherits all the attributes and properties of a :class:`~numpy.ndarray` instance. diff --git a/doc/source/reference/random/bit_generators/index.rst b/doc/source/reference/random/bit_generators/index.rst index d93f38d0b..e523b4339 100644 --- a/doc/source/reference/random/bit_generators/index.rst +++ b/doc/source/reference/random/bit_generators/index.rst @@ -1,5 +1,7 @@ .. currentmodule:: numpy.random +.. _random-bit-generators: + Bit Generators ============== @@ -50,6 +52,8 @@ The included BitGenerators are: Philox <philox> SFC64 <sfc64> +.. _seeding_and_entropy: + Seeding and Entropy =================== @@ -127,6 +131,16 @@ of 12 instances: .. end_block +If you already have an initial random generator instance, you can shorten +the above by using the `~BitGenerator.spawn` method: + +.. code-block:: python + + from numpy.random import PCG64, SeedSequence + # High quality initial entropy + entropy = 0x87351080e25cb0fad77a44a3be03b491 + base_bitgen = PCG64(entropy) + generators = base_bitgen.spawn(12) An alternative way is to use the fact that a `~SeedSequence` can be initialized by a tuple of elements. Here we use a base entropy value and an integer diff --git a/doc/source/reference/random/compatibility.rst b/doc/source/reference/random/compatibility.rst new file mode 100644 index 000000000..138f03ca3 --- /dev/null +++ b/doc/source/reference/random/compatibility.rst @@ -0,0 +1,87 @@ +.. _random-compatibility: + +.. currentmodule:: numpy.random + +Compatibility Policy +==================== + +`numpy.random` has a somewhat stricter compatibility policy than the rest of +NumPy. Users of pseudorandomness often have use cases for being able to +reproduce runs in fine detail given the same seed (so-called "stream +compatibility"), and so we try to balance those needs with the flexibility to +enhance our algorithms. :ref:`NEP 19 <NEP19>` describes the evolution of this +policy. + +The main kind of compatibility that we enforce is stream-compatibility from run +to run under certain conditions. If you create a `Generator` with the same +`BitGenerator`, with the same seed, perform the same sequence of method calls +with the same arguments, on the same build of ``numpy``, in the same +environment, on the same machine, you should get the same stream of numbers. +Note that these conditions are very strict. There are a number of factors +outside of NumPy's control that limit our ability to guarantee much more than +this. For example, different CPUs implement floating point arithmetic +differently, and this can cause differences in certain edge cases that cascade +to the rest of the stream. `Generator.multivariate_normal`, for another +example, uses a matrix decomposition from ``numpy.linalg``. Even on the same +platform, a different build of ``numpy`` may use a different version of this +matrix decomposition algorithm from the LAPACK that it links to, causing +`Generator.multivariate_normal` to return completely different (but equally +valid!) results. We strive to prefer algorithms that are more resistant to +these effects, but this is always imperfect. + +.. note:: + + Most of the `Generator` methods allow you to draw multiple values from + a distribution as arrays. The requested size of this array is a parameter, + for the purposes of the above policy. Calling ``rng.random()`` 5 times is + not *guaranteed* to give the same numbers as ``rng.random(5)``. We reserve + the ability to decide to use different algorithms for different-sized + blocks. In practice, this happens rarely. + +Like the rest of NumPy, we generally maintain API source +compatibility from version to version. If we *must* make an API-breaking +change, then we will only do so with an appropriate deprecation period and +warnings, according to :ref:`general NumPy policy <NEP23>`. + +Breaking stream-compatibility in order to introduce new features or +improve performance in `Generator` or `default_rng` will be *allowed* with +*caution*. Such changes will be considered features, and as such will be no +faster than the standard release cadence of features (i.e. on ``X.Y`` releases, +never ``X.Y.Z``). Slowness will not be considered a bug for this purpose. +Correctness bug fixes that break stream-compatibility can happen on bugfix +releases, per usual, but developers should consider if they can wait until the +next feature release. We encourage developers to strongly weight user’s pain +from the break in stream-compatibility against the improvements. One example +of a worthwhile improvement would be to change algorithms for a significant +increase in performance, for example, moving from the `Box-Muller transform +<https://en.wikipedia.org/wiki/Box%E2%80%93Muller_transform>`_ method of +Gaussian variate generation to the faster `Ziggurat algorithm +<https://en.wikipedia.org/wiki/Ziggurat_algorithm>`_. An example of +a discouraged improvement would be tweaking the Ziggurat tables just a little +bit for a small performance improvement. + +.. note:: + + In particular, `default_rng` is allowed to change the default + `BitGenerator` that it uses (again, with *caution* and plenty of advance + warning). + +In general, `BitGenerator` classes have stronger guarantees of +version-to-version stream compatibility. This allows them to be a firmer +building block for downstream users that need it. Their limited API surface +makes it easier for them to maintain this compatibility from version to version. See +the docstrings of each `BitGenerator` class for their individual compatibility +guarantees. + +The legacy `RandomState` and the :ref:`associated convenience functions +<functions-in-numpy-random>` have a stricter version-to-version +compatibility guarantee. For reasons outlined in :ref:`NEP 19 <NEP19>`, we had made +stronger promises about their version-to-version stability early in NumPy's +development. There are still some limited use cases for this kind of +compatibility (like generating data for tests), so we maintain as much +compatibility as we can. There will be no more modifications to `RandomState`, +not even to fix correctness bugs. There are a few gray areas where we can make +minor fixes to keep `RandomState` working without segfaulting as NumPy's +internals change, and some docstring fixes. However, the previously-mentioned +caveats about the variability from machine to machine and build to build still +apply to `RandomState` just as much as it does to `Generator`. diff --git a/doc/source/reference/random/extending.rst b/doc/source/reference/random/extending.rst index 6bb941496..998faf80a 100644 --- a/doc/source/reference/random/extending.rst +++ b/doc/source/reference/random/extending.rst @@ -25,7 +25,7 @@ provided by ``ctypes.next_double``. Both CTypes and CFFI allow the more complicated distributions to be used directly in Numba after compiling the file distributions.c into a ``DLL`` or ``so``. An example showing the use of a more complicated distribution is in -the `examples` section below. +the `Examples`_ section below. .. _random_cython: @@ -113,6 +113,6 @@ Examples .. toctree:: Numba <examples/numba> - CFFI + Numba <examples/numba_cffi> + CFFI + Numba <examples/numba_cffi> Cython <examples/cython/index> CFFI <examples/cffi> diff --git a/doc/source/reference/random/generator.rst b/doc/source/reference/random/generator.rst index dc71cb1f9..e08395b17 100644 --- a/doc/source/reference/random/generator.rst +++ b/doc/source/reference/random/generator.rst @@ -18,12 +18,13 @@ can be changed by passing an instantized BitGenerator to ``Generator``. :members: __init__ :exclude-members: __init__ -Accessing the BitGenerator --------------------------- +Accessing the BitGenerator and Spawning +--------------------------------------- .. autosummary:: :toctree: generated/ ~numpy.random.Generator.bit_generator + ~numpy.random.Generator.spawn Simple random data ------------------ diff --git a/doc/source/reference/random/index.rst b/doc/source/reference/random/index.rst index 83a27d80c..38555b133 100644 --- a/doc/source/reference/random/index.rst +++ b/doc/source/reference/random/index.rst @@ -7,210 +7,124 @@ Random sampling (:mod:`numpy.random`) ===================================== -Numpy's random number routines produce pseudo random numbers using -combinations of a `BitGenerator` to create sequences and a `Generator` -to use those sequences to sample from different statistical distributions: - -* BitGenerators: Objects that generate random numbers. These are typically - unsigned integer words filled with sequences of either 32 or 64 random bits. -* Generators: Objects that transform sequences of random bits from a - BitGenerator into sequences of numbers that follow a specific probability - distribution (such as uniform, Normal or Binomial) within a specified - interval. - -Since Numpy version 1.17.0 the Generator can be initialized with a -number of different BitGenerators. It exposes many different probability -distributions. See `NEP 19 <https://www.numpy.org/neps/ -nep-0019-rng-policy.html>`_ for context on the updated random Numpy number -routines. The legacy `RandomState` random number routines are still -available, but limited to a single BitGenerator. See :ref:`new-or-different` -for a complete list of improvements and differences from the legacy -``RandomState``. - -For convenience and backward compatibility, a single `RandomState` -instance's methods are imported into the numpy.random namespace, see -:ref:`legacy` for the complete list. - .. _random-quick-start: Quick Start ----------- -Call `default_rng` to get a new instance of a `Generator`, then call its -methods to obtain samples from different distributions. By default, -`Generator` uses bits provided by `PCG64` which has better statistical -properties than the legacy `MT19937` used in `RandomState`. - -.. code-block:: python - - # Do this (new version) - from numpy.random import default_rng - rng = default_rng() - vals = rng.standard_normal(10) - more_vals = rng.standard_normal(10) - - # instead of this (legacy version) - from numpy import random - vals = random.standard_normal(10) - more_vals = random.standard_normal(10) - -`Generator` can be used as a replacement for `RandomState`. Both class -instances hold an internal `BitGenerator` instance to provide the bit -stream, it is accessible as ``gen.bit_generator``. Some long-overdue API -cleanup means that legacy and compatibility methods have been removed from -`Generator` - -=================== ============== ============ -`RandomState` `Generator` Notes -------------------- -------------- ------------ -``random_sample``, ``random`` Compatible with `random.random` -``rand`` -------------------- -------------- ------------ -``randint``, ``integers`` Add an ``endpoint`` kwarg -``random_integers`` -------------------- -------------- ------------ -``tomaxint`` removed Use ``integers(0, np.iinfo(np.int_).max,`` - ``endpoint=False)`` -------------------- -------------- ------------ -``seed`` removed Use `SeedSequence.spawn` -=================== ============== ============ - -See :ref:`new-or-different` for more information. - -Something like the following code can be used to support both ``RandomState`` -and ``Generator``, with the understanding that the interfaces are slightly -different - -.. code-block:: python - - try: - rng_integers = rng.integers - except AttributeError: - rng_integers = rng.randint - a = rng_integers(1000) - -Seeds can be passed to any of the BitGenerators. The provided value is mixed -via `SeedSequence` to spread a possible sequence of seeds across a wider -range of initialization states for the BitGenerator. Here `PCG64` is used and -is wrapped with a `Generator`. - -.. code-block:: python - - from numpy.random import Generator, PCG64 - rng = Generator(PCG64(12345)) - rng.standard_normal() - -Here we use `default_rng` to create an instance of `Generator` to generate a -random float: - ->>> import numpy as np ->>> rng = np.random.default_rng(12345) ->>> print(rng) -Generator(PCG64) ->>> rfloat = rng.random() ->>> rfloat -0.22733602246716966 ->>> type(rfloat) -<class 'float'> - -Here we use `default_rng` to create an instance of `Generator` to generate 3 -random integers between 0 (inclusive) and 10 (exclusive): - ->>> import numpy as np ->>> rng = np.random.default_rng(12345) ->>> rints = rng.integers(low=0, high=10, size=3) ->>> rints -array([6, 2, 7]) ->>> type(rints[0]) -<class 'numpy.int64'> - -Introduction ------------- -The new infrastructure takes a different approach to producing random numbers -from the `RandomState` object. Random number generation is separated into -two components, a bit generator and a random generator. - -The `BitGenerator` has a limited set of responsibilities. It manages state -and provides functions to produce random doubles and random unsigned 32- and -64-bit values. - -The `random generator <Generator>` takes the -bit generator-provided stream and transforms them into more useful -distributions, e.g., simulated normal random values. This structure allows -alternative bit generators to be used with little code duplication. - -The `Generator` is the user-facing object that is nearly identical to the -legacy `RandomState`. It accepts a bit generator instance as an argument. -The default is currently `PCG64` but this may change in future versions. -As a convenience NumPy provides the `default_rng` function to hide these -details: - ->>> from numpy.random import default_rng ->>> rng = default_rng(12345) ->>> print(rng) -Generator(PCG64) ->>> print(rng.random()) -0.22733602246716966 - -One can also instantiate `Generator` directly with a `BitGenerator` instance. - -To use the default `PCG64` bit generator, one can instantiate it directly and -pass it to `Generator`: - ->>> from numpy.random import Generator, PCG64 ->>> rng = Generator(PCG64(12345)) ->>> print(rng) -Generator(PCG64) - -Similarly to use the older `MT19937` bit generator (not recommended), one can -instantiate it directly and pass it to `Generator`: - ->>> from numpy.random import Generator, MT19937 ->>> rng = Generator(MT19937(12345)) ->>> print(rng) -Generator(MT19937) - -What's New or Different -~~~~~~~~~~~~~~~~~~~~~~~ +The :mod:`numpy.random` module implements pseudo-random number generators +(PRNGs or RNGs, for short) with the ability to draw samples from a variety of +probability distributions. In general, users will create a `Generator` instance +with `default_rng` and call the various methods on it to obtain samples from +different distributions. + +:: + + >>> import numpy as np + >>> rng = np.random.default_rng() + # Generate one random float uniformly distributed over the range [0, 1) + >>> rng.random() #doctest: +SKIP + 0.06369197489564249 # may vary + # Generate an array of 10 numbers according to a unit Gaussian distribution. + >>> rng.standard_normal(10) #doctest: +SKIP + array([-0.31018314, -1.8922078 , -0.3628523 , -0.63526532, 0.43181166, # may vary + 0.51640373, 1.25693945, 0.07779185, 0.84090247, -2.13406828]) + # Generate an array of 5 integers uniformly over the range [0, 10). + >>> rng.integers(low=0, high=10, size=5) #doctest: +SKIP + array([8, 7, 6, 2, 0]) # may vary + +Our RNGs are deterministic sequences and can be reproduced by specifying a seed integer to +derive its initial state. By default, with no seed provided, `default_rng` will create +seed the RNG from nondeterministic data from the operating system and therefore +generate different numbers each time. The pseudo-random sequences will be +independent for all practical purposes, at least those purposes for which our +pseudo-randomness was good for in the first place. + +:: + + >>> rng1 = np.random.default_rng() + >>> rng1.random() #doctest: +SKIP + 0.6596288841243357 # may vary + >>> rng2 = np.random.default_rng() + >>> rng2.random() #doctest: +SKIP + 0.11885628817151628 # may vary + .. warning:: - The Box-Muller method used to produce NumPy's normals is no longer available - in `Generator`. It is not possible to reproduce the exact random - values using Generator for the normal distribution or any other - distribution that relies on the normal such as the `RandomState.gamma` or - `RandomState.standard_t`. If you require bitwise backward compatible - streams, use `RandomState`. - -* The Generator's normal, exponential and gamma functions use 256-step Ziggurat - methods which are 2-10 times faster than NumPy's Box-Muller or inverse CDF - implementations. -* Optional ``dtype`` argument that accepts ``np.float32`` or ``np.float64`` - to produce either single or double precision uniform random variables for - select distributions -* Optional ``out`` argument that allows existing arrays to be filled for - select distributions -* All BitGenerators can produce doubles, uint64s and uint32s via CTypes - (`PCG64.ctypes`) and CFFI (`PCG64.cffi`). This allows the bit generators - to be used in numba. -* The bit generators can be used in downstream projects via - :ref:`Cython <random_cython>`. -* `Generator.integers` is now the canonical way to generate integer - random numbers from a discrete uniform distribution. The ``rand`` and - ``randn`` methods are only available through the legacy `RandomState`. - The ``endpoint`` keyword can be used to specify open or closed intervals. - This replaces both ``randint`` and the deprecated ``random_integers``. -* `Generator.random` is now the canonical way to generate floating-point - random numbers, which replaces `RandomState.random_sample`, - `RandomState.sample`, and `RandomState.ranf`. This is consistent with - Python's `random.random`. -* All BitGenerators in numpy use `SeedSequence` to convert seeds into - initialized states. -* The addition of an ``axis`` keyword argument to methods such as - `Generator.choice`, `Generator.permutation`, and `Generator.shuffle` - improves support for sampling from and shuffling multi-dimensional arrays. - -See :ref:`new-or-different` for a complete list of improvements and -differences from the traditional ``Randomstate``. + The pseudo-random number generators implemented in this module are designed + for statistical modeling and simulation. They are not suitable for security + or cryptographic purposes. See the :py:mod:`secrets` module from the + standard library for such use cases. + +Seeds should be large positive integers. `default_rng` can take positive +integers of any size. We recommend using very large, unique numbers to ensure +that your seed is different from anyone else's. This is good practice to ensure +that your results are statistically independent from theirs unless you are +intentionally *trying* to reproduce their result. A convenient way to get +such a seed number is to use :py:func:`secrets.randbits` to get an +arbitrary 128-bit integer. + +:: + + >>> import secrets + >>> import numpy as np + >>> secrets.randbits(128) #doctest: +SKIP + 122807528840384100672342137672332424406 # may vary + >>> rng1 = np.random.default_rng(122807528840384100672342137672332424406) + >>> rng1.random() + 0.5363922081269535 + >>> rng2 = np.random.default_rng(122807528840384100672342137672332424406) + >>> rng2.random() + 0.5363922081269535 + +See the documentation on `default_rng` and `SeedSequence` for more advanced +options for controlling the seed in specialized scenarios. + +`Generator` and its associated infrastructure was introduced in NumPy version +1.17.0. There is still a lot of code that uses the older `RandomState` and the +functions in `numpy.random`. While there are no plans to remove them at this +time, we do recommend transitioning to `Generator` as you can. The algorithms +are faster, more flexible, and will receive more improvements in the future. +For the most part, `Generator` can be used as a replacement for `RandomState`. +See :ref:`legacy` for information on the legacy infrastructure, +:ref:`new-or-different` for information on transitioning, and :ref:`NEP 19 +<NEP19>` for some of the reasoning for the transition. + +Design +------ + +Users primarily interact with `Generator` instances. Each `Generator` instance +owns a `BitGenerator` instance that implements the core RNG algorithm. The +`BitGenerator` has a limited set of responsibilities. It manages state and +provides functions to produce random doubles and random unsigned 32- and 64-bit +values. + +The `Generator` takes the bit generator-provided stream and transforms them +into more useful distributions, e.g., simulated normal random values. This +structure allows alternative bit generators to be used with little code +duplication. + +NumPy implements several different `BitGenerator` classes implementing +different RNG algorithms. `default_rng` currently uses `~PCG64` as the +default `BitGenerator`. It has better statistical properties and performance +than the `~MT19937` algorithm used in the legacy `RandomState`. See +:ref:`random-bit-generators` for more details on the supported BitGenerators. + +`default_rng` and BitGenerators delegate the conversion of seeds into RNG +states to `SeedSequence` internally. `SeedSequence` implements a sophisticated +algorithm that intermediates between the user's input and the internal +implementation details of each `BitGenerator` algorithm, each of which can +require different amounts of bits for its state. Importantly, it lets you use +arbitrary-sized integers and arbitrary sequences of such integers to mix +together into the RNG state. This is a useful primitive for constructing +a :ref:`flexible pattern for parallel RNG streams <seedsequence-spawn>`. + +For backward compatibility, we still maintain the legacy `RandomState` class. +It continues to use the `~MT19937` algorithm by default, and old seeds continue +to reproduce the same results. The convenience :ref:`functions-in-numpy-random` +are still aliases to the methods on a single global `RandomState` instance. See +:ref:`legacy` for the complete details. See :ref:`new-or-different` for +a detailed comparison between `Generator` and `RandomState`. Parallel Generation ~~~~~~~~~~~~~~~~~~~ @@ -235,6 +149,7 @@ Concepts Legacy Generator (RandomState) <legacy> BitGenerators, SeedSequences <bit_generators/index> Upgrading PCG64 with PCG64DXSM <upgrading-pcg64> + compatibility Features -------- diff --git a/doc/source/reference/random/legacy.rst b/doc/source/reference/random/legacy.rst index b1fce49a1..00921c477 100644 --- a/doc/source/reference/random/legacy.rst +++ b/doc/source/reference/random/legacy.rst @@ -52,7 +52,7 @@ using the state of the `RandomState`: :exclude-members: __init__ Seeding and State ------------------ +================= .. autosummary:: :toctree: generated/ @@ -62,7 +62,7 @@ Seeding and State ~RandomState.seed Simple random data ------------------- +================== .. autosummary:: :toctree: generated/ @@ -75,7 +75,7 @@ Simple random data ~RandomState.bytes Permutations ------------- +============ .. autosummary:: :toctree: generated/ @@ -83,7 +83,7 @@ Permutations ~RandomState.permutation Distributions -------------- +============== .. autosummary:: :toctree: generated/ @@ -123,8 +123,10 @@ Distributions ~RandomState.weibull ~RandomState.zipf +.. _functions-in-numpy-random: + Functions in `numpy.random` ---------------------------- +=========================== Many of the RandomState methods above are exported as functions in `numpy.random` This usage is discouraged, as it is implemented via a global `RandomState` instance which is not advised on two counts: @@ -133,8 +135,7 @@ Many of the RandomState methods above are exported as functions in - It uses a `RandomState` rather than the more modern `Generator`. -For backward compatible legacy reasons, we cannot change this. See -:ref:`random-quick-start`. +For backward compatible legacy reasons, we will not change this. .. autosummary:: :toctree: generated/ diff --git a/doc/source/reference/random/new-or-different.rst b/doc/source/reference/random/new-or-different.rst index 8f4a70540..9b5bf38e5 100644 --- a/doc/source/reference/random/new-or-different.rst +++ b/doc/source/reference/random/new-or-different.rst @@ -5,17 +5,9 @@ What's New or Different ----------------------- -.. warning:: - - The Box-Muller method used to produce NumPy's normals is no longer available - in `Generator`. It is not possible to reproduce the exact random - values using ``Generator`` for the normal distribution or any other - distribution that relies on the normal such as the `Generator.gamma` or - `Generator.standard_t`. If you require bitwise backward compatible - streams, use `RandomState`, i.e., `RandomState.gamma` or - `RandomState.standard_t`. - -Quick comparison of legacy :ref:`mtrand <legacy>` to the new `Generator` +NumPy 1.17.0 introduced `Generator` as an improved replacement for +the :ref:`legacy <legacy>` `RandomState`. Here is a quick comparison of the two +implementations. ================== ==================== ============= Feature Older Equivalent Notes @@ -44,21 +36,17 @@ Feature Older Equivalent Notes ``high`` interval endpoint ================== ==================== ============= -And in more detail: - -* Simulate from the complex normal distribution - (`~.Generator.complex_normal`) * The normal, exponential and gamma generators use 256-step Ziggurat methods which are 2-10 times faster than NumPy's default implementation in `~.Generator.standard_normal`, `~.Generator.standard_exponential` or - `~.Generator.standard_gamma`. - + `~.Generator.standard_gamma`. Because of the change in algorithms, it is not + possible to reproduce the exact random values using ``Generator`` for these + distributions or any distribution method that relies on them. .. ipython:: python - from numpy.random import Generator, PCG64 import numpy.random - rng = Generator(PCG64()) + rng = np.random.default_rng() %timeit -n 1 rng.standard_normal(100000) %timeit -n 1 numpy.random.standard_normal(100000) @@ -74,18 +62,25 @@ And in more detail: * `~.Generator.integers` is now the canonical way to generate integer - random numbers from a discrete uniform distribution. The ``rand`` and - ``randn`` methods are only available through the legacy `~.RandomState`. - This replaces both ``randint`` and the deprecated ``random_integers``. -* The Box-Muller method used to produce NumPy's normals is no longer available. + random numbers from a discrete uniform distribution. This replaces both + ``randint`` and the deprecated ``random_integers``. +* The ``rand`` and ``randn`` methods are only available through the legacy + `~.RandomState`. +* `Generator.random` is now the canonical way to generate floating-point + random numbers, which replaces `RandomState.random_sample`, + `sample`, and `ranf`, all of which were aliases. This is consistent with + Python's `random.random`. * All bit generators can produce doubles, uint64s and uint32s via CTypes (`~PCG64.ctypes`) and CFFI (`~PCG64.cffi`). This allows these bit generators to be used in numba. * The bit generators can be used in downstream projects via Cython. +* All bit generators use `SeedSequence` to :ref:`convert seed integers to + initialized states <seeding_and_entropy>`. * Optional ``dtype`` argument that accepts ``np.float32`` or ``np.float64`` to produce either single or double precision uniform random variables for - select distributions + select distributions. `~.Generator.integers` accepts a ``dtype`` argument + with any signed or unsigned integer dtype. * Uniforms (`~.Generator.random` and `~.Generator.integers`) * Normals (`~.Generator.standard_normal`) @@ -94,9 +89,10 @@ And in more detail: .. ipython:: python - rng = Generator(PCG64(0)) - rng.random(3, dtype='d') - rng.random(3, dtype='f') + rng = np.random.default_rng() + rng.random(3, dtype=np.float64) + rng.random(3, dtype=np.float32) + rng.integers(0, 256, size=3, dtype=np.uint8) * Optional ``out`` argument that allows existing arrays to be filled for select distributions @@ -111,6 +107,7 @@ And in more detail: .. ipython:: python + rng = np.random.default_rng() existing = np.zeros(4) rng.random(out=existing[:2]) print(existing) @@ -121,9 +118,12 @@ And in more detail: .. ipython:: python - rng = Generator(PCG64(123456789)) + rng = np.random.default_rng() a = np.arange(12).reshape((3, 4)) a rng.choice(a, axis=1, size=5) rng.shuffle(a, axis=1) # Shuffle in-place a + +* Added a method to sample from the complex normal distribution + (`~.Generator.complex_normal`) diff --git a/doc/source/reference/random/parallel.rst b/doc/source/reference/random/parallel.rst index b625d34b7..b4934a0ca 100644 --- a/doc/source/reference/random/parallel.rst +++ b/doc/source/reference/random/parallel.rst @@ -12,6 +12,11 @@ or distributed). `~SeedSequence` spawning ------------------------ +NumPy allows you to spawn new (with very high probability) independent +`~BitGenerator` and `~Generator` instances via their ``spawn()`` method. +This spawning is implemented by the `~SeedSequence` used for initializing +the bit generators random stream. + `~SeedSequence` `implements an algorithm`_ to process a user-provided seed, typically as an integer of some size, and to convert it into an initial state for a `~BitGenerator`. It uses hashing techniques to ensure that low-quality seeds @@ -53,15 +58,25 @@ wrap this together into an API that is easy to use and difficult to misuse. .. end_block -Child `~SeedSequence` objects can also spawn to make grandchildren, and so on. -Each `~SeedSequence` has its position in the tree of spawned `~SeedSequence` -objects mixed in with the user-provided seed to generate independent (with very -high probability) streams. +For convenience the direct use of `~SeedSequence` is not necessary. +The above ``streams`` can be spawned directly from a parent generator +via `~Generator.spawn`: + +.. code-block:: python + + parent_rng = default_rng(12345) + streams = parent_rng.spawn(10) + +.. end_block + +Child objects can also spawn to make grandchildren, and so on. +Each child has a `~SeedSequence` with its position in the tree of spawned +child objects mixed in with the user-provided seed to generate independent +(with very high probability) streams. .. code-block:: python - grandchildren = child_seeds[0].spawn(4) - grand_streams = [default_rng(s) for s in grandchildren] + grandchildren = streams[0].spawn(4) .. end_block diff --git a/doc/source/reference/routines.ctypeslib.rst b/doc/source/reference/routines.ctypeslib.rst index c6127ca64..fe5ffb5a8 100644 --- a/doc/source/reference/routines.ctypeslib.rst +++ b/doc/source/reference/routines.ctypeslib.rst @@ -1,7 +1,7 @@ .. module:: numpy.ctypeslib *********************************************************** -C-Types Foreign Function Interface (:mod:`numpy.ctypeslib`) +C-Types foreign function interface (:mod:`numpy.ctypeslib`) *********************************************************** .. currentmodule:: numpy.ctypeslib diff --git a/doc/source/reference/routines.datetime.rst b/doc/source/reference/routines.datetime.rst index 966ed5a47..72513882b 100644 --- a/doc/source/reference/routines.datetime.rst +++ b/doc/source/reference/routines.datetime.rst @@ -1,6 +1,6 @@ .. _routines.datetime: -Datetime Support Functions +Datetime support functions ************************** .. currentmodule:: numpy @@ -12,7 +12,7 @@ Datetime Support Functions datetime_data -Business Day Functions +Business day functions ====================== .. currentmodule:: numpy diff --git a/doc/source/reference/routines.dtype.rst b/doc/source/reference/routines.dtype.rst index e9189ca07..6cb036031 100644 --- a/doc/source/reference/routines.dtype.rst +++ b/doc/source/reference/routines.dtype.rst @@ -30,7 +30,6 @@ Data type information finfo iinfo - MachAr Data type testing ----------------- diff --git a/doc/source/reference/routines.dual.rst b/doc/source/reference/routines.dual.rst deleted file mode 100644 index 18c7791d0..000000000 --- a/doc/source/reference/routines.dual.rst +++ /dev/null @@ -1,47 +0,0 @@ -Optionally SciPy-accelerated routines (:mod:`numpy.dual`) -========================================================= - -.. automodule:: numpy.dual - -Linear algebra --------------- - -.. currentmodule:: numpy.linalg - -.. autosummary:: - - cholesky - det - eig - eigh - eigvals - eigvalsh - inv - lstsq - norm - pinv - solve - svd - -FFT ---- - -.. currentmodule:: numpy.fft - -.. autosummary:: - - fft - fft2 - fftn - ifft - ifft2 - ifftn - -Other ------ - -.. currentmodule:: numpy - -.. autosummary:: - - i0 diff --git a/doc/source/reference/routines.io.rst b/doc/source/reference/routines.io.rst index 2542b336f..1ec2ccb5e 100644 --- a/doc/source/reference/routines.io.rst +++ b/doc/source/reference/routines.io.rst @@ -83,7 +83,7 @@ Data sources DataSource -Binary Format Description +Binary format description ------------------------- .. autosummary:: :toctree: generated/ diff --git a/doc/source/reference/routines.ma.rst b/doc/source/reference/routines.ma.rst index d503cc243..fd22a74aa 100644 --- a/doc/source/reference/routines.ma.rst +++ b/doc/source/reference/routines.ma.rst @@ -31,6 +31,7 @@ From existing data ma.fromfunction ma.MaskedArray.copy + ma.diagflat Ones and zeros @@ -72,6 +73,9 @@ Inspecting the array ma.isMaskedArray ma.isMA ma.isarray + ma.isin + ma.in1d + ma.unique ma.MaskedArray.all @@ -394,6 +398,17 @@ Clipping and rounding ma.MaskedArray.clip ma.MaskedArray.round +Set operations +~~~~~~~~~~~~~~ +.. autosummary:: + :toctree: generated/ + + + ma.intersect1d + ma.setdiff1d + ma.setxor1d + ma.union1d + Miscellanea ~~~~~~~~~~~ diff --git a/doc/source/reference/routines.math.rst b/doc/source/reference/routines.math.rst index a454841b3..4cc8719fb 100644 --- a/doc/source/reference/routines.math.rst +++ b/doc/source/reference/routines.math.rst @@ -39,8 +39,8 @@ Rounding .. autosummary:: :toctree: generated/ + round around - round_ rint fix floor @@ -149,13 +149,15 @@ Extrema Finding :toctree: generated/ maximum - fmax + max amax + fmax nanmax minimum - fmin + min amin + fmin nanmin diff --git a/doc/source/reference/routines.other.rst b/doc/source/reference/routines.other.rst index e980406eb..769b3d910 100644 --- a/doc/source/reference/routines.other.rst +++ b/doc/source/reference/routines.other.rst @@ -58,9 +58,6 @@ Matlab-like Functions who disp -Exceptions ----------- -.. autosummary:: - :toctree: generated/ +.. automodule:: numpy.exceptions - AxisError +.. automodule:: numpy.dtypes diff --git a/doc/source/reference/routines.rst b/doc/source/reference/routines.rst index 593d017cc..72ed80972 100644 --- a/doc/source/reference/routines.rst +++ b/doc/source/reference/routines.rst @@ -24,7 +24,6 @@ indentation. routines.ctypeslib routines.datetime routines.dtype - routines.dual routines.emath routines.err routines.fft @@ -44,4 +43,5 @@ indentation. routines.sort routines.statistics routines.testing + routines.testing.overrides routines.window diff --git a/doc/source/reference/routines.testing.overrides.rst b/doc/source/reference/routines.testing.overrides.rst new file mode 100644 index 000000000..262852633 --- /dev/null +++ b/doc/source/reference/routines.testing.overrides.rst @@ -0,0 +1,18 @@ +.. module:: numpy.testing.overrides + +Support for testing overrides (:mod:`numpy.testing.overrides`) +============================================================== + +.. currentmodule:: numpy.testing.overrides + +Support for testing custom array container implementations. + +Utility Functions +----------------- +.. autosummary:: + :toctree: generated/ + + allows_array_function_override + allows_array_ufunc_override + get_overridable_numpy_ufuncs + get_overridable_numpy_array_functions diff --git a/doc/source/reference/routines.testing.rst b/doc/source/reference/routines.testing.rst index 16d53bb4e..82e43dfdb 100644 --- a/doc/source/reference/routines.testing.rst +++ b/doc/source/reference/routines.testing.rst @@ -51,11 +51,6 @@ Decorators .. autosummary:: :toctree: generated/ - dec.deprecated - dec.knownfailureif - dec.setastest - dec.skipif - dec.slow decorate_methods Test Running @@ -63,10 +58,8 @@ Test Running .. autosummary:: :toctree: generated/ - Tester clear_and_catch_warnings measure - run_module_suite rundocs suppress_warnings diff --git a/doc/source/reference/simd/build-options.rst b/doc/source/reference/simd/build-options.rst index 0994f15aa..d1e2e6b8e 100644 --- a/doc/source/reference/simd/build-options.rst +++ b/doc/source/reference/simd/build-options.rst @@ -333,7 +333,7 @@ and here is how it looks on x86_64/gcc: .. literalinclude:: log_example.txt :language: bash -As you see, there is a separate report for each of ``build_ext`` and ``build_clib`` +There is a separate report for each of ``build_ext`` and ``build_clib`` that includes several sections, and each section has several values, representing the following: **Platform**: @@ -371,6 +371,17 @@ that includes several sections, and each section has several values, representin - The lines that come after the above property and end with a ':' on a separate line, represent the paths of c/c++ sources that define the generated optimizations. -Runtime Trace -------------- -To be completed. +.. _runtime-simd-dispatch: + +Runtime dispatch +---------------- +Importing NumPy triggers a scan of the available CPU features from the set +of dispatchable features. This can be further restricted by setting the +environment variable ``NPY_DISABLE_CPU_FEATURES`` to a comma-, tab-, or +space-separated list of features to disable. This will raise an error if +parsing fails or if the feature was not enabled. For instance, on ``x86_64`` +this will disable ``AVX2`` and ``FMA3``:: + + NPY_DISABLE_CPU_FEATURES="AVX2,FMA3" + +If the feature is not available, a warning will be emitted. diff --git a/doc/source/reference/simd/gen_features.py b/doc/source/reference/simd/gen_features.py index 9a38ef5c9..b141e23d0 100644 --- a/doc/source/reference/simd/gen_features.py +++ b/doc/source/reference/simd/gen_features.py @@ -168,7 +168,7 @@ if __name__ == '__main__': gen_path = path.join( path.dirname(path.realpath(__file__)), "generated_tables" ) - with open(path.join(gen_path, 'cpu_features.inc'), 'wt') as fd: + with open(path.join(gen_path, 'cpu_features.inc'), 'w') as fd: fd.write(f'.. generated via {__file__}\n\n') for arch in ( ("x86", "PPC64", "PPC64LE", "ARMHF", "AARCH64", "S390X") @@ -177,7 +177,7 @@ if __name__ == '__main__': table = Features(arch, 'gcc').table() fd.write(wrapper_section(title, table)) - with open(path.join(gen_path, 'compilers-diff.inc'), 'wt') as fd: + with open(path.join(gen_path, 'compilers-diff.inc'), 'w') as fd: fd.write(f'.. generated via {__file__}\n\n') for arch, cc_names in ( ("x86", ("clang", "ICC", "MSVC")), diff --git a/doc/source/reference/simd/generated_tables/compilers-diff.inc b/doc/source/reference/simd/generated_tables/compilers-diff.inc index 4b9009a68..d5a87da3c 100644 --- a/doc/source/reference/simd/generated_tables/compilers-diff.inc +++ b/doc/source/reference/simd/generated_tables/compilers-diff.inc @@ -1,33 +1,35 @@ -.. generated via /home/seiko/work/repos/numpy/doc/source/reference/simd/./gen_features.py +.. generated via /numpy/numpy/./doc/source/reference/simd/gen_features.py On x86::Intel Compiler ~~~~~~~~~~~~~~~~~~~~~~ .. table:: :align: left - ================ ========================================================================================================================================== - Name Implies - ================ ========================================================================================================================================== - FMA3 SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C :enabled:`AVX2` - AVX2 SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C :enabled:`FMA3` - AVX512F SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 :enabled:`AVX512CD` - :disabled:`XOP` :disabled:`SSE` :disabled:`SSE2` :disabled:`SSE3` :disabled:`SSSE3` :disabled:`SSE41` :disabled:`POPCNT` :disabled:`SSE42` :disabled:`AVX` - :disabled:`FMA4` :disabled:`SSE` :disabled:`SSE2` :disabled:`SSE3` :disabled:`SSSE3` :disabled:`SSE41` :disabled:`POPCNT` :disabled:`SSE42` :disabled:`AVX` - ================ ========================================================================================================================================== + ====================== ================================================================================================================================================================================================================================================================================================================================== ====================== + Name Implies Gathers + ====================== ================================================================================================================================================================================================================================================================================================================================== ====================== + FMA3 SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C :enabled:`AVX2` + AVX2 SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C :enabled:`FMA3` + AVX512F SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 :enabled:`AVX512CD` + :disabled:`XOP` :disabled:`SSE` :disabled:`SSE2` :disabled:`SSE3` :disabled:`SSSE3` :disabled:`SSE41` :disabled:`POPCNT` :disabled:`SSE42` :disabled:`AVX` + :disabled:`FMA4` :disabled:`SSE` :disabled:`SSE2` :disabled:`SSE3` :disabled:`SSSE3` :disabled:`SSE41` :disabled:`POPCNT` :disabled:`SSE42` :disabled:`AVX` + :disabled:`AVX512_SPR` :disabled:`SSE` :disabled:`SSE2` :disabled:`SSE3` :disabled:`SSSE3` :disabled:`SSE41` :disabled:`POPCNT` :disabled:`SSE42` :disabled:`AVX` :disabled:`F16C` :disabled:`FMA3` :disabled:`AVX2` :disabled:`AVX512F` :disabled:`AVX512CD` :disabled:`AVX512_SKX` :disabled:`AVX512_CLX` :disabled:`AVX512_CNL` :disabled:`AVX512_ICL` :disabled:`AVX512FP16` + ====================== ================================================================================================================================================================================================================================================================================================================================== ====================== On x86::Microsoft Visual C/C++ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. table:: :align: left - ====================== ============================================================================================================================================================================================================================================================= ============================================================================= - Name Implies Gathers - ====================== ============================================================================================================================================================================================================================================================= ============================================================================= - FMA3 SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C :enabled:`AVX2` - AVX2 SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C :enabled:`FMA3` - AVX512F SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 :enabled:`AVX512CD` :enabled:`AVX512_SKX` - AVX512CD SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F :enabled:`AVX512_SKX` - :disabled:`AVX512_KNL` :disabled:`SSE` :disabled:`SSE2` :disabled:`SSE3` :disabled:`SSSE3` :disabled:`SSE41` :disabled:`POPCNT` :disabled:`SSE42` :disabled:`AVX` :disabled:`F16C` :disabled:`FMA3` :disabled:`AVX2` :disabled:`AVX512F` :disabled:`AVX512CD` :disabled:`AVX512ER` :disabled:`AVX512PF` - :disabled:`AVX512_KNM` :disabled:`SSE` :disabled:`SSE2` :disabled:`SSE3` :disabled:`SSSE3` :disabled:`SSE41` :disabled:`POPCNT` :disabled:`SSE42` :disabled:`AVX` :disabled:`F16C` :disabled:`FMA3` :disabled:`AVX2` :disabled:`AVX512F` :disabled:`AVX512CD` :disabled:`AVX512_KNL` :disabled:`AVX5124FMAPS` :disabled:`AVX5124VNNIW` :disabled:`AVX512VPOPCNTDQ` - ====================== ============================================================================================================================================================================================================================================================= ============================================================================= + ====================== ================================================================================================================================================================================================================================================================================================================================== ============================================================================= + Name Implies Gathers + ====================== ================================================================================================================================================================================================================================================================================================================================== ============================================================================= + FMA3 SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C :enabled:`AVX2` + AVX2 SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C :enabled:`FMA3` + AVX512F SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 :enabled:`AVX512CD` :enabled:`AVX512_SKX` + AVX512CD SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F :enabled:`AVX512_SKX` + :disabled:`AVX512_KNL` :disabled:`SSE` :disabled:`SSE2` :disabled:`SSE3` :disabled:`SSSE3` :disabled:`SSE41` :disabled:`POPCNT` :disabled:`SSE42` :disabled:`AVX` :disabled:`F16C` :disabled:`FMA3` :disabled:`AVX2` :disabled:`AVX512F` :disabled:`AVX512CD` :disabled:`AVX512ER` :disabled:`AVX512PF` + :disabled:`AVX512_KNM` :disabled:`SSE` :disabled:`SSE2` :disabled:`SSE3` :disabled:`SSSE3` :disabled:`SSE41` :disabled:`POPCNT` :disabled:`SSE42` :disabled:`AVX` :disabled:`F16C` :disabled:`FMA3` :disabled:`AVX2` :disabled:`AVX512F` :disabled:`AVX512CD` :disabled:`AVX512_KNL` :disabled:`AVX5124FMAPS` :disabled:`AVX5124VNNIW` :disabled:`AVX512VPOPCNTDQ` + :disabled:`AVX512_SPR` :disabled:`SSE` :disabled:`SSE2` :disabled:`SSE3` :disabled:`SSSE3` :disabled:`SSE41` :disabled:`POPCNT` :disabled:`SSE42` :disabled:`AVX` :disabled:`F16C` :disabled:`FMA3` :disabled:`AVX2` :disabled:`AVX512F` :disabled:`AVX512CD` :disabled:`AVX512_SKX` :disabled:`AVX512_CLX` :disabled:`AVX512_CNL` :disabled:`AVX512_ICL` :disabled:`AVX512FP16` + ====================== ================================================================================================================================================================================================================================================================================================================================== ============================================================================= diff --git a/doc/source/reference/simd/generated_tables/cpu_features.inc b/doc/source/reference/simd/generated_tables/cpu_features.inc index 7782172d2..603370e21 100644 --- a/doc/source/reference/simd/generated_tables/cpu_features.inc +++ b/doc/source/reference/simd/generated_tables/cpu_features.inc @@ -1,35 +1,36 @@ -.. generated via /home/seiko/work/repos/review/numpy/doc/source/reference/simd/gen_features.py +.. generated via /numpy/numpy/./doc/source/reference/simd/gen_features.py On x86 ~~~~~~ .. table:: :align: left - ============== =========================================================================================================================================================================== ===================================================== - Name Implies Gathers - ============== =========================================================================================================================================================================== ===================================================== - ``SSE`` ``SSE2`` - ``SSE2`` ``SSE`` - ``SSE3`` ``SSE`` ``SSE2`` - ``SSSE3`` ``SSE`` ``SSE2`` ``SSE3`` - ``SSE41`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` - ``POPCNT`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` - ``SSE42`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` - ``AVX`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` - ``XOP`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` - ``FMA4`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` - ``F16C`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` - ``FMA3`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` - ``AVX2`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` - ``AVX512F`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` - ``AVX512CD`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` - ``AVX512_KNL`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512ER`` ``AVX512PF`` - ``AVX512_KNM`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512_KNL`` ``AVX5124FMAPS`` ``AVX5124VNNIW`` ``AVX512VPOPCNTDQ`` - ``AVX512_SKX`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512VL`` ``AVX512BW`` ``AVX512DQ`` - ``AVX512_CLX`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512_SKX`` ``AVX512VNNI`` - ``AVX512_CNL`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512_SKX`` ``AVX512IFMA`` ``AVX512VBMI`` - ``AVX512_ICL`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512_SKX`` ``AVX512_CLX`` ``AVX512_CNL`` ``AVX512VBMI2`` ``AVX512BITALG`` ``AVX512VPOPCNTDQ`` - ============== =========================================================================================================================================================================== ===================================================== + ============== ========================================================================================================================================================================================== ===================================================== + Name Implies Gathers + ============== ========================================================================================================================================================================================== ===================================================== + ``SSE`` ``SSE2`` + ``SSE2`` ``SSE`` + ``SSE3`` ``SSE`` ``SSE2`` + ``SSSE3`` ``SSE`` ``SSE2`` ``SSE3`` + ``SSE41`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` + ``POPCNT`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` + ``SSE42`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` + ``AVX`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` + ``XOP`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` + ``FMA4`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` + ``F16C`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` + ``FMA3`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` + ``AVX2`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` + ``AVX512F`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` + ``AVX512CD`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` + ``AVX512_KNL`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512ER`` ``AVX512PF`` + ``AVX512_KNM`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512_KNL`` ``AVX5124FMAPS`` ``AVX5124VNNIW`` ``AVX512VPOPCNTDQ`` + ``AVX512_SKX`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512VL`` ``AVX512BW`` ``AVX512DQ`` + ``AVX512_CLX`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512_SKX`` ``AVX512VNNI`` + ``AVX512_CNL`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512_SKX`` ``AVX512IFMA`` ``AVX512VBMI`` + ``AVX512_ICL`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512_SKX`` ``AVX512_CLX`` ``AVX512_CNL`` ``AVX512VBMI2`` ``AVX512BITALG`` ``AVX512VPOPCNTDQ`` + ``AVX512_SPR`` ``SSE`` ``SSE2`` ``SSE3`` ``SSSE3`` ``SSE41`` ``POPCNT`` ``SSE42`` ``AVX`` ``F16C`` ``FMA3`` ``AVX2`` ``AVX512F`` ``AVX512CD`` ``AVX512_SKX`` ``AVX512_CLX`` ``AVX512_CNL`` ``AVX512_ICL`` ``AVX512FP16`` + ============== ========================================================================================================================================================================================== ===================================================== On IBM/POWER big-endian ~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/source/reference/simd/how-it-works.rst b/doc/source/reference/simd/how-it-works.rst index 19b3dba45..dac80dd02 100644 --- a/doc/source/reference/simd/how-it-works.rst +++ b/doc/source/reference/simd/how-it-works.rst @@ -1,6 +1,6 @@ -********************************** +********************************* How does the CPU dispatcher work? -********************************** +********************************* NumPy dispatcher is based on multi-source compiling, which means taking a certain source and compiling it multiple times with different compiler |
