diff options
| author | Ruediger Meier <ruediger.meier@ga-group.nl> | 2015-09-26 14:01:26 +0200 |
|---|---|---|
| committer | Ruediger Meier <ruediger.meier@ga-group.nl> | 2015-09-30 23:18:55 +0200 |
| commit | 08e8c1415670b029c26ae8ce0585fb9ea0b11e63 (patch) | |
| tree | 38bce83f7f4bcf6ffa1c23894996135321d27939 /numpy/lib/function_base.py | |
| parent | e6dbcead884019bd2aee802f78c41f6a9c46cd61 (diff) | |
| download | numpy-08e8c1415670b029c26ae8ce0585fb9ea0b11e63.tar.gz | |
MAINT: corrcoef, memory usage optimization
We calculate sqrt on the small vector rather than on that huge
product matrix and we combine the "outer" product with element-wise
devision. So even though we have a slower loop over the rows now ... this
code snippet runs about 3 times faster than before.
However the speed improvement of the whole function is not really
significant because cov() takes 80-99% of the time (dependent on
blas/lapack implementation and number of CPU cores).
More important is that we will safe 1/3 memory. For example corrcoef()
for a [23k, m] matrix needs 8GB now instead of 12GB.
Diffstat (limited to 'numpy/lib/function_base.py')
| -rw-r--r-- | numpy/lib/function_base.py | 6 |
1 files changed, 5 insertions, 1 deletions
diff --git a/numpy/lib/function_base.py b/numpy/lib/function_base.py index 007ff42a4..e649807e3 100644 --- a/numpy/lib/function_base.py +++ b/numpy/lib/function_base.py @@ -2349,7 +2349,11 @@ def corrcoef(x, y=None, rowvar=1, bias=np._NoValue, ddof=np._NoValue): except ValueError: # scalar covariance # nan if incorrect value (nan, inf, 0), 1 otherwise return c / c - return c / sqrt(multiply.outer(d, d)) + d = sqrt(d) + # calculate "c / multiply.outer(d, d)" row-wise ... for memory and speed + for i in range(0, d.size): + c[i,:] /= (d * d[i]) + return c def blackman(M): |