Merge pull request #5914 from sirtom67/master

New typemap for in-place arrays of arbitrary number of dimensions:

10 files changed, 406 insertions, 26 deletions

diff --git a/doc/source/reference/swig.interface-file.rst b/doc/source/reference/swig.interface-file.rst
index c381feb85..e5d369d0e 100644
--- a/doc/source/reference/swig.interface-file.rst
+++ b/doc/source/reference/swig.interface-file.rst
@@ -320,6 +320,17 @@ signatures are
 These typemaps now check to make sure that the ``INPLACE_ARRAY``
 arguments use native byte ordering.  If not, an exception is raised.
 
+There is also a "flat" in-place array for situations in which
+you would like to modify or process each element, regardless of the
+number of dimensions. One example is a "quantization" function that
+quantizes each element of an array in-place, be it 1D, 2D or whatever.
+This form checks for continuity but allows either C or Fortran ordering.
+
+ND:
+
+ * ``(DATA_TYPE* INPLACE_ARRAY_FLAT, DIM_TYPE DIM_FLAT)``
+
+
 Argout Arrays
 `````````````
 
diff --git a/doc/source/reference/swig.testing.rst b/doc/source/reference/swig.testing.rst
index c0daaec66..13642a52e 100644
--- a/doc/source/reference/swig.testing.rst
+++ b/doc/source/reference/swig.testing.rst
@@ -57,6 +57,7 @@ Two-dimensional arrays are tested in exactly the same manner.  The
 above description applies, but with ``Matrix`` substituted for
 ``Vector``.  For three-dimensional tests, substitute ``Tensor`` for
 ``Vector``.  For four-dimensional tests, substitute ``SuperTensor``
+for ``Vector``. For flat in-place array tests, substitute ``Flat``
 for ``Vector``.
 For the descriptions that follow, we will reference the
 ``Vector`` tests, but the same information applies to ``Matrix``,
diff --git a/tools/swig/README b/tools/swig/README
index 1f05b106c..7fa0599c6 100644
--- a/tools/swig/README
+++ b/tools/swig/README
@@ -37,6 +37,11 @@ The files related to testing are are in the test subdirectory::
     SuperTensor.i
     testSuperTensor.py
 
+    Flat.h
+    Flat.cxx
+    Flat.i
+    testFlat.py
+
 The header files contain prototypes for functions that illustrate the
 wrapping issues we wish to address.  Right now, this consists of
 functions with argument signatures of the following forms.  Vector.h::
@@ -89,9 +94,12 @@ SuperTensor.h::
 
     (type ARGOUT_ARRAY4[ANY][ANY][ANY][ANY])
 
+Flat.h::
+    (type* INPLACE_ARRAY_FLAT, int DIM_FLAT)
+
 These function signatures take a pointer to an array of type "type",
 whose length is specified by the integer(s) DIM1 (and DIM2, and DIM3,
-and DIM4).
+and DIM4, or DIM_FLAT).
 
 The objective for the IN_ARRAY signatures is for SWIG to generate
 python wrappers that take a container that constitutes a valid
@@ -105,30 +113,32 @@ The objective for the INPLACE_ARRAY signatures is for SWIG to generate
 python wrappers that accept a numpy array of any of the above-listed
 types.
 
-The source files Vector.cxx, Matrix.cxx Tensor.cxx and SuperTensor.cxx
-contain the actual implementations of the functions described in
-Vector.h, Matrix.h Tensor.h and SuperTensor.h.  The python scripts
-testVector.py, testMatrix.py testTensor.py and testSuperTensor.py
-test the resulting python wrappers using the unittest module.
-
-The SWIG interface files Vector.i, Matrix.i Tensor.i and SuperTensor.i
-are used to generate the wrapper code.  The SWIG_FILE_WITH_INIT macro
-allows numpy.i to be used with multiple python modules.  If it is
-specified, then the %init block found in Vector.i, Matrix.i Tensor.i
-and SuperTensor.i are required.  The other things done in Vector.i,
-Matrix.i Tensor.i and SuperTensor.i are the inclusion of the
-appropriate header file and numpy.i file, and the "%apply" directives
-to force the functions to use the typemaps.
+The source files Vector.cxx, Matrix.cxx, Tensor.cxx, SuperTensor.cxx
+and Flat.cxx contain the actual implementations of the functions
+described in Vector.h, Matrix.h Tensor.h, SuperTensor.h and Flat.h.
+The python scripts testVector.py, testMatrix.py testTensor.py,
+testSuperTensor.py and testFlat.py test the resulting python wrappers
+using the unittest module.
+
+The SWIG interface files Vector.i, Matrix.i, Tensor.i, SuperTensor.i
+and Flat.i are used to generate the wrapper code.  The
+SWIG_FILE_WITH_INIT macro allows numpy.i to be used with multiple
+python modules.  If it is specified, then the %init block found in
+Vector.i, Matrix.i Tensor.i, SuperTensor.i and Flat.i are required.
+The other things done in Vector.i, Matrix.i, Tensor.i, SuperTensor.i
+and Flat.i are the inclusion of the appropriate header file and
+numpy.i file, and the "%apply" directives to force the functions to
+use the typemaps.
 
 The setup.py script is a standard python distutils script.  It defines
-_Vector, _Matrix _Tensor and _SuperTensor extension modules and Vector
-, Matrix, Tensor and SuperTensor python modules.  The Makefile
+_Vector, _Matrix, _Tensor, _SuperTensor, _Flat extension modules and Vector,
+Matrix, Tensor, SuperTensor and Flat python modules.  The Makefile
 automates everything, setting up the dependencies, calling swig to
 generate the wrappers, and calling setup.py to compile the wrapper
-code and generate the shared objects.
-Targets "all" (default), "test", "doc" and "clean" are supported.  The
-"doc" target creates HTML documentation (with make target "html"), and
-PDF documentation (with make targets "tex" and "pdf").
+code and generate the shared objects.  Targets "all" (default), "test",
+"doc" and "clean" are supported.  The "doc" target creates HTML
+documentation (with make target "html"), and PDF documentation
+(with make targets "tex" and "pdf").
 
 To build and run the test code, simply execute from the shell::
 
diff --git a/tools/swig/numpy.i b/tools/swig/numpy.i
index b9a7ce7f4..b6a588c03 100644
--- a/tools/swig/numpy.i
+++ b/tools/swig/numpy.i
@@ -381,6 +381,22 @@
     return contiguous;
   }
 
+  /* Test whether a python object is (C_ or F_) contiguous.  If array is
+   * contiguous, return 1.  Otherwise, set the python error string and
+   * return 0.
+   */
+  int require_c_or_f_contiguous(PyArrayObject* ary)
+  {
+    int contiguous = 1;
+    if (!(array_is_contiguous(ary) || array_is_fortran(ary)))
+    {
+      PyErr_SetString(PyExc_TypeError,
+                      "Array must be contiguous (C_ or F_).  A non-contiguous array was given");
+      contiguous = 0;
+    }
+    return contiguous;
+  }
+
   /* Require that a numpy array is not byte-swapped.  If the array is
    * not byte-swapped, return 1.  Otherwise, set the python error string
    * and return 0.
@@ -543,7 +559,7 @@
 
 /* %numpy_typemaps() macro
  *
- * This macro defines a family of 74 typemaps that allow C arguments
+ * This macro defines a family of 75 typemaps that allow C arguments
  * of the form
  *
  *    1. (DATA_TYPE IN_ARRAY1[ANY])
@@ -640,6 +656,8 @@
  *   73. (DATA_TYPE** ARGOUTVIEWM_FARRAY4, DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3, DIM_TYPE* DIM4)
  *   74. (DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3, DIM_TYPE* DIM4, DATA_TYPE** ARGOUTVIEWM_FARRAY4)
  *
+ *   75. (DATA_TYPE* INPLACE_ARRAY_FLAT, DIM_TYPE DIM_FLAT)
+ *
  * where "DATA_TYPE" is any type supported by the NumPy module, and
  * "DIM_TYPE" is any int-like type suitable for specifying dimensions.
  * The difference between "ARRAY" typemaps and "FARRAY" typemaps is
@@ -3072,6 +3090,32 @@
   $result = SWIG_Python_AppendOutput($result,obj);
 }
 
+/**************************************/
+/* In-Place Array Typemap - flattened */
+/**************************************/
+
+/* Typemap suite for (DATA_TYPE* INPLACE_ARRAY_FLAT, DIM_TYPE DIM_FLAT)
+ */
+%typecheck(SWIG_TYPECHECK_DOUBLE_ARRAY,
+           fragment="NumPy_Macros")
+  (DATA_TYPE* INPLACE_ARRAY_FLAT, DIM_TYPE DIM_FLAT)
+{
+  $1 = is_array($input) && PyArray_EquivTypenums(array_type($input),
+                                                 DATA_TYPECODE);
+}
+%typemap(in,
+         fragment="NumPy_Fragments")
+  (DATA_TYPE* INPLACE_ARRAY_FLAT, DIM_TYPE DIM_FLAT)
+  (PyArrayObject* array=NULL, int i=1)
+{
+  array = obj_to_array_no_conversion($input, DATA_TYPECODE);
+  if (!array || !require_c_or_f_contiguous(array)
+      || !require_native(array)) SWIG_fail;
+  $1 = (DATA_TYPE*) array_data(array);
+  $2 = 1;
+  for (i=0; i < array_numdims(array); ++i) $2 *= array_size(array,i);
+}
+
 %enddef    /* %numpy_typemaps() macro */
 /* *************************************************************** */
 
diff --git a/tools/swig/test/Flat.cxx b/tools/swig/test/Flat.cxx
new file mode 100644
index 000000000..1a7f42b25
--- /dev/null
+++ b/tools/swig/test/Flat.cxx
@@ -0,0 +1,36 @@
+#include <stdlib.h>
+#include <math.h>
+#include <iostream>
+#include "Flat.h"
+
+// The following macro defines a family of functions that work with 1D
+// arrays with the forms
+//
+//     void SNAMEProcess(TYPE * array,  int size);
+//
+// for any specified type TYPE (for example: short, unsigned int, long
+// long, etc.) with given short name SNAME (for example: short, uint,
+// longLong, etc.).  The macro is then expanded for the given
+// TYPE/SNAME pairs.  The resulting functions are for testing numpy
+// interfaces for:
+//
+//  * in-place arrays (arbitrary number of dimensions) with a fixed number of elements
+//
+#define TEST_FUNCS(TYPE, SNAME) \
+\
+void SNAME ## Process(TYPE * array, int size) {          \
+  for (int i=0; i<size; ++i) array[i] += 1;              \
+}
+
+TEST_FUNCS(signed char       , schar    )
+TEST_FUNCS(unsigned char     , uchar    )
+TEST_FUNCS(short             , short    )
+TEST_FUNCS(unsigned short    , ushort   )
+TEST_FUNCS(int               , int      )
+TEST_FUNCS(unsigned int      , uint     )
+TEST_FUNCS(long              , long     )
+TEST_FUNCS(unsigned long     , ulong    )
+TEST_FUNCS(long long         , longLong )
+TEST_FUNCS(unsigned long long, ulongLong)
+TEST_FUNCS(float             , float    )
+TEST_FUNCS(double            , double   )
diff --git a/tools/swig/test/Flat.h b/tools/swig/test/Flat.h
new file mode 100644
index 000000000..87e6998c6
--- /dev/null
+++ b/tools/swig/test/Flat.h
@@ -0,0 +1,34 @@
+#ifndef FLAT_H
+#define FLAT_H
+
+// The following macro defines the prototypes for a family of
+// functions that work with arrays with the forms
+//
+//     void SNAMEProcess(TYPE * array,  int size);
+//
+// for any specified type TYPE (for example: short, unsigned int, long
+// long, etc.) with given short name SNAME (for example: short, uint,
+// longLong, etc.).  The macro is then expanded for the given
+// TYPE/SNAME pairs.  The resulting functions are for testing numpy
+// interfaces for:
+//
+//  * in-place arrays (arbitrary number of dimensions) with a fixed number of elements
+//
+#define TEST_FUNC_PROTOS(TYPE, SNAME) \
+\
+void SNAME ## Process(TYPE * array, int size); \
+
+TEST_FUNC_PROTOS(signed char       , schar    )
+TEST_FUNC_PROTOS(unsigned char     , uchar    )
+TEST_FUNC_PROTOS(short             , short    )
+TEST_FUNC_PROTOS(unsigned short    , ushort   )
+TEST_FUNC_PROTOS(int               , int      )
+TEST_FUNC_PROTOS(unsigned int      , uint     )
+TEST_FUNC_PROTOS(long              , long     )
+TEST_FUNC_PROTOS(unsigned long     , ulong    )
+TEST_FUNC_PROTOS(long long         , longLong )
+TEST_FUNC_PROTOS(unsigned long long, ulongLong)
+TEST_FUNC_PROTOS(float             , float    )
+TEST_FUNC_PROTOS(double            , double   )
+
+#endif
diff --git a/tools/swig/test/Flat.i b/tools/swig/test/Flat.i
new file mode 100644
index 000000000..20a911b50
--- /dev/null
+++ b/tools/swig/test/Flat.i
@@ -0,0 +1,36 @@
+// -*- c++ -*-
+%module Flat
+
+%{
+#define SWIG_FILE_WITH_INIT
+#include "Flat.h"
+%}
+
+// Get the NumPy typemaps
+%include "../numpy.i"
+
+%init %{
+  import_array();
+%}
+
+%define %apply_numpy_typemaps(TYPE)
+
+%apply (TYPE* INPLACE_ARRAY_FLAT, int DIM_FLAT) {(TYPE* array, int size)};
+
+%enddef    /* %apply_numpy_typemaps() macro */
+
+%apply_numpy_typemaps(signed char       )
+%apply_numpy_typemaps(unsigned char     )
+%apply_numpy_typemaps(short             )
+%apply_numpy_typemaps(unsigned short    )
+%apply_numpy_typemaps(int               )
+%apply_numpy_typemaps(unsigned int      )
+%apply_numpy_typemaps(long              )
+%apply_numpy_typemaps(unsigned long     )
+%apply_numpy_typemaps(long long         )
+%apply_numpy_typemaps(unsigned long long)
+%apply_numpy_typemaps(float             )
+%apply_numpy_typemaps(double            )
+
+// Include the header file to be wrapped
+%include "Flat.h"
diff --git a/tools/swig/test/Makefile b/tools/swig/test/Makefile
index 5632e7ad0..a01ac409a 100644
--- a/tools/swig/test/Makefile
+++ b/tools/swig/test/Makefile
@@ -1,5 +1,5 @@
 # SWIG
-INTERFACES = Array.i Farray.i Vector.i Matrix.i Tensor.i Fortran.i
+INTERFACES = Array.i Farray.i Vector.i Matrix.i Tensor.i Fortran.i Flat.i
 WRAPPERS   = $(INTERFACES:.i=_wrap.cxx)
 PROXIES    = $(INTERFACES:.i=.py      )
 
@@ -7,7 +7,8 @@ PROXIES    = $(INTERFACES:.i=.py      )
 .PHONY : all
 all: $(WRAPPERS) Array1.cxx Array1.h Array2.cxx Array2.h ArrayZ.cxx ArrayZ.h \
     Farray.cxx Farray.h Vector.cxx Vector.h \
-	Matrix.cxx Matrix.h Tensor.cxx Tensor.h Fortran.h Fortran.cxx
+	Matrix.cxx Matrix.h Tensor.cxx Tensor.h Fortran.h Fortran.cxx \
+	Flat.h Flat.cxx
 	./setup.py build_ext -i
 
 # Test target: run the tests
@@ -19,6 +20,7 @@ test: all
 	python testArray.py
 	python testFarray.py
 	python testFortran.py
+	python testFlat.py
 
 # Rule: %.i -> %_wrap.cxx
 %_wrap.cxx: %.i %.h ../numpy.i
diff --git a/tools/swig/test/setup.py b/tools/swig/test/setup.py
index 81df1b8ed..4ff870e19 100755
--- a/tools/swig/test/setup.py
+++ b/tools/swig/test/setup.py
@@ -54,12 +54,18 @@ _Fortran = Extension("_Fortran",
                     include_dirs = [numpy_include],
                     )
 
+_Flat = Extension("_Flat",
+                    ["Flat_wrap.cxx",
+                     "Flat.cxx"],
+                    include_dirs = [numpy_include],
+                    )
+
 # NumyTypemapTests setup
 setup(name        = "NumpyTypemapTests",
       description = "Functions that work on arrays",
       author      = "Bill Spotz",
       py_modules  = ["Array", "Farray", "Vector", "Matrix", "Tensor",
-                     "Fortran"],
+                     "Fortran", "Flat"],
       ext_modules = [_Array, _Farray, _Vector, _Matrix, _Tensor,
-                     _Fortran]
+                     _Fortran, _Flat]
       )
diff --git a/tools/swig/test/testFlat.py b/tools/swig/test/testFlat.py
new file mode 100755
index 000000000..bd96bc778
--- /dev/null
+++ b/tools/swig/test/testFlat.py
@@ -0,0 +1,200 @@
+#! /usr/bin/env python
+from __future__ import division, absolute_import, print_function
+
+# System imports
+from   distutils.util import get_platform
+import os
+import sys
+import unittest
+
+import struct
+
+# Import NumPy
+import numpy as np
+major, minor = [ int(d) for d in np.__version__.split(".")[:2] ]
+if major == 0: BadListError = TypeError
+else:          BadListError = ValueError
+
+import Flat
+
+######################################################################
+
+class FlatTestCase(unittest.TestCase):
+
+    def __init__(self, methodName="runTest"):
+        unittest.TestCase.__init__(self, methodName)
+        self.typeStr  = "double"
+        self.typeCode = "d"
+
+    # Test the (type* INPLACE_ARRAY_FLAT, int DIM_FLAT) typemap
+    def testProcess1D(self):
+        "Test Process function 1D array"
+        print(self.typeStr, "... ", end=' ', file=sys.stderr)
+        process = Flat.__dict__[self.typeStr + "Process"]
+        pack_output = ''
+        for i in range(10):
+            pack_output += struct.pack(self.typeCode,i)
+        x = np.frombuffer(pack_output, dtype=self.typeCode)
+        y = x.copy()
+        process(y)
+        self.assertEquals(np.all((x+1)==y),True)
+
+    def testProcess3D(self):
+        "Test Process function 3D array"
+        print(self.typeStr, "... ", end=' ', file=sys.stderr)
+        process = Flat.__dict__[self.typeStr + "Process"]
+        pack_output = ''
+        for i in range(24):
+            pack_output += struct.pack(self.typeCode,i)
+        x = np.frombuffer(pack_output, dtype=self.typeCode)
+        x.shape = (2,3,4)
+        y = x.copy()
+        process(y)
+        self.assertEquals(np.all((x+1)==y),True)
+
+    def testProcess3DTranspose(self):
+        "Test Process function 3D array, FORTRAN order"
+        print(self.typeStr, "... ", end=' ', file=sys.stderr)
+        process = Flat.__dict__[self.typeStr + "Process"]
+        pack_output = ''
+        for i in range(24):
+            pack_output += struct.pack(self.typeCode,i)
+        x = np.frombuffer(pack_output, dtype=self.typeCode)
+        x.shape = (2,3,4)
+        y = x.copy()
+        process(y.T)
+        self.assertEquals(np.all((x.T+1)==y.T),True)
+
+    def testProcessNoncontiguous(self):
+        "Test Process function with non-contiguous array, which should raise an error"
+        print(self.typeStr, "... ", end=' ', file=sys.stderr)
+        process = Flat.__dict__[self.typeStr + "Process"]
+        pack_output = ''
+        for i in range(24):
+            pack_output += struct.pack(self.typeCode,i)
+        x = np.frombuffer(pack_output, dtype=self.typeCode)
+        x.shape = (2,3,4)
+        self.assertRaises(TypeError, process, x[:,:,0])
+
+
+######################################################################
+
+class scharTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "schar"
+        self.typeCode = "b"
+
+######################################################################
+
+class ucharTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "uchar"
+        self.typeCode = "B"
+
+######################################################################
+
+class shortTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "short"
+        self.typeCode = "h"
+
+######################################################################
+
+class ushortTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "ushort"
+        self.typeCode = "H"
+
+######################################################################
+
+class intTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "int"
+        self.typeCode = "i"
+
+######################################################################
+
+class uintTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "uint"
+        self.typeCode = "I"
+
+######################################################################
+
+class longTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "long"
+        self.typeCode = "l"
+
+######################################################################
+
+class ulongTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "ulong"
+        self.typeCode = "L"
+
+######################################################################
+
+class longLongTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "longLong"
+        self.typeCode = "q"
+
+######################################################################
+
+class ulongLongTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "ulongLong"
+        self.typeCode = "Q"
+
+######################################################################
+
+class floatTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "float"
+        self.typeCode = "f"
+
+######################################################################
+
+class doubleTestCase(FlatTestCase):
+    def __init__(self, methodName="runTest"):
+        FlatTestCase.__init__(self, methodName)
+        self.typeStr  = "double"
+        self.typeCode = "d"
+
+######################################################################
+
+if __name__ == "__main__":
+
+    # Build the test suite
+    suite = unittest.TestSuite()
+    suite.addTest(unittest.makeSuite(    scharTestCase))
+    suite.addTest(unittest.makeSuite(    ucharTestCase))
+    suite.addTest(unittest.makeSuite(    shortTestCase))
+    suite.addTest(unittest.makeSuite(   ushortTestCase))
+    suite.addTest(unittest.makeSuite(      intTestCase))
+    suite.addTest(unittest.makeSuite(     uintTestCase))
+    suite.addTest(unittest.makeSuite(     longTestCase))
+    suite.addTest(unittest.makeSuite(    ulongTestCase))
+    suite.addTest(unittest.makeSuite( longLongTestCase))
+    suite.addTest(unittest.makeSuite(ulongLongTestCase))
+    suite.addTest(unittest.makeSuite(    floatTestCase))
+    suite.addTest(unittest.makeSuite(   doubleTestCase))
+
+    # Execute the test suite
+    print("Testing 1D Functions of Module Flat")
+    print("NumPy version", np.__version__)
+    print()
+    result = unittest.TextTestRunner(verbosity=2).run(suite)
+    sys.exit(bool(result.errors + result.failures))