bug 839327 Crash in Gcalc_operation_reducer::end_couple with ST_UNION and MULTIPOLYGONs in 5.3-gis.

        When edges of a polygon coicide, it can form an pike, that is turned into a line after an operation.
        In this case a former polygon point can be an end of a single line, and that case wasn't properly handled.


per-file comments:
  mysql-test/r/gis-precise.result
bug 839327 Crash in Gcalc_operation_reducer::end_couple with ST_UNION and MULTIPOLYGONs in 5.3-gis.
        test result updated.

  mysql-test/t/gis-precise.test
bug 839327 Crash in Gcalc_operation_reducer::end_couple with ST_UNION and MULTIPOLYGONs in 5.3-gis.
        test case added.

  sql/gcalc_tools.cc
bug 839327 Crash in Gcalc_operation_reducer::end_couple with ST_UNION and MULTIPOLYGONs in 5.3-gis.
        in the scev_two_ends case check if we have single line ending on a polygon node.

1 files changed, 13 insertions, 3 deletions

diff --git a/sql/gcalc_tools.cc b/sql/gcalc_tools.cc
index e502d54a160..3ee58646ca4 100644
--- a/sql/gcalc_tools.cc
+++ b/sql/gcalc_tools.cc
@@ -626,9 +626,19 @@ int Gcalc_operation_reducer::count_slice(Gcalc_scan_iterator *si)
       }
       case scev_two_ends:
       {
-        if (cur_t->enabled() &&
-            end_couple(cur_t, cur_t->get_next(), events->pi))
-          return 1;
+        if (cur_t->enabled() && cur_t->get_next()->enabled())
+        {
+          /* When two threads are ended here */
+          if (end_couple(cur_t, cur_t->get_next(), events->pi))
+            return 1;
+        }
+        else if (cur_t->enabled() || cur_t->get_next()->enabled())
+        {
+          /* Rare case when edges of a polygon coincide */
+          if (end_line(cur_t->enabled() ? cur_t : cur_t->get_next(),
+                       events->pi, si))
+            return 1;
+        }
         *cur_t_hook= cur_t->get_next()->get_next();
         free_item(cur_t->next);
         free_item(cur_t);