Refactor purge_deleted, operate on batches of stacks

Avoid large sql "in" clauses by operating on smaller batches of stacks
at a time.

To avoid transaction overhead and contention on the resource table,
the first deletions occur outside of a transaction (are
autocommitted). This is OK because the purge is re-rentrant -- we
won't lose any stack_id's to delete if something goes wrong before the
conn.begin() block. That is, we will not orphan any rows if the purge
is run multiple times where an error occurs.

Change-Id: I9edf0558ed54820842193560e323df6501411d1d

1 files changed, 9 insertions, 1 deletions

diff --git a/heat/cmd/manage.py b/heat/cmd/manage.py
index 0bda99907..6f1b6e1f5 100644
--- a/heat/cmd/manage.py
+++ b/heat/cmd/manage.py
@@ -133,7 +133,8 @@ def purge_deleted():
     """Remove database records that have been previously soft deleted."""
     utils.purge_deleted(CONF.command.age,
                         CONF.command.granularity,
-                        CONF.command.project_id)
+                        CONF.command.project_id,
+                        CONF.command.batch_size)
 
 
 def do_crypt_parameters_and_properties():
@@ -179,6 +180,13 @@ def add_command_parsers(subparsers):
     parser.add_argument(
         '-p', '--project-id',
         help=_('Project ID to purge deleted stacks.'))
+    # optional parameter, can be skipped. default='20'
+    parser.add_argument(
+        '-b', '--batch_size', default='20',
+        help=_('Number of stacks to delete at a time (per transaction). '
+               'Note that a single stack may have many db rows '
+               '(events, etc.) associated with it.'))
+
     # update_params parser
     parser = subparsers.add_parser('update_params')
     parser.set_defaults(func=do_crypt_parameters_and_properties)