Can run swift-bench across multiple cores/servers.

You run one or more swift-bench-client processes like this:
    $ swift-bench-client 127.0.0.1 20001
    $ swift-bench-client 127.0.0.1 20002

Then you run swift-bench with a new option, --bench-clients (-b), which is
specified once for each swift-bench-client:
    $ swift-bench -b 127.0.0.1:20001 -b 127.0.0.1:20002

You get log lines from each client (interleaved) along with a final report
for all clients:
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:44:06,148 INFO Auth version: 1.0
    127.0.0.1:20001 swift-bench-server 2012-08-25 22:44:06,148 INFO Auth version: 1.0
    127.0.0.1:20001 swift-bench-server 2012-08-25 22:44:12,249 INFO 83 PUTS [0 failures], 41.5/s
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:44:14,430 INFO 74 PUTS [0 failures], 34.3/s
    ...
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:45:18,942 INFO Auth version: 1.0
    127.0.0.1:20002 swift-bench-server 2012-08-25 22:45:20,946 INFO 238 DEL [2 failures], 118.9/s
    swift-bench 2012-08-25 22:45:27,549 INFO 2000 PUTS **FINAL** [0 failures], 56.8/s
    swift-bench 2012-08-25 22:45:27,550 INFO 30000 GETS **FINAL** [50 failures], 974.6/s
    swift-bench 2012-08-25 22:45:27,550 INFO 2000 DEL **FINAL** [20 failures], 237.1/s

The concurrency, PUT count, and GET count config settings are divided by
the number of bench_clients.  In other words, the same volume of work is
attempted (vs. not specifying --bench-clients), but it can now span
servers and CPU cores.

Benchmark containers are created (if use_proxy = yes) and deleted (if
delete = yes), with appropriate concurrency, in the initiating
swift-bench process, not any of the swift-bench-client processes.

Change-Id: Idbf31a23093244ab357a9bf77e6031257774f24a

1 files changed, 22 insertions, 2 deletions

diff --git a/bin/swift-bench b/bin/swift-bench
index b69671a..edeb511 100755
--- a/bin/swift-bench
+++ b/bin/swift-bench
@@ -21,7 +21,8 @@ import signal
 import uuid
 from optparse import OptionParser
 
-from swift.common.bench import BenchController
+from swift.common.bench import (BenchController, DistributedBenchController,
+                                create_containers, delete_containers)
 from swift.common.utils import readconf, LogAdapter
 
 # The defaults should be sufficient to run swift-bench on a SAIO
@@ -49,6 +50,8 @@ CONF_DEFAULTS = {
     'devices': 'sdb1',  # space-sep list
     'log_level': 'INFO',
     'timeout': '10',
+    'auth_version': '1.0',
+    'bench_clients': [],
 }
 
 SAIO_DEFAULTS = {
@@ -81,6 +84,13 @@ if __name__ == '__main__':
                       help='User name for obtaining an auth token')
     parser.add_option('-K', '--key', dest='key',
                       help='Key for obtaining an auth token')
+    parser.add_option('-b', '--bench-clients', action='append',
+                      metavar='<ip>:<port>',
+                      help=('A string of the form "<ip>:<port>" which matches '
+                            'the arguments supplied to a swift-bench-client '
+                            'process.  This argument must be specified '
+                            'once per swift-bench-client you want to '
+                            'utilize.'))
     parser.add_option('-u', '--url', dest='url',
                       help='Storage URL')
     parser.add_option('-c', '--concurrency', dest='concurrency',
@@ -125,6 +135,8 @@ if __name__ == '__main__':
         options.put_concurrency = options.concurrency
         options.get_concurrency = options.concurrency
         options.del_concurrency = options.concurrency
+    options.containers = ['%s_%d' % (options.container_name, i)
+                          for i in xrange(int(options.num_containers))]
 
     def sigterm(signum, frame):
         sys.exit('Termination signal received.')
@@ -145,5 +157,13 @@ if __name__ == '__main__':
                                   '%(message)s')
     loghandler.setFormatter(logformat)
 
-    controller = BenchController(logger, options)
+    if options.use_proxy:
+        create_containers(logger, options)
+
+    controller_class = DistributedBenchController if options.bench_clients \
+        else BenchController
+    controller = controller_class(logger, options)
     controller.run()
+
+    if options.delete:
+        delete_containers(logger, options)