summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorAlex Rudyy <orudyy@apache.org>2015-04-15 10:02:01 +0000
committerAlex Rudyy <orudyy@apache.org>2015-04-15 10:02:01 +0000
commite0ee147cf7646988a8876860c151e8c593ef378a (patch)
tree18e56b4c49b9ad4bc03943cac1c6e8075cc43553
parentdb3ef0f8eab2d3e6bc2001c6dcbf0fd896537314 (diff)
downloadqpid-python-e0ee147cf7646988a8876860c151e8c593ef378a.tar.gz
QPID-6481: Move perftests docbook into java source tree
git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1673705 13f79535-47bb-0310-9956-ffa450edef68
-rw-r--r--qpid/doc/book/src/java-perftests/JMS-Performance-Test-Framework.xml304
-rw-r--r--qpid/doc/book/src/java-perftests/Makefile20
2 files changed, 0 insertions, 324 deletions
diff --git a/qpid/doc/book/src/java-perftests/JMS-Performance-Test-Framework.xml b/qpid/doc/book/src/java-perftests/JMS-Performance-Test-Framework.xml
deleted file mode 100644
index 51a777c94b..0000000000
--- a/qpid/doc/book/src/java-perftests/JMS-Performance-Test-Framework.xml
+++ /dev/null
@@ -1,304 +0,0 @@
-<?xml version="1.0"?>
-<!--
-
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
--->
-
-<book xmlns:xi="http://www.w3.org/2001/XInclude">
- <title>JMS Performance Test Framework</title>
- <chapter>
- <para>
- The distributed test (aka Perf Test) framework was written for
- testing the performance of a JMS provider in various common scenarios.
- Although it was originally written for the purpose of testing Qpid,
- it can be used to test the performance of any JMS provider with minimal configuration changes.
- <para>
- </para>
- This document explains how to use the framework.
- </para>
-
- <section id="how-it-works">
- <title>How it works</title>
- <para>
- First, you need to <emphasis>run a message broker</emphasis>. This can be Qpid, ActiveMQ etc
- (although see <xref linkend="caveats-for-non-qpid-jms-providers"/>).
- All messages are sent using the JMS API.
- <para>
- </para>
- Then run a <emphasis>Perf Test Controller</emphasis>, providing the details of the test in one or more
- JSON or Javascript files (see <xref linkend="test-definitions">Test definitions</xref>).
- <para>
- </para>
- Now <emphasis>run one or more Perf Test Client processes</emphasis>. These will be responsible for
- sending/receiving the messages once the test starts. For convenience, you can
- instead configure the Controller to start clients in-process. The clients and
- the controller communicate using queues on the message broker.
- <para>
- </para>
- The test results are written to CSV files or optionally to a
- database (see <xref linkend="writing-results-to-a-jdbc-database"/>).
- <para>
- </para>
- You can <emphasis>use the qpid-perftests-visualisation tool</emphasis>
- (<xref linkend="visualising-test-results"/>) to create charts from the results.
- </para>
- </section>
-
- <section id="test-definitions">
- <title>Test definitions</title>
- <para>
- Test definition files specify details about the messages to send,
- how many connections and sessions to use etc. There are a lot of options
- available - see the .js and .json files under the <filename>perftests/etc/testdefs/</filename> folder for examples.
- </para>
- <para>
- Each JSON file contains a list of tests, expressed as a JSON structure. Alternatively, JavaScript can be used
- to generate this data structure. If a file has a .js extension it is parsed as JavaScript and the
- resulting object <code>jsonObject</code> used as the test specification. JavaScript is useful for reducing
- duplication in test specifications (e.g. by looping or by moving repeating literals into variables).
- </para>
- <para>
- If the ControllerRunner is pointed at a directory instead of a file, each test specification file in that
- directory is used.
- </para>
- </section>
-
- <section id="example-usage">
- <title>Example usage</title>
- <para>
- The <filename>perftests/etc/</filename> folder contains shell scripts that can be used to run the performance
- tests and visualise the results. It also contains sub-folders for test config
- and chart definitions.
- </para>
- </section>
-
- <section id="instructions">
- <title>Instructions</title>
- <para>
- <itemizedlist>
- <listitem>Extract the perftests archive</listitem>
- <listitem>Start your JMS broker</listitem>
- <listitem>cd into the <filename>perftests/etc/</filename> folder</listitem>
- <listitem>
- <para>To run the Controller and clients in a single process, run the following command:
- </para>
- <screen>
- $ java -cp ".:../lib/*:/path/to/your-jms-client-jars/*" \
- -Dqpid.dest_syntax=BURL \ # used if the test specifications use Qpid's BURL format for queues
- org.apache.qpid.disttest.ControllerRunner \
- jndi-config=perftests-jndi.properties \
- test-config=/path/to/test-config.json \
- distributed=false
- </screen>
- <para>
- When the test is complete, the CSV files containing the results are written to
- the current directory.
- </para>
- </listitem>
- </itemizedlist>
- </para>
-
- <section id="running-clients-in-a-separate-process">
- <title>Running the clients in a separate process</title>
- <para>
- When using a large number of clients, you may get more representative
- performance results if the clients are distributed among multiple processes,
- potentially on multiple machines. To do this:
- <itemizedlist>
- <listitem>Run the Controller, providing <envar>distributed=true</envar>.</listitem>
- <listitem>
- <para>Run your clients (assuming you want to launch 10 logical clients in this process):</para>
- <screen>
- $ cd perftests/etc
- $ java -cp ".:../lib/*:/path/to/your-jms-client-jars/*" \
- -Dqpid.dest_syntax=BURL \
- org.apache.qpid.disttest.ClientRunner \
- jndi-config=perftests-jndi.properties \
- number-of-clients=10
- </screen>
- </listitem>
- </itemizedlist>
- </para>
- </section>
- </section>
-
- <section id="writing-results-to-a-jdbc-database">
- <title>Writing results to a JDBC database</title>
- <para>
- For most use cases, writing results to a CSV file is acceptable. However, there are some cases
- where it is desired to write to a database instead. For example, if you need to keep track of how
- results have varied over time, writing to a database table allows this to be easily discovered by
- running a SQL query and/or producing a time series chart with the visualisation tool
- (see <xref linkend="visualising-test-results-queries"/>).
- </para>
- <para>
- To write results to a database:
- <itemizedlist>
- <listitem>Add <code>writeToDb=true</code> to the <code>ControllerRunner</code> parameters</listitem>
- <listitem>
- Add <code>jdbcDriverClass</code> and <code>jdbcUrl</code> properties to your JNDI configuration file
- (obviously adding the JDBC driver class to your classpath too).
- </listitem>
- <listitem>
- Note that the framework automatically creates the results table <code>RESULTS</code>
- if it does not already exist.
- </listitem>
- </itemizedlist>
- </para>
- </section>
-
- <section id="caveats-for-non-qpid-jms-providers">
- <title>Caveats for non-Qpid JMS providers</title>
- <para>
- If you are not using the Qpid broker, you must create one or more queues before
- running the test. This is necessary because you can't use Qpid's API to create
- queues on the broker. The queues are:
- <itemizedlist>
- <listitem>
- The controller queue. You can specify the physical name of this in
- <filename>perftests/etc/perftests-jndi.properties</filename>.
- This queue is used by the clients to register with the Controller and to send results to it.
- </listitem>
- <listitem>
- The queue(s) used by your JSON test configuration
- (unless you have configured a vendor-specific queue creator).
- </listitem>
- </itemizedlist>
- </para>
- <para>
- You must also override the Controller's default queue creator using the system
- property <envar>qpid.disttest.queue.creator.class</envar>. Provide the class name of an
- implementation of <classname>org.apache.qpid.disttest.jms.QueueCreator</classname>, or
- <classname>org.apache.qpid.disttest.jms.NoOpQueueCreator</classname> if you are going to create and
- delete the queues manually.
- </para>
- <para>
- You can also omit the <envar>qpid.dest_syntax</envar> system property if your JMS provider is
- not Qpid.
- </para>
- </section>
-
- <section id="when-production-is-slower-than-consumption">
- <title>When message production is slower than consumption</title>
- <para>
- A given test configuration may cause messages to be produced faster than they are consumed.
- Unless you deal with this, the broker may become overwhelmed (e.g. running out of disk space or memory).
- The steps you can take to mitigate this are:
- <itemizedlist>
- <listitem>
- Apply a small delay between each message publication, using the Producer JSON property <code>_interval</code>.
- </listitem>
- <listitem>
- Use your JMS provider's built-in features for throttling incoming messages.
- In the case of Qpid, this is done by configuring the queue to enforce flow control, like so:
- <screen><![CDATA[
- {
- "_tests":[
- {
- "_name": "My test",
- "_queues":[
- {
- "_name": "direct://amq.direct//myTestQueue",
- "_attributes":
- {
- "x-qpid-capacity": 10000000,
- "x-qpid-flow-resume-capacity": 8000000
- }
- }
- ],
- ...
- }
- }]]>
- </screen>
- </listitem>
- </itemizedlist>
- </para>
- </section>
-
- <section id="visualising-test-results">
- <title>Visualising test results</title>
- <para>
- The module visualisation-jfc can be used to generate charts as .png files from the results
- produced by running the perf tests (or in fact from any CSV file with a header row or database table).
- At runtime the visualisation module accepts the following input:
- <itemizedlist>
- <listitem>The test results (either a CSV file or a database table)</listitem>
- <listitem>
- The chart definition, which is a properties file specifying settings such as the query to run against the data,
- and the type of chart to generate.
- <para>
- The quickest way to create a new chart definition is to base it on an existing example in
- <filename>perftests/etc/chartdefs/</filename>.
- </para>
- </listitem>
- </itemizedlist>
- </para>
- <section>
- <title>Chart types</title>
- <para>
- Currently, the available chart types are:
- <itemizedlist>
- <listitem>LINE</listitem>
- <listitem>LINE3D</listitem>
- <listitem>BAR</listitem>
- <listitem>BAR3D</listitem>
- <listitem>XYLINE</listitem>
- <listitem>TIMELINE</listitem>
- <listitem>STATISTICAL_BAR</listitem>
- </itemizedlist>
- </para>
- </section>
- <section id="visualising-test-results-queries">
- <title>Queries</title>
- <para>
- The query in the chart definition file is an ANSI SQL query. If reading from a CSV file the SQL table name
- is simply the file name and the SQL column names are the CSV header row entries.
- </para>
- </section>
- <section>
- <title>Running the visualisation tool</title>
- <para>
- Here is an example of how to run the visualisation tool.
- <screen>
-$ cd perftests/etc
-$ BASE_DIR=`pwd`
-$ java -cp "${BASE_DIR}:/path/to/extracted/visualistion-module/lib/*" \
- -Djava.awt.headless=true -Dlog4j.configuration=file:log4j.properties \
- -DcsvCurrentDir=/path/to/csv-files-directory/ \ # referenced in chart definition file
- -DcsvBaselineDir=/path/to/baseline-csv-files-directory/ \ # referenced in chart definition file
- -DbaselineName="My baseline name" \ # referenced in chart definition file
- org.apache.qpid.disttest.charting.ChartingUtil \
- chart-defs=/path/to/chartdefs-files/ \
- </screen>
- </para>
- <para>
- To fetch the results from a database table instead of a CSV file, modify the visualistion tool
- parameters like so:
- <screen>
- $ java -cp "${BASE_DIR}:/path/to/extracted/visualistion-module/lib/*" \
- ...
- org.apache.qpid.disttest.charting.ChartingUtil \
- jdbcUrl=jdbc:your-jdbc-url
- jdbcDriverClass=your.jdbc.driver.Classname
- </screen>
- </para>
- </section>
- </section>
- </chapter>
-</book>
diff --git a/qpid/doc/book/src/java-perftests/Makefile b/qpid/doc/book/src/java-perftests/Makefile
deleted file mode 100644
index 0266a0f54d..0000000000
--- a/qpid/doc/book/src/java-perftests/Makefile
+++ /dev/null
@@ -1,20 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-
-include ../Makefile.inc