1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
|
$Id$
Release 1.0 - January 2003
***********************************
* Performance Test Overview *
***********************************
This directory contains performance tests for measuring round-trip
latency statistics of ACE synchronous messaging using unmarshalled
ACE_CDR::Octet.
In terms of protocol and service type, three different tests are
available:
* TCP over SOCK_STREAM
* SCTP over SOCK_STREAM
* SCTP over SOCK_SEQPACK
All three of these tests share the same architecture, which is
described in the following section.
***************************
* Test Architecture *
***************************
As mentioned above, the performance tests measure round-trip latency
statistics. We are talking about the round trip between two service
endpoints, a client and a server. The client sends the server an
arbitrary message with a specific length---up to 2^16 bytes---and the
server responds with a 2-byte reply. The client measures and records
the time it takes to complete this exchange. The whole test is then
repeated through many iterations, and finally the client outputs the
resulting statistics.
In detail,
1. The server program is started first. It creates a passive-mode
socket and runs an infinite loop waiting for clients to connect.
2. The client program is started next. It may be configured with
numerous parameters, which include:
* the hostname and port where the server's passive-mode socket
is listening,
* the desired message length, which may be any power of two
between 2^0 and 2^16,
* the desired number of iterations (by default 1 million)
3. The client connects to the server's passive-mode socket. If all
goes well, the client and server will each obtain a data-mode
socket that serves as a communication endpoint.
4. The client sends an initial header message containing the intended
number of iterations and the message length (which will be
the same for each iteration). Armed with this information, the
server spawns a separate thread to handle these iterations.
5. The client creates an in-memory data structure that serves as a
histogram. The histogram is configured with a range of possible
round-trip latencies, and this range is divided into a number of
"bins". The latency measured for each test entry will either
fall into one of these "bins", or it will fall outside the entire
range and will be logged as an outlier.
6. The client and server repeat the following interaction for each
test iteration.
a. The client starts a stopwatch.
b. The client sends the server an arbitrary message of the
established length.
c. After receiving this entire message from the client, the
server sends the client a 2-byte reply.
d. After receiving the 2-byte reply from the server, the client
stops the stopwatch.
e. The round-trip latency, as measureed by the stopwatch, is
logged to the in-memory histogram.
7. After all these iterations are complete, the client disconnects
from the server. The client then takes the in-memory histogram
and dumps its text representation to STDOUT.
8. The client exits. The server thread dedicated to this client
exits. The main thread of the server continues to run an infinite
loop waiting for more clients to connect to the passive-mode
socket.
**************************
* Test Executables *
**************************
You must use a different pair of client and server executables
depending on whether you desire to use the SOCK_STREAM service or the
SOCK_SEQPACK service.
The SOCK_STREAM client and server are called SOCK_STREAM_clt and
SOCK_STREAM_srv, respectively. The SOCK_SEQPACK client and server are
called SOCK_SEQPACK_clt and SOCK_SEQPACK_srv, respectively.
Each of these executables uses SCTP as its default protocol. As an
extra feature, SOCK_STREAM_clt and SOCK_STREAM_srv offer TCP as an
alternate protocol. The choice between TCP and SCTP is exposed via
the '-t' option. (Individual instances of SOCK_STREAM_clt and
SOCK_STREAM_srv must use the same protocol in order to connect and
function.)
Any of the four executables will show usage information if you run it
with the '-h' option. For instance, here is the usage message
produced by SOCK_STREAM_clt:
./SOCK_STREAM_clt - Measures round trip latency statistics of ACE synchronous
messaging (SOCK_Stream) using unmarshalled ACE_CDR::Octet.
USAGE: ./SOCK_STREAM_clt [ -<flag> [<val>] | --<name> [<val>] ]...
Flag Args Option-Name Default
-c int test-iterations 1000000
-n none test-enable-nagle NO NAGLING
-t str (sctp|tcp) test-transport-protocol sctp
-m dbl histogram-min-bin 0
-M dbl histogram-max-bin 10000
-x int histogram-num-outliers 100
-b int histogram-bin-count 1000
-C int client-port assigned by kernel
-i str client-connect-addr INADDR_ANY
-p int server-port 45453
-H str server-host localhost
-s int (0-16) payload-size-power-of-2 <MUST SET VALUE>
-h none help
For each option, the long "Option-Name" may be prefixed by two dashes
and used on the command line in place of the shorter "Flag". Hence
./SOCK_STREAM_clt -s 5
is equivalent to
./SOCK_STREAM_clt --payload-size-power-of-2 5
The options shown above for SOCK_STREAM_clt (and the similar options
available for SOCK_SEQPACK_clt) are discussed in the next section.
*****************************
* Client-Side Options *
*****************************
The following options are available for both SOCK_STREAM_clt and
SOCK_SEQPACK_clt.
test-iterations [ -c int ]
the number of times to repeat the round-trip-latency test
loop. The default is 1000000.
test-enable-nagle [ -n ]
Enable Nagle's algorithm (which is disabled by default).
histogram-min-bin [ -m double ]
the lower boundary on the range of the histogram.
The default is 0. The unit of measurement is the millisecond.
histogram-max-bin [ -M double ]
the upper boundary on the range of the histogram.
The default is 10000. The unit of measurement is the
millisecond.
histogram-num-outliers [ -x int ]
the maximum number of outliers to maintain in the histogram.
The default is 100.
histogram-bin-count [ -b int ]
the number of histogram bins. The default is 1000. The width
of each bin will be
(histogram-max-bin - histogram-min-bin) / histogram-bin-count
client-port [ -C int ]
client-connect-addr [ -i string ]
the port and network address where the client's data-mode
socket is to be bound. By default, the port number is
assigned by the kernel, and the socket is bound to all network
interfaces.
server-port [ -p int ]
server-host [ -H string ]
the port and hostname where the server's passive-mode socket
is bound. The default port is 45453 and the default host is
localhost.
payload-size-power-of-2 [ -s int ]
an integer X. The size of the payload will be 2^X bytes.
help [ -h ]
Show usage message.
The following option is available only for SOCK_STREAM_clt:
test-transport-protocol [ -t (sctp|tcp) ]
the protocol for the test. The default is sctp.
In contrast, SOCK_SEQPACK_clt always uses SCTP as its protocol.
*****************************
* Server-Side Options *
*****************************
The following options are available for both SOCK_STREAM_srv and
SOCK_SEQPACK_srv:
test-enable-nagle [ -n ]
Enable Nagle's algorithm (which is disabled by default).
server-port [ -p int ]
the port where the server's passive-mode socket will be
bound. The default is 45453.
help [ -h ]
Show usage message.
The following option is available for both SOCK_STREAM_srv and
SOCK_SEQPACK_srv, but the latter offers additional functionality:
server-accept-addr [ -a w.x.y.z ] (for SOCK_STREAM_srv)
[ -a w.x.y.z,a.b.c.d,... ] (for SOCK_SEQPACK_srv)
the network address (or addresses) to which the server's
passive-mode socket will be bound.
The default value of INADDR_ANY should be sufficient for
machines that have only one network interface.
If your machine has two interfaces, and you wish to bind the
socket only to one of these, then you may explicitly specify
the desired interface with an expression such as
"-a 192.168.221.104" or "-a 192.168.1.43".
If your machine has three or more interfaces, and you wish to
bind the socket only to a subset of two or more, AND you are
using SOCK_SEQPACK_srv, THEN you may explicitly specify the
desired interface with an expression such as
"-a 192.168.221.104,192.168.1.43". The argument here is a
comma-delimited list of dotted-decimal IPv4 addresses with no
whitespace interspersed. This level of control is not
possible with SOCK_STREAM_srv.
The following option is available only for SOCK_STREAM_srv:
test-transport-protocol [ -t (sctp|tcp) ]
the protocol for the test. The default is sctp.
In contrast, SOCK_SEQPACK_srv always uses SCTP as its protocol.
***************************
* Test Walk-Through *
***************************
This section shows a walk-through of a typical scenario using
SOCK_STREAM_clt and SOCK_STREAM_srv to run an SCTP performance test.
You can run the equivalent TCP performance test by adding "-t tcp" to
the command line of both programs. Alternately, you can run a
SOCK_SEQPACK-based SCTP performance test by substituting
SOCK_SEQPACK_clt and SOCK_SEQPACK_srv for SOCK_STREAM_clt and
SOCK_STREAM_srv, respectively.
In this scenario, both client and server run on the same machine. We
use a message size of 2^7 bytes and a relatively small number of
iterations, 1000, in order to make the test go quickly. The server's
passive-mode socket binds to the default port, 45453, which is also
the default port expected by the client. We invoke no other special
or unusual options.
For clarity, we envision starting the client and server in two
separate shells.
The steps of the walk-through are as follows:
1. Start the server.
$ ./SOCK_STREAM_srv
(12761|1024) Accepting connections on port 45453
on interface INADDR_ANY using IPPROTO_SCTP
(12761|1024) starting server at port 45453
(12761|1024) select timed out
(12761|1024) select timed out
(12761|1024) select timed out
(12761|1024) select timed out
...
The server will continue to print, periodically, the message
"select timed out" as it waits for clients to connect.
2. Start the client.
$ ./SOCK_STREAM_clt -s 7 -c 1000
3. Observing the debugging output from the server, you should see
evidence that the client has connected.
(12761|1024) select timed out
(12761|1024) select timed out
(12761|1024) select timed out
(12761|1024) spawning server
(12761|1024) client utica-b connected from 32768
(12761|1024) Test for 1100 iterations
(12761|1024) select timed out
(12761|1024) select timed out
Note that the server expects 1100 iterations, even though we
configured the client for only 1000 iterations. The reason for
this discrepencany is that the client always runs 100 primer
iterations before the requested test iterations.
4. When the 1100 iterations are complete, observing the output from
the client, you should see the histogram:
Histogram ACE Unmarshalled Msg Synchronous Latency Test
(Message Size 128, Message Type octet)
version: 1.1
minimum: 41
maximum: 60182
mean: 106.858
variance: 3.62659e+06
num_points: 1000
num_bins: 1000 0 10000
Low - High Count Fraction Cumulative
below - 0.000 : 0 0.000 0.000
0.000 - 10.000 : 0 0.000 0.000
10.000 - 20.000 : 0 0.000 0.000
20.000 - 30.000 : 0 0.000 0.000
30.000 - 40.000 : 0 0.000 0.000
40.000 - 50.000 : 988 0.988 0.988
50.000 - 60.000 : 5 0.005 0.993
60.000 - 70.000 : 2 0.002 0.995
70.000 - 80.000 : 0 0.000 0.995
80.000 - 90.000 : 1 0.001 0.996
90.000 - 100.000 : 0 0.000 0.996
100.000 - 110.000 : 0 0.000 0.996
...
9960.000 - 9970.000 : 0 0.000 0.999
9970.000 - 9980.000 : 0 0.000 0.999
9980.000 - 9990.000 : 0 0.000 0.999
9990.000 - 10000.000 : 0 0.000 0.999
10000.000 - above : 1 0.001 1.000
outliers:
60182.000
******************************************
* Running a Complete Test Spectrum *
******************************************
People who are interested in round-trip latency often want to see a
"spectrum" of statistics for a range of payload sizes. This directory
includes a script, run_spectrum.pl, that automates running
SOCK_STREAM_clt or SOCK_SEQPACK_clt multiple times in order to
generate a spectrum. (The appropriate server program must be started
manually.)
The run_spectrum.pl script offers embedded documentation. To see the
full documentation, please run
./run_spectrum.pl --manual
************************
* Sample Results *
************************
The file sample-spectrum.png plots the spectrum data for a test run at
LM ATL facilities on the following systems: two Linux 2.4.18, 1600 MHz
Pentium 4 machines on an isolated 100Mbps Ethernet network.
The test conditions include several critical parameters that were set
as follows:
sctp_rto_initial = 20
sctp_rto_min = 20
sctp_rto_max = 20
Testing has shown that these parameters affect maximum round-trip
latency.
|