1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
|
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.5: http://docutils.sourceforge.net/" />
<title>Plac: Parsing the Command Line the Easy Way</title>
<meta name="author" content="Michele Simionato" />
<meta name="date" content="June 2010" />
<style type="text/css">
.first {
margin-top: 0 }
.last {
margin-bottom: 0 }
a.toc-backref {
text-decoration: none ;
color: black }
dd {
margin-bottom: 0.5em }
div.abstract {
margin: 2em 5em }
div.abstract p.topic-title {
font-weight: bold ;
text-align: center }
div.attention, div.caution, div.danger, div.error, div.hint,
div.important, div.note, div.tip, div.warning {
margin: 2em ;
border: medium outset ;
padding: 1em }
div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title {
color: red ;
font-weight: bold ;
font-family: sans-serif }
div.hint p.admonition-title, div.important p.admonition-title,
div.note p.admonition-title, div.tip p.admonition-title {
font-weight: bold ;
font-family: sans-serif }
div.dedication {
margin: 2em 5em ;
text-align: center ;
font-style: italic }
div.dedication p.topic-title {
font-weight: bold ;
font-style: normal }
div.figure {
margin-left: 2em }
div.footer, div.header {
font-size: smaller }
div.system-messages {
margin: 5em }
div.system-messages h1 {
color: red }
div.system-message {
border: medium outset ;
padding: 1em }
div.system-message p.system-message-title {
color: red ;
font-weight: bold }
div.topic {
margin: 2em }
hr {
width: 75% }
ol.simple, ul.simple {
margin-bottom: 1em }
ol.arabic {
list-style: decimal }
ol.loweralpha {
list-style: lower-alpha }
ol.upperalpha {
list-style: upper-alpha }
ol.lowerroman {
list-style: lower-roman }
ol.upperroman {
list-style: upper-roman }
p.caption {
font-style: italic }
p.credits {
font-style: italic ;
font-size: smaller }
p.label {
white-space: nowrap }
p.topic-title {
font-weight: bold }
pre.address {
margin-bottom: 0 ;
margin-top: 0 ;
font-family: serif ;
font-size: 100% }
pre.line-block {
font-family: serif ;
font-size: 100% }
pre.literal-block, pre.doctest-block {
background-color: #eeeeee }
span.classifier {
font-family: sans-serif ;
font-style: oblique }
span.classifier-delimiter {
font-family: sans-serif ;
font-weight: bold }
span.interpreted {
font-family: sans-serif }
span.option-argument {
font-style: italic }
span.pre {
white-space: pre }
span.problematic {
color: red }
table {
margin-top: 0.5em ;
margin-bottom: 0.5em }
table.citation {
border-left: solid thin gray ;
padding-left: 0.5ex }
table.docinfo {
margin: 2em 4em }
table.footnote {
border-left: solid thin black ;
padding-left: 0.5ex }
td, th {
padding-left: 0.5em ;
padding-right: 0.5em ;
vertical-align: top }
th.docinfo-name, th.field-name {
font-weight: bold ;
text-align: left ;
white-space: nowrap }
h1 tt, h2 tt, h3 tt, h4 tt, h5 tt, h6 tt {
font-size: 100% }
tt {
background-color: #eeeeee }
ul.auto-toc {
list-style-type: none }
/*
Additional styles for "modern"-style of DocFactory.
:Author: Gunnar Schwant
:Contact: g.schwant@gmx.de
*/
.first {
font-size: 10pt }
.last {
font-size: 10pt }
a {
text-decoration: none }
a.reference {
color: #00009F }
a:hover {
background-color: #00009F ;
color: white }
body {
font-family: arial,helvetica,univers ;
font-size: 10pt ;
padding-top: 0.6cm ;
margin-left:0.5cm ;
margin-right:0.5cm ;
margin-bottom:0.5cm }
dd {
font-size: 10pt ;
padding-top: 0.1cm
}
dt {
font-size: 10pt ;
font-weight: bold ;
background-color: #6FC7FB ;
padding-left: 0.1cm ;
padding-top: 0.1cm ;
padding-bottom: 0.1cm }
div.abstract {
font-size: 10pt }
div.abstract p.topic-title {
font-size: 10pt }
div.attention, div.caution, div.danger, div.error, div.hint,
div.important, div.note, div.tip, div.warning {
font-size: 10pt }
div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
margin-top: 0em ;
font-size: 12pt ;
font-family: arial,helvetica,univers }
div.dedication {
font-size: 10pt }
div.dedication p.topic-title {
font-size: 10pt }
div.figure {
font-size: 10pt }
div.footer, div.header {
font-size: 8pt }
div.system-messages {
font-size: 10pt }
div.system-messages h1 {
font-size: 12pt }
div.system-message {
font-size: 10pt }
div.system-message p.system-message-title {
font-size: 10pt }
div.topic {
font-size: 10pt }
h1, h2, h3, h4, h5, h6 {
padding-top: 0.5cm ;
page-break-after: avoid ;
font-family: arial,helvetica,univers }
h1 {
font-size: 18pt }
h1.title {
color: white ;
background-color: #00009F ;
padding-top: 0cm }
h2 {
font-size: 16pt }
h2.subtitle {
padding-top: 0cm }
h3 {
font-size: 14pt }
h4 {
font-size: 12pt }
h5, h6 {
font-size: 10pt }
hr {
width: 100%;
page-break-after: always }
li {
padding-top: 1mm ;
padding-bottom: 1mm }
ol.simple, ul.simple {
font-size: 10pt }
ol.arabic {
font-size: 10pt }
ol.loweralpha {
font-size: 10pt }
ol.upperalpha {
font-size: 10pt }
ol.lowerroman {
font-size: 10pt }
ol.upperroman {
font-size: 10pt }
p.caption {
font-size: 10pt }
p.credits {
font-style: italic ;
font-size: 8pt }
p.label {
font-size: 10pt }
p.topic-title {
font-size: 10pt }
pre.address {
font-family: arial,helvetica,univers ;
font-size: 10pt }
pre.line-block {
font-size: 10pt }
pre.literal-block, pre.doctest-block {
border-width: 1pt ;
border-style: solid ;
border-color: #999999 ;
color: #0000C0 ;
background-color: #ffffe0 ;
font-size: 9pt }
span.classifier {
font-size: 10pt ;
font-family: arial,helvetica,univers }
span.classifier-delimiter {
font-size: 10pt ;
font-family: arial,helvetica,univers }
span.field-argument {
font-size: 10pt }
span.interpreted {
font-size: 10pt ;
font-family: arial,helvetica,univers }
span.option-argument {
font-size: 10pt }
span.problematic {
font-size: 10pt }
table {
font-size: 10pt ;
border-collapse: collapse ;
border-width: 1.5pt ;
border-color: #003366 }
table.citation {
font-size: 10pt }
table.docinfo {
font-size: 10pt }
table.footnote {
font-size: 8pt ;
text-align: left }
table.table {
width: 100% }
th {
border-width: 1.5pt }
td {
border-width: 1pt }
td, th {
font-size: 10pt ;
border-style: thin ;
border-color: #003366 }
td.docinfo-name, th.field-name {
font-size: 10pt }
h1 tt, h2 tt, h3 tt, h4 tt, h5 tt, h6 tt {
font-size: 10pt }
</style>
</head>
<body>
<div class="document" id="plac-parsing-the-command-line-the-easy-way">
<h1 class="title">Plac: Parsing the Command Line the Easy Way</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Michele Simionato</td></tr>
<tr class="field"><th class="docinfo-name">E-mail:</th><td class="field-body"><a class="reference external" href="mailto:michele.simionato@gmail.com">michele.simionato@gmail.com</a></td>
</tr>
<tr><th class="docinfo-name">Date:</th>
<td>June 2010</td></tr>
<tr class="field"><th class="docinfo-name">Download page:</th><td class="field-body"><a class="reference external" href="http://pypi.python.org/pypi/plac">http://pypi.python.org/pypi/plac</a></td>
</tr>
<tr class="field"><th class="docinfo-name">Project page:</th><td class="field-body"><a class="reference external" href="http://micheles.googlecode.com/hg/plac/doc/plac.html">http://micheles.googlecode.com/hg/plac/doc/plac.html</a></td>
</tr>
<tr class="field"><th class="docinfo-name">Requires:</th><td class="field-body">Python 2.3+</td>
</tr>
<tr class="field"><th class="docinfo-name">Installation:</th><td class="field-body"><tt class="docutils literal"><span class="pre">easy_install</span> <span class="pre">-U</span> <span class="pre">plac</span></tt></td>
</tr>
<tr class="field"><th class="docinfo-name">License:</th><td class="field-body">BSD license</td>
</tr>
</tbody>
</table>
<div class="contents topic" id="contents">
<p class="topic-title first">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#the-importance-of-scaling-down" id="id1">The importance of scaling down</a></li>
<li><a class="reference internal" href="#scripts-with-required-arguments" id="id2">Scripts with required arguments</a></li>
<li><a class="reference internal" href="#scripts-with-default-arguments" id="id3">Scripts with default arguments</a></li>
<li><a class="reference internal" href="#scripts-with-options-and-smart-options" id="id4">Scripts with options (and smart options)</a></li>
<li><a class="reference internal" href="#scripts-with-flags" id="id5">Scripts with flags</a></li>
<li><a class="reference internal" href="#plac-for-python-2-x-users" id="id6">plac for Python 2.X users</a></li>
<li><a class="reference internal" href="#more-features" id="id7">More features</a></li>
<li><a class="reference internal" href="#a-realistic-example" id="id8">A realistic example</a></li>
<li><a class="reference internal" href="#keyword-arguments" id="id9">Keyword arguments</a></li>
<li><a class="reference internal" href="#final-example-a-shelve-interface" id="id10">Final example: a shelve interface</a></li>
<li><a class="reference internal" href="#plac-vs-argparse" id="id11">plac vs argparse</a></li>
<li><a class="reference internal" href="#plac-vs-the-rest-of-the-world" id="id12">plac vs the rest of the world</a></li>
<li><a class="reference internal" href="#the-future" id="id13">The future</a></li>
<li><a class="reference internal" href="#trivia-the-story-behind-the-name" id="id14">Trivia: the story behind the name</a></li>
</ul>
</div>
<div class="section" id="the-importance-of-scaling-down">
<h1><a class="toc-backref" href="#id1">The importance of scaling down</a></h1>
<p>There is no want of command line arguments parsers in the Python
world. The standard library alone contains three different modules:
<a class="reference external" href="http://docs.python.org/library/getopt.html">getopt</a> (from the stone age),
<a class="reference external" href="http://docs.python.org/library/optparse.html">optparse</a> (from Python 2.3) and <a class="reference external" href="http://argparse.googlecode.com">argparse</a> (from Python 2.7). All of
them are quite powerful and especially <a class="reference external" href="http://argparse.googlecode.com">argparse</a> is an industrial
strength solution; unfortunately, all of them feature a non-zero learning
curve and a certain verbosity. They do not scale down well, at
least in my opinion.</p>
<p>It should not be necessary to stress the importance <a class="reference external" href="http://www.welton.it/articles/scalable_systems">scaling down</a>;
nevertheless, a lot of people are obsessed with features and concerned with
the possibility of scaling up, forgetting the equally important
issue of scaling down. This is an old meme in
the computing world: programs should address the common cases simply and
simple things should be kept simple, while at the same keeping
difficult things possible. <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> adhere as much as possible to this
philosophy and it is designed to handle well the simple cases, while
retaining the ability to handle complex cases by relying on the
underlying power of <a class="reference external" href="http://argparse.googlecode.com">argparse</a>.</p>
<p>Technically <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> is just a simple wrapper over <a class="reference external" href="http://argparse.googlecode.com">argparse</a> which hides
most of its complexity by using a declarative interface: the argument
parser is inferred rather than written down by imperatively. Still, <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> is
surprisingly scalable upwards, even without using the underlying
<a class="reference external" href="http://argparse.googlecode.com">argparse</a>. I have been using Python for 8 years and in my experience
it is extremely unlikely that you will ever need to go beyond the
features provided by the declarative interface of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>: they should
be more than enough for 99.9% of the use cases.</p>
<p><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> is targetting especially unsophisticated users,
programmers, sys-admins, scientists and in general people writing
throw-away scripts for themselves, choosing the command line
interface because it is the quick and simple. Such users are not
interested in features, they are interested in a small learning curve:
they just want to be able to write a simple command line tool from a
simple specification, not to build a command-line parser by
hand. Unfortunately, the modules in the standard library forces them
to go the hard way. They are designed to implement power user tools
and they have a non-trivial learning curve. On the contrary, <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>
is designed to be simple to use and extremely concise, as the examples
below will show.</p>
</div>
<div class="section" id="scripts-with-required-arguments">
<h1><a class="toc-backref" href="#id2">Scripts with required arguments</a></h1>
<p>Let me start with the simplest possible thing: a script that takes a
single argument and does something to it. It cannot get simpler
than that, unless you consider a script without command-line
arguments, where there is nothing to parse. Still, it is a use
case <em>extremely common</em>: I need to write scripts like that nearly
every day, I wrote hundreds of them in the last few years and I have
never been happy. Here is a typical example of code I have been
writing by hand for years:</p>
<pre class="literal-block">
# example1.py
def main(dsn):
"Do something with the database"
print(dsn)
# ...
if __name__ == '__main__':
import sys
n = len(sys.argv[1:])
if n == 0:
sys.exit('usage: python %s dsn' % sys.argv[0])
elif n == 1:
main(sys.argv[1])
else:
sys.exit('Unrecognized arguments: %s' % ' '.join(sys.argv[2:]))
</pre>
<p>As you see the whole <tt class="docutils literal"><span class="pre">if</span> <span class="pre">__name__</span> <span class="pre">==</span> <span class="pre">'__main__'</span></tt> block (nine lines)
is essentially boilerplate that should not exist. Actually I think
the language should recognize the main function and pass to it the
command-line arguments automatically; unfortunaly this is unlikely to
happen. I have been writing boilerplate like this in hundreds of
scripts for years, and every time I <em>hate</em> it. The purpose of using a
scripting language is convenience and trivial things should be
trivial. Unfortunately the standard library does not help for this
incredibly common use case. Using <a class="reference external" href="http://docs.python.org/library/getopt.html">getopt</a> and <a class="reference external" href="http://docs.python.org/library/optparse.html">optparse</a> does not help,
since they are intended to manage options and not positional
arguments; the <a class="reference external" href="http://argparse.googlecode.com">argparse</a> module helps a bit and it is able to reduce
the boilerplate from nine lines to six lines:</p>
<pre class="literal-block">
# example2.py
def main(dsn):
"Do something on the database"
print(dsn)
# ...
if __name__ == '__main__':
import argparse
p = argparse.ArgumentParser()
p.add_argument('dsn')
arg = p.parse_args()
main(arg.dsn)
</pre>
<p>However saving three lines does not justify introducing the external
dependency: most people will not switch to Python 2.7, which at the time of
this writing is just about to be released, for many years.
Moreover, it just feels too complex to instantiate a class and to
define a parser by hand for such a trivial task.</p>
<p>The <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> module is designed to manage well such use cases, and it is able
to reduce the original nine lines of boiler plate to two lines. With the
<a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> module all you need to write is</p>
<pre class="literal-block">
# example3.py
def main(dsn):
"Do something with the database"
print(dsn)
# ...
if __name__ == '__main__':
import plac; plac.call(main)
</pre>
<p>The <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> module provides for free (actually the work is done by the
underlying <a class="reference external" href="http://argparse.googlecode.com">argparse</a> module) a nice usage message:</p>
<pre class="literal-block">
$ python example3.py -h
</pre>
<pre class="literal-block">
usage: example3.py [-h] dsn
Do something with the database
positional arguments:
dsn
optional arguments:
-h, --help show this help message and exit
</pre>
<p>Moreover <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> manages the case of missing arguments and of too many arguments.
This is only the tip of the iceberg: <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> is able to do much more than that.</p>
</div>
<div class="section" id="scripts-with-default-arguments">
<h1><a class="toc-backref" href="#id3">Scripts with default arguments</a></h1>
<p>The need to have suitable defaults for command-line scripts is quite
common. For instance I have encountered this use case at work hundreds
of times:</p>
<pre class="literal-block">
# example4.py
from datetime import datetime
def main(dsn, table='product', today=datetime.today()):
"Do something on the database"
print(dsn, table, today)
if __name__ == '__main__':
import sys
args = sys.argv[1:]
if not args:
sys.exit('usage: python %s dsn' % sys.argv[0])
elif len(args) > 2:
sys.exit('Unrecognized arguments: %s' % ' '.join(argv[2:]))
main(*args)
</pre>
<p>Here I want to perform a query on a database table, by extracting the
most recent data: it makes sense for <tt class="docutils literal"><span class="pre">today</span></tt> to be a default argument.
If there is a most used table (in this example a table called <tt class="docutils literal"><span class="pre">'product'</span></tt>)
it also makes sense to make it a default argument. Performing the parsing
of the command-line arguments by hand takes 8 ugly lines of boilerplate
(using <a class="reference external" href="http://argparse.googlecode.com">argparse</a> would require about the same number of lines).
With <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> the entire <tt class="docutils literal"><span class="pre">__main__</span></tt> block reduces to the usual two lines:</p>
<pre class="literal-block">
if __name__ == '__main__':
import plac; plac.call(main)
</pre>
<p>In other words, six lines of boilerplate have been removed, and we get
the usage message for free:</p>
<pre class="literal-block">
usage: example5.py [-h] dsn [table] [today]
Do something on the database
positional arguments:
dsn
table
today
optional arguments:
-h, --help show this help message and exit
</pre>
<p><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> manages transparently even the case when you want to pass a
variable number of arguments. Here is an example, a script running
on a database a series of SQL scripts:</p>
<pre class="literal-block">
# example7.py
from datetime import datetime
def main(dsn, *scripts):
"Run the given scripts on the database"
for script in scripts:
print('executing %s' % script)
# ...
if __name__ == '__main__':
import plac; plac.call(main)
</pre>
<p>Here is the usage message:</p>
<pre class="literal-block">
usage: example7.py [-h] dsn [scripts [scripts ...]]
Run the given scripts on the database
positional arguments:
dsn
scripts
optional arguments:
-h, --help show this help message and exit
</pre>
<p>The examples here should have made clear that <em>plac is able to figure out
the command-line arguments parser to use from the signature of the main
function</em>. This is the whole idea behind <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>: if the intent is clear,
let's the machine take care of the details.</p>
<p><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> is inspired to an old Python Cookbook recipe (<a class="reference external" href="http://code.activestate.com/recipes/278844-parsing-the-command-line/">optionparse</a>), in
the sense that it delivers the programmer from the burden of writing
the parser, but is less of a hack: instead of extracting the parser
from the docstring of the module, it extracts it from the signature of
the <tt class="docutils literal"><span class="pre">main</span></tt> function.</p>
<p>The idea comes from the <cite>function annotations</cite> concept, a new
feature of Python 3. An example is worth a thousand words, so here
it is:</p>
<pre class="literal-block">
# example7_.py
from datetime import datetime
def main(dsn: "Database dsn", *scripts: "SQL scripts"):
"Run the given scripts on the database"
for script in scripts:
print('executing %s' % script)
# ...
if __name__ == '__main__':
import plac; plac.call(main)
</pre>
<p>Here the arguments of the <tt class="docutils literal"><span class="pre">main</span></tt> function have been annotated with
strings which are intented to be used in the help message:</p>
<pre class="literal-block">
usage: example7_.py [-h] dsn [scripts [scripts ...]]
Run the given scripts on the database
positional arguments:
dsn Database dsn
scripts SQL scripts
optional arguments:
-h, --help show this help message and exit
</pre>
<p><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> is able to recognize much more complex annotations, as
I will show in the next paragraphs.</p>
</div>
<div class="section" id="scripts-with-options-and-smart-options">
<h1><a class="toc-backref" href="#id4">Scripts with options (and smart options)</a></h1>
<p>It is surprising how few command-line scripts with options I have
written over the years (probably less than a hundred), compared to the
number of scripts with positional arguments I wrote (certainly more
than a thousand of them). Still, this use case cannot be neglected.
The standard library modules (all of them) are quite verbose when it
comes to specifying the options and frankly I have never used them
directly. Instead, I have always relied on the
<a class="reference external" href="http://code.activestate.com/recipes/278844-parsing-the-command-line/">optionparse</a> recipe, which provides a convenient wrapper over
<a class="reference external" href="http://code.activestate.com/recipes/278844-parsing-the-command-line/">optionparse</a>. Alternatively, in the simplest cases, I have just
performed the parsing by hand. In <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> the parser is inferred by the
function annotations. Here is an example:</p>
<pre class="literal-block">
# example8.py
def main(command: ("SQL query", 'option', 'c'), dsn):
if command:
print('executing %s on %s' % (command, dsn))
# ...
if __name__ == '__main__':
import plac; plac.call(main)
</pre>
<p>Here the argument <tt class="docutils literal"><span class="pre">command</span></tt> has been annotated with the tuple
<tt class="docutils literal"><span class="pre">("SQL</span> <span class="pre">query",</span> <span class="pre">'option',</span> <span class="pre">'c')</span></tt>: the first string is the help string
which will appear in the usage message, the second string tells <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>
that <tt class="docutils literal"><span class="pre">command</span></tt> is an option and the third string that there is also
a short form of the option <tt class="docutils literal"><span class="pre">-c</span></tt>, the long form being <tt class="docutils literal"><span class="pre">--command</span></tt>.
The usage message is the following:</p>
<pre class="literal-block">
usage: example8.py [-h] [-c COMMAND] dsn
positional arguments:
dsn
optional arguments:
-h, --help show this help message and exit
-c COMMAND, --command COMMAND
SQL query
</pre>
<p>Here are two examples of usage:</p>
<pre class="literal-block">
$ python3 example8.py -c"select * from table" dsn
executing select * from table on dsn
$ python3 example8.py --command="select * from table" dsn
executing select * from table on dsn
</pre>
<p>The third argument in the function annotation can be omitted: in such
case it will be assumed to be <tt class="docutils literal"><span class="pre">None</span></tt>. The consequence is that
the usual dichotomy between long and short options (GNU-style options)
disappears: we get <em>smart options</em>, which have the single character prefix
of short options and behave like both long and short options, since
they can be abbreviated. Here is an example featuring smart options:</p>
<pre class="literal-block">
# example6.py
def main(dsn, command: ("SQL query", 'option')):
print('executing %r on %s' % (command, dsn))
if __name__ == '__main__':
import plac; plac.call(main)
</pre>
<pre class="literal-block">
usage: example6.py [-h] [-command COMMAND] dsn
positional arguments:
dsn
optional arguments:
-h, --help show this help message and exit
-command COMMAND SQL query
</pre>
<p>The following are all valid invocations ot the script:</p>
<pre class="literal-block">
$ python3 example6.py -c "select" dsn
executing 'select' on dsn
$ python3 example6.py -com "select" dsn
executing 'select' on dsn
$ python3 example6.py -command="select" dsn
executing 'select' on dsn
</pre>
<p>Notice that the form <tt class="docutils literal"><span class="pre">-command=SQL</span></tt> is recognized only for the full
option, not for its abbreviations:</p>
<pre class="literal-block">
$ python3 example6.py -com="select" dsn
usage: example6.py [-h] [-command COMMAND] dsn
example6.py: error: unrecognized arguments: -com=select
</pre>
<p>If the option is not passed, the variable <tt class="docutils literal"><span class="pre">command</span></tt>
will get the value <tt class="docutils literal"><span class="pre">None</span></tt>. However, it is possible to specify a non-trivial
default. Here is an example:</p>
<pre class="literal-block">
# example8_.py
def main(dsn, command: ("SQL query", 'option')='select * from table'):
print('executing %r on %s' % (command, dsn))
if __name__ == '__main__':
import plac; plac.call(main)
</pre>
<p>Notice that the default value appears in the help message:</p>
<pre class="literal-block">
usage: example8_.py [-h] [-command select * from table] dsn
positional arguments:
dsn
optional arguments:
-h, --help show this help message and exit
-command select * from table
SQL query
</pre>
<p>When you run the script and you do not pass the <tt class="docutils literal"><span class="pre">-command</span></tt> option, the
default query will be executed:</p>
<pre class="literal-block">
$ python3 example8_.py dsn
executing 'select * from table' on dsn
</pre>
</div>
<div class="section" id="scripts-with-flags">
<h1><a class="toc-backref" href="#id5">Scripts with flags</a></h1>
<p><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> is able to recognize flags, i.e. boolean options which are
<tt class="docutils literal"><span class="pre">True</span></tt> if they are passed to the command line and <tt class="docutils literal"><span class="pre">False</span></tt>
if they are absent. Here is an example:</p>
<pre class="literal-block">
# example9.py
def main(verbose: ('prints more info', 'flag', 'v'), dsn: 'connection string'):
if verbose:
print('connecting to %s' % dsn)
# ...
if __name__ == '__main__':
import plac; plac.call(main)
</pre>
<pre class="literal-block">
usage: example9.py [-h] [-v] dsn
positional arguments:
dsn connection string
optional arguments:
-h, --help show this help message and exit
-v, --verbose prints more info
</pre>
<pre class="literal-block">
$ python3 example9.py -v dsn
connecting to dsn
</pre>
<p>Notice that it is an error trying to specify a default for flags: the
default value for a flag is always <tt class="docutils literal"><span class="pre">False</span></tt>. If you feel the need to
implement non-boolean flags, you should use an option with two
choices, as explained in the "more features" section.</p>
<p>For consistency with the way the usage message is printed, I suggest
you to follow the Flag-Option-Required-Default (FORD) convention: in
the <tt class="docutils literal"><span class="pre">main</span></tt> function write first the flag arguments, then the option
arguments, then the required arguments and finally the default
arguments. This is just a convention and you are not forced to use it,
except for the default arguments (including the varargs) which must
stay at the end as it is required by the Python syntax.</p>
<p>I also suggests to specify a one-character abbreviation for flags: in
this way you can use the GNU-style composition of flags (i.e. <tt class="docutils literal"><span class="pre">-zxvf</span></tt>
is an abbreviation of <tt class="docutils literal"><span class="pre">-z</span> <span class="pre">-x</span> <span class="pre">-v</span> <span class="pre">-f</span></tt>). I usually do not provide
the one-character abbreviation for options, since it does not make sense
to compose them.</p>
</div>
<div class="section" id="plac-for-python-2-x-users">
<h1><a class="toc-backref" href="#id6">plac for Python 2.X users</a></h1>
<p>I do not use Python 3. At work we are just starting to think about
migrating to Python 2.6. It will take years before we
think to migrate to Python 3. I am pretty much sure most Pythonistas
are in the same situation. Therefore <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> provides a way to work
with function annotations even in Python 2.X (including Python 2.3).
There is no magic involved; you just need to add the annotations
by hand. For instance the annotated function declaration</p>
<pre class="literal-block">
def main(dsn: "Database dsn", *scripts: "SQL scripts"):
...
</pre>
<p>is equivalent to the following code:</p>
<pre class="literal-block">
def main(dsn, *scripts):
...
main.__annotations__ = dict(
dsn="Database dsn",
scripts="SQL scripts")
</pre>
<p>One should be careful to match the keys of the annotation dictionary
with the names of the arguments in the annotated function; for lazy
people with Python 2.4 available the simplest way is to use the
<tt class="docutils literal"><span class="pre">plac.annotations</span></tt> decorator that performs the check for you:</p>
<pre class="literal-block">
@plac.annotations(
dsn="Database dsn",
scripts="SQL scripts")
def main(dsn, *scripts):
...
</pre>
<p>In the rest of this article I will assume that you are using Python 2.X with
X >= 4 and I will use the <tt class="docutils literal"><span class="pre">plac.annotations</span></tt> decorator. Notice however
that the core features of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> run even on Python 2.3.</p>
</div>
<div class="section" id="more-features">
<h1><a class="toc-backref" href="#id7">More features</a></h1>
<p>One of the goals of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> is to have a learning curve of <em>minutes</em> for
its core features, compared to the learning curve of <em>hours</em> of
<a class="reference external" href="http://argparse.googlecode.com">argparse</a>. In order to reach this goal, I have <em>not</em> sacrificed all
the features of <a class="reference external" href="http://argparse.googlecode.com">argparse</a>. Actually a lot of <a class="reference external" href="http://argparse.googlecode.com">argparse</a> power persists
in <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>. Until now, I have only showed simple annotations, but in
general an annotation is a 6-tuple of the form</p>
<blockquote>
<tt class="docutils literal"><span class="pre">(help,</span> <span class="pre">kind,</span> <span class="pre">abbrev,</span> <span class="pre">type,</span> <span class="pre">choices,</span> <span class="pre">metavar)</span></tt></blockquote>
<p>where <tt class="docutils literal"><span class="pre">help</span></tt> is the help message, <tt class="docutils literal"><span class="pre">kind</span></tt> is a string in the set {
<tt class="docutils literal"><span class="pre">"flag"</span></tt>, <tt class="docutils literal"><span class="pre">"option"</span></tt>, <tt class="docutils literal"><span class="pre">"positional"</span></tt>}, <tt class="docutils literal"><span class="pre">abbrev</span></tt> is a
one-character string or <tt class="docutils literal"><span class="pre">None</span></tt>, <tt class="docutils literal"><span class="pre">type</span></tt> is a callable taking a
string in input,
<tt class="docutils literal"><span class="pre">choices</span></tt> is a discrete sequence of values and <tt class="docutils literal"><span class="pre">metavar</span></tt> is a string.</p>
<p><tt class="docutils literal"><span class="pre">type</span></tt> is used to automagically convert the command line arguments
from the string type to any Python type; by default there is no
conversion and <tt class="docutils literal"><span class="pre">type=None</span></tt>.</p>
<p><tt class="docutils literal"><span class="pre">choices</span></tt> is used to restrict the number of the valid
options; by default there is no restriction i.e. <tt class="docutils literal"><span class="pre">choices=None</span></tt>.</p>
<p><tt class="docutils literal"><span class="pre">metavar</span></tt> is used to change the argument name in the usage message
(and only there); by default the metavar is <tt class="docutils literal"><span class="pre">None</span></tt>: this means that
the name in the usage message is the same as the argument name,
unless the argument has a default and in such a case is
equal to the stringified form of the default.</p>
<p>Here is an example showing many of the features (copied from the
<a class="reference external" href="http://argparse.googlecode.com">argparse</a> documentation):</p>
<pre class="literal-block">
# example10.py
import plac
@plac.annotations(
operator=("The name of an operator", 'positional', None, str, ['add', 'mul']),
numbers=("A number", 'positional', None, float, None, "n"))
def main(operator, *numbers):
"A script to add and multiply numbers"
if operator == 'mul':
op = float.__mul__
result = 1.0
else: # operator == 'add'
op = float.__add__
result = 0.0
for n in numbers:
result = op(result, n)
return result
if __name__ == '__main__':
print(plac.call(main))
</pre>
<p>Here is the usage:</p>
<pre class="literal-block">
usage: example10.py [-h] {add,mul} [n [n ...]]
A script to add and multiply numbers
positional arguments:
{add,mul} The name of an operator
n A number
optional arguments:
-h, --help show this help message and exit
</pre>
<p>Notice that the docstring of the <tt class="docutils literal"><span class="pre">main</span></tt> function has been automatically added
to the usage message. Here are a couple of examples of use:</p>
<pre class="literal-block">
$ python example10.py add 1 2 3 4
10.0
$ python example10.py mul 1 2 3 4
24.0
$ python example10.py ad 1 2 3 4 # a mispelling error
usage: example10.py [-h] {add,mul} [n [n ...]]
example10.py: error: argument operator: invalid choice: 'ad' (choose from 'add', 'mul')
</pre>
<p><tt class="docutils literal"><span class="pre">plac.call</span></tt> can also be used in doctests like this:</p>
<pre class="doctest-block">
>>> import plac, example10
>>> plac.call(example10.main, ['add', '1', '2'])
3.0
</pre>
<p><tt class="docutils literal"><span class="pre">plac.call</span></tt> works for generators too:</p>
<pre class="doctest-block">
>>> def main(n):
... for i in range(int(n)):
... yield i
>>> plac.call(main, ['3'])
[0, 1, 2]
</pre>
<p>Internally <tt class="docutils literal"><span class="pre">plac.call</span></tt> tries to convert the output of the main function
into a list, if possible. If the output is not iterable or it is a
string, it is left unchanged, but if it is iterable it is converted.
In particular, generator objects are exhausted by <tt class="docutils literal"><span class="pre">plac.call</span></tt>.</p>
<p>This behavior avoids mistakes like forgetting of applying
<tt class="docutils literal"><span class="pre">list(result)</span></tt> to the result of <tt class="docutils literal"><span class="pre">plac.call</span></tt>; moreover it makes
errors visible early, and avoids mistakes in code like the following:</p>
<pre class="literal-block">
try:
result = plac.call(main, args)
except:
# do something
</pre>
<p>Without the "listify" functionality, a main function returning a
generator object would not raise any exception until the generator
is iterated over.</p>
<p>If you are a fan of lazyness, you can still have it by setting the <tt class="docutils literal"><span class="pre">eager</span></tt>
flag to <tt class="docutils literal"><span class="pre">False</span></tt>, as in the following example:</p>
<pre class="literal-block">
for line in plac.call(main, args, eager=False):
print(line)
</pre>
<p>If <tt class="docutils literal"><span class="pre">main</span></tt> returns a generator object this example will print each
line as soon as available, whereas the default behaviour is to print
all the lines together and the end of the computation.</p>
</div>
<div class="section" id="a-realistic-example">
<h1><a class="toc-backref" href="#id8">A realistic example</a></h1>
<p>Here is a more realistic script using most of the features of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> to
run SQL queries on a database by relying on <a class="reference external" href="http://www.sqlalchemy.org/">SQLAlchemy</a>. Notice the usage
of the <tt class="docutils literal"><span class="pre">type</span></tt> feature to automagically convert a SQLAlchemy connection
string into a <a class="reference external" href="http://www.sqlalchemy.org/docs/reference/ext/sqlsoup.html">SqlSoup</a> object:</p>
<pre class="literal-block">
# dbcli.py
import plac
from sqlalchemy.ext.sqlsoup import SqlSoup
@plac.annotations(
db=("Connection string", 'positional', None, SqlSoup),
header=("Header", 'flag', 'H'),
sqlcmd=("SQL command", 'option', 'c', str, None, "SQL"),
delimiter=("Column separator", 'option', 'd'),
scripts="SQL scripts",
)
def main(db, header, sqlcmd, delimiter="|", *scripts):
"A script to run queries and SQL scripts on a database"
yield 'Working on %s' % db.bind.url
if sqlcmd:
result = db.bind.execute(sqlcmd)
if header: # print the header
yield delimiter.join(result.keys())
for row in result: # print the rows
yield delimiter.join(map(str, row))
for script in scripts:
db.bind.execute(file(script).read())
yield 'executed %s' % script
if __name__ == '__main__':
for output in plac.call(main):
print(output)
</pre>
<p>You can see the <em>yield-is-print</em> pattern here: instead of using
<tt class="docutils literal"><span class="pre">print</span></tt> in the main function, I use <tt class="docutils literal"><span class="pre">yield</span></tt>, and I perform the
print in the <tt class="docutils literal"><span class="pre">__main__</span></tt> block. The advantage of the pattern is that
tests invoking <tt class="docutils literal"><span class="pre">plac.call</span></tt> and checking the result become trivial:
had I performed the printing in the main function, the test would have
involved an ugly hack like redirecting <tt class="docutils literal"><span class="pre">sys.stdout</span></tt> to a
<tt class="docutils literal"><span class="pre">StringIO</span></tt> object.</p>
<p>Here is the usage message:</p>
<pre class="literal-block">
usage: dbcli.py [-h] [-H] [-c SQL] [-d |] db [scripts [scripts ...]]
A script to run queries and SQL scripts on a database
positional arguments:
db Connection string
scripts SQL scripts
optional arguments:
-h, --help show this help message and exit
-H, --header Header
-c SQL, --sqlcmd SQL SQL command
-d |, --delimiter | Column separator
</pre>
<p>You can check for yourself that the script works.</p>
</div>
<div class="section" id="keyword-arguments">
<h1><a class="toc-backref" href="#id9">Keyword arguments</a></h1>
<p>Starting from release 0.4, <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> supports keyword arguments. In
practice that means that if your main function has keyword arguments,
<a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> treats specially arguments of the form <tt class="docutils literal"><span class="pre">"name=value"</span></tt> in the
command line. Here is an example:</p>
<pre class="literal-block">
# example12.py
import plac
@plac.annotations(
opt=('some option', 'option'),
args='default arguments',
kw='keyword arguments')
def main(opt, *args, **kw):
if opt:
yield 'opt=%s' % opt
if args:
yield 'args=%s' % str(args)
if kw:
yield 'kw=%s' % kw
if __name__ == '__main__':
for output in plac.call(main):
print(output)
</pre>
<p>Here is the generated usage message:</p>
<pre class="literal-block">
usage: example12.py [-h] [-opt OPT] [args [args ...]] [kw [kw ...]]
positional arguments:
args default arguments
kw keyword arguments
optional arguments:
-h, --help show this help message and exit
-opt OPT some option
</pre>
<p>Here is how you call the script:</p>
<pre class="literal-block">
$ python example12.py -o X a1 a2 name=value
opt=X
args=('a1', 'a2')
kw={'name': 'value'}
</pre>
<p>When using keyword arguments, one must be careful to use names which
are not alreay taken; for instance in this examples the name <tt class="docutils literal"><span class="pre">opt</span></tt>
is taken:</p>
<pre class="literal-block">
$ python example12.py 1 2 kw1=1 kw2=2 opt=0
usage: example12.py [-h] [-o OPT] [args [args ...]] [kw [kw ...]]
example12.py: error: colliding keyword arguments: opt
</pre>
<p>The names taken are the names of the flags, of the options, and of the
positional arguments, excepted varargs and keywords. This limitation
is a consequence of the way the argument names are managed in function calls
by the Python language.</p>
</div>
<div class="section" id="final-example-a-shelve-interface">
<h1><a class="toc-backref" href="#id10">Final example: a shelve interface</a></h1>
<p>Here is a less trivial example for the keyword arguments feature.
The use case is the following: suppose we have stored the
configuration parameters of a given application into a Python shelve
and we need a command-line tool to edit the shelve.
A possible implementation using <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> could be the following:</p>
<pre class="literal-block">
# ishelve.py
import os, shelve, plac
DEFAULT_SHELVE = os.path.expanduser('~/conf.shelve')
@plac.annotations(
help=('show help', 'flag'),
showall=('show all parameters in the shelve', 'flag'),
clear=('clear the shelve', 'flag'),
delete=('delete an element', 'option'),
filename=('filename of the shelve', 'option'),
params='names of the parameters in the shelve',
setters='setters param=value')
def main(help, showall, clear, delete, filename=DEFAULT_SHELVE,
*params, **setters):
"A simple interface to a shelve. Use .help to see the available commands."
sh = shelve.open(filename)
try:
if not any([help, showall, clear, delete, params, setters]):
yield 'no arguments passed, use .help to see the available commands'
elif help: # custom help
yield 'Commands: .help, .showall, .clear, .delete'
yield '<param> ...'
yield '<param=value> ...'
elif showall:
for param, name in sh.items():
yield '%s=%s' % (param, name)
elif clear:
sh.clear()
yield 'cleared the shelve'
elif delete:
try:
del sh[delete]
except KeyError:
yield '%s: not found' % delete
else:
yield 'deleted %s' % delete
for param in params:
try:
yield sh[param]
except KeyError:
yield '%s: not found' % param
for param, value in setters.items():
sh[param] = value
yield 'setting %s=%s' % (param, value)
finally:
sh.close()
main.add_help = False # there is a custom help, remove the default one
main.prefix_chars = '.' # use dot-prefixed commands
if __name__ == '__main__':
for output in plac.call(main):
print(output)
</pre>
<p>A few notes are in order:</p>
<ol class="arabic simple">
<li>I have disabled the ordinary help provided by <a class="reference external" href="http://argparse.googlecode.com">argparse</a> and I have
implemented a custom help command.</li>
<li>I have changed the prefix character used to recognize the options
to a dot.</li>
<li>Keyword arguments recognition (in the <tt class="docutils literal"><span class="pre">**setters</span></tt>) is used to make it
possible to store a value in the shelve with the syntax
<tt class="docutils literal"><span class="pre">param_name=param_value</span></tt>.</li>
<li><tt class="docutils literal"><span class="pre">*params</span></tt> are used to retrieve parameters from the shelve and some
error checking is performed in the case of missing parameters</li>
<li>A command to clear the shelve is implemented as a flag (<tt class="docutils literal"><span class="pre">.clear</span></tt>).</li>
<li>A command to delete a given parameter is implemented as an option
(<tt class="docutils literal"><span class="pre">.delete</span></tt>).</li>
<li>There is an option with default (<tt class="docutils literal"><span class="pre">.filename=conf.shelve</span></tt>) to store
the filename of the shelve.</li>
<li>All things considered, the code looks like a poor man object oriented
interface implemented with a chain of elifs instead of methods. Of course,
<a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> can do better than that, but let me start from a low-level approach
first.</li>
</ol>
<p>If you run <tt class="docutils literal"><span class="pre">ishelve.py</span></tt> without arguments you get the following
message:</p>
<pre class="literal-block">
$ python ishelve.py
no arguments passed, use .help to see the available commands
</pre>
<p>If you run <tt class="docutils literal"><span class="pre">ishelve.py</span></tt> with the option <tt class="docutils literal"><span class="pre">.h</span></tt> (or any abbreviation
of <tt class="docutils literal"><span class="pre">.help</span></tt>) you get:</p>
<pre class="literal-block">
$ python ishelve.py .h
Commands: .help, .showall, .clear, .delete
<param> ...
<param=value> ...
</pre>
<p>You can check by hand that the tool work:</p>
<pre class="literal-block">
$ python ishelve.py .clear # start from an empty shelve
cleared the shelve
$ python ishelve.py a=1 b=2
setting a=1
setting b=2
$ python ishelve.py .showall
b=2
a=1
$ python ishelve.py .del b # abbreviation for .delete
deleted b
$ python ishelve.py a
1
$ python ishelve.py b
b: not found
$ python ishelve.py .cler # mispelled command
usage: ishelve.py [.help] [.showall] [.clear] [.delete DELETE]
[.filename /home/micheles/conf.shelve]
[params [params ...]] [setters [setters ...]]
ishelve.py: error: unrecognized arguments: .cler
</pre>
</div>
<div class="section" id="plac-vs-argparse">
<h1><a class="toc-backref" href="#id11">plac vs argparse</a></h1>
<p><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> is opinionated and by design it does not try to make available
all of the features of <a class="reference external" href="http://argparse.googlecode.com">argparse</a> in an easy way. In particular you
should be aware of the following limitations/differences (the
following assumes knowledge of <a class="reference external" href="http://argparse.googlecode.com">argparse</a>):</p>
<ul class="simple">
<li>plac does not support the destination concept: the destination
coincides with the name of the argument, always. This restriction
has some drawbacks. For instance, suppose you want to define a long
option called <tt class="docutils literal"><span class="pre">--yield</span></tt>. In this case the destination would be <tt class="docutils literal"><span class="pre">yield</span></tt>,
which is a Python keyword, and since you cannot introduce an
argument with that name in a function definition, it is impossible
to implement it. Your choices are to change the name of the long
option, or to use <a class="reference external" href="http://argparse.googlecode.com">argparse</a> with a suitable destination.</li>
<li><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> does not support "required options". As the <a class="reference external" href="http://argparse.googlecode.com">argparse</a>
documentation puts it: <em>Required options are generally considered bad
form - normal users expect options to be optional. You should avoid
the use of required options whenever possible.</em> Notice that since
<a class="reference external" href="http://argparse.googlecode.com">argparse</a> supports them, <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> can manage them too, but not directly.</li>
<li><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> supports only regular boolean flags. <a class="reference external" href="http://argparse.googlecode.com">argparse</a> has the ability to
define generalized two-value flags with values different from <tt class="docutils literal"><span class="pre">True</span></tt>
and <tt class="docutils literal"><span class="pre">False</span></tt>. An earlier version of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> had this feature too, but
since you can use options with two choices instead, and in any case
the conversion from <tt class="docutils literal"><span class="pre">{True,</span> <span class="pre">False}</span></tt> to any couple of values
can be trivially implemented with a ternary operator
(<tt class="docutils literal"><span class="pre">value1</span> <span class="pre">if</span> <span class="pre">flag</span> <span class="pre">else</span> <span class="pre">value2</span></tt>), I have removed it (KISS rules!).</li>
<li><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> does not support <tt class="docutils literal"><span class="pre">nargs</span></tt> options directly (it uses them internally,
though, to implement flag recognition). The reason it that all the use
cases of interest to me are covered by <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> and did not feel the need
to increase the learning curve by adding direct support for <tt class="docutils literal"><span class="pre">nargs</span></tt>.</li>
<li><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> does support subparsers, but you must read the <a class="reference external" href="in-writing">advanced usage
document</a> to see how it works.</li>
<li><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> does not support actions directly. This also
looks like a feature too advanced for the goals of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>. Notice however
that the ability to define your own annotation objects (again, see
the <a class="reference external" href="in-writing">advanced usage document</a>) may mitigate the need for custom actions.</li>
</ul>
<p><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> can leverage directly on many <a class="reference external" href="http://argparse.googlecode.com">argparse</a> features.</p>
<p>For instance, you can make invisible an argument in the usage message
simply by using <tt class="docutils literal"><span class="pre">'==SUPPRESS=='</span></tt> as help string (or
<tt class="docutils literal"><span class="pre">argparse.SUPPRESS</span></tt>). Similarly, you can use <a class="reference external" href="http://argparse.googlecode.com/svn/tags/r11/doc/other-utilities.html?highlight=filetype#FileType">argparse.FileType</a>
directly.</p>
<p>It is also possible to pass options to the underlying
<tt class="docutils literal"><span class="pre">argparse.ArgumentParser</span></tt> object (currently it accepts the default
arguments <tt class="docutils literal"><span class="pre">description</span></tt>, <tt class="docutils literal"><span class="pre">epilog</span></tt>, <tt class="docutils literal"><span class="pre">prog</span></tt>, <tt class="docutils literal"><span class="pre">usage</span></tt>,
<tt class="docutils literal"><span class="pre">add_help</span></tt>, <tt class="docutils literal"><span class="pre">argument_default</span></tt>, <tt class="docutils literal"><span class="pre">parents</span></tt>, <tt class="docutils literal"><span class="pre">prefix_chars</span></tt>,
<tt class="docutils literal"><span class="pre">fromfile_prefix_chars</span></tt>, <tt class="docutils literal"><span class="pre">conflict_handler</span></tt>, <tt class="docutils literal"><span class="pre">formatter_class</span></tt>).
It is enough to set such attributes on the <tt class="docutils literal"><span class="pre">main</span></tt> function. For
instance</p>
<pre class="literal-block">
def main(...):
pass
main.add_help = False
</pre>
<p>disables the recognition of the help flag <tt class="docutils literal"><span class="pre">-h,</span> <span class="pre">--help</span></tt>. This
mechanism does not look particularly elegant, but it works well
enough. I assume that the typical user of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> will be happy with
the defaults and would not want to change them; still it is possible
if she wants to.</p>
<p>For instance, by setting the <tt class="docutils literal"><span class="pre">description</span></tt> attribute, it is possible
to add a comment to the usage message (by default the docstring of the
<tt class="docutils literal"><span class="pre">main</span></tt> function is used as description).</p>
<p>It is also possible to change the option prefix; for
instance if your script must run under Windows and you want to use "/"
as option prefix you can add the line:</p>
<pre class="literal-block">
main.prefix_chars='/-'
</pre>
<p>The first prefix char (<tt class="docutils literal"><span class="pre">/</span></tt>) is used
as the default for the recognition of options and flags;
the second prefix char (<tt class="docutils literal"><span class="pre">-</span></tt>) is kept to keep the <tt class="docutils literal"><span class="pre">-h/--help</span></tt> option
working: however you can disable it and reimplement it, if you like,
as seen in the <tt class="docutils literal"><span class="pre">ishelve</span></tt> example.</p>
<p>It is possible to access directly the underlying <a class="reference external" href="http://argparse.googlecode.com/svn/tags/r11/doc/ArgumentParser.html">ArgumentParser</a> object, by
invoking the <tt class="docutils literal"><span class="pre">plac.parser_from</span></tt> utility function:</p>
<pre class="doctest-block">
>>> import plac
>>> def main(arg):
... pass
...
>>> print(plac.parser_from(main)) #doctest: +ELLIPSIS
ArgumentParser(prog=...)
</pre>
<p>Internally <tt class="docutils literal"><span class="pre">plac.call</span></tt> uses <tt class="docutils literal"><span class="pre">plac.parser_from</span></tt> and adds the parser
to the main function as an attribute. When <tt class="docutils literal"><span class="pre">plac.call(func)</span></tt> is
invoked multiple time, the parser is re-used and not rebuilt from scratch again.</p>
<p>I use <tt class="docutils literal"><span class="pre">plac.parser_from</span></tt> in the unit tests of the module, but regular
users should not need to use it, unless they want to access <em>all</em>
of the features of <a class="reference external" href="http://argparse.googlecode.com">argparse</a> directly without calling the main function.</p>
<p>Interested readers should read the documentation of <a class="reference external" href="http://argparse.googlecode.com">argparse</a> to
understand the meaning of the other options. If there is a set of
options that you use very often, you may consider writing a decorator
adding such options to the <tt class="docutils literal"><span class="pre">main</span></tt> function for you. For simplicity,
<a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> does not perform any magic except the addition of the <tt class="docutils literal"><span class="pre">.p</span></tt>
attribute.</p>
</div>
<div class="section" id="plac-vs-the-rest-of-the-world">
<h1><a class="toc-backref" href="#id12">plac vs the rest of the world</a></h1>
<p>Originally <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> boasted about being "the easiest command-line
arguments parser in the world". Since then, people started pointing
out to me various projects which are based on the same idea
(extracting the parser from the main function signature) and are
arguably even easier than <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>:</p>
<ul class="simple">
<li><a class="reference external" href="http://pypi.python.org/pypi/opterator">opterator</a> by Dusty Phillips</li>
<li><a class="reference external" href="http://pypi.python.org/pypi/CLIArgs">CLIArgs</a> by Pavel Panchekha</li>
</ul>
<p>Luckily for me none of such projects had the idea of using
function annotations and <a class="reference external" href="http://argparse.googlecode.com">argparse</a>; as a consequence, they are
no match for the capabilities of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>.</p>
<p>Of course, there are tons of other libraries to parse the command
line. For instance <a class="reference external" href="http://pypi.python.org/pypi/Clap/0.7">Clap</a> by Matthew Frazier which appeared on PyPI
just the day before <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>; <a class="reference external" href="http://pypi.python.org/pypi/Clap/0.7">Clap</a> is fine but it is certainly not
easier than <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>.</p>
<p><a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> can also be used as a replacement of the <a class="reference external" href="http://docs.python.org/library/cmd.html">cmd</a> module in the standard
library and as such it shares many features with the module <a class="reference external" href="http://packages.python.org/cmd2/">cmd2</a> by
Catherine Devlin. However, this is completely coincidental, since I became
aware of the <a class="reference external" href="http://packages.python.org/cmd2/">cmd2</a> module only after writing <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>.</p>
</div>
<div class="section" id="the-future">
<h1><a class="toc-backref" href="#id13">The future</a></h1>
<p>Currently the core of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> is around 200 lines of code, not counting blanks,
comments and docstrings. I do not plan to extend the core much in the
future. The idea is to keep the module short: it is and it should
remain a little wrapper over <a class="reference external" href="http://argparse.googlecode.com">argparse</a>. Actually I have thought about
contributing the core back to <a class="reference external" href="http://argparse.googlecode.com">argparse</a> if <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> becomes successfull
and gains a reasonable number of users. For the moment it should be
considered in alpha status.</p>
<p>Notice that even if <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> has been designed to be simple to use for
simple stuff, its power should not be underestimated; it is actually a
quite advanced tool with a domain of applicability which far exceeds
the realm of command-line arguments parsers.</p>
<p>Version 0.5 of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> doubled the code base and the documentation: it is
based on the idea of using <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> to implement command-line interpreters,
i.e. something akin to the <tt class="docutils literal"><span class="pre">cmd</span></tt> module in the standard library, only better.
The new features of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> are described in the <a class="reference external" href="in-writing">advanced usage document</a> .
They are implemented in a separated module (<tt class="docutils literal"><span class="pre">plac_ext.py</span></tt>), since
they require Python 2.5 to work, whereas <tt class="docutils literal"><span class="pre">plac_core.py</span></tt> only requires
Python 2.3.</p>
</div>
<div class="section" id="trivia-the-story-behind-the-name">
<h1><a class="toc-backref" href="#id14">Trivia: the story behind the name</a></h1>
<p>The <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> project started very humble: I just wanted to make
easy_installable my old <a class="reference external" href="http://code.activestate.com/recipes/278844-parsing-the-command-line/">optionparse</a> recipe, and to publish it on PyPI.
The original name of <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a> was optionparser and the idea behind it was
to build an <a class="reference external" href="http://docs.python.org/library/optparse.html?highlight=optionparser#optparse.OptionParser">OptionParser</a> object from the docstring of the module.
However, before doing that, I decided to check out the <a class="reference external" href="http://argparse.googlecode.com">argparse</a> module,
since I knew it was going into Python 2.7 and Python 2.7 was coming out.
Soon enough I realized two things:</p>
<ol class="arabic simple">
<li>the single greatest idea of <a class="reference external" href="http://argparse.googlecode.com">argparse</a> was unifying the positional arguments
and the options in a single namespace object;</li>
<li>parsing the docstring was so old-fashioned, considering the existence
of functions annotations in Python 3.</li>
</ol>
<p>Putting together these two observations with the original idea of inferring the
parser I decided to build an <a class="reference external" href="http://argparse.googlecode.com/svn/tags/r11/doc/ArgumentParser.html">ArgumentParser</a> object from function
annotations. The <tt class="docutils literal"><span class="pre">optionparser</span></tt> name was ruled out, since I was
now using <a class="reference external" href="http://argparse.googlecode.com">argparse</a>; a name like <tt class="docutils literal"><span class="pre">argparse_plus</span></tt> was also ruled out,
since the typical usage was completely different from the <a class="reference external" href="http://argparse.googlecode.com">argparse</a> usage.</p>
<p>I made a research on PyPI and the name <em>clap</em> (Command Line Arguments Parser)
was not taken, so I renamed everything to clap. After two days
a <a class="reference external" href="http://pypi.python.org/pypi/Clap/0.7">Clap</a> module appeared on PyPI <expletives deleted>!</p>
<p>Having little imagination, I decided to rename everything again to plac,
an anagram of clap: since it is a non-existing English name, I hope nobody
will steal it from me!</p>
<p>That's all, I hope you will enjoy working with <a class="reference external" href="http://pypi.python.org/pypi/plac">plac</a>!</p>
</div>
</div>
</body>
</html>
|
