diff options
| -rw-r--r-- | MANIFEST | 1 | ||||
| -rw-r--r-- | pod/Makefile | 4 | ||||
| -rw-r--r-- | pod/buildtoc | 2 | ||||
| -rw-r--r-- | pod/perl.pod | 1 | ||||
| -rw-r--r-- | pod/perlcompile.pod | 443 | ||||
| -rw-r--r-- | pod/roffitall | 1 |
6 files changed, 451 insertions, 1 deletions
@@ -991,6 +991,7 @@ pod/perlapio.pod IO API info pod/perlbook.pod Book info pod/perlbot.pod Object-oriented Bag o' Tricks pod/perlcall.pod Callback info +pod/perlcompile.pod Info on using the Compiler suite pod/perldata.pod Data structure info pod/perldbmfilter.pod Info about DBM Filters pod/perldebug.pod Debugger info diff --git a/pod/Makefile b/pod/Makefile index f28b9d43a1..8a96236acf 100644 --- a/pod/Makefile +++ b/pod/Makefile @@ -59,6 +59,7 @@ POD = \ perlxstut.pod \ perlguts.pod \ perlcall.pod \ + perlcompile.pod \ perltodo.pod \ perlhist.pod \ perlfaq.pod \ @@ -118,6 +119,7 @@ MAN = \ perlxstut.man \ perlguts.man \ perlcall.man \ + perlcompile.man \ perltodo.man \ perlhist.man \ perlfaq.man \ @@ -177,6 +179,7 @@ HTML = \ perlxstut.html \ perlguts.html \ perlcall.html \ + perlcompile.html \ perltodo.html \ perlhist.html \ perlfaq.html \ @@ -236,6 +239,7 @@ TEX = \ perlxstut.tex \ perlguts.tex \ perlcall.tex \ + perlcompile.tex \ perltodo.tex \ perlhist.tex \ perlfaq.tex \ diff --git a/pod/buildtoc b/pod/buildtoc index 2574b1096f..1a9a24bb2d 100644 --- a/pod/buildtoc +++ b/pod/buildtoc @@ -13,7 +13,7 @@ sub output ($); perllol perltoot perltootc perlobj perltie perlbot perlipc perldbmfilter perldebug perldiag perlsec perltrap perlport perlstyle perlpod perlbook - perlembed perlapio perlxs perlxstut perlguts perlcall + perlembed perlapio perlxs perlxstut perlguts perlcall perlcompile perlhist ); diff --git a/pod/perl.pod b/pod/perl.pod index 0275543c31..abf3a7bff3 100644 --- a/pod/perl.pod +++ b/pod/perl.pod @@ -50,6 +50,7 @@ sections: perlthrtut Perl threads tutorial perldbmfilter Perl DBM Filters + perlcompile Perl compiler suite intro perldebug Perl debugging perldiag Perl diagnostic messages perlsec Perl security diff --git a/pod/perlcompile.pod b/pod/perlcompile.pod new file mode 100644 index 0000000000..0ba94187db --- /dev/null +++ b/pod/perlcompile.pod @@ -0,0 +1,443 @@ +=head1 NAME + +perlcompile - Introduction to the Perl Compiler-Translator + +=head1 DESCRIPTION + +Perl has always had a compiler: your source is compiled into an +internal form (a parse tree) which is then optimized before being +run. Since version 5.005, Perl has shipped with a module +capable of inspecting the optimized parse tree (C<B>), and this has +been used to write many useful utilities, including a module that lets +you turn your Perl into C source code that can be compiled into an +native executable. + +The C<B> module provides access to the parse tree, and other modules +("back ends") do things with the tree. Some write it out as +bytecode, C source code, or a semi-human-readable text. Another +traverses the parse tree to build a cross-reference of which +subroutines, formats, and variables are used where. Another checks +your code for dubious constructs. Yet another back end dumps the +parse tree back out as Perl source, acting as a source code beautifier +or deobfuscator. + +Because its original purpose was to be a way to produce C code +corresponding to a Perl program, and in turn a native executable, the +C<B> module and its associated back ends are known as "the +compiler", even though they don't really compile anything. +Different parts of the compiler are more accurately a "translator", +or an "inspector", but people want Perl to have a "compiler +option" not an "inspector gadget". What can you do? + +This document covers the use of the Perl compiler: which modules +it comprises, how to use the most important of the back end modules, +what problems there are, and how to work around them. + +=head2 Layout + +The compiler back ends are in the C<B::> hierarchy, and the front-end +(the module that you, the user of the compiler, will sometimes +interact with) is the O module. Some back ends (e.g., C<B::C>) have +programs (e.g., I<perlcc>) to hide the modules' complexity. + +Here are the important back ends to know about, with their status +expressed as a number from 0 (outline for later implementation) to +10 (if there's a bug in it, we're very surprised): + +=over 4 + +=item B::Bytecode + +Stores the parse tree in a machine-independent format, suitable +for later reloading through the ByteLoader module. Status: 5 (some +things work, some things don't, some things are untested). + +=item B::C + +Creates a C source file containing code to rebuild the parse tree +and resume the interpreter. Status: 6 (many things work adequately, +including programs using Tk). + +=item B::CC + +Creates a C source file corresponding to the run time code path in +the parse tree. This is the closest to a Perl-to-C translator there +is, but the code it generates is almost incomprehensible because it +translates the parse tree into a giant switch structure that +manipulates Perl structures. Eventual goal is to reduce (given +sufficient type information in the Perl program) some of the +Perl data structure manipulations into manipulations of C-level +ints, floats, etc. Status: 5 (some things work, including +uncomplicated Tk examples). + +=item B::Lint + +Complains if it finds dubious constructs in your source code. Status: +6 (it works adequately, but only has a very limited number of areas +that it checks). + +=item B::Deparse + +Recreates the Perl source, making an attempt to format it coherently. +Status: 8 (it works nicely, but a few obscure things are missing). + +=item B::Xref + +Reports on the declaration and use of subroutines and variables. +Status: 8 (it works nicely, but still has a few lingering bugs). + +=back + +=head1 Using The Back Ends + +The following sections describe how to use the various compiler back +ends. They're presented roughly in order of maturity, so that the +most stable and proven back ends are described first, and the most +experimental and incomplete back ends are described last. + +The O module automatically enabled the B<-c> flag to Perl, which +prevents Perl from executing your code once it has been compiled. +This is why all the back ends print: + + myperlprogram syntax OK + +before producing any other output. + +=head2 The Cross Referencing Back End (B::Xref) + +The cross referencing back end produces a report on your program, +breaking down declarations and uses of subroutines and variables (and +formats) by file and subroutine. For instance, here's part of the +report from the I<pod2man> program that comes with Perl: + + Subroutine clear_noremap + Package (lexical) + $ready_to_print i1069, 1079 + Package main + $& 1086 + $. 1086 + $0 1086 + $1 1087 + $2 1085, 1085 + $3 1085, 1085 + $ARGV 1086 + %HTML_Escapes 1085, 1085 + +This shows the variables used in the subroutine C<clear_noremap>. The +variable C<$ready_to_print> is a my() (lexical) variable, +B<i>ntroduced (first declared with my()) on line 1069, and used on +line 1079. The variable C<$&> from the main package is used on 1086, +and so on. + +A line number may be prefixed by a single letter: + +=over 4 + +=item i + +Lexical variable introduced (declared with my()) for the first time. + +=item & + +Subroutine or method call. + +=item s + +Subroutine defined. + +=item r + +Format defined. + +=back + +The most useful option the cross referencer has is to save the report +to a separate file. For instance, to save the report on +I<myperlprogram> to the file I<report>: + + $ perl -MO=Xref,-oreport myperlprogram + +=head2 The Decompiling Back End + +The Deparse back end turns your Perl source back into Perl source. It +can reformat along the way, making it useful as a de-obfuscator. The +most basic way to use it is: + + $ perl -MO=Deparse myperlprogram + +You'll notice immediately that Perl has no idea of how to paragraph +your code. You'll have to separate chunks of code from each other +with newlines by hand. However, watch what it will do with +one-liners: + + $ perl -MO=Deparse -e '$op=shift||die "usage: $0 + code [...]";chomp(@ARGV=<>)unless@ARGV; for(@ARGV){$was=$_;eval$op; + die$@ if$@; rename$was,$_ unless$was eq $_}' + -e syntax OK + $op = shift @ARGV || die("usage: $0 code [...]"); + chomp(@ARGV = <ARGV>) unless @ARGV; + foreach $_ (@ARGV) { + $was = $_; + eval $op; + die $@ if $@; + rename $was, $_ unless $was eq $_; + } + +(this is the I<rename> program that comes in the I<eg/> directory +of the Perl source distribution). + +The decompiler has several options for the code it generates. For +instance, you can set the size of each indent from 4 (as above) to +2 with: + + $ perl -MO=Deparse,-si2 myperlprogram + +The B<-p> option adds parentheses where normally they are omitted: + + $ perl -MO=Deparse -e 'print "Hello, world\n"' + -e syntax OK + print "Hello, world\n"; + $ perl -MO=Deparse,-p -e 'print "Hello, world\n"' + -e syntax OK + print("Hello, world\n"); + +See L<B::Deparse> for more information on the formatting options. + +=head2 The Lint Back End (B::Lint) + +The lint back end inspects programs for poor style. One programmer's +bad style is another programmer's useful tool, so options let you +select what is complained about. + +To run the style checker across your source code: + + $ perl -MO=Lint myperlprogram + +To disable context checks and undefined subroutines: + + $ perl -MO=Lint,-context,-undefined-subs myperlprogram + +See L<B::Lint> for information on the options. + +=head2 The Simple C Back End + +This module saves the internal compiled state of your Perl program +to a C source file, which can be turned into a native executable +for that particular platform using a C compiler. The resulting +program links against the Perl interpreter library, so it +will not save you disk space (unless you build Perl with a shared +library) or program size. It may, however, save you startup time. + +The C<perlcc> tool generates such executables by default. + + perlcc myperlprogram.pl + +=head2 The Bytecode Back End + +This back end is only useful if you also have a way to load and +execute the bytecode that it produces. The ByteLoader module provides +this functionality. + +To turn a Perl program into executable byte code, you can use C<perlcc> +with the C<-b> switch: + + perlcc -b myperlprogram.pl + +The byte code is machine independent, so once you have a compiled +module or program, it is as portable as Perl source (assuming that +the user of the module or program has a modern-enough Perl interpreter +to decode the byte code). + +See B<B::Bytecode> for information on options to control the +optimization and nature of the code generated by the Bytecode module. + +=head2 The Optimized C Back End + +The optimized C back end will turn your Perl program's run time +code-path into an equivalent (but optimized) C program that manipulates +the Perl data structures directly. The program will still link against +the Perl interpreter library, to allow for eval(), C<s///e>, +C<require>, etc. + +The C<perlcc> tool generates such executables when using the -opt +switch. To compile a Perl program (ending in C<.pl> +or C<.p>): + + perlcc -opt myperlprogram.pl + +To produce a shared library from a Perl module (ending in C<.pm>): + + perlcc -opt Myperlmodule.pm + +For more information, see L<perlcc> and L<B::CC>. + +=over 4 + +=item B + +This module is the introspective ("reflective" in Java terms) +module, which allows a Perl program to inspect its innards. The +back end modules all use this module to gain access to the compiled +parse tree. You, the user of a back end module, will not need to +interact with B. + +=item O + +This module is the front-end to the compiler's back ends. Normally +called something like this: + + $ perl -MO=Deparse myperlprogram + +This is like saying C<use O 'Deparse'> in your Perl program. + +=item B::Asmdata + +This module is used by the B::Assembler module, which is in turn used +by the B::Bytecode module, which stores a parse-tree as +bytecode for later loading. It's not a back end itself, but rather a +component of a back end. + +=item B::Assembler + +This module turns a parse-tree into data suitable for storing +and later decoding back into a parse-tree. It's not a back end +itself, but rather a component of a back end. It's used by the +I<assemble> program that produces bytecode. + +=item B::Bblock + +This module is used by the B::CC back end. It walks "basic blocks", +whatever they may be. + +=item B::Bytecode + +This module is a back end that generates bytecode from a +program's parse tree. This bytecode is written to a file, from where +it can later be reconstructed back into a parse tree. The goal is to +do the expensive program compilation once, save the interpreter's +state into a file, and then restore the state from the file when the +program is to be executed. See L</"The Bytecode Back End"> +for details about usage. + +=item B::C + +This module writes out C code corresponding to the parse tree and +other interpreter internal structures. You compile the corresponding +C file, and get an executable file that will restore the internal +structures and the Perl interpreter will begin running the +program. See L</"The Simple C Back End"> for details about usage. + +=item B::CC + +This module writes out C code corresponding to your program's +operations. Unlike the B::C module, which merely stores the +interpreter and its state in a C program, the B::CC module makes a +C program that does not involve the interpreter. As a consequence, +programs translated into C by B::CC can execute faster than normal +interpreted programs. See L</"The Optimized C Back End"> for +details about usage. + +=item B::Debug + +This module dumps the Perl parse tree in verbose detail to STDOUT. +It's useful for people who are writing their own back end, or who +are learning about the Perl internals. It's not useful to the +average programmer. + +=item B::Deparse + +This module produces Perl source code from the compiled parse tree. +It is useful in debugging and deconstructing other people's code, +also as a pretty-printer for your own source. See +L</"The Decompiling Back End"> for details about usage. + +=item B::Disassembler + +This module turns bytecode back into a parse tree. It's not a back +end itself, but rather a component of a back end. It's used by the +I<disassemble> program that comes with the bytecode. + +=item B::Lint + +This module inspects the compiled form of your source code for things +which, while some people frown on them, aren't necessarily bad enough +to justify a warning. For instance, use of an array in scalar context +without explicitly saying C<scalar(@array)> is something that Lint +can identify. See L</"The Lint Back End"> for details about usage. + +=item B::Showlex + +This module prints out the my() variables used in a function or a +file. To gt a list of the my() variables used in the subroutine +mysub() defined in the file myperlprogram: + + $ perl -MO=Showlex,mysub myperlprogram + +To gt a list of the my() variables used in the file myperlprogram: + + $ perl -MO=Showlex myperlprogram + +[BROKEN] + +=item B::Stackobj + +This module is used by the B::CC module. It's not a back end itself, +but rather a component of a back end. + +=item B::Stash + +This module is used by the L<perlcc> program, which compiles a module +into an executable. B::Stash prints the symbol tables in use by a +program, and is used to prevent B::CC from producing C code for the +B::* and O modules. It's not a back end itself, but rather a +component of a back end. + +=item B::Terse + +This module prints the contents of the parse tree, but without as much +information as B::Debug. For comparison, C<print "Hello, world."> +produced 96 lines of output from B::Debug, but only 6 from B::Terse. + +This module is useful for people who are writing their own back end, +or who are learning about the Perl internals. It's not useful to the +average programmer. + +=item B::Xref + +This module prints a report on where the variables, subroutines, and +formats are defined and used within a program and the modules it +loads. See L</"The Cross Referencing Back End"> for details about +usage. + +=cut + +=head1 KNOWN PROBLEMS + +The simple C backend currently only saves typeglobs with alphanumeric +names. + +The optimized C backend outputs code for more modules than it should +(e.g., DirHandle). It also has little hope of properly handling +C<goto LABEL> outside the running subroutine (C<goto &sub> is ok). +C<goto LABEL> currently does not work at all in this backend. +It also creates a huge initialization function that gives +C compilers headaches. Splitting the initialization function gives +better results. Other problems include: unsigned math does not +work correctly; some opcodes are handled incorrectly by default +opcode handling mechanism. + +BEGIN{} blocks are executed while compiling your code. Any external +state that is initialized in BEGIN{}, such as opening files, initiating +database connections etc., do not behave properly. To work around +this, Perl has an INIT{} block that corresponds to code being executed +before your program begins running but after your program has finished +being compiled. Execution order: BEGIN{}, (possible save of state +through compiler back-end), INIT{}, program runs, END{}. + +=head1 AUTHOR + +This document was originally written by Nathan Torkington, and is now +maintained by the perl5-porters mailing list +I<perl5-porters@perl.org>. + +=cut diff --git a/pod/roffitall b/pod/roffitall index afb4432e2a..bcf58642b7 100644 --- a/pod/roffitall +++ b/pod/roffitall @@ -68,6 +68,7 @@ toroff=` $mandir/perlxstut.1 \ $mandir/perlguts.1 \ $mandir/perlcall.1 \ + $mandir/perlcompile.1 \ $mandir/perltodo.1 \ $mandir/perlhist.1 \ $mandir/perldelta.1 \ |