diff options
author | Daniel Dragan <bulk88@hotmail.com> | 2015-08-15 16:03:49 -0400 |
---|---|---|
committer | Tony Cook <tony@develop-help.com> | 2015-09-07 11:23:24 +1000 |
commit | 3bdc51af3f9be1ab22a176fefca16c329755b094 (patch) | |
tree | 4dc01f7473566fa6f02db862fa7aa239b9316d36 /README.win32 | |
parent | 48458f6935316bb1d6fa1746b41a9d05bc87a286 (diff) | |
download | perl-3bdc51af3f9be1ab22a176fefca16c329755b094.tar.gz |
add parallel support 4 Win32 dmake building part 1
-if building a 32 bit Perl, with a 64bit dmake, force
PROCESSOR_ARCHITECTURE to x86, otherwise the next couple macros will
try to build a 64 bit perl with 32 bit CCs
-PDBOUT is required to run multiple VC cl.exe processes, otherwise all
VC cl.exe processes will error out trying to lock and write to a
file called "vc*0.pdb", PDBOUT is empty for GCC builds since they dont
have PDB files
-to reduce excess IO calls checking for miniperl.exe plus remove a
"Found file corresponding to virtual target" warning that dmake emits,
make this makefile unaware that miniperl.exe exists. dmake has a very bad
exponential number of IO/stat() calls for every target that is yet
unbuilt, see procmon logs in [perl #123854], so instead of a stat on
../miniperl.exe, then ../lib/buildcustomize.pl, it will be just a stat on
../lib/buildcustomize.pl
-remove makefile awareness of ..\lib\Config_heavy.pl,
if ..\lib\Config_heavy.pl is ever updated, so is ..\lib\Config.pm
less IO calls for dmake, see also commit 962e59f394
-to break up the sequential nature of this makefile, allow (XS) Extensions
to build, before (AKA parallel with) perl5**.dll and perl.exe are built.
This is achieved by running makedef.pl very early, and generating
perl5**.lib/libperl5**.a from the def file, and NOT generating
perl5**.lib/libperl5**.a from perl5**.dll at link time of perl5**.dll.
The conquence of this is, declspec(dllexport) is now unusable, but exports
should be centrally managed anyways (embed.fnc/etc) so this isn't a big
issue.
-EUMM makefiles shouldn't be subject to parallelism, untested and disable
for now, plus creating PLMAKE allows "dmake -n" to work for diagnosing
this makefile
-slim down all target. Extensions* and UNIDATAFILES now know how to build
themselves, the parallel nature says you can't rely on left to right
execution of deps in a parent node to make a child (dep) node build
-miniperl.exe used to be unbuildable from a clean tree except from all
target, since miniperl.exe didn't depend on mini config.h. Also mini dir
can't be a target, since each .obj built will dirty the mini dir's time
stamp and infinite loop happens, instead use a .exists file
-dmake rescans for all outstanding targets, at each recipe line to run,
this early in the build process, there are an enormous amount of files to
test for, so the echos are very slow, 350ms each, so combine as many of
the lines of mini/.exists AKA mini config.h together as possible. ".IF"
can't be put inside "one line", so not all lines were merged. Shell "if"
could be used to further group the echos but this enough to make the pause
/cpu spining not really noticable. USE_CPLUSPLUS contains an unrolled line
(the #endif) on both sides. See also procmon logs in [perl #123854]
-perllib.obj/.o needs perllibst.h which is built by miniperl+a script,
perllibst.h target doesn't need any Extensions*, so perllib.obj is still
quick to build
-perldll.def doesn't need perllibst.h since makedef.pl uses
$Config{static_ext} to find boot xsubs to export, makedef.pl does not
read perllibst.h, remove perllibst.h for more parallelism/less things to
build before perldll.def runs, and therefore Extensions (XS) starts to
run quicker, and Extensions (XS) is the longest target to build
-perlmain and perlmainst .obj/.o needs full perl headers to compile, they
aren't like generate_uudmap.exe and perlglob.exe which are perlapi unaware
-perl.exe doesn't need perl5**.dll to build, just the imp lib to
perl5**.dll, perl.exe is in the same category as XS modules, more
parallelism
-ppport.h isn't needed in blead perl, since blead is the newest perl in the
world, this allows Extensions (XS) to run sooner, ppport.h is replaced by
a dummy empty file, delete ppport.h from final installed CORE dir during
installation so the dummy ppport.h doesn't wind up in installed perl and isnt
seen by CPAN XS modules during their building by the perl user after perl
was build and installed, note mkppport script and mkppport.lst are unused
in makefile.mk now
-break up the dependencies of all the Extensions* targets, static
(just Win32CORE normally), dynamic XS and non-XS, these 3 run in parallel
now, non-XS doesn't need perl5**.lib/.a or any C headers, just PP
Config.pm
-DLL XS requires PP DynaLoader.pm for dl_findfile() otherwise
Mkbootstrap.pm fatally errors, Mkbootstrap.pm could be revised to not load
DynaLoader on Win32, since " if ($Config{'dlsrc'} =~ /^dl_dld/){" is
false, but this is the easier path right now
-DynaLoader requires module lib to build itself, but Extensions_nonxs
happens to not need mod lib, so move where mod lib is built
-in utils target, change it so it can run with miniperl, more parallelism,
pl2bat doesn't require full perl, the full perl stuff is from
commit 3fe9a6f19e from 5.003 and no ML archive is available to explain why
full perl was used instead of mini perl
-add 4 modules to buildcustomize so nonxs and xs build simultaneously
-remove ancient cruft from commit 26618a56da from 1997 from README.w32
dmake is automatically set inside CFG_VARS today. OSRELEASE is irrelavent
today, Perl doesn't use any of the defaults (like name of CC) from the
maketool.
-Reduce 21 calls of "cmd.exe /x/d/c cd" by miniperl to just 1 by saving and
reusing cwd. This reduced the amount of time to run
"..\miniperl.exe -I..\lib -f ..\write_buildcustomize.pl .." from 312 ms
to 62 ms elapsed time using timeit.exe for me.
Diffstat (limited to 'README.win32')
-rw-r--r-- | README.win32 | 27 |
1 files changed, 15 insertions, 12 deletions
diff --git a/README.win32 b/README.win32 index d6c50d7b46..f78f11377b 100644 --- a/README.win32 +++ b/README.win32 @@ -96,11 +96,16 @@ See L<Usage Hints for Perl on Windows> below for general hints about this. =item Make You need a "make" program to build the sources. If you are using -Visual C++ or the Windows SDK tools, nmake will work. Builds using -the gcc need dmake. - -dmake is a freely available make that has very nice macro features -and parallelability. +Visual C++ or the Windows SDK tools, you can use nmake supplied with Visual C++ +or Windows SDK. You may also use, for Visual C++ or Windows SDK, dmake instead +of nmake. dmake is open source software, but is not included with Visual C++ or +Windows SDK. If you want parallel building with Visual C++ or +Windows SDK you must use dmake instead of nmake. Builds using gcc need dmake. +nmake is not supported for gcc builds. gmake is not supported, but might be +added in the future. It is recommended to use dmake 4.13 or newer for parallel +building. Older dmakes, in parallel mode, have very high CPU usage and pound +the disk/filing system with duplicate I/O calls in an aggressive polling +loop. A port of dmake for Windows is available from: @@ -135,13 +140,6 @@ console already set up for your target architecture (x86-32 or x86-64 or IA64). With the newer compilers, you may also use the older batch files if you choose so. -You can also use dmake to build using Visual C++; provided, however, -you set OSRELEASE to "microsft" (or whatever the directory name -under which the Visual C dmake configuration lives) in your environment -and edit win32/config.vc to change "make=nmake" into "make=dmake". The -latter step is only essential if you want to use dmake as your default -make for building extensions using MakeMaker. - =item Microsoft Visual C++ 2008-2013 Express Edition These free versions of Visual C++ 2008-2013 Professional contain the same @@ -400,6 +398,11 @@ perl523.dll at the perl toplevel, and various other extension dll's under the lib\auto directory. If the build fails for any reason, make sure you have done the previous steps correctly. +To try dmake's parallel mode, type "dmake -P2", where 2, is the maximum number +of parallel jobs you want to run. A number of things in the build process will +run in parallel, but there are serialization points where you will see just 1 +CPU maxed out. This is normal. + If you are advanced enough with building C code, here is a suggestion to speed up building perl, and the later C<make test>. Try to keep your PATH enviromental variable with the least number of folders possible (remember to keep your C |