Enhanced "zip safety" analysis (including scan of win32.exe's) and have

EasyInstall act on zip safety flags.  Add a lot more docs for setuptools.

--HG--
branch : setuptools
extra : convert_revision : svn%3A6015fed2-1504-0410-9fe1-9d1591cc4771/sandbox/trunk/setuptools%4041115

4 files changed, 463 insertions, 198 deletions

diff --git a/EasyInstall.txt b/EasyInstall.txt
index 6c9d1079..91d2e5c3 100755
--- a/EasyInstall.txt
+++ b/EasyInstall.txt
@@ -303,6 +303,53 @@ versions managed by EasyInstall, you won't have any more conflicts to worry
 about!
 
 
+Compressed Installation
+-----------------------
+
+EasyInstall tries to install packages in zipped form, if it can.  Zipping
+packages can significantly increase Python's overall import performance if
+you're installing to``site-packages`` and not using the ``--multi`` option,
+because Python processes zipfile entries on ``sys.path`` much faster than it
+does directories.
+
+As of version 0.5a9, EasyInstall analyzes packages to determine whether they
+can be safely installed as a zipfile, and then acts on its analysis.  (Previous
+versions would not install a package as a zipfile unless you used the
+``--zip-ok`` option.)
+
+The current analysis approach is very conservative; it currenly looks for:
+
+ * Any use of the ``__file__`` or ``__path__`` variables (which should be
+   replaced with ``pkg_resources`` API calls)
+
+ * Possible use of ``inspect`` functions that expect to manipulate source files
+   (e.g. ``inspect.getsource()``)
+
+ * Any data files or C extensions (this restriction will be removed in a future
+   release, once the ``pkg_resources`` runtime has been hardened for multi-user
+   environments)
+
+If any of the above are found in the package being installed, EasyInstall will
+assume that the package cannot be safely run from a zipfile, and unzip it to
+a directory instead.  You can override this analysis with the ``-zip-ok`` flag,
+which will tell EasyInstall to install the package as a zipfile anyway.  Or,
+you can use the ``--always-unzip`` flag, in which case EasyInstall will always
+unzip, even if its analysis says the package is safe to run as a zipfile.
+
+Normally, however, it is simplest to let EasyInstall handle the determination
+of whether to zip or unzip, and only specify overrides when needed to work
+around a problem.  If you find you need to override EasyInstall's guesses, you
+may want to contact the package author and the EasyInstall maintainers, so that
+they can make appropriate changes in future versions.
+
+(Note: If a package uses ``setuptools`` in its setup script, the package author
+has the option to declare the package safe or unsafe for zipped usage via the
+``zip_safe`` argument to ``setup()``.  If the package author makes such a
+declaration, EasyInstall believes the package's author and does not perform its
+own analysis.  However, your command-line option, if any, will still override
+the package author's choice.)
+
+
 Reference Manual
 ================
 
@@ -348,27 +395,26 @@ Command-Line Options
 --------------------
 
 ``--zip-ok, -z``
-    Enable installing the package as a zip file. This can significantly
-    increase Python's overall import performance if you're installing to
-    ``site-packages`` and not using the ``--multi`` option, because Python
-    process zipfile entries on ``sys.path`` much faster than it does
-    directories. So, if you don't use this option, and you install a lot of
-    packages, some of them may be slower to import.
-
-    But, this option is disabled by default, unless you're installing from an
-    already-built binary zipfile (``.egg`` file). This is to avoid problems
-    when using packages that dosn't support running from a zip file. Such
-    packages usually access data files in their package directories using the
-    Python ``__file__`` or ``__path__`` attribute, instead of the
-    ``pkg_resources`` API. So, if you find that a package doesn't work properly
-    when used with this option, you may want to suggest to the author that they
-    switch to using the ``pkg_resources`` resource API, which will allow their
-    package to work whether it's installed as a zipfile or not.
-
-    (Note: this option only affects the installation of newly-built packages
-    that are not already installed in the target directory; if you want to
-    convert an existing installed version from zipped to unzipped or vice
-    versa, you'll need to delete the existing version first.)
+    Install all packages as zip files, even if they are marked as unsafe for
+    running as a zipfile.  This can be useful when EasyInstall's analysis
+    of a non-setuptools package is too conservative, but keep in mind that
+    the package may not work correctly.  (Changed in 0.5a9; previously this
+    option was required in order for zipped installation to happen at all.)
+
+``--always-unzip, -Z``
+    Don't install any packages as zip files, even if the packages are marked
+    as safe for running as a zipfile.  This can be useful if a package does
+    something unsafe, but not in a way that EasyInstall can easily detect.
+    EasyInstall's default analysis is currently very conservative, however, so
+    you should only use this option if you've had problems with a particular
+    package, and *after* reporting the problem to the package's maintainer and
+    to the EasyInstall maintainers.
+
+    (Note: the ``-z/-Z`` options only affect the installation of newly-built
+    or downloaded packages that are not already installed in the target
+    directory; if you want to convert an existing installed version from
+    zipped to unzipped or vice versa, you'll need to delete the existing
+    version first, and re-run EasyInstall.)
 
 ``--multi-version, -m``
     "Multi-version" mode. Specifying this option prevents ``easy_install`` from
@@ -564,6 +610,15 @@ Known Issues
  * EasyInstall can now be given a path to a directory containing a setup
    script, and it will attempt to build and install the package there.
 
+ * EasyInstall now performs a safety analysis on module contents to determine
+   whether a package is likely to run in zipped form, and displays
+   information about what modules may be doing introspection that would break
+   when running as a zipfile.
+
+ * Added the ``--always-unzip/-Z`` option, to force unzipping of packages that
+   would ordinarily be considered safe to unzip, and changed the meaning of
+   ``--zip-ok/-z`` to "always leave everything zipped".
+   
 0.5a8
  * There is now a separate documentation page for `setuptools`_; revision
    history that's not specific to EasyInstall has been moved to that page.
diff --git a/setuptools.txt b/setuptools.txt
index a2e21e7f..83483fac 100755
--- a/setuptools.txt
+++ b/setuptools.txt
@@ -183,6 +183,48 @@ unless you need the associated ``setuptools`` feature.
     for more information.
 
 
+Using ``find_packages()``
+-------------------------
+
+For simple projects, it's usually easy enough to manually add packages to
+the ``packages`` argument of ``setup()``.  However, for very large projects
+(Twisted, PEAK, Zope, Chandler, etc.), it can be a big burden to keep the
+package list updated.  That's what ``setuptools.find_packages()`` is for.
+
+``find_packages()`` takes a source directory, and a list of package names or
+patterns to exclude.  If omitted, the source directory defaults to the same
+directory as the setup script.  Some projects use a ``src`` or ``lib``
+directory as the root of their source tree, and those projects would of course
+use ``"src"`` or ``"lib"`` as the first argument to ``find_packages()``.  (And
+such projects also need something like ``package_dir = {'':'src'}`` in their
+``setup()`` arguments, but that's just a normal distutils thing.)
+
+Anyway, ``find_packages()`` walks the target directory, and finds Python
+packages by looking for ``__init__.py`` files.  It then filters the list of
+packages using the exclusion patterns.  
+
+Exclusion patterns are package names, optionally including wildcards.  For
+example, ``find_packages(exclude=["*.tests"])`` will exclude all packages whose
+last name part is ``tests``.   Or, ``find_packages(exclude=["*.tests",
+"*.tests.*"])`` will also exclude any subpackages of packages named ``tests``,
+but it still won't exclude a top-level ``tests`` package or the children
+thereof.  In fact, if you really want no ``tests`` packages at all, you'll need
+something like this::
+
+    find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"])
+
+in order to cover all the bases.  Really, the exclusion patterns are intended
+to cover simpler use cases than this, like excluding a single, specified
+package and its subpackages.
+
+Regardless of the target directory or exclusions, the ``find_packages()``
+function returns a list of package names suitable for use as the ``packages``
+argument to ``setup()``, and so is usually the easiest way to set that
+argument in your setup script.  Especially since it frees you from having to
+remember to modify your setup script whenever your project grows additional
+top-level packages or subpackages.
+
+
 Declaring Dependencies
 ======================
 
@@ -305,71 +347,6 @@ so that Package B doesn't have to remove the ``[PDF]`` from its requirement
 specifier.
 
 
-Distributing a ``setuptools``-based package
-===========================================
-
-Your users might not have ``setuptools`` installed on their machines, or even
-if they do, it might not be the right version.  Fixing this is easy; just
-download `ez_setup.py`_, and put it in the same directory as your ``setup.py``
-script.  (Be sure to add it to your revision control system, too.)  Then add
-these two lines to the very top of your setup script, before the script imports
-anything from setuptools::
-
-    import ez_setup
-    ez_setup.use_setuptools()
-
-That's it.  The ``ez_setup`` module will automatically download a matching
-version of ``setuptools`` from PyPI, if it isn't present on the target system.
-Whenever you install an updated version of setuptools, you should also update
-your projects' ``ez_setup.py`` files, so that a matching version gets installed
-on the target machine(s).
-
-By the way, setuptools supports the new PyPI "upload" command, so you can use
-``setup.py sdist upload`` or ``setup.py bdist_egg upload`` to upload your
-source or egg distributions respectively.  Your project's current version must
-be registered with PyPI first, of course; you can use ``setup.py register`` to
-do that.  Or you can do it all in one step, e.g. ``setup.py register sdist
-bdist_egg upload`` will register the package, build source and egg
-distributions, and then upload them both to PyPI, where they'll be easily
-found by other projects that depend on them.
-
-
-Managing Multiple Projects
---------------------------
-
-If you're managing several projects that need to use ``ez_setup``, and you are
-using Subversion as your revision control system, you can use the
-"svn:externals" property to share a single copy of ``ez_setup`` between
-projects, so that it will always be up-to-date whenever you check out or update
-an individual project, without having to manually update each project to use
-a new version.
-
-However, because Subversion only supports using directories as externals, you
-have to turn ``ez_setup.py`` into ``ez_setup/__init__.py`` in order to do this,
-then create "externals" definitions that map the ``ez_setup`` directory into
-each project.  Also, if any of your projects use ``find_packages()`` on their
-setup directory, you will need to exclude the resulting ``ez_setup`` package,
-to keep it from being included in your distributions, e.g.::
-
-    setup(
-        ...
-        packages = find_packages(exclude=['ez_setup']),
-    )
-
-Of course, the ``ez_setup`` package will still be included in your packages'
-source distributions, as it needs to be.
-
-For your convenience, you may use the following external definition, which will
-track the latest version of setuptools::
-
-    ez_setup svn://svn.eby-sarna.com/svnroot/ez_setup
-
-You can set this by executing this command in your project directory::
-
-    svn propedit svn:externals .
-
-And then adding the line shown above to the file that comes up for editing.
-
 
 Including Data Files
 ====================
@@ -457,12 +434,6 @@ a quick example of converting code that uses ``__file__`` to use
 .. _Accessing Package Resources: http://peak.telecommunity.com/DevCenter/PythonEggs#accessing-package-resources
 
 
-Setting the ``zip_safe`` flag
------------------------------
-
-XXX put doc about zip_safe flag here, once it's implemented
-
-
 "Development Mode"
 ==================
 
@@ -510,35 +481,128 @@ There are several options to control the precise behavior of the ``develop``
 command; see the section on the `develop`_ command below for more details.
 
 
-Tagging and "Daily Build" or "Snapshot" Releases
-================================================
+Distributing a ``setuptools``-based package
+===========================================
 
-Sorry, this section isn't written yet, and neither are the next few sections,
-until you get to the `Command Reference`_ section below.  You might want to
-`subscribe to changes in this page <setuptools?action=subscribe>`_ to see when
-new documentation is added or updated.
 
+Using ``setuptools``...  Without bundling it!
+---------------------------------------------
 
-Generating Source Distributions
-===============================
+Your users might not have ``setuptools`` installed on their machines, or even
+if they do, it might not be the right version.  Fixing this is easy; just
+download `ez_setup.py`_, and put it in the same directory as your ``setup.py``
+script.  (Be sure to add it to your revision control system, too.)  Then add
+these two lines to the very top of your setup script, before the script imports
+anything from setuptools::
+
+    import ez_setup
+    ez_setup.use_setuptools()
 
-XXX ``sdist`` - auto-include files from CVS or Subversion
+That's it.  The ``ez_setup`` module will automatically download a matching
+version of ``setuptools`` from PyPI, if it isn't present on the target system.
+Whenever you install an updated version of setuptools, you should also update
+your projects' ``ez_setup.py`` files, so that a matching version gets installed
+on the target machine(s).
 
+By the way, setuptools supports the new PyPI "upload" command, so you can use
+``setup.py sdist upload`` or ``setup.py bdist_egg upload`` to upload your
+source or egg distributions respectively.  Your project's current version must
+be registered with PyPI first, of course; you can use ``setup.py register`` to
+do that.  Or you can do it all in one step, e.g. ``setup.py register sdist
+bdist_egg upload`` will register the package, build source and egg
+distributions, and then upload them both to PyPI, where they'll be easily
+found by other projects that depend on them.
 
-Using ``find_packages()``
-=========================
 
-XXX
+Managing Multiple Projects
+--------------------------
+
+If you're managing several projects that need to use ``ez_setup``, and you are
+using Subversion as your revision control system, you can use the
+"svn:externals" property to share a single copy of ``ez_setup`` between
+projects, so that it will always be up-to-date whenever you check out or update
+an individual project, without having to manually update each project to use
+a new version.
 
+However, because Subversion only supports using directories as externals, you
+have to turn ``ez_setup.py`` into ``ez_setup/__init__.py`` in order to do this,
+then create "externals" definitions that map the ``ez_setup`` directory into
+each project.  Also, if any of your projects use ``find_packages()`` on their
+setup directory, you will need to exclude the resulting ``ez_setup`` package,
+to keep it from being included in your distributions, e.g.::
+
+    setup(
+        ...
+        packages = find_packages(exclude=['ez_setup']),
+    )
 
-Building Extensions written with Pyrex
-======================================
+Of course, the ``ez_setup`` package will still be included in your packages'
+source distributions, as it needs to be.
 
-XXX
+For your convenience, you may use the following external definition, which will
+track the latest version of setuptools::
+
+    ez_setup svn://svn.eby-sarna.com/svnroot/ez_setup
+
+You can set this by executing this command in your project directory::
+
+    svn propedit svn:externals .
+
+And then adding the line shown above to the file that comes up for editing.
+
+
+Setting the ``zip_safe`` flag
+-----------------------------
+
+For maximum performance, Python packages are best installed as zip files.
+Not all packages, however, are capable of running in compressed form, because
+they may expect to be able to access either source code or data files as
+normal operating system files.  So, ``setuptools`` can install your project
+as a zipfile or a directory, and its default choice is determined by the
+project's ``zip_safe`` flag.
+
+You can pass a True or False value for the ``zip_safe`` argument to the
+``setup()`` function, or you can omit it.  If you omit it, the ``bdist_egg``
+command will analyze your project's contents to see if it can detect any
+conditions that would prevent it from working in a zipfile.  It will output
+notices to the console about any such conditions that it finds.
+
+Currently, this analysis is extremely conservative: it will consider the
+project unsafe if it contains any C extensions or datafiles whatsoever.  This
+does *not* mean that the project can't or won't work as a zipfile!  It just
+means that the ``bdist_egg`` authors aren't yet comfortable asserting that
+the project *will* work.  If the project contains no C or data files, and does
+no ``__file__`` or ``__path__`` introspection or source code manipulation, then
+there is an extremely solid chance the project will work when installed as a
+zipfile.  (And if the project uses ``pkg_resources`` for all its data file
+access, then C extensions and other data files shouldn't be a problem at all.
+See the `Accessing Data Files at Runtime`_ section above for more information.)
+
+However, if ``bdist_egg`` can't be *sure* that your package will work, but
+you've checked over all the warnings it issued, and you are either satisfied it
+*will* work (or if you want to try it for yourself), then you should set
+``zip_safe`` to ``True`` in your ``setup()`` call.  If it turns out that it
+doesn't work, you can always change it to ``False``, which will force
+``setuptools`` to install your project as a directory rather than as a zipfile.
+
+Of course, the end-user can still override either decision, if they are using
+EasyInstall to install your package.  And, if you want to override for testing
+purposes, you can just run ``setup.py easy_install --zip-ok .`` or ``setup.py
+easy_install --always-unzip .`` in your project directory. to install the
+package as a zipfile or directory, respectively.
+
+In the future, as we gain more experience with different packages and become
+more satisfied with the robustness of the ``pkg_resources`` runtime, the
+"zip safety" analysis may become less conservative.  However, we strongly
+recommend that you determine for yourself whether your project functions
+correctly when installed as a zipfile, correct any problems if you can, and
+then make an explicit declaration of ``True`` or ``False`` for the ``zip_safe``
+flag, so that it will not be necessary for ``bdist_egg`` or ``EasyInstall`` to
+try to guess whether your project can work as a zipfile.
 
 
 Namespace Packages
-==================
+------------------
 
 Sometimes, a large package is more useful if distributed as a collection of
 smaller eggs.  However, Python does not normally allow the contents of a
@@ -593,6 +657,109 @@ like ``org.apache`` as a namespace for packages that are part of apache.org
 projects.)
 
 
+Tagging and "Daily Build" or "Snapshot" Releases
+------------------------------------------------
+
+When a set of related projects are under development, it may be important to
+track finer-grained version increments than you would normally use for e.g.
+"stable" releases.  While stable releases might be measured in dotted numbers
+with alpha/beta/etc. status codes, development versions of a project often
+need to be tracked by revision or build number or even build date.  This is
+especially true when projects in development need to refer to one another, and
+therefore may literally need an up-to-the-minute version of something!
+
+To support these scenarios, ``setuptools`` allows you to "tag" your source and
+egg distributions by adding one or more of the following to the project's
+"official" version identifier:
+
+* An identifying string, such as "build" or "dev", or a manually-tracked build
+  or revision number (``--tag-build=STRING, -bSTRING``)
+
+* A "last-modified revision number" string generated automatically from 
+  Subversion's metadata (assuming your project is being built from a Subversion
+  "working copy")  (``--tag-svn-revision, -r``)
+
+* An 8-character representation of the build date (``--tag-date, -d``)
+
+You can add these tags by adding ``egg_info`` and the desired options to
+the command line ahead of the ``sdist`` or ``bdist`` commands that you want
+to generate a daily build or snapshot for.  See the section below on the
+`egg_info`_ command for more details.
+
+Also, if you are creating builds frequently, and either building them in a
+downloadable location or are copying them to a distribution server, you should
+probably also check out the `rotate`_ command, which lets you automatically
+delete all but the N most-recently-modified distributions matching a glob
+pattern.  So, you can use a command line like::
+
+    setup.py egg_info -rbDEV bdist_egg rotate -m.egg -k3
+
+to build an egg whose version info includes 'DEV-rNNNN' (where NNNN is the
+most recent Subversion revision that affected the source tree), and then
+delete any egg files from the distribution directory except for the three
+that were built most recently.
+
+If you have to manage automated builds for multiple packages, each with
+different tagging and rotation policies, you may also want to check out the
+`alias`_ command, which would let each package define an alias like ``daily``
+that would perform the necessary tag, build, and rotate commands.  Then, a
+simpler scriptor cron job could just run ``setup.py daily`` in each project
+directory.  (And, you could also define sitewide or per-user default versions
+of the ``daily`` alias, so that projects that didn't define their own would
+use the appropriate defaults.)
+
+
+Generating Source Distributions
+-------------------------------
+
+``setuptools`` enhances the distutils' default algorithm for source file
+selection, so that all files managed by CVS or Subversion in your project tree
+are included in any source distribution you build.  This is a big improvement
+over having to manually write a ``MANIFEST.in`` file and try to keep it in
+sync with your project.  So, if you are using CVS or Subversion, and your
+source distributions only need to include files that you're tracking in
+revision control, don't create a a ``MANIFEST.in`` file for your project.
+(And, if you already have one, you might consider deleting it the next time
+you would otherwise have to change it.)
+
+Unlike the distutils, ``setuptools`` regenerates the source distribution
+``MANIFEST`` file every time you build a source distribution, as long as you
+*don't* have a ``MANIFEST.in`` file.  If you do have a ``MANIFEST.in`` (e.g.
+because you aren't using CVS or Subversion), then you'll have to follow the
+normal distutils procedures for managing what files get included in a source
+distribution, and setuptools' enhanced algorithms will *not* be used.
+
+(Note, by the way, that if you're using some other revision control system, you
+might consider submitting a patch to the ``setuptools.command.sdist`` module
+so we can include support for it, too.)
+
+
+Distributing Extensions compiled with Pyrex
+-------------------------------------------
+
+``setuptools`` includes transparent support for building Pyrex extensions, as
+long as you define your extensions using ``setuptools.Extension``, *not*
+``distutils.Extension``.  You must also not import anything from Pyrex in
+your setup script.
+
+If you follow these rules, you can safely list ``.pyx`` files as the source
+of your ``Extension`` objects in the setup script.  ``setuptools`` will detect
+at build time whether Pyrex is installed or not.  If it is, then ``setuptools``
+will use it.  If not, then ``setuptools`` will silently change the
+``Extension`` objects to refer to the ``.c`` counterparts of the ``.pyx``
+files, so that the normal distutils C compilation process will occur.
+
+Of course, for this to work, your source distributions must include the C
+code generated by Pyrex, as well as your original ``.pyx`` files.  This means
+that you will probably want to include current ``.c`` files in your revision
+control system, rebuilding them whenever you check changes in for the ``.pyx``
+source files.  This will ensure that people tracking your project in CVS or
+Subversion will be able to build it even if they don't have Pyrex installed,
+and that your source releases will be similarly usable with or without Pyrex.
+
+
+
+
 -----------------
 Command Reference
 -----------------
@@ -1192,6 +1359,8 @@ Release Notes/Change History
    "unmanaged" packages when installing the distribution.
 
  * Added ``zip_safe`` and ``namespace_packages`` arguments to ``setup()``.
+   Added package analysis to determine zip-safety if the ``zip_safe`` flag
+   is not given, and advise the author regarding what code might need changing.
 
  * Fixed the swapped ``-d`` and ``-b`` options of ``bdist_egg``.
 
diff --git a/setuptools/command/bdist_egg.py b/setuptools/command/bdist_egg.py
index 621bbb18..e75a4a9c 100644
--- a/setuptools/command/bdist_egg.py
+++ b/setuptools/command/bdist_egg.py
@@ -221,9 +221,7 @@ class bdist_egg(Command):
             if os.path.isfile(path):
                 self.copy_file(path,os.path.join(egg_info,filename))
 
-        # Write a zip safety flag file
-        flag = self.zip_safe() and 'zip-safe' or 'not-zip-safe'
-        open(os.path.join(archive_root,'EGG-INFO',flag),'w').close()
+        write_safety_flag(archive_root, self.zip_safe())
 
         if os.path.exists(os.path.join(self.egg_info,'depends.txt')):
             log.warn(
@@ -231,8 +229,9 @@ class bdist_egg(Command):
                 "Use the install_requires/extras_require setup() args instead."
             )
 
-        if self.exclude_source_files: self.zap_pyfiles()
-
+        if self.exclude_source_files:
+            self.zap_pyfiles()
+        
         # Make the archive
         make_zipfile(self.egg_output, archive_root, verbose=self.verbose,
                           dry_run=self.dry_run)
@@ -244,77 +243,119 @@ class bdist_egg(Command):
             ('bdist_egg',get_python_version(),self.egg_output))
 
 
+
     def zap_pyfiles(self):
         log.info("Removing .py files from temporary directory")
-        for base,dirs,files in self.walk_contents():
+        for base,dirs,files in walk_egg(self.bdist_dir):
             for name in files:
                 if name.endswith('.py'):
                     path = os.path.join(base,name)
                     log.debug("Deleting %s", path)
                     os.unlink(path)
 
-    def walk_contents(self):
-        """Walk files about to be archived, skipping the metadata directory"""
-        walker = os.walk(self.bdist_dir)
-        base,dirs,files = walker.next()       
-        if 'EGG-INFO' in dirs:
-            dirs.remove('EGG-INFO')
-
-        yield base,dirs,files
-        for bdf in walker:
-            yield bdf
-            
     def zip_safe(self):
         safe = getattr(self.distribution,'zip_safe',None)
         if safe is not None:
             return safe
         log.warn("zip_safe flag not set; analyzing archive contents...")
-        safe = True
-        for base, dirs, files in self.walk_contents():
-            for name in files:
-                if name.endswith('.py') or name.endswith('.pyw'):
-                    continue
-                elif name.endswith('.pyc') or name.endswith('.pyo'):
-                    # always scan, even if we already know we're not safe
-                    safe = self.scan_module(base, name) and safe
-                elif safe:
-                    log.warn(
-                        "Distribution contains data or extensions; assuming "
-                        "it's unsafe (set zip_safe=True in setup() to change"
-                    )
-                    safe = False    # XXX
-        return safe
-
-    def scan_module(self, base, name):
-        """Check whether module possibly uses unsafe-for-zipfile stuff"""
-        filename = os.path.join(base,name)
-        if filename[:-1] in self.stubs:
-            return True     # Extension module
-
-        pkg = base[len(self.bdist_dir)+1:].replace(os.sep,'.')
-        module = pkg+(pkg and '.' or '')+os.path.splitext(name)[0]
-
-        f = open(filename,'rb'); f.read(8)   # skip magic & date
-        code = marshal.load(f);  f.close()
-        safe = True
-
-        symbols = dict.fromkeys(iter_symbols(code))
-        for bad in ['__file__', '__path__']:
-            if bad in symbols:
-                log.warn("%s: module references %s", module, bad)
-                safe = False
-        if 'inspect' in symbols:
-            for bad in [
-                'getsource', 'getabsfile', 'getsourcefile', 'getfile'
-                'getsourcelines', 'findsource', 'getcomments', 'getframeinfo',
-                'getinnerframes', 'getouterframes', 'stack', 'trace'
-            ]:
-                if bad in symbols:
-                    log.warn("%s: module MAY be using inspect.%s", module, bad)
-                    safe = False           
-        return safe
+        return analyze_egg(self.bdist_dir, self.stubs)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
 
 
+
+
+
+
+
+def walk_egg(egg_dir):
+    """Walk an unpacked egg's contents, skipping the metadata directory"""
+    walker = os.walk(egg_dir)
+    base,dirs,files = walker.next()       
+    if 'EGG-INFO' in dirs:
+        dirs.remove('EGG-INFO')
+    yield base,dirs,files
+    for bdf in walker:
+        yield bdf
+
+def analyze_egg(egg_dir, stubs):
+    safe = True
+    for base, dirs, files in walk_egg(egg_dir):
+        for name in files:
+            if name.endswith('.py') or name.endswith('.pyw'):
+                continue
+            elif name.endswith('.pyc') or name.endswith('.pyo'):
+                # always scan, even if we already know we're not safe
+                safe = scan_module(egg_dir, base, name, stubs) and safe
+            '''elif safe:
+                log.warn(
+                    "Distribution contains data or extensions; assuming "
+                    "it's unsafe (set zip_safe=True in setup() to change"
+                )
+                safe = False    # XXX'''
+    return safe
+
+def write_safety_flag(egg_dir, safe):
+    # Write a zip safety flag file
+    flag = safe and 'zip-safe' or 'not-zip-safe'
+    open(os.path.join(egg_dir,'EGG-INFO',flag),'w').close()
+
+
+
+
+
+
+
+
+
+
+def scan_module(egg_dir, base, name, stubs):
+    """Check whether module possibly uses unsafe-for-zipfile stuff"""
+
+    filename = os.path.join(base,name)
+    if filename[:-1] in stubs:
+        return True     # Extension module
+
+    pkg = base[len(egg_dir)+1:].replace(os.sep,'.')
+    module = pkg+(pkg and '.' or '')+os.path.splitext(name)[0]
+
+    f = open(filename,'rb'); f.read(8)   # skip magic & date
+    code = marshal.load(f);  f.close()
+    safe = True
+
+    symbols = dict.fromkeys(iter_symbols(code))
+    for bad in ['__file__', '__path__']:
+        if bad in symbols:
+            log.warn("%s: module references %s", module, bad)
+            safe = False
+    if 'inspect' in symbols:
+        for bad in [
+            'getsource', 'getabsfile', 'getsourcefile', 'getfile'
+            'getsourcelines', 'findsource', 'getcomments', 'getframeinfo',
+            'getinnerframes', 'getouterframes', 'stack', 'trace'
+        ]:
+            if bad in symbols:
+                log.warn("%s: module MAY be using inspect.%s", module, bad)
+                safe = False           
+    return safe
+
 def iter_symbols(code):
     """Yield names and strings used by `code` and its nested code objects"""
     for name in code.co_names: yield name
diff --git a/setuptools/command/easy_install.py b/setuptools/command/easy_install.py
index b80dcb8d..1b3b72a6 100755
--- a/setuptools/command/easy_install.py
+++ b/setuptools/command/easy_install.py
@@ -67,6 +67,7 @@ class easy_install(Command):
          "-O2 for \"python -OO\", and -O0 to disable [default: -O0]"),
         ('record=', None,
          "filename in which to record list of installed files"),
+        ('always-unzip', 'Z', "don't install as a zipfile, no matter what")
     ]
 
     boolean_options = [
@@ -74,12 +75,11 @@ class easy_install(Command):
         'delete-conflicting', 'ignore-conflicts-at-my-risk',
     ]
 
+    negative_opt = {'always-unzip': 'zip-ok'}
     create_index = PackageIndex
 
 
 
-
-
     def initialize_options(self):
         self.zip_ok = None
         self.install_dir = self.script_dir = self.exclude_scripts = None
@@ -291,7 +291,7 @@ class easy_install(Command):
         install_needed = install_needed or not download.endswith('.egg')
         log.info("Processing %s", os.path.basename(download))
         if install_needed or self.always_copy:
-            dists = self.install_eggs(download, self.zip_ok, tmpdir)
+            dists = self.install_eggs(download, tmpdir)
             for dist in dists:
                 self.process_distribution(spec, dist)
         else:
@@ -337,14 +337,14 @@ class easy_install(Command):
                 metadata.get_metadata('scripts/'+script_name).replace('\r','\n')
             )
 
-
-
-
-
-
-
-
-
+    def should_unzip(self, dist):
+        if self.zip_ok is not None:
+            return not self.zip_ok
+        if dist.metadata.has_metadata('not-zip-safe'):
+            return True
+        if not dist.metadata.has_metadata('zip-safe'):
+            return True
+        return False
 
 
 
@@ -408,10 +408,10 @@ class easy_install(Command):
                 pass
 
 
-    def install_eggs(self, dist_filename, zip_ok, tmpdir):
+    def install_eggs(self, dist_filename, tmpdir):
         # .egg dirs or files are already built, so just return them
         if dist_filename.lower().endswith('.egg'):
-            return [self.install_egg(dist_filename, True, tmpdir)]
+            return [self.install_egg(dist_filename, tmpdir)]
         elif dist_filename.lower().endswith('.exe'):
             return [self.install_exe(dist_filename, tmpdir)]
 
@@ -438,7 +438,7 @@ class easy_install(Command):
             setup_script = setups[0]
 
         # Now run it, and return the result
-        return self.build_and_install(setup_script, setup_base, zip_ok)
+        return self.build_and_install(setup_script, setup_base)
 
 
 
@@ -456,13 +456,14 @@ class easy_install(Command):
             metadata = EggMetadata(zipimport.zipimporter(egg_path))
         return Distribution.from_filename(egg_path,metadata=metadata)
 
-    def install_egg(self, egg_path, zip_ok, tmpdir):
+    def install_egg(self, egg_path, tmpdir):
         destination = os.path.join(self.install_dir,os.path.basename(egg_path))
         destination = os.path.abspath(destination)
         if not self.dry_run:
             ensure_directory(destination)
 
-        self.check_conflicts(self.egg_distribution(egg_path))
+        dist = self.egg_distribution(egg_path)
+        self.check_conflicts(dist)
         if not samefile(egg_path, destination):
             if os.path.isdir(destination):
                 dir_util.remove_tree(destination, dry_run=self.dry_run)
@@ -474,14 +475,13 @@ class easy_install(Command):
                     f,m = shutil.move, "Moving"
                 else:
                     f,m = shutil.copytree, "Copying"
-            elif zip_ok:
-                if egg_path.startswith(tmpdir):
-                    f,m = shutil.move, "Moving"
-                else:
-                    f,m = shutil.copy2, "Copying"
-            else:
+            elif self.should_unzip(dist):
                 self.mkpath(destination)
                 f,m = self.unpack_and_compile, "Extracting"
+            elif egg_path.startswith(tmpdir):
+                f,m = shutil.move, "Moving"
+            else:
+                f,m = shutil.copy2, "Copying"
 
             self.execute(f, (egg_path, destination),
                 (m+" %s to %s") %
@@ -526,17 +526,15 @@ class easy_install(Command):
         )
 
         # install the .egg
-        return self.install_egg(egg_path, self.zip_ok, tmpdir)
+        return self.install_egg(egg_path, tmpdir)
 
 
 
 
     def exe_to_egg(self, dist_filename, egg_tmp):
         """Extract a bdist_wininst to the directories an egg would use"""
-
         # Check for .pth file and set up prefix translations
         prefixes = get_exe_prefixes(dist_filename)
-
         to_compile = []
         native_libs = []
         top_level = {}
@@ -561,16 +559,18 @@ class easy_install(Command):
 
         # extract, tracking .pyd/.dll->native_libs and .py -> to_compile
         unpack_archive(dist_filename, egg_tmp, process)
-
+        stubs = []
         for res in native_libs:
             if res.lower().endswith('.pyd'):    # create stubs for .pyd's
                 parts = res.split('/')
                 resource, parts[-1] = parts[-1], parts[-1][:-1]
                 pyfile = os.path.join(egg_tmp, *parts)
-                to_compile.append(pyfile)
+                to_compile.append(pyfile); stubs.append(pyfile)
                 bdist_egg.write_stub(resource, pyfile)
 
         self.byte_compile(to_compile)   # compile .py's
+        bdist_egg.write_safety_flag(egg_tmp,
+            bdist_egg.analyze_egg(egg_tmp, stubs))  # write zip-safety flag
 
         for name in 'top_level','native_libs':
             if locals()[name]:
@@ -695,7 +695,7 @@ PYTHONPATH, or by being added to sys.path by your code.)
 
 
 
-    def build_and_install(self, setup_script, setup_base, zip_ok):
+    def build_and_install(self, setup_script, setup_base):
         sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg)
         sys.modules.setdefault('distutils.command.bdist_egg', egg_info)
 
@@ -724,7 +724,7 @@ PYTHONPATH, or by being added to sys.path by your code.)
 
             eggs = []
             for egg in glob(os.path.join(dist_dir,'*.egg')):
-                eggs.append(self.install_egg(egg, zip_ok, setup_base))
+                eggs.append(self.install_egg(egg, setup_base))
 
             if not eggs and not self.dry_run:
                 log.warn("No eggs found in %s (setup script problem?)",