Work on my talk committed

3 files changed, 368 insertions, 170 deletions

diff --git a/pypers/scheme/my-lib.sls b/pypers/scheme/my-lib.sls
deleted file mode 100644
index 6149ddc..0000000
--- a/pypers/scheme/my-lib.sls
+++ /dev/null
@@ -1,8 +0,0 @@
- #!r6rs
- (library (my-lib)
-           (export a b)
-            (import (rnrs))
-             (define a 42)
-              (define b 0)
-               (display "my-lib instantiated!\n")
- )
diff --git a/pypers/scheme/talk.txt b/pypers/scheme/talk.txt
index d1df883..67c772e 100644
--- a/pypers/scheme/talk.txt
+++ b/pypers/scheme/talk.txt
@@ -6,128 +6,177 @@ The R6RS module system
 :date: 2009-05-28
 
 .. include:: <s5defs.txt>
-.. footer:: EuroLisp 2009
+.. footer:: Michele Simionato, EuroLisp 2009
 
-Introduction
-------------------------------------------------
+Part I
+--------------------------
 
-This is as a talk for macro writers who want to write
-portable R6RS code. It assumes:
+The easy things about the R6RS module system
 
-.. class:: incremental
+.. image:: Jigsaw.png
 
- - knowledge of (R5RS) Scheme
- - some knowledge of syntax-case macros
- - not much knowledge of R6RS Scheme
- - no knowledge of the R6RS module system
+Part II
+----------------------------------------
+
+Explicit phasing and the tower of meta-levels
+
+.. image:: exploding-head.jpg
+   :width: 400
+
+Part III
+--------------------------
+
+Porting libraries between different R6RS implementations
+
+.. image:: aps_lib_03.jpg
+   :width: 500
 
-Ok, now about me
+Self-presentation
 ----------------------------
 
+Who am I?
+
+.. image:: faccina.jpg
+
+
+About me
+----------------------------
+
+A hobbyist Scheme programmer
+
 .. class:: incremental
 
-- I am a hobbyist Scheme programmer
-- my background is in Python
+- I use Python at work
 - I started programming in Scheme 5+ years ago
-- author of "The Adventures of a Pythonista in Schemeland"
-- I love/hate Scheme
+- I am writing "The Adventures of a Pythonista in Schemeland" on Artima
+- the blog posts will become a book
+- I think my experience is relevant for new 
+  R6RS programmers
 
 What I have done in Scheme
 --------------------------------------
 
+- written a module called sweet-macros as sugar over syntax-case
+
 .. class:: incremental
 
-- written sweet-macros as a sugar over syntax-case
 - written various support libraries for the Adventures
 - fought a lot with portability issues
 - experimented with cutting edge features (found bugs!)
 - made a lot of noise in various mailing lists
-- got *impressive* support from Scheme implementors
-- my experience relevant for new R6RS programmers
+- got *impressive* support from Scheme implementors :-)
 
-R6RS modules in absence of macros
-----------------------------------
+Disclaimer
+------------------------------------------------
 
-- everything is (nearly) trivial
+This is as a talk for **macro** writers who want to write
+portable R6RS code. It assumes:
 
-::
+.. class:: incremental
+
+ - knowledge of (R5RS) Scheme
+ - knowledge of Scheme macros
+ - no knowledge of R6RS Scheme
+ - no knowledge of the R6RS module system
+
+Part I: no macros or simple macros
+----------------------------------------------------
 
- $ cat mylib.ss
+.. code-block:: scheme
+
+ $ cat my-lib.sls
  #!r6rs
  (library (my-lib)
  (export a b)
- (import (rnrs))
+ (import (rnrs)); standard R6RS bindings
  (define a 42)
  (define b 0)
  (display "my-lib instantiated!\n")
  )
 
- $ cat example.sls
- (import (rnrs) (my-lib))
- (display (+ a b))
- (newline)
-
-Easy features (I)
+Import syntax (I)
 --------------------------------------
 
+.. code-block:: scheme
+
+   (import (rnrs) (my-lib))
+   (display (+ a b)) ;=> 42
+
 .. class:: incremental
 
-- import with a prefix::
+- import with a prefix:
+
+  .. code-block:: scheme
 
-   (import (my-lib (prefix my-lib:)))
-   (display my-lib:a)
+   (import (rnrs) (my-lib (prefix my-lib:)))
+   (display my-lib:a) ;=> 42
 
-- import only a specific sets of names::
+- import only a specific sets of names:
 
-   (import (only (my-lib) a))
-   (display a)
+  .. code-block:: scheme
 
-Easy features (II)
+   (import (rnrs) (only (my-lib) a))
+   (display a) ;=> 42
+
+Import syntax (II)
 ----------------------------------------
 
+There are other easy features
+
 .. class:: incremental
 
-- rename a set of identifiers::
+- renaming a set of identifiers:
 
-   (import (rename (my-lib) (a ml:a)))
-   (display ml:a)
+  .. code-block:: scheme
 
-- exclude a set of identifiers::
+   (import (rnrs) (rename (my-lib) (a ml:a)))
+   (display ml:a) ;=> 42
 
-   (import (exclude (my-lib) a))
-   (display b)
+- excluding a set of identifiers:
 
-Easy features (III)
-----------------------------------------
+  .. code-block:: scheme
 
-- support for importing a specific version
+   (import (rnrs) (exclude (my-lib) a))
+   (display b) ;=> 0
 
-Missing features
+Limitations
 ----------------------------------------
 
+- how to map libraries to the file system is not specified
+
 .. class:: incremental
 
+- support for implementation-specific files (.IMPL.sls convention)
+  is in its infancy
+
+- support for importing a specific version is in fieri
+
 - ``(export *)`` not available
+
+  + you must list explicitly all the exported identifiers
+  + at least, it makes writing IDEs easier
+
 - no introspection API 
 
-Compatibility caveats
-------------------------------------------
+  + no (exported-vars my-lib)
+  + no (exported-macros my-lib)
 
-.. class:: incremental
+Let's start with macros now
+----------------------------------
 
-- the  module search path is implementation dependent
-- do not forget the #!r6rs line in PLT
-- each module needs a directory in PLT Scheme
-- instantiation semantics is implementation-dependent
-  (for instance non-used libraries are not instantiated
-  in Ikarus)
+They are not so scary ...
 
-R6RS modules with simple macros
--------------------------------------
+.. image:: mantid.jpg
+   :width: 560
 
-- no problem here::
+R6RS modules and syntax-rules
+---------------------------------------
 
- $ cat mylib.ss
+A simple example:
+
+.. code-block:: scheme
+
+ $ cat my-lib.ss
 
  #!r6rs
  (library (my-lib)
@@ -141,160 +190,220 @@ R6RS modules with simple macros
 Macro usage
 -------------------------
 
+99.99% of times there are no problems with syntax-rules macros:
+
+.. code-block:: scheme
+
  $ cat example.sls
  (import (rnrs) (my-lib))
  (display (double 1));  prints (1 1)
 
-Where is all the fuss?
--------------------------------------------------
+I will show an issue with a second order syntax-rules macro
+later on
 
-.. class:: incremental
+Why all the fuss?
+-------------------------------------------------
 
-- the problem is when you have macros depending on helper
-  variables/functions/macros
-- then you must understand *phase separation*
-- there are *three* different concepts of phase separation
-- all the possibilities are accepted by the R6RS
-- but they are incompatible!
+- the most common problem is when you have macro transformers
+  depending on helper variables:
 
-Explaining phase separation
-------------------------------------------------
+.. code-block:: scheme
 
-Here is the problem::
+   > (let ()
+       (define a 42)
+       (define-syntax m (lambda (x) a))
+       (m))
+   error: identifier a out of context
 
- > (let ((a 42))
-     (let-syntax ((m (lambda (x) a)))
-       (m)))
- error: identifer a out of context
+.. class:: incremental
 
-Why the identifier is not available to the macro?
+- Why the identifier is not available to the macro?
 
-Expand-time vs run-time
+Because of phase separation
 -------------------------------
 
-.. class:: incremental
+.. code-block:: scheme
 
- - because it is defined too late!
+  (let ()
+   (define a 42)                    ; run-time
+   (define-syntax m (lambda (x) a)) ; macro-def-time
+   (m))
 
-::
+.. class:: incremental
 
- (let ((a 42))                      ; run-time
-   (let-syntax ((m (lambda (x) a))) ; expand-time
-     (m)))                          ; expand-time
+ - ``a`` is defined too late!
 
-- regular definitions (both define and let) are performed
-  at runtime, whereas macro definitions (define-syntax
-  and let-syntax) are performed at expand-time.
+ - regular definitions (both define and let) are performed
+   at run-time
 
-The solution
---------------------------------------
+ - macro definitions (both define-syntax
+   and let-syntax) are performed at compile-time
 
-- put the helper object (value, function, macro)
-  in a different module and import it at expand time
+Phase errors
+-------------------------------------
 
-::
+One must be careful not to mix expand-time
+and run-time
 
- > (import (for (my-lib) expand))
- > (let-syntax ((m (lambda (x) a))) (m))
-   42
+.. image:: salvador-dali-clock.jpg 
 
 Beware! the REPL may betray you
 -------------------------------------------
-  
-.. class:: incremental
 
-- Ikarus, Ypsilon and others
+.. code-block:: scheme
 
-::
+ $ mzscheme # or larceny
+ > (define a 42)
+ > (let-syntax ((m (lambda (x) a))) (m))
+ reference to undefined identifier: a
+  
+but
+
+.. code-block:: scheme
 
-  $ ikarus
+  $ ikarus # or ypsilon
   > (define a 42)
   > (let-syntax ((m (lambda (x) a))) (m))
     42
 
-PLT REPL is smarter
----------------------------
+Old Scheme behavior
+----------------------------------------------------
 
- $ mzscheme
- > (define a 42)
+Phase separation is a "new" thing; for 
+instance Guile 1.8 has not phase separation:
+
+.. code-block:: scheme
+
+ guile> (let ()
+          (define a 42)
+          (define-macro (m) a) 
+          (m))
+ 42
+
+(the next version of Guile will have phase separation
+and some support for R6RS Scheme).
+
+To cope with phase separation
+--------------------------------------
+
+Put the helper object (value, function, macro)
+in a different module and import it at expand time
+
+.. code-block:: scheme
+
+ > (import (for (my-lib) expand))
  > (let-syntax ((m (lambda (x) a))) (m))
- reference to undefined identifier: a
-  
+   42
+
 .. class:: incremental
 
-- Notice: define-for-syntax is missing
+- slightly inconvenient
+- I wish we could include libraries in scripts :-(
 
-Weak phase separation vs strong phase separation
------------------------------------------------------
+So everthing is settled, right?
+-------------------------------------
 
-This script
+Not really, there are a few R6RS surprises ...
 
-::
+.. image:: joker_13.jpg
+   :width: 300
+
+The R6RS specification is loose
+-------------------------------------------------------
+
+An example:
 
-  (import (for (my-lib) expand))
+.. code-block:: scheme
+
+  (import (for (only (my-lib) a) expand))
   (display (let-syntax ((m (lambda (x) a))) (m)))
   (display a)
   
 .. class:: incremental
 
-- compiles and runs fine on system based on psyntax
-- does not even compile on PLT Scheme and Larceny!
+- R6RS implementations are required to support 
+  ``(import (for (my-lib) expand))`` *syntactically*
+- they are not required to honor it!
+- this code does not compile on PLT Scheme and Larceny
+- but it compiles and runs fine on all other systems!
+
+Lack of phase specification
+---------------------------------------------
+
+- systems based on psyntax (and Ypsilon) import the identifiers at 
+  all phases simultaneously
 
-The R6RS document is schizophrenic
--------------------------------------------------------
-  
 .. class:: incremental
 
-- R6RS implementations are required to support the syntax
- (import (for (my-lib) expand)) *formally*
+- you cannot import a name into a specific phase
+- I will argue this a good thing because it avoids the tower
+  of meta-levels (later)
+- the R6RS forbids reusing the same
+  name with different bindings in different phases anyway
 
-- they are not required to honor it!
 
-- it does not say weak phase separation is bad
-- it does not say strong phase separation is bad
-- it does not say anything about instantiation semantics
+Part II
+----------------------------
 
-You get the worse of two worlds
---------------------------------------------
+Fasten your seatbelts now ...
 
-Writers of portable code
+The Dark Tower of meta-levels
+-----------------------------------------------------
+
+.. image:: DarkTower.jpg
+
+Meta-levels in macros
+-------------------------------------------
+
+.. code-block:: scheme
+
+  (define-syntax macro                ; meta-level 0
+     (lambda (x)                      ; meta-level 1
+       (syntax-case x (literals ...)  ; meta-level 1
+         (pattern                     ; meta-level 1
+          fender                      ; meta-level 1
+          #'template))))              ; meta-level 0
 
 .. class:: incremental
 
-- cannot rely on the power of explicit phasing
-- cannot rely on the simplicity implicit phasing
-- cannot rely on single instantiation nor on multiple instantiation
-- apparently it was politically impossible to do better
+- the right hand side of macro definition refers to names which are
+  one meta-level up
+
+- inside a template one goes back one meta-level
+
+- there is an arbitrary number of positive meta-levels,
+  depending on the level of nesting
 
-The tower of metalevels
+An example at meta-level 2
 -----------------------------------------------------
 
-System with strong phase separation have a tower of metalevels
+.. code-block:: scheme
 
-  (define-syntax m1       ;; level 1
+  (import (for (only (my-lib) a) (meta 2))
+          (for (only (my-lib) b) (meta 1)))
+  (define-syntax m1       ;; level 0
    (lambda (x1)           ;; level 1
-      (define-syntax m2   ;; level 2
+      (define-syntax m2   ;; level 1
          (lambda (x2) a)) ;; level 2
       (+ (m2) b)))        ;; level 1
-
   (display                ;; level 0
     (m1)                  ;; level 1
   )                       ;; level 0
 
-Negative levels
--------------------------------------------
+Levels/Phases are ordered
+---------------------------------------
 
-- there is an arbitrary number of positive metalevels,
-  depending of how level of nesting you have
-  in macro definitions
+- innermost levels are compiled first
+- outermost templates are evaluated later
 
-- there is also an arbitrary number of negative
-  metalevels!!
+.. image:: list-comprehension.jpg 
 
-A second order macro
--------------------------
+A subtle second order macro
+---------------------------------------------
+
+.. code-block:: scheme
 
-(define-syntax very-static-table
+ (define-syntax very-static-table
   (syntax-rules ()
     ((_ (name value) ...) 
       (syntax-rules (<names> name ...)
@@ -310,39 +419,135 @@ A second order macro
 Phase levels
 ---------------------------------------
 
-(define-syntax very-static-table       ;; level 1
+.. code-block:: scheme
+
+ (define-syntax very-static-table      ;; level 0
   (syntax-rules ()                     ;; level 1
-    ((_ (name value) ...)              ;; level 0
+    ((_ (name value) ...)              ;; level 1
       (syntax-rules (<names> name ...) ;; level 0
         ((_ <names>)                   ;; level 0
-          '                            ;; level -1
-          (name ...))                  ;; level 0
+          '(name ...))                 ;; level -1
         ((_ name)                      ;; level 0
           value)                       ;; level -1
           ...))))
 
-(import (for (only (rnrs) quote) expand))
+Needs ``(import (for (only (rnrs) quote) (meta -1)))``
+
+Ikarus, Mosh, IronScheme ...
+---------------------------------------------------
+
+Such implementations do not need to worry about
+the Dark Tower. I think they will have a great future.
+
+.. image:: tower_of_babel.jpg
+
+You get the worse of two worlds
+--------------------------------------------
 
-Multiple instantiation
+Writers of portable code
+
+.. class:: incremental
+
+- cannot rely on the simplicity of implicit phasing
+- cannot rely on the full power of explicit phasing
+- cannot rely on a clearly defined import semantics
+- apparently it was politically impossible to do better
+
+Part III
+-------------------------
+
+Porting macro-rich R6RS libraries can be a rather heavy task ...
+
+.. image:: mule.jpg
+
+Beware of negative metalevels
+-------------------------------
+
+.. code-block:: scheme
+
+ (import (rnrs) (for (rnrs) (meta -1))
+ (for (sweet-macros helper1) (meta -1) (meta 0) (meta 1)))
+
+.. image:: sweet-macros.png
+   :width: 695
+
+Beware of bugs
+---------------------------------------------------
+
+I have found many bugs in different R6RS implementations
+while porting my ``sweet-macros`` library:
+
+- Ikarus (1)
+- Ypsilon (4)
+- PLT (3)
+- Larceny (1)
+
+*All fixed within hours!*
+
+Other difficulties I encountered
+------------------------------------------
+
+- I had to wait for the .IMPL.sls convention to be implemented
+
+- I am generating the helper modules required by PLT/Larceny
+  from the Ikarus/Ypsilon module
+
+While writing the APS libraries I have found many non-portable
+behaviours:
+
+.. class:: incremental
+
+- in implementations based on psyntax and in Ypsilon a module
+  is visited only if one of its macros is used
+
+More non-portable behavior
 ----------------------------------------------------
-                         
+
 .. class:: incremental
 
-- PLT Scheme has a special kind of extra strong phase separation
-- not only you can instantiate a module in one or more phases
-- each instance is totally independent
+- in implementations others than PLT, side-effects
+  can leak through phases
+
+- the number of times a library is instantiated is
+  totally implementation-dependent
+
+- it also depends if you are using separate compilation
+  or not.
+
+An example:
+
+.. code-block:: scheme
 
  $ cat two-phases.ss
- (import (my-lib) expand run)
+ (import (for (my-lib) expand run))
 
-Example in Larceny
--------------------
+Different runs
+--------------------------------------------
+
+::
+
+ $ plt-r6rs two-phases.ss
+ my-lib instantiated!
+ my-lib instantiated!
+
+ $ larceny -r6rs -program two-phases.ss
+ my-lib instantiated!
+
+ $ ypsilon --r6rs two-phases.ss
+ my-lib instantiated!
+
+ $ ikarus --r6rs-script two-phases.ss
 
 
 References
 ------------------------------
 
-- http://wsgi.org/wsgi
-- http://www.python.org/dev/peps/pep-0333
-- http://pythonpaste.org/do-it-yourself-framework.html
-- http://pylonshq.com/
+You can find all the details in my Adventures
+
+- http://www.artima.com
+- http://www.phyast.pitt.edu/~micheles/scheme/TheAdventuresofaPythonistainSchemeland.pdf
+- http://www.phyast.pitt.edu/~micheles/scheme/sweet-macros.zip
+
+and of course in the R6RS document
+
+- http://www.r6rs.org/final/html/r6rs/r6rs.html
diff --git a/pypers/scheme/two-phases.ss b/pypers/scheme/two-phases.ss
new file mode 100644
index 0000000..4461ffa
--- /dev/null
+++ b/pypers/scheme/two-phases.ss
@@ -0,0 +1 @@
+(import (for (experimental my-lib) expand run))