Merge branch 'fl/cvsserver'

* fl/cvsserver:
  config.txt: Add gitcvs.db* variables
  cvsserver: Document the GIT branches -> CVS modules mapping more prominently
  cvsserver: Reword documentation on necessity of write access
  cvsserver: Allow to "add" a removed file
  cvsserver: Add asciidoc documentation for new database backend configuration
  cvsserver: Corrections to the database backend configuration
  cvsserver: Use DBI->table_info instead of DBI->tables
  cvsserver: Abort if connect to database fails
  cvsserver: Make the database backend configurable
  cvsserver: Allow to override the configuration per access method
  cvsserver: Handle three part keys in git config correctly
  cvsserver: Introduce new state variable 'method'

Conflicts:

	Documentation/config.txt

3 files changed, 224 insertions, 18 deletions

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 7e41ca6a0d..2c0a666323 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -423,8 +423,34 @@ gitcvs.allbinary::
 	causes the client to treat all files as binary files which suppresses
 	any newline munging it otherwise might do. A work-around for the
 	fact that there is no way yet to set single files to mode '-kb'.
+
+gitcvs.dbname::
+	Database used by git-cvsserver to cache revision information
+	derived from the git repository. The exact meaning depends on the
+	used database driver, for SQLite (which is the default driver) this
+	is a filename. Supports variable substitution (see
+	gitlink:git-cvsserver[1] for details). May not contain semicolons (`;`).
+	Default: '%Ggitcvs.%m.sqlite'
+
+gitcvs.dbdriver::
+	Used Perl DBI driver. You can specify any available driver
+        for this here, but it might not work. git-cvsserver is tested
+	with 'DBD::SQLite', reported to work with 'DBD::Pg', and
+	reported *not* to work with 'DBD::mysql'. Experimental feature.
+	May not contain double colons (`:`). Default: 'SQLite'.
 	See gitlink:git-cvsserver[1].
 
+gitcvs.dbuser, gitcvs.dbpass::
+	Database user and password. Only useful if setting 'gitcvs.dbdriver',
+	since SQLite has no concept of database users and/or passwords.
+	'gitcvs.dbuser' supports variable substitution (see
+	gitlink:git-cvsserver[1] for details).
+
+All gitcvs variables except for 'gitcvs.allbinary' can also specifed
+as 'gitcvs.<access_method>.<varname>' (where 'access_method' is one
+of "ext" and "pserver") to make them apply only for the given access
+method.
+
 http.sslVerify::
 	Whether to verify the SSL certificate when fetching or pushing
 	over HTTPS. Can be overridden by the 'GIT_SSL_NO_VERIFY' environment
diff --git a/Documentation/git-cvsserver.txt b/Documentation/git-cvsserver.txt
index f9e0c77379..d22844ba49 100644
--- a/Documentation/git-cvsserver.txt
+++ b/Documentation/git-cvsserver.txt
@@ -31,6 +31,10 @@ over pserver for anonymous CVS access.
 
 CVS clients cannot tag, branch or perform GIT merges.
 
+git-cvsserver maps GIT branches to CVS modules. This is very different
+from what most CVS users would expect since in CVS modules usually represent
+one or more directories.
+
 INSTALLATION
 ------------
 
@@ -65,9 +69,22 @@ env variable, you can rename git-cvsserver to cvs.
 
 ------
 Note: you need to ensure each user that is going to invoke git-cvsserver has
-write access to the log file and to the git repository. When offering anon
-access via pserver, this means that the nobody user should have write access
-to at least the sqlite database at the root of the repository.
+write access to the log file and to the database (see
+<<dbbackend,Database Backend>>. If you want to offer write access over
+SSH, the users of course also need write access to the git repository itself.
+
+[[configaccessmethod]]
+All configuration variables can also be overriden for a specific method of
+access. Valid method names are "ext" (for SSH access) and "pserver". The
+following example configuration would disable pserver access while still
+allowing access over SSH.
+------
+   [gitcvs]
+        enabled=0
+
+   [gitcvs "ext"]
+        enabled=1
+------
 --
 3. On the client machine you need to set the following variables.
    CVSROOT should be set as per normal, but the directory should point at the
@@ -93,6 +110,90 @@ Example:
      cvs co -d project-master master
 ------
 
+[[dbbackend]]
+Database Backend
+----------------
+
+git-cvsserver uses one database per git head (i.e. CVS module) to
+store information about the repository for faster access. The
+database doesn't contain any persitent data and can be completly
+regenerated from the git repository at any time. The database
+needs to be updated (i.e. written to) after every commit.
+
+If the commit is done directly by using git (as opposed to
+using git-cvsserver) the update will need to happen on the
+next repository access by git-cvsserver, independent of
+access method and requested operation.
+
+That means that even if you offer only read access (e.g. by using
+the pserver method), git-cvsserver should have write access to
+the database to work reliably (otherwise you need to make sure
+that the database if up-to-date all the time git-cvsserver is run).
+
+By default it uses SQLite databases in the git directory, named
+`gitcvs.<module_name>.sqlite`. Note that the SQLite backend creates
+temporary files in the same directory as the database file on
+write so it might not be enough to grant the users using
+git-cvsserver write access to the database file without granting
+them write access to the directory, too.
+
+You can configure the database backend with the following
+configuration variables:
+
+Configuring database backend
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+git-cvsserver uses the Perl DBI module. Please also read
+its documentation if changing these variables, especially
+about `DBI->connect()`.
+
+gitcvs.dbname::
+	Database name. The exact meaning depends on the
+	used database driver, for SQLite this is a filename.
+	Supports variable substitution (see below). May
+	not contain semicolons (`;`).
+	Default: '%Ggitcvs.%m.sqlite'
+
+gitcvs.dbdriver::
+	Used DBI driver. You can specify any available driver
+	for this here, but it might not work. cvsserver is tested
+	with 'DBD::SQLite', reported to work with
+	'DBD::Pg', and reported *not* to work with 'DBD::mysql'.
+	Please regard this as an experimental feature. May not
+	contain double colons (`:`).
+	Default: 'SQLite'
+
+gitcvs.dbuser::
+	Database user. Only useful if setting `dbdriver`, since
+	SQLite has no concept of database users. Supports variable
+	substitution (see below).
+
+gitcvs.dbpass::
+	Database password.  Only useful if setting `dbdriver`, since
+	SQLite has no concept of database passwords.
+
+All variables can also be set per access method, see <<configaccessmethod,above>>.
+
+Variable substitution
+^^^^^^^^^^^^^^^^^^^^^
+In `dbdriver` and `dbuser` you can use the following variables:
+
+%G::
+	git directory name
+%g::
+	git directory name, where all characters except for
+	alpha-numeric ones, `.`, and `-` are replaced with
+	`_` (this should make it easier to use the directory
+	name in a filename if wanted)
+%m::
+	CVS module/git head name
+%a::
+	access method (one of "ext" or "pserver")
+%u::
+	Name of the user running git-cvsserver.
+	If no name can be determined, the
+	numeric uid is used.
+
 Eclipse CVS Client Notes
 ------------------------
 
diff --git a/git-cvsserver.perl b/git-cvsserver.perl
index 25816c5a21..087e3abaef 100755
--- a/git-cvsserver.perl
+++ b/git-cvsserver.perl
@@ -91,7 +91,9 @@ $log->debug("Temporary directory is '$TEMP_DIR'");
 # if we are called with a pserver argument,
 # deal with the authentication cat before entering the
 # main loop
+$state->{method} = 'ext';
 if (@ARGV && $ARGV[0] eq 'pserver') {
+    $state->{method} = 'pserver';
     my $line = <STDIN>; chomp $line;
     unless( $line eq 'BEGIN AUTH REQUEST') {
        die "E Do not understand $line - expecting BEGIN AUTH REQUEST\n";
@@ -181,11 +183,18 @@ sub req_Root
     }
     foreach my $line ( @gitvars )
     {
-        next unless ( $line =~ /^(.*?)\.(.*?)=(.*)$/ );
-        $cfg->{$1}{$2} = $3;
+        next unless ( $line =~ /^(.*?)\.(.*?)(?:\.(.*?))?=(.*)$/ );
+        unless ($3) {
+            $cfg->{$1}{$2} = $4;
+        } else {
+            $cfg->{$1}{$2}{$3} = $4;
+        }
     }
 
-    unless ( defined ( $cfg->{gitcvs}{enabled} ) and $cfg->{gitcvs}{enabled} =~ /^\s*(1|true|yes)\s*$/i )
+    unless ( ($cfg->{gitcvs}{$state->{method}}{enabled}
+	      and $cfg->{gitcvs}{$state->{method}}{enabled} =~ /^\s*(1|true|yes)\s*$/i)
+	     or ($cfg->{gitcvs}{enabled}
+	      and $cfg->{gitcvs}{enabled} =~ /^\s*(1|true|yes)\s*$/i) )
     {
         print "E GITCVS emulation needs to be enabled on this repo\n";
         print "E the repo config file needs a [gitcvs] section added, and the parameter 'enabled' set to 1\n";
@@ -194,9 +203,10 @@ sub req_Root
         return 0;
     }
 
-    if ( defined ( $cfg->{gitcvs}{logfile} ) )
+    my $logfile = $cfg->{gitcvs}{$state->{method}}{logfile} || $cfg->{gitcvs}{logfile};
+    if ( $logfile )
     {
-        $log->setfile($cfg->{gitcvs}{logfile});
+        $log->setfile($logfile);
     } else {
         $log->nofile();
     }
@@ -350,12 +360,52 @@ sub req_add
 
     argsplit("add");
 
+    my $updater = GITCVS::updater->new($state->{CVSROOT}, $state->{module}, $log);
+    $updater->update();
+
+    argsfromdir($updater);
+
     my $addcount = 0;
 
     foreach my $filename ( @{$state->{args}} )
     {
         $filename = filecleanup($filename);
 
+        my $meta = $updater->getmeta($filename);
+        my $wrev = revparse($filename);
+
+        if ($wrev && $meta && ($wrev < 0))
+        {
+            # previously removed file, add back
+            $log->info("added file $filename was previously removed, send 1.$meta->{revision}");
+
+            print "MT +updated\n";
+            print "MT text U \n";
+            print "MT fname $filename\n";
+            print "MT newline\n";
+            print "MT -updated\n";
+
+            unless ( $state->{globaloptions}{-n} )
+            {
+                my ( $filepart, $dirpart ) = filenamesplit($filename,1);
+
+                print "Created $dirpart\n";
+                print $state->{CVSROOT} . "/$state->{module}/$filename\n";
+
+                # this is an "entries" line
+                my $kopts = kopts_from_path($filepart);
+                $log->debug("/$filepart/1.$meta->{revision}//$kopts/");
+                print "/$filepart/1.$meta->{revision}//$kopts/\n";
+                # permissions
+                $log->debug("SEND : u=$meta->{mode},g=$meta->{mode},o=$meta->{mode}");
+                print "u=$meta->{mode},g=$meta->{mode},o=$meta->{mode}\n";
+                # transmit file
+                transmitfile($meta->{filehash});
+            }
+
+            next;
+        }
+
         unless ( defined ( $state->{entries}{$filename}{modified_filename} ) )
         {
             print "E cvs add: nothing known about `$filename'\n";
@@ -1027,7 +1077,7 @@ sub req_ci
 
     $log->info("req_ci : " . ( defined($data) ? $data : "[NULL]" ));
 
-    if ( @ARGV && $ARGV[0] eq 'pserver')
+    if ( $state->{method} eq 'pserver')
     {
         print "error 1 pserver access cannot commit\n";
         exit;
@@ -2132,25 +2182,40 @@ sub new
 
     bless $self, $class;
 
-    $self->{dbdir} = $config . "/";
-    die "Database dir '$self->{dbdir}' isn't a directory" unless ( defined($self->{dbdir}) and -d $self->{dbdir} );
-
     $self->{module} = $module;
-    $self->{file} = $self->{dbdir} . "/gitcvs.$module.sqlite";
-
     $self->{git_path} = $config . "/";
 
     $self->{log} = $log;
 
     die "Git repo '$self->{git_path}' doesn't exist" unless ( -d $self->{git_path} );
 
-    $self->{dbh} = DBI->connect("dbi:SQLite:dbname=" . $self->{file},"","");
+    $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} ||
+        $cfg->{gitcvs}{dbdriver} || "SQLite";
+    $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||
+        $cfg->{gitcvs}{dbname} || "%Ggitcvs.%m.sqlite";
+    $self->{dbuser} = $cfg->{gitcvs}{$state->{method}}{dbuser} ||
+        $cfg->{gitcvs}{dbuser} || "";
+    $self->{dbpass} = $cfg->{gitcvs}{$state->{method}}{dbpass} ||
+        $cfg->{gitcvs}{dbpass} || "";
+    my %mapping = ( m => $module,
+                    a => $state->{method},
+                    u => getlogin || getpwuid($<) || $<,
+                    G => $self->{git_path},
+                    g => mangle_dirname($self->{git_path}),
+                    );
+    $self->{dbname} =~ s/%([mauGg])/$mapping{$1}/eg;
+    $self->{dbuser} =~ s/%([mauGg])/$mapping{$1}/eg;
+
+    die "Invalid char ':' in dbdriver" if $self->{dbdriver} =~ /:/;
+    die "Invalid char ';' in dbname" if $self->{dbname} =~ /;/;
+    $self->{dbh} = DBI->connect("dbi:$self->{dbdriver}:dbname=$self->{dbname}",
+                                $self->{dbuser},
+                                $self->{dbpass});
+    die "Error connecting to database\n" unless defined $self->{dbh};
 
     $self->{tables} = {};
-    foreach my $table ( $self->{dbh}->tables )
+    foreach my $table ( keys %{$self->{dbh}->table_info(undef,undef,undef,'TABLE')->fetchall_hashref('TABLE_NAME')} )
     {
-        $table =~ s/^"//;
-        $table =~ s/"$//;
         $self->{tables}{$table} = 1;
     }
 
@@ -2848,5 +2913,19 @@ sub safe_pipe_capture {
     return wantarray ? @output : join('',@output);
 }
 
+=head2 mangle_dirname
+
+create a string from a directory name that is suitable to use as
+part of a filename, mainly by converting all chars except \w.- to _
+
+=cut
+sub mangle_dirname {
+    my $dirname = shift;
+    return unless defined $dirname;
+
+    $dirname =~ s/[^\w.-]/_/g;
+
+    return $dirname;
+}
 
 1;