cmd/go: allow go get with arbitrary URLs

This CL permits using arbitrary, non-VCS-qualified URLs as
aliases for fully VCS-qualified and/or well-known code hosting
sites.

Example 1) A VCS-qualified URL can now be shorter.

Before:
$ go get camlistore.org/r/p/camlistore.git/pkg/blobref
After:
$ go get camlistore.org/pkg/blobref

Example 2) A custom domain can be used as the import,
referencing a well-known code hosting site.

Before:
$ go get github.com/bradfitz/sonden
After:
$ go get bradfitz.com/pkg/sonden

The mechanism used is a <meta> tag in the HTML document
retrieved from fetching:

    https://<import>?go-get=1  (preferred)
    http://<import>?go-get=1   (fallback)

The meta tag should look like:

<meta name="go-import" content="import-alias-prefix vcs full-repo-root">

The full-repo-root must be a full URL root to a repository containing
a scheme and *not* containing a ".vcs" qualifier.

The vcs is one of "git", "hg", "svn", etc.

The import-alias-prefix must be a prefix or exact match of the
package being fetched with "go get".

If there are multiple meta tags, only the one with a prefix
matching the import path is used. It is an error if multiple
go-import values match the import prefix.

If the import-alias-prefix is not an exact match for the import,
another HTTP fetch is performed, at the declared root (which does
*not* need to be the domain's root).

For example, assuming that "camlistore.org/pkg/blobref" declares
in its HTML head:

<meta name="go-import" content="camlistore.org git https://camlistore.org/r/p/camlistore" />

... then:

$ go get camlistore.org/pkg/blobref

... looks at the following URLs:

   https://camlistore.org/pkg/blobref?go-get=1
   http://camlistore.org/pkg/blobref?go-get=1
   https://camlistore.org/?go-get=1
   http://camlistore.org/?go-get=1

Ultimately it finds, at the root (camlistore.org/), the same go-import:

<meta name="go-import" content="camlistore.org git https://camlistore.org/r/p/camlistore" />

... and proceeds to trust it, checking out git //camlistore.org/r/p/camlistore at
the import path of "camlistore.org" on disk.

Fixes issue 3099

R=r, rsc, gary.burd, eikeon, untheoretic, n13m3y3r, rsc
CC=golang-dev
http://codereview.appspot.com/5660051

1 files changed, 5 insertions, 4 deletions

diff --git a/src/cmd/go/get.go b/src/cmd/go/get.go
index 0ad22adb0..aa0ab7bd7 100644
--- a/src/cmd/go/get.go
+++ b/src/cmd/go/get.go
@@ -162,10 +162,11 @@ func downloadPackage(p *Package) error {
 	} else {
 		// Analyze the import path to determine the version control system,
 		// repository, and the import path for the root of the repository.
-		vcs, repo, rootPath, err = vcsForImportPath(p.ImportPath)
-	}
-	if err != nil {
-		return err
+		rr, err := repoRootForImportPath(p.ImportPath)
+		if err != nil {
+			return err
+		}
+		vcs, repo, rootPath = rr.vcs, rr.repo, rr.root
 	}
 
 	if p.build.SrcRoot == "" {