summaryrefslogtreecommitdiff
path: root/doc/container_registry
diff options
context:
space:
mode:
authorAchilleas Pipinellis <axilleas@axilleas.me>2016-08-03 17:33:37 +0300
committerAchilleas Pipinellis <axilleas@axilleas.me>2016-08-03 17:34:55 +0300
commitc91168c04a71301d8423b881c1219cfd510d5784 (patch)
tree26d3a73d02a2894a6f4982339d02a9508936e03b /doc/container_registry
parent2c9cce0feb8bd4e10f3406493eff30e783782d15 (diff)
downloadgitlab-ce-c91168c04a71301d8423b881c1219cfd510d5784.tar.gz
Small refactor on Registry troubleshooting
[ci skip]
Diffstat (limited to 'doc/container_registry')
-rw-r--r--doc/container_registry/troubleshooting.md49
1 files changed, 23 insertions, 26 deletions
diff --git a/doc/container_registry/troubleshooting.md b/doc/container_registry/troubleshooting.md
index 8008bf29935..14c4a7d9a63 100644
--- a/doc/container_registry/troubleshooting.md
+++ b/doc/container_registry/troubleshooting.md
@@ -5,27 +5,27 @@
1. Check to make sure that the system clock on your Docker client and GitLab server have
been synchronized (e.g. via NTP).
-2. If you are using an S3-backed registry, double check that the IAM
+2. If you are using an S3-backed Registry, double check that the IAM
permissions and the S3 credentials (including region) are correct. See [the
sample IAM policy](https://docs.docker.com/registry/storage-drivers/s3/)
for more details.
-3. Check the registry logs (e.g. `/var/log/gitlab/registry/current`) and the GitLab production logs
+3. Check the Registry logs (e.g. `/var/log/gitlab/registry/current`) and the GitLab production logs
for errors (e.g. `/var/log/gitlab/gitlab-rails/production.log`). You may be able to find clues
there.
-# Advanced Troubleshooting
+## Advanced Troubleshooting
-NOTE: The following section is only recommended for experts.
+>**NOTE:** The following section is only recommended for experts.
Sometimes it's not obvious what is wrong, and you may need to dive deeper into
-the communication between the Docker client and the registry to find out
+the communication between the Docker client and the Registry to find out
what's wrong. We will use a concrete example in the past to illustrate how to
diagnose a problem with the S3 setup.
-## Example: Unexpected 403 error during push
+### Unexpected 403 error during push
-A user attempted to enable an S3-backed registry. The `docker login` step went
+A user attempted to enable an S3-backed Registry. The `docker login` step went
fine. However, when pushing an image, the output showed:
```
@@ -44,11 +44,11 @@ error parsing HTTP 403 response body: unexpected end of JSON input: ""
```
This error is ambiguous, as it's not clear whether the 403 is coming from the
-GitLab Rails application, the Docker registry, or something else. In this
+GitLab Rails application, the Docker Registry, or something else. In this
case, since we know that since the login succeeded, we probably need to look
-at the communication between the client and the registry.
+at the communication between the client and the Registry.
-The REST API between the Docker client and registry is [described
+The REST API between the Docker client and Registry is [described
here](https://docs.docker.com/registry/spec/api/). Normally, one would just
use Wireshark or tcpdump to capture the traffic and see where things went
wrong. However, since all communication between Docker clients and servers
@@ -56,12 +56,12 @@ are done over HTTPS, it's a bit difficult to decrypt the traffic quickly even
if you know the private key. What can we do instead?
One way would be to disable HTTPS by setting up an [insecure
-registry](https://docs.docker.com/registry/insecure/). This could introduce a
+Registry](https://docs.docker.com/registry/insecure/). This could introduce a
security hole and is only recommended for local testing. If you have a
production system and can't or don't want to do this, there is another way:
use mitmproxy, which stands for Man-in-the-Middle Proxy.
-## mitmproxy
+### mitmproxy
[mitmproxy](https://mitmproxy.org/) allows you to place a proxy between your
client and server to inspect all traffic. One wrinkle is that your system
@@ -70,10 +70,9 @@ needs to trust the mitmproxy SSL certificates for this to work.
The following installation instructions assume you are running Ubuntu:
1. Install mitmproxy (see http://docs.mitmproxy.org/en/stable/install.html)
-
-2. Run `mitmproxy --port 9000` to generate its certificates. Enter CTRL-C to quit.
-
-3. Install the certificate from ~/.mitmproxy to your system:
+1. Run `mitmproxy --port 9000` to generate its certificates.
+ Enter <kbd>CTRL</kbd>-<kbd>C</kbd> to quit.
+1. Install the certificate from `~/.mitmproxy` to your system:
```sh
sudo cp ~/.mitmproxy/mitmproxy-ca-cert.pem /usr/local/share/ca-certificates/mitmproxy-ca-cert.crt
@@ -87,24 +86,22 @@ Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....done.
```
-## Verifying mitmproxy certifiactes
-
-To verify that the certificates are properly install, run:
+To verify that the certificates are properly installed, run:
```sh
mitmproxy --port 9000
```
-This will run mitmproxy on port 9000. In another window, run:
+This will run mitmproxy on port `9000`. In another window, run:
```sh
curl --proxy http://localhost:9000 https://httpbin.org/status/200
```
-If everything is setup correctly, then you will see information on the mitmproxy window and
+If everything is setup correctly, you will see information on the mitmproxy window and
no errors from the curl commands.
-## Running the Docker daemon with a proxy
+### Running the Docker daemon with a proxy
For Docker to connect through a proxy, you must start the Docker daemon with the
proper environment variables. The easiest way is to shutdown Docker (e.g. `sudo initctl stop docker`)
@@ -118,10 +115,10 @@ docker daemon --debug
This will launch the Docker daemon and proxy all connections through mitmproxy.
-## Running the Docker client
+### Running the Docker client
-Now that we have mitmproxy and Docker running, we can now attempt to login and push a container
-image. You may need to run as root to do this. For example:
+Now that we have mitmproxy and Docker running, we can attempt to login and push
+a container image. You may need to run as root to do this. For example:
```sh
docker login s3-testing.myregistry.com:4567
@@ -141,4 +138,4 @@ The above image shows:
What does this mean? This strongly suggests that the S3 user does not have the right
[permissions to perform a HEAD request](http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectHEAD.html).
The solution: check the [IAM permissions again](https://docs.docker.com/registry/storage-drivers/s3/).
-Once the right permissions were set, the error went away.
+Once the right permissions were set, the error will go away.