summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorStan Hu <stanhu@gmail.com>2017-11-10 17:14:54 -0800
committerStan Hu <stanhu@gmail.com>2017-11-18 00:06:15 -0800
commitb001487d94a131fa698d7feb1be8d9168c877e55 (patch)
tree55f0f009c84ac1b412f93f872c89d1e17e955949 /doc
parent4a89018e00465cb43ceae3eb218616d5d59f8e41 (diff)
downloadgitlab-ce-b001487d94a131fa698d7feb1be8d9168c877e55.tar.gz
Document how to troubleshoot internal API calls
[ci skip] iFoo
Diffstat (limited to 'doc')
-rw-r--r--doc/administration/troubleshooting/debug.md28
1 files changed, 28 insertions, 0 deletions
diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md
index be538ea250a..83a714810c1 100644
--- a/doc/administration/troubleshooting/debug.md
+++ b/doc/administration/troubleshooting/debug.md
@@ -163,6 +163,34 @@ separate Rails process to debug the issue:
1. In a new window, run `top`. It should show this ruby process using 100% CPU. Write down the PID.
1. Follow step 2 from the previous section on using gdb.
+### GitLab: API is not accessible
+
+This often occurs when gitlab-shell attempts to request authorization via the
+internal API (e.g., `http://localhost:8080/api/v4/internal/allowed`), and
+something in the check fails. There are many reasons why this may happen:
+
+1. Timeout connecting to a database (e.g., PostgreSQL or Redis)
+1. Error in Git hooks or push rules
+1. Error accessing the repository (e.g., stale NFS handles)
+
+To diagnose this problem, try to reproduce the problem and then see if there
+is a unicorn worker that is spinning via `top`. Try to use the `gdb`
+techniques above. In addition, using `strace` may help isolate issues:
+
+```shell
+strace -tt -T -f -s 1024 -p <PID of unicorn worker> -o /tmp/unicorn.txt
+```
+
+If you cannot isolate which Unicorn worker is the issue, try to run `strace`
+on all the Unicorn workers to see where the `/internal/allowed` endpoint gets
+stuck:
+
+```shell
+ps auwx | grep unicorn | awk '{ print " -p " $2}' | xargs strace -tt -T -f -s 1024 -o /tmp/unicorn.txt
+```
+
+The output in `/tmp/unicorn.txt` may help diagnose the root cause.
+
# More information
* [Debugging Stuck Ruby Processes](https://blog.newrelic.com/2013/04/29/debugging-stuck-ruby-processes-what-to-do-before-you-kill-9/)