diff options
author | Stan Hu <stanhu@gmail.com> | 2017-11-10 17:14:54 -0800 |
---|---|---|
committer | Stan Hu <stanhu@gmail.com> | 2017-11-18 00:06:15 -0800 |
commit | b001487d94a131fa698d7feb1be8d9168c877e55 (patch) | |
tree | 55f0f009c84ac1b412f93f872c89d1e17e955949 /doc/administration/troubleshooting | |
parent | 4a89018e00465cb43ceae3eb218616d5d59f8e41 (diff) | |
download | gitlab-ce-b001487d94a131fa698d7feb1be8d9168c877e55.tar.gz |
Document how to troubleshoot internal API calls
[ci skip]
iFoo
Diffstat (limited to 'doc/administration/troubleshooting')
-rw-r--r-- | doc/administration/troubleshooting/debug.md | 28 |
1 files changed, 28 insertions, 0 deletions
diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index be538ea250a..83a714810c1 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -163,6 +163,34 @@ separate Rails process to debug the issue: 1. In a new window, run `top`. It should show this ruby process using 100% CPU. Write down the PID. 1. Follow step 2 from the previous section on using gdb. +### GitLab: API is not accessible + +This often occurs when gitlab-shell attempts to request authorization via the +internal API (e.g., `http://localhost:8080/api/v4/internal/allowed`), and +something in the check fails. There are many reasons why this may happen: + +1. Timeout connecting to a database (e.g., PostgreSQL or Redis) +1. Error in Git hooks or push rules +1. Error accessing the repository (e.g., stale NFS handles) + +To diagnose this problem, try to reproduce the problem and then see if there +is a unicorn worker that is spinning via `top`. Try to use the `gdb` +techniques above. In addition, using `strace` may help isolate issues: + +```shell +strace -tt -T -f -s 1024 -p <PID of unicorn worker> -o /tmp/unicorn.txt +``` + +If you cannot isolate which Unicorn worker is the issue, try to run `strace` +on all the Unicorn workers to see where the `/internal/allowed` endpoint gets +stuck: + +```shell +ps auwx | grep unicorn | awk '{ print " -p " $2}' | xargs strace -tt -T -f -s 1024 -o /tmp/unicorn.txt +``` + +The output in `/tmp/unicorn.txt` may help diagnose the root cause. + # More information * [Debugging Stuck Ruby Processes](https://blog.newrelic.com/2013/04/29/debugging-stuck-ruby-processes-what-to-do-before-you-kill-9/) |