diff options
| -rw-r--r-- | doc/administration/troubleshooting/debug.md | 28 |
1 files changed, 28 insertions, 0 deletions
diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index be538ea250a..83a714810c1 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -163,6 +163,34 @@ separate Rails process to debug the issue: 1. In a new window, run `top`. It should show this ruby process using 100% CPU. Write down the PID. 1. Follow step 2 from the previous section on using gdb. +### GitLab: API is not accessible + +This often occurs when gitlab-shell attempts to request authorization via the +internal API (e.g., `http://localhost:8080/api/v4/internal/allowed`), and +something in the check fails. There are many reasons why this may happen: + +1. Timeout connecting to a database (e.g., PostgreSQL or Redis) +1. Error in Git hooks or push rules +1. Error accessing the repository (e.g., stale NFS handles) + +To diagnose this problem, try to reproduce the problem and then see if there +is a unicorn worker that is spinning via `top`. Try to use the `gdb` +techniques above. In addition, using `strace` may help isolate issues: + +```shell +strace -tt -T -f -s 1024 -p <PID of unicorn worker> -o /tmp/unicorn.txt +``` + +If you cannot isolate which Unicorn worker is the issue, try to run `strace` +on all the Unicorn workers to see where the `/internal/allowed` endpoint gets +stuck: + +```shell +ps auwx | grep unicorn | awk '{ print " -p " $2}' | xargs strace -tt -T -f -s 1024 -o /tmp/unicorn.txt +``` + +The output in `/tmp/unicorn.txt` may help diagnose the root cause. + # More information * [Debugging Stuck Ruby Processes](https://blog.newrelic.com/2013/04/29/debugging-stuck-ruby-processes-what-to-do-before-you-kill-9/) |