Added Sherlock, a custom profiling tool for GitLab

Sherlock will be a new GitLab specific tool for measuring the
performance of Rails requests (and SideKiq jobs at some point). Some of
the things that are currently tracked:

* SQL queries along with their timings, backtraces and query plans
  (using "EXPLAIN ANALYZE" for PostgreSQL and regular "EXPLAIN" for
  MySQL)
* Timings of application files (including views) on a per line basis
* Some meta data such as the request method, path, total duration, etc

More tracking (e.g. Rugged or gitlab-shell timings) might be added in
the future.

Sherlock will replace any existing tools we have used so far (e.g.
active_record_query_trace and rack-mini-profiler), hence the
corresponding Gems have been removed from the Gemfile.

Sherlock can be enabled by starting Rails as following:

    ENABLE_SHERLOCK=1 bundle exec rails s

Recorded transactions can be found at `/sherlock/transactions`.

9 files changed, 313 insertions, 0 deletions

diff --git a/app/views/sherlock/file_samples/show.html.haml b/app/views/sherlock/file_samples/show.html.haml
new file mode 100644
index 00000000000..cfd11e45b6a
--- /dev/null
+++ b/app/views/sherlock/file_samples/show.html.haml
@@ -0,0 +1,55 @@
+- page_title t('sherlock.title'), t('sherlock.transaction'),
+  t('sherlock.file_sample')
+
+- header_title t('sherlock.title'), sherlock_transactions_path
+
+.gray-content-block
+  .pull-right
+    = link_to(sherlock_transaction_path(@transaction), class: 'btn') do
+      %i.fa.fa-arrow-left
+      = t('sherlock.transaction')
+  .oneline
+    = t('sherlock.file_sample')
+    = @file_sample.id
+
+.prepend-top-default
+  %p
+    %span.light
+      #{t('sherlock.time')}:
+    %strong
+      = @file_sample.duration.round(2)
+      = t('sherlock.milliseconds')
+  %p
+    %span.light
+      #{t('sherlock.events')}:
+    %strong
+      = @file_sample.events
+
+%article.file-holder
+  .file-title
+    %i.fa.fa-file-text-o.fa-fw
+    %strong
+      = @file_sample.file
+  .code.file-content.js-syntax-highlight
+    .line-numbers
+      %table.sherlock-line-samples-table
+        %thead
+          %tr
+            %th= t('sherlock.line_capitalized')
+            %th= t('sherlock.events')
+            %th= t('sherlock.time')
+            %th= t('sherlock.percent')
+        %tbody
+          - @file_sample.line_samples.each_with_index do |sample, index|
+            %tr{class: sample.majority_of?(@file_sample.duration) ? 'slow' : ''}
+              %td= index + 1
+              %td= sample.events
+              %td
+                = sample.duration.round(2)
+                = t('sherlock.milliseconds')
+              %td
+                = sample.percentage_of(@file_sample.duration).round
+                = t('sherlock.percent')
+
+    .sherlock-file-sample
+      = highlight(@file_sample.file, @file_sample.source)
diff --git a/app/views/sherlock/queries/_backtrace.html.haml b/app/views/sherlock/queries/_backtrace.html.haml
new file mode 100644
index 00000000000..5c9294c0ab5
--- /dev/null
+++ b/app/views/sherlock/queries/_backtrace.html.haml
@@ -0,0 +1,27 @@
+.prepend-top-default
+  .panel.panel-default
+    .panel-heading
+      %strong
+        = t('sherlock.application_backtrace')
+    %ul.well-list
+      - @query.application_backtrace.each do |location|
+        %li
+          = location.path
+          %small.light
+            = t('sherlock.line')
+            = location.line
+
+  .panel.panel-default
+    .panel-heading
+      %strong
+        = t('sherlock.full_backtrace')
+    %ul.well-list
+      - @query.backtrace.each do |location|
+        %li
+          - if location.application?
+            %strong= location.path
+          - else
+            = location.path
+          %small.light
+            = t('sherlock.line')
+            = location.line
diff --git a/app/views/sherlock/queries/_general.html.haml b/app/views/sherlock/queries/_general.html.haml
new file mode 100644
index 00000000000..549b47430e6
--- /dev/null
+++ b/app/views/sherlock/queries/_general.html.haml
@@ -0,0 +1,50 @@
+.prepend-top-default
+  .panel.panel-default
+    .panel-heading
+      %strong
+        = t('sherlock.general')
+    %ul.well-list
+      %li
+        %span.light
+          #{t('sherlock.time')}:
+        %strong
+          = @query.duration.round(4)
+          = t('sherlock.milliseconds')
+      %li
+        %span.light
+          #{t('sherlock.origin')}:
+        %strong
+          = @query.last_application_frame.path
+        %small.light
+          = t('sherlock.line')
+          = @query.last_application_frame.line
+
+  .panel.panel-default
+    .panel-heading
+      .pull-right
+        %button.js-clipboard-trigger.btn.btn-xs{title: t('sherlock.copy_to_clipboard'), type: :button}
+          %i.fa.fa-clipboard
+        %pre.hidden
+          = @query.formatted_query
+      %strong
+        = t('sherlock.query')
+    %ul.well-list
+      %li
+        .code.js-syntax-highlight.sherlock-code
+          :preserve
+            #{highlight("#{@query.id}.sql", @query.formatted_query)}
+
+  .panel.panel-default
+    .panel-heading
+      .pull-right
+        %button.js-clipboard-trigger.btn.btn-xs{title: t('sherlock.copy_to_clipboard'), type: :button}
+          %i.fa.fa-clipboard
+        %pre.hidden
+          = @query.explain
+      %strong
+        = t('sherlock.query_plan')
+    %ul.well-list
+      %li
+        .code.js-syntax-highlight.sherlock-code
+          %pre
+            %code= @query.explain
diff --git a/app/views/sherlock/queries/show.html.haml b/app/views/sherlock/queries/show.html.haml
new file mode 100644
index 00000000000..4a84348ac82
--- /dev/null
+++ b/app/views/sherlock/queries/show.html.haml
@@ -0,0 +1,26 @@
+- page_title t('sherlock.title'), t('sherlock.transaction'), t('sherlock.query')
+- header_title t('sherlock.title'), sherlock_transactions_path
+
+%ul.center-top-menu
+  %li.active
+    %a(href="#tab-general" data-toggle="tab")
+      = t('sherlock.general')
+  %li
+    %a(href="#tab-backtrace" data-toggle="tab")
+      = t('sherlock.backtrace')
+
+.gray-content-block
+  .pull-right
+    = link_to(sherlock_transaction_path(@transaction), class: 'btn') do
+      %i.fa.fa-arrow-left
+      = t('sherlock.transaction')
+  .oneline
+    = t('sherlock.query')
+    = @query.id
+
+.tab-content
+  .tab-pane.active#tab-general
+    = render(partial: 'general')
+
+  .tab-pane#tab-backtrace
+    = render(partial: 'backtrace')
diff --git a/app/views/sherlock/transactions/_file_samples.html.haml b/app/views/sherlock/transactions/_file_samples.html.haml
new file mode 100644
index 00000000000..0afdbc8dffa
--- /dev/null
+++ b/app/views/sherlock/transactions/_file_samples.html.haml
@@ -0,0 +1,22 @@
+- if @transaction.file_samples.empty?
+  .nothing-here-block
+    = t('sherlock.no_file_samples')
+- else
+  .table-holder
+    %table.table
+      %thead
+        %tr
+          %th= t('sherlock.time_inclusive')
+          %th= t('sherlock.path')
+          %th
+      %tbody
+        - @transaction.sorted_file_samples.each do |sample|
+          %tr
+            %td
+              = sample.duration.round(2)
+              = t('sherlock.milliseconds')
+            %td= sample.relative_path
+            %td
+              = link_to(t('sherlock.view'),
+                sherlock_transaction_file_sample_path(@transaction, sample),
+                class: 'btn btn-xs')
diff --git a/app/views/sherlock/transactions/_general.html.haml b/app/views/sherlock/transactions/_general.html.haml
new file mode 100644
index 00000000000..4287a0c3203
--- /dev/null
+++ b/app/views/sherlock/transactions/_general.html.haml
@@ -0,0 +1,33 @@
+.prepend-top-default
+  .panel.panel-default
+    .panel-heading
+      %strong
+        = t('sherlock.general')
+    %ul.well-list
+      %li
+        %span.light
+          #{t('sherlock.id')}:
+        %strong
+          = @transaction.id
+      %li
+        %span.light
+          #{t('sherlock.type')}:
+        %strong
+          = @transaction.type
+      %li
+        %span.light
+          #{t('sherlock.path')}:
+        %strong
+          = @transaction.path
+      %li
+        %span.light
+          #{t('sherlock.time')}:
+        %strong
+          = @transaction.duration.round(2)
+          = t('sherlock.seconds')
+      %li
+        %span.light
+          #{t('sherlock.finished_at')}:
+        %strong
+          = time_ago_in_words(@transaction.finished_at)
+          = t('sherlock.ago')
diff --git a/app/views/sherlock/transactions/_queries.html.haml b/app/views/sherlock/transactions/_queries.html.haml
new file mode 100644
index 00000000000..b7e0162e80d
--- /dev/null
+++ b/app/views/sherlock/transactions/_queries.html.haml
@@ -0,0 +1,24 @@
+- if @transaction.queries.empty?
+  .nothing-here-block
+    = t('sherlock.no_queries')
+- else
+  .table-holder
+    %table.table#sherlock-queries
+      %thead
+        %tr
+          %th= t('sherlock.time')
+          %th= t('sherlock.query')
+          %td
+      %tbody
+        - @transaction.sorted_queries.each do |query|
+          %tr
+            %td
+              = query.duration.round(2)
+              = t('sherlock.milliseconds')
+            %td
+              .code.js-syntax-highlight.sherlock-code
+                = highlight("#{query.id}.sql", query.formatted_query)
+            %td
+              = link_to(t('sherlock.view'),
+                sherlock_transaction_query_path(@transaction, query),
+                class: 'btn btn-xs')
diff --git a/app/views/sherlock/transactions/index.html.haml b/app/views/sherlock/transactions/index.html.haml
new file mode 100644
index 00000000000..fb31131ba88
--- /dev/null
+++ b/app/views/sherlock/transactions/index.html.haml
@@ -0,0 +1,40 @@
+- page_title t('sherlock.title')
+- header_title t('sherlock.title'), sherlock_transactions_path
+
+.gray-content-block
+  .pull-right
+    = link_to(destroy_all_sherlock_transactions_path,
+      class: 'btn btn-danger',
+      method: :delete) do
+      %i.fa.fa-trash
+      = t('sherlock.delete_all_transactions')
+  .oneline= t('sherlock.introduction')
+
+- if @transactions.empty?
+  .nothing-here-block= t('sherlock.no_transactions')
+- else
+  .table-holder
+    %table.table
+      %thead
+        %tr
+          %th= t('sherlock.type')
+          %th= t('sherlock.path')
+          %th= t('sherlock.time')
+          %th= t('sherlock.queries')
+          %th= t('sherlock.finished_at')
+          %th
+      %tbody
+        - @transactions.each do |trans|
+          %tr
+            %td= trans.type
+            %td= trans.path
+            %td
+              = trans.duration.round(2)
+              = t('sherlock.seconds')
+            %td= trans.queries.length
+            %td
+              = time_ago_in_words(trans.finished_at)
+              = t('sherlock.ago')
+            %td
+              = link_to(sherlock_transaction_path(trans), class: 'btn btn-xs') do
+                = t('sherlock.view')
diff --git a/app/views/sherlock/transactions/show.html.haml b/app/views/sherlock/transactions/show.html.haml
new file mode 100644
index 00000000000..3c8ffb06648
--- /dev/null
+++ b/app/views/sherlock/transactions/show.html.haml
@@ -0,0 +1,36 @@
+- page_title t('sherlock.title'), t('sherlock.transaction')
+- header_title t('sherlock.title'), sherlock_transactions_path
+
+%ul.center-top-menu
+  %li.active
+    %a(href="#tab-general" data-toggle="tab")
+      = t('sherlock.general')
+  %li
+    %a(href="#tab-queries" data-toggle="tab")
+      = t('sherlock.queries')
+      %span.badge
+        #{@transaction.queries.length}
+  %li
+    %a(href="#tab-file-samples" data-toggle="tab")
+      = t('sherlock.file_samples')
+      %span.badge
+        #{@transaction.file_samples.length}
+
+.gray-content-block
+  .pull-right
+    = link_to(sherlock_transactions_path, class: 'btn') do
+      %i.fa.fa-arrow-left
+      = t('sherlock.all_transactions')
+  .oneline
+    = t('sherlock.transaction')
+    = @transaction.id
+
+.tab-content
+  .tab-pane.active#tab-general
+    = render(partial: 'general')
+
+  .tab-pane#tab-queries
+    = render(partial: 'queries')
+
+  .tab-pane#tab-file-samples
+    = render(partial: 'file_samples')