Improve performance of the cycle analytics page.

1. These changes bring down page load time for 100 issues from more than
   a minute to about 1.5 seconds.

2. This entire commit is composed of these types of performance
   enhancements:

     - Cache relevant data in `IssueMetrics` wherever possible.
     - Cache relevant data in `MergeRequestMetrics` wherever possible.
     - Preload metrics

3. Given these improvements, we now only need to make 4 SQL calls:

    - Load all issues
    - Load all merge requests
    - Load all metrics for the issues
    - Load all metrics for the merge requests

4. A list of all the data points that are now being pre-calculated:

    a. The first time an issue is mentioned in a commit

      - In `GitPushService`, find all issues mentioned by the given commit
        using `ReferenceExtractor`. Set the `first_mentioned_in_commit_at`
        flag for each of them.

      - There seems to be a (pre-existing) bug here - files (and
        therefore commits) created using the Web CI don't have
        cross-references created, and issues are not closed even when
        the commit title is "Fixes #xx".

    b. The first time a merge request is deployed to production

      When a `Deployment` is created, find all merge requests that
      were merged in before the deployment, and set the
      `first_deployed_to_production_at` flag for each of them.

    c. The start / end time for a merge request pipeline

      Hook into the `Pipeline` state machine. When the `status` moves to
      `running`, find the merge requests whose tip commit matches the
      pipeline, and record the `latest_build_started_at` time for each
      of them. When the `status` moves to `success`, record the
      `latest_build_finished_at` time.

    d. The merge requests that close an issue

      - This was a big cause of the performance problems we were having
        with Cycle Analytics. We need to use `ReferenceExtractor` to make
        this calculation, which is slow when we have to run it on a large
        number of merge requests.

      - When a merge request is created, updated, or refreshed, find the
        issues it closes, and create an instance of
        `MergeRequestsClosingIssues`, which acts as a join model between
        merge requests and issues.

      - If a `MergeRequestsClosingIssues` instance links a merge request
        and an issue, that issue closes that merge request.

5. The `Queries` module was changed into a class, so we can cache the
   results of `issues` and `merge_requests_closing_issues` across
   various cycle analytics stages.

6. The code added in this commit is untested. Tests will be added in the
   next commit.

1 files changed, 14 insertions, 1 deletions

diff --git a/db/schema.rb b/db/schema.rb
index 2c380de3d50..26d49e7b261 100644
--- a/db/schema.rb
+++ b/db/schema.rb
@@ -11,7 +11,7 @@
 #
 # It's strongly recommended that you check this file into your version control system.
 
-ActiveRecord::Schema.define(version: 20160901141443) do
+ActiveRecord::Schema.define(version: 20160915081353) do
 
   # These are extensions that must be enabled in order to support this database
   enable_extension "plpgsql"
@@ -442,6 +442,7 @@ ActiveRecord::Schema.define(version: 20160901141443) do
     t.datetime "first_added_to_board_at"
     t.datetime "created_at",                         null: false
     t.datetime "updated_at",                         null: false
+    t.datetime "first_mentioned_in_commit_at"
   end
 
   add_index "issue_metrics", ["issue_id"], name: "index_issue_metrics", using: :btree
@@ -596,6 +597,9 @@ ActiveRecord::Schema.define(version: 20160901141443) do
     t.datetime "first_closed_at"
     t.datetime "created_at",                               null: false
     t.datetime "updated_at",                               null: false
+    t.datetime "first_deployed_to_production_at"
+    t.datetime "latest_build_started_at"
+    t.datetime "latest_build_finished_at"
   end
 
   add_index "merge_request_metrics", ["merge_request_id"], name: "index_merge_request_metrics", using: :btree
@@ -641,6 +645,13 @@ ActiveRecord::Schema.define(version: 20160901141443) do
   add_index "merge_requests", ["title"], name: "index_merge_requests_on_title", using: :btree
   add_index "merge_requests", ["title"], name: "index_merge_requests_on_title_trigram", using: :gin, opclasses: {"title"=>"gin_trgm_ops"}
 
+  create_table "merge_requests_closing_issues", force: :cascade do |t|
+    t.integer  "merge_request_id", null: false
+    t.integer  "issue_id",         null: false
+    t.datetime "created_at",       null: false
+    t.datetime "updated_at",       null: false
+  end
+
   create_table "milestones", force: :cascade do |t|
     t.string   "title",       null: false
     t.integer  "project_id",  null: false
@@ -1170,6 +1181,8 @@ ActiveRecord::Schema.define(version: 20160901141443) do
   add_foreign_key "lists", "boards"
   add_foreign_key "lists", "labels"
   add_foreign_key "merge_request_metrics", "merge_requests"
+  add_foreign_key "merge_requests_closing_issues", "issues"
+  add_foreign_key "merge_requests_closing_issues", "merge_requests"
   add_foreign_key "personal_access_tokens", "users"
   add_foreign_key "protected_branch_merge_access_levels", "protected_branches"
   add_foreign_key "protected_branch_push_access_levels", "protected_branches"