Improve performance of the cycle analytics page.

1. These changes bring down page load time for 100 issues from more than
   a minute to about 1.5 seconds.

2. This entire commit is composed of these types of performance
   enhancements:

     - Cache relevant data in `IssueMetrics` wherever possible.
     - Cache relevant data in `MergeRequestMetrics` wherever possible.
     - Preload metrics

3. Given these improvements, we now only need to make 4 SQL calls:

    - Load all issues
    - Load all merge requests
    - Load all metrics for the issues
    - Load all metrics for the merge requests

4. A list of all the data points that are now being pre-calculated:

    a. The first time an issue is mentioned in a commit

      - In `GitPushService`, find all issues mentioned by the given commit
        using `ReferenceExtractor`. Set the `first_mentioned_in_commit_at`
        flag for each of them.

      - There seems to be a (pre-existing) bug here - files (and
        therefore commits) created using the Web CI don't have
        cross-references created, and issues are not closed even when
        the commit title is "Fixes #xx".

    b. The first time a merge request is deployed to production

      When a `Deployment` is created, find all merge requests that
      were merged in before the deployment, and set the
      `first_deployed_to_production_at` flag for each of them.

    c. The start / end time for a merge request pipeline

      Hook into the `Pipeline` state machine. When the `status` moves to
      `running`, find the merge requests whose tip commit matches the
      pipeline, and record the `latest_build_started_at` time for each
      of them. When the `status` moves to `success`, record the
      `latest_build_finished_at` time.

    d. The merge requests that close an issue

      - This was a big cause of the performance problems we were having
        with Cycle Analytics. We need to use `ReferenceExtractor` to make
        this calculation, which is slow when we have to run it on a large
        number of merge requests.

      - When a merge request is created, updated, or refreshed, find the
        issues it closes, and create an instance of
        `MergeRequestsClosingIssues`, which acts as a join model between
        merge requests and issues.

      - If a `MergeRequestsClosingIssues` instance links a merge request
        and an issue, that issue closes that merge request.

5. The `Queries` module was changed into a class, so we can cache the
   results of `issues` and `merge_requests_closing_issues` across
   various cycle analytics stages.

6. The code added in this commit is untested. Tests will be added in the
   next commit.

4 files changed, 50 insertions, 1 deletions

diff --git a/app/services/create_deployment_service.rb b/app/services/create_deployment_service.rb
index efeb9df9527..cc63eeae9c3 100644
--- a/app/services/create_deployment_service.rb
+++ b/app/services/create_deployment_service.rb
@@ -6,7 +6,7 @@ class CreateDeploymentService < BaseService
       name: params[:environment]
     )
 
-    project.deployments.create(
+    deployment = project.deployments.create(
       environment: environment,
       ref: params[:ref],
       tag: params[:tag],
@@ -14,5 +14,26 @@ class CreateDeploymentService < BaseService
       user: current_user,
       deployable: deployable
     )
+
+    update_merge_request_metrics(deployment, environment)
+
+    deployment
+  end
+
+  private
+
+  def update_merge_request_metrics(deployment, environment)
+    # TODO: Test cases:
+    # 1. Merge request with metrics available and `first_deployed_to_production_at` is nil
+    # 2. Merge request with metrics available and `first_deployed_to_production_at` is set
+    # 3. Merge request with metrics unavailable
+    # 4. Only applies to merge requests merged AFTER the previous production deploy to this branch
+    if environment.name == "production"
+      merge_requests_deployed_to_production_for_first_time = project.merge_requests.joins("LEFT OUTER JOIN merge_request_metrics ON merge_request_metrics.merge_request_id = merge_requests.id").
+                                                             where(target_branch: params[:ref], "merge_request_metrics.first_deployed_to_production_at" => nil).
+                                                             where("merge_request_metrics.merged_at < ?", deployment.created_at)
+
+      merge_requests_deployed_to_production_for_first_time.each { |merge_request| merge_request.metrics.record_production_deploy!(deployment.created_at) }
+    end
   end
 end
diff --git a/app/services/git_push_service.rb b/app/services/git_push_service.rb
index 78feb37aa2a..0f4f805155f 100644
--- a/app/services/git_push_service.rb
+++ b/app/services/git_push_service.rb
@@ -134,6 +134,7 @@ class GitPushService < BaseService
       end
 
       commit.create_cross_references!(authors[commit], closed_issues)
+      update_issue_metrics(commit, authors)
     end
   end
 
@@ -186,4 +187,10 @@ class GitPushService < BaseService
   def branch_name
     @branch_name ||= Gitlab::Git.ref_name(params[:ref])
   end
+
+  # TODO: What about commits created using the Web UI? Cross references are not created there.
+  def update_issue_metrics(commit, authors)
+    mentioned_issues = commit.all_references(authors[commit]).issues
+    mentioned_issues.each { |issue| issue.metrics.record_commit_mention!(commit.committed_date) if issue.metrics.present? }
+  end
 end
diff --git a/app/services/merge_requests/base_service.rb b/app/services/merge_requests/base_service.rb
index ba424b09463..9704a7050eb 100644
--- a/app/services/merge_requests/base_service.rb
+++ b/app/services/merge_requests/base_service.rb
@@ -35,6 +35,18 @@ module MergeRequests
       end
     end
 
+    def create(merge_request)
+      merge_request = super(merge_request)
+      merge_request.cache_merge_request_closes_issues!(current_user)
+      merge_request
+    end
+
+    def update(merge_request)
+      merge_request = super(merge_request)
+      merge_request.cache_merge_request_closes_issues!(current_user)
+      merge_request
+    end
+
     private
 
     def filter_params
diff --git a/app/services/merge_requests/refresh_service.rb b/app/services/merge_requests/refresh_service.rb
index 5cedd6f11d9..cd4dd967404 100644
--- a/app/services/merge_requests/refresh_service.rb
+++ b/app/services/merge_requests/refresh_service.rb
@@ -13,6 +13,7 @@ module MergeRequests
       reload_merge_requests
       reset_merge_when_build_succeeds
       mark_pending_todos_done
+      cache_merge_requests_closing_issues
 
       # Leave a system note if a branch was deleted/added
       if branch_added? || branch_removed?
@@ -141,6 +142,14 @@ module MergeRequests
       end
     end
 
+    # If the merge requests closes any issues, save this information in the
+    # `MergeRequestsClosingIssues` model (as a performance optimization).
+    def cache_merge_requests_closing_issues
+      merge_requests_for_source_branch.each do |merge_request|
+        merge_request.cache_merge_request_closes_issues!(@current_user)
+      end
+    end
+
     def filter_merge_requests(merge_requests)
       merge_requests.uniq.select(&:source_project)
     end