Adapt functions to work for external Knative

Remove Kn services cache from Clusters::Application::Knative

Knative function can exist even if user did not installed Knative via
GitLab managed apps.

-> Move responsibility of finding services into the Cluster
-> Responsability is inside Clusters::Cluster::KnativeServiceFinder
-> Projects::Serverless::FunctionsFinder now calls depends solely on a
cluster to find the Kn services.
-> Detect Knative by resource presence instead of service presence
-> Mock knative_installed response temporarily for frontend to develop

Display loader while `installed === 'checking'`

Added frontend work to determine if Knative is installed

Memoize with_reactive_cache(*args, &block) to avoid race conditions

When calling with_reactive_cache more than once, it's possible that the
second call will already have the value populated. Therefore, in cases
where we need the sequential calls to have consistent results, we'd fall
under a race condition.

Check knative installation via Knative resource presence

Only load pods if Knative is discovered

Always return a response in FunctionsController#index

- Always indicate if Knative is installed, not installed or checking
- Always indicate the partial response for functions. Final response is
guaranteed when knative_installed is either true | false.

Adds specs for Clusters::Cluster#knative_services_finder

Fix method name when calling on specs

Add an explicit check for functions

Added an explicit check to see if there are any functions available

Fix Serverless feature spec

- we don't find knative installation via database anymore,
rather via Knative resource

Display error message for request timeouts

Display an error message if the request times out

Adds feature specs for when functions exist

Remove a test purposed hardcoded flag

Add ability to partially load functions

Added the ability to partially load functions on the frontend

Add frontend unit tests

Added tests for the new frontend additions

Generate new translations

Generated new frontend translations

Address review comments

Cleaned up the frontend unit test.
Added computed prop for `isInstalled`.

Move string to constant

Simplify nil to array conversion

Put knative_installed states in a frozen hash for better read

Pluralize list of Knative states

Quey services and pods filtering name

This way we don't need to filter the namespace in memory.
Also, the data we get from the network is much smaller.

Simplify cache_key and fix bug

- Simplifies the cache_key by removing namespace duplicate
- Fixes a bug with reactive_cache memoization

12 files changed, 212 insertions, 93 deletions

diff --git a/app/assets/javascripts/serverless/components/functions.vue b/app/assets/javascripts/serverless/components/functions.vue
index f9b4e789563..94341050b86 100644
--- a/app/assets/javascripts/serverless/components/functions.vue
+++ b/app/assets/javascripts/serverless/components/functions.vue
@@ -4,6 +4,7 @@ import { GlLoadingIcon } from '@gitlab/ui';
 import FunctionRow from './function_row.vue';
 import EnvironmentRow from './environment_row.vue';
 import EmptyState from './empty_state.vue';
+import { CHECKING_INSTALLED } from '../constants';
 
 export default {
   components: {
@@ -13,10 +14,6 @@ export default {
     GlLoadingIcon,
   },
   props: {
-    installed: {
-      type: Boolean,
-      required: true,
-    },
     clustersPath: {
       type: String,
       required: true,
@@ -31,8 +28,15 @@ export default {
     },
   },
   computed: {
-    ...mapState(['isLoading', 'hasFunctionData']),
+    ...mapState(['installed', 'isLoading', 'hasFunctionData']),
     ...mapGetters(['getFunctions']),
+
+    checkingInstalled() {
+      return this.installed === CHECKING_INSTALLED;
+    },
+    isInstalled() {
+      return this.installed === true;
+    },
   },
   created() {
     this.fetchFunctions({
@@ -47,15 +51,16 @@ export default {
 
 <template>
   <section id="serverless-functions">
-    <div v-if="installed">
+    <gl-loading-icon
+      v-if="checkingInstalled"
+      :size="2"
+      class="prepend-top-default append-bottom-default"
+    />
+
+    <div v-else-if="isInstalled">
       <div v-if="hasFunctionData">
-        <gl-loading-icon
-          v-if="isLoading"
-          :size="2"
-          class="prepend-top-default append-bottom-default"
-        />
-        <template v-else>
-          <div class="groups-list-tree-container">
+        <template>
+          <div class="groups-list-tree-container js-functions-wrapper">
             <ul class="content-list group-list-tree">
               <environment-row
                 v-for="(env, index) in getFunctions"
@@ -66,6 +71,11 @@ export default {
             </ul>
           </div>
         </template>
+        <gl-loading-icon
+          v-if="isLoading"
+          :size="2"
+          class="prepend-top-default append-bottom-default js-functions-loader"
+        />
       </div>
       <div v-else class="empty-state js-empty-state">
         <div class="text-content">
diff --git a/app/assets/javascripts/serverless/constants.js b/app/assets/javascripts/serverless/constants.js
index 35f77205f2c..2fa15e56ccb 100644
--- a/app/assets/javascripts/serverless/constants.js
+++ b/app/assets/javascripts/serverless/constants.js
@@ -1,3 +1,7 @@
 export const MAX_REQUESTS = 3; // max number of times to retry
 
 export const X_INTERVAL = 5; // Reflects the number of verticle bars on the x-axis
+
+export const CHECKING_INSTALLED = 'checking'; // The backend is still determining whether or not Knative is installed
+
+export const TIMEOUT = 'timeout';
diff --git a/app/assets/javascripts/serverless/serverless_bundle.js b/app/assets/javascripts/serverless/serverless_bundle.js
index 2d3f086ffee..ed3b633d766 100644
--- a/app/assets/javascripts/serverless/serverless_bundle.js
+++ b/app/assets/javascripts/serverless/serverless_bundle.js
@@ -45,7 +45,7 @@ export default class Serverless {
         },
       });
     } else {
-      const { statusPath, clustersPath, helpPath, installed } = document.querySelector(
+      const { statusPath, clustersPath, helpPath } = document.querySelector(
         '.js-serverless-functions-page',
       ).dataset;
 
@@ -56,7 +56,6 @@ export default class Serverless {
         render(createElement) {
           return createElement(Functions, {
             props: {
-              installed: installed !== undefined,
               clustersPath,
               helpPath,
               statusPath,
diff --git a/app/assets/javascripts/serverless/store/actions.js b/app/assets/javascripts/serverless/store/actions.js
index 826501c9022..a0a9fdf7ace 100644
--- a/app/assets/javascripts/serverless/store/actions.js
+++ b/app/assets/javascripts/serverless/store/actions.js
@@ -3,13 +3,18 @@ import axios from '~/lib/utils/axios_utils';
 import statusCodes from '~/lib/utils/http_status';
 import { backOff } from '~/lib/utils/common_utils';
 import createFlash from '~/flash';
-import { MAX_REQUESTS } from '../constants';
+import { __ } from '~/locale';
+import { MAX_REQUESTS, CHECKING_INSTALLED, TIMEOUT } from '../constants';
 
 export const requestFunctionsLoading = ({ commit }) => commit(types.REQUEST_FUNCTIONS_LOADING);
 export const receiveFunctionsSuccess = ({ commit }, data) =>
   commit(types.RECEIVE_FUNCTIONS_SUCCESS, data);
-export const receiveFunctionsNoDataSuccess = ({ commit }) =>
-  commit(types.RECEIVE_FUNCTIONS_NODATA_SUCCESS);
+export const receiveFunctionsPartial = ({ commit }, data) =>
+  commit(types.RECEIVE_FUNCTIONS_PARTIAL, data);
+export const receiveFunctionsTimeout = ({ commit }, data) =>
+  commit(types.RECEIVE_FUNCTIONS_TIMEOUT, data);
+export const receiveFunctionsNoDataSuccess = ({ commit }, data) =>
+  commit(types.RECEIVE_FUNCTIONS_NODATA_SUCCESS, data);
 export const receiveFunctionsError = ({ commit }, error) =>
   commit(types.RECEIVE_FUNCTIONS_ERROR, error);
 
@@ -25,18 +30,25 @@ export const receiveMetricsError = ({ commit }, error) =>
 export const fetchFunctions = ({ dispatch }, { functionsPath }) => {
   let retryCount = 0;
 
+  const functionsPartiallyFetched = data => {
+    if (data.functions !== null && data.functions.length) {
+      dispatch('receiveFunctionsPartial', data);
+    }
+  };
+
   dispatch('requestFunctionsLoading');
 
   backOff((next, stop) => {
     axios
       .get(functionsPath)
       .then(response => {
-        if (response.status === statusCodes.NO_CONTENT) {
+        if (response.data.knative_installed === CHECKING_INSTALLED) {
           retryCount += 1;
           if (retryCount < MAX_REQUESTS) {
+            functionsPartiallyFetched(response.data);
             next();
           } else {
-            stop(null);
+            stop(TIMEOUT);
           }
         } else {
           stop(response.data);
@@ -45,10 +57,13 @@ export const fetchFunctions = ({ dispatch }, { functionsPath }) => {
       .catch(stop);
   })
     .then(data => {
-      if (data !== null) {
+      if (data === TIMEOUT) {
+        dispatch('receiveFunctionsTimeout');
+        createFlash(__('Loading functions timed out. Please reload the page to try again.'));
+      } else if (data.functions !== null && data.functions.length) {
         dispatch('receiveFunctionsSuccess', data);
       } else {
-        dispatch('receiveFunctionsNoDataSuccess');
+        dispatch('receiveFunctionsNoDataSuccess', data);
       }
     })
     .catch(error => {
diff --git a/app/assets/javascripts/serverless/store/mutation_types.js b/app/assets/javascripts/serverless/store/mutation_types.js
index 25b2f7ac38a..b8fa9ea1a01 100644
--- a/app/assets/javascripts/serverless/store/mutation_types.js
+++ b/app/assets/javascripts/serverless/store/mutation_types.js
@@ -1,5 +1,7 @@
 export const REQUEST_FUNCTIONS_LOADING = 'REQUEST_FUNCTIONS_LOADING';
 export const RECEIVE_FUNCTIONS_SUCCESS = 'RECEIVE_FUNCTIONS_SUCCESS';
+export const RECEIVE_FUNCTIONS_PARTIAL = 'RECEIVE_FUNCTIONS_PARTIAL';
+export const RECEIVE_FUNCTIONS_TIMEOUT = 'RECEIVE_FUNCTIONS_TIMEOUT';
 export const RECEIVE_FUNCTIONS_NODATA_SUCCESS = 'RECEIVE_FUNCTIONS_NODATA_SUCCESS';
 export const RECEIVE_FUNCTIONS_ERROR = 'RECEIVE_FUNCTIONS_ERROR';
 
diff --git a/app/assets/javascripts/serverless/store/mutations.js b/app/assets/javascripts/serverless/store/mutations.js
index 991f32a275d..2685a5b11ff 100644
--- a/app/assets/javascripts/serverless/store/mutations.js
+++ b/app/assets/javascripts/serverless/store/mutations.js
@@ -5,12 +5,23 @@ export default {
     state.isLoading = true;
   },
   [types.RECEIVE_FUNCTIONS_SUCCESS](state, data) {
-    state.functions = data;
+    state.functions = data.functions;
+    state.installed = data.knative_installed;
     state.isLoading = false;
     state.hasFunctionData = true;
   },
-  [types.RECEIVE_FUNCTIONS_NODATA_SUCCESS](state) {
+  [types.RECEIVE_FUNCTIONS_PARTIAL](state, data) {
+    state.functions = data.functions;
+    state.installed = true;
+    state.isLoading = true;
+    state.hasFunctionData = true;
+  },
+  [types.RECEIVE_FUNCTIONS_TIMEOUT](state) {
+    state.isLoading = false;
+  },
+  [types.RECEIVE_FUNCTIONS_NODATA_SUCCESS](state, data) {
     state.isLoading = false;
+    state.installed = data.knative_installed;
     state.hasFunctionData = false;
   },
   [types.RECEIVE_FUNCTIONS_ERROR](state, error) {
diff --git a/app/assets/javascripts/serverless/store/state.js b/app/assets/javascripts/serverless/store/state.js
index afc3f37d7ba..fdd29299749 100644
--- a/app/assets/javascripts/serverless/store/state.js
+++ b/app/assets/javascripts/serverless/store/state.js
@@ -1,5 +1,6 @@
 export default () => ({
   error: null,
+  installed: 'checking',
   isLoading: true,
 
   // functions
diff --git a/app/controllers/projects/serverless/functions_controller.rb b/app/controllers/projects/serverless/functions_controller.rb
index 79030da64d3..4b0d001fca6 100644
--- a/app/controllers/projects/serverless/functions_controller.rb
+++ b/app/controllers/projects/serverless/functions_controller.rb
@@ -10,15 +10,13 @@ module Projects
           format.json do
             functions = finder.execute
 
-            if functions.any?
-              render json: serialize_function(functions)
-            else
-              head :no_content
-            end
+            render json: {
+              knative_installed: finder.knative_installed,
+              functions: serialize_function(functions)
+            }.to_json
           end
 
           format.html do
-            @installed = finder.installed?
             render
           end
         end
diff --git a/app/finders/clusters/cluster/knative_services_finder.rb b/app/finders/clusters/cluster/knative_services_finder.rb
new file mode 100644
index 00000000000..85b0967e935
--- /dev/null
+++ b/app/finders/clusters/cluster/knative_services_finder.rb
@@ -0,0 +1,114 @@
+# frozen_string_literal: true
+module Clusters
+  class Cluster
+    class KnativeServicesFinder
+      include ReactiveCaching
+      include Gitlab::Utils::StrongMemoize
+
+      KNATIVE_STATES = {
+        'checking' => 'checking',
+        'installed' => 'installed',
+        'not_found' => 'not_found'
+      }.freeze
+
+      self.reactive_cache_key = ->(finder) { finder.model_name }
+      self.reactive_cache_worker_finder = ->(_id, *cache_args) { from_cache(*cache_args) }
+
+      attr_reader :cluster, :project
+
+      def initialize(cluster, project)
+        @cluster = cluster
+        @project = project
+      end
+
+      def with_reactive_cache_memoized(*cache_args, &block)
+        strong_memoize(:reactive_cache) do
+          with_reactive_cache(*cache_args, &block)
+        end
+      end
+
+      def clear_cache!
+        clear_reactive_cache!(*cache_args)
+      end
+
+      def self.from_cache(cluster_id, project_id)
+        cluster = Clusters::Cluster.find(cluster_id)
+        project = ::Project.find(project_id)
+
+        new(cluster, project)
+      end
+
+      def calculate_reactive_cache(*)
+        # read_services calls knative_client.discover implicitily. If we stop
+        # detecting services but still want to detect knative, we'll need to
+        # explicitily call: knative_client.discover
+        #
+        # We didn't create it separately to avoid 2 cluster requests.
+        ksvc = read_services
+        pods = knative_client.discovered ? read_pods : []
+        { services: ksvc, pods: pods, knative_detected: knative_client.discovered }
+      end
+
+      def services
+        return [] unless search_namespace
+
+        cached_data = with_reactive_cache_memoized(*cache_args) { |data| data }
+        cached_data.to_h.fetch(:services, [])
+      end
+
+      def cache_args
+        [cluster.id, project.id]
+      end
+
+      def service_pod_details(service)
+        cached_data = with_reactive_cache_memoized(*cache_args) { |data| data }
+        cached_data.to_h.fetch(:pods, []).select do |pod|
+          filter_pods(pod, service)
+        end
+      end
+
+      def knative_detected
+        cached_data = with_reactive_cache_memoized(*cache_args) { |data| data }
+
+        knative_state = cached_data.to_h[:knative_detected]
+
+        return KNATIVE_STATES['checking'] if knative_state.nil?
+        return KNATIVE_STATES['installed'] if knative_state
+
+        KNATIVE_STATES['uninstalled']
+      end
+
+      def model_name
+        self.class.name.underscore.tr('/', '_')
+      end
+
+      private
+
+      def search_namespace
+        @search_namespace ||= cluster.kubernetes_namespace_for(project)
+      end
+
+      def knative_client
+        cluster.kubeclient.knative_client
+      end
+
+      def filter_pods(pod, service)
+        pod["metadata"]["labels"]["serving.knative.dev/service"] == service
+      end
+
+      def read_services
+        knative_client.get_services(namespace: search_namespace).as_json
+      rescue Kubeclient::ResourceNotFoundError
+        []
+      end
+
+      def read_pods
+        cluster.kubeclient.core_client.get_pods(namespace: search_namespace).as_json
+      end
+
+      def id
+        nil
+      end
+    end
+  end
+end
diff --git a/app/finders/projects/serverless/functions_finder.rb b/app/finders/projects/serverless/functions_finder.rb
index e5bffccabfe..9b8d7ed5a58 100644
--- a/app/finders/projects/serverless/functions_finder.rb
+++ b/app/finders/projects/serverless/functions_finder.rb
@@ -14,8 +14,15 @@ module Projects
         knative_services.flatten.compact
       end
 
-      def installed?
-        clusters_with_knative_installed.exists?
+      # Possible return values: Clusters::Cluster::KnativeServicesFinder::KNATIVE_STATE
+      def knative_installed
+        states = @clusters.map do |cluster|
+          cluster.knative_services_finder(project).knative_detected.tap do |state|
+            return state if state == ::Clusters::Cluster::KnativeServicesFinder::KNATIVE_STATES['checking'] # rubocop:disable Cop/AvoidReturnFromBlocks
+          end
+        end
+
+        states.any? { |state| state == ::Clusters::Cluster::KnativeServicesFinder::KNATIVE_STATES['installed'] }
       end
 
       def service(environment_scope, name)
@@ -25,7 +32,7 @@ module Projects
       def invocation_metrics(environment_scope, name)
         return unless prometheus_adapter&.can_query?
 
-        cluster = clusters_with_knative_installed.preload_knative.find do |c|
+        cluster = @clusters.find do |c|
           environment_scope == c.environment_scope
         end
 
@@ -34,7 +41,7 @@ module Projects
       end
 
       def has_prometheus?(environment_scope)
-        clusters_with_knative_installed.preload_knative.to_a.any? do |cluster|
+        @clusters.any? do |cluster|
           environment_scope == cluster.environment_scope && cluster.application_prometheus_available?
         end
       end
@@ -42,10 +49,12 @@ module Projects
       private
 
       def knative_service(environment_scope, name)
-        clusters_with_knative_installed.preload_knative.map do |cluster|
+        @clusters.map do |cluster|
           next if environment_scope != cluster.environment_scope
 
-          services = cluster.application_knative.services_for(ns: cluster.kubernetes_namespace_for(project))
+          services = cluster
+            .knative_services_finder(project)
+            .services
             .select { |svc| svc["metadata"]["name"] == name }
 
           add_metadata(cluster, services).first unless services.nil?
@@ -53,8 +62,11 @@ module Projects
       end
 
       def knative_services
-        clusters_with_knative_installed.preload_knative.map do |cluster|
-          services = cluster.application_knative.services_for(ns: cluster.kubernetes_namespace_for(project))
+        @clusters.map do |cluster|
+          services = cluster
+            .knative_services_finder(project)
+            .services
+
           add_metadata(cluster, services) unless services.nil?
         end
       end
@@ -65,17 +77,14 @@ module Projects
           s["cluster_id"] = cluster.id
 
           if services.length == 1
-            s["podcount"] = cluster.application_knative.service_pod_details(
-              cluster.kubernetes_namespace_for(project),
-              s["metadata"]["name"]).length
+            s["podcount"] = cluster
+              .knative_services_finder(project)
+              .service_pod_details(s["metadata"]["name"])
+              .length
           end
         end
       end
 
-      def clusters_with_knative_installed
-        @clusters.with_knative_installed
-      end
-
       # rubocop: disable CodeReuse/ServiceClass
       def prometheus_adapter
         @prometheus_adapter ||= ::Prometheus::AdapterService.new(project).prometheus_adapter
diff --git a/app/models/clusters/applications/knative.rb b/app/models/clusters/applications/knative.rb
index 9fbf5d8af04..d5a3bd62e3d 100644
--- a/app/models/clusters/applications/knative.rb
+++ b/app/models/clusters/applications/knative.rb
@@ -15,9 +15,6 @@ module Clusters
       include ::Clusters::Concerns::ApplicationVersion
       include ::Clusters::Concerns::ApplicationData
       include AfterCommitQueue
-      include ReactiveCaching
-
-      self.reactive_cache_key = ->(knative) { [knative.class.model_name.singular, knative.id] }
 
       def set_initial_status
         return unless not_installable?
@@ -41,8 +38,6 @@ module Clusters
 
       scope :for_cluster, -> (cluster) { where(cluster: cluster) }
 
-      after_save :clear_reactive_cache!
-
       def chart
         'knative/knative'
       end
@@ -77,55 +72,12 @@ module Clusters
         ClusterWaitForIngressIpAddressWorker.perform_async(name, id)
       end
 
-      def client
-        cluster.kubeclient.knative_client
-      end
-
-      def services
-        with_reactive_cache do |data|
-          data[:services]
-        end
-      end
-
-      def calculate_reactive_cache
-        { services: read_services, pods: read_pods }
-      end
-
       def ingress_service
         cluster.kubeclient.get_service('istio-ingressgateway', 'istio-system')
       end
 
-      def services_for(ns: namespace)
-        return [] unless services
-        return [] unless ns
-
-        services.select do |service|
-          service.dig('metadata', 'namespace') == ns
-        end
-      end
-
-      def service_pod_details(ns, service)
-        with_reactive_cache do |data|
-          data[:pods].select { |pod| filter_pods(pod, ns, service) }
-        end
-      end
-
       private
 
-      def read_pods
-        cluster.kubeclient.core_client.get_pods.as_json
-      end
-
-      def filter_pods(pod, namespace, service)
-        pod["metadata"]["namespace"] == namespace && pod["metadata"]["labels"]["serving.knative.dev/service"] == service
-      end
-
-      def read_services
-        client.get_services.as_json
-      rescue Kubeclient::ResourceNotFoundError
-        []
-      end
-
       def install_knative_metrics
         ["kubectl apply -f #{METRICS_CONFIG}"] if cluster.application_prometheus_available?
       end
diff --git a/app/models/clusters/cluster.rb b/app/models/clusters/cluster.rb
index 57a1e461b2d..e1d6b2a802b 100644
--- a/app/models/clusters/cluster.rb
+++ b/app/models/clusters/cluster.rb
@@ -223,6 +223,10 @@ module Clusters
       end
     end
 
+    def knative_services_finder(project)
+      @knative_services_finder ||= KnativeServicesFinder.new(self, project)
+    end
+
     private
 
     def instance_domain