diff options
Diffstat (limited to 'doc/development/integrations')
| -rw-r--r-- | doc/development/integrations/codesandbox.md | 140 | ||||
| -rw-r--r-- | doc/development/integrations/img/copy_cookies.png | bin | 0 -> 44311 bytes | |||
| -rw-r--r-- | doc/development/integrations/img/copy_curl.png | bin | 0 -> 60175 bytes | |||
| -rw-r--r-- | doc/development/integrations/img/example_vuln.png (renamed from doc/development/integrations/example_vuln.png) | bin | 36208 -> 36208 bytes | |||
| -rw-r--r-- | doc/development/integrations/jenkins.md | 8 | ||||
| -rw-r--r-- | doc/development/integrations/jira_connect.md | 36 | ||||
| -rw-r--r-- | doc/development/integrations/secure.md | 18 | ||||
| -rw-r--r-- | doc/development/integrations/secure_partner_integration.md | 14 |
8 files changed, 197 insertions, 19 deletions
diff --git a/doc/development/integrations/codesandbox.md b/doc/development/integrations/codesandbox.md new file mode 100644 index 00000000000..1641f4656a0 --- /dev/null +++ b/doc/development/integrations/codesandbox.md @@ -0,0 +1,140 @@ +# Set up local Codesandbox development environment + +This guide walks through setting up a local [Codesandbox repository](https://github.com/codesandbox/codesandbox-client) and integrating it with a local GitLab instance. Codesandbox +is used to power the Web IDE's [Live Preview feature](../../user/project/web_ide/index.md#live-preview). Having a local Codesandbox setup is useful for debugging upstream issues or +creating upstream contributions like [this one](https://github.com/codesandbox/codesandbox-client/pull/5137). + +## Initial setup + +Before using Codesandbox with your local GitLab instance, you must: + +1. Enable HTTPS on your GDK. Codesandbox uses Service Workers that require `https`. + Follow the GDK [NGINX configuration instructions](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/master/doc/howto/nginx.md) to enable HTTPS for GDK. +1. Clone the [`codesandbox-client` project](https://github.com/codesandbox/codesandbox-client) + locally. If you plan on contributing upstream, you might want to fork and clone first. +1. (Optional) Use correct `python` and `nodejs` versions. Otherwise, `yarn` may fail to + install or build some packages. If you're using `asdf` you can run the following commands: + + ```shell + asdf local nodejs 10.14.2 + asdf local python 2.7.18 + ``` + +1. Run the following commands in the `codesandbox-client` project checkout: + + ```shell + # This might be necessary for the `prepublishOnly` job that is run later + yarn global add lerna + + # Install packages + yarn + ``` + + You can run `yarn build:clean` to clean up the build assets. + +## Use local GitLab instance with local Codesandbox + +GitLab integrates with two parts of Codesandbox: + +- An NPM package called `smooshpack` (called `sandpack` in the `codesandbox-client` project). + This exposes an entrypoint for us to kick off Codesandbox's bundler. +- A server that houses Codesandbox assets for bundling and previewing. This is hosted + on a separate server for security. + +Each time you want to run GitLab and Codesandbox together, you need to perform the +steps in the following sections. + +### Use local `smooshpack` for GitLab + +GitLab usually satisfies its `smooshpack` dependency with a remote module, but we want +to use a locally-built module. To build and use a local `smooshpack` module: + +1. In the `codesandbox-client` project directory, run: + + ```shell + cd standalone-packages/sandpack + yarn link + + # (Optional) you might want to start a development build + yarn run start + ``` + + Now, in the GitLab project, you can run `yarn link "smooshpack"`. `yarn` looks + for `smooshpack` **on disk** as opposed to the one hosted remotely. + +1. In the `gitlab` project directory, run: + + ```shell + # Remove and reinstall node_modules just to be safe + rm -rf node_modules + yarn install + + # Use the "smooshpack" package on disk + yarn link "smooshpack" + ``` + +### Fix possible GDK webpack problem + +`webpack` in GDK can fail to find packages inside a linked package. This step can help +you avoid `webpack` breaking with messages saying that it can't resolve packages from +`smooshpack/dist/sandpack.es5.js`. + +In the `codesandbox-client` project directory, run: + +```shell +cd standalone-packages + +mkdir node_modules +ln -s $PATH_TO_LOCAL_GITLAB/node_modules/core-js ./node_modules/core-js +``` + +### Start building codesandbox app assets + +In the `codesandbox-client` project directory: + +```shell +cd packages/app + +yarn start:sandpack-sandbox +``` + +### Create HTTPS proxy for Codesandbox `sandpack` assets + +Because we need `https`, we need to create a proxy to the webpack server. We can use +[`http-server`](https://www.npmjs.com/package/http-server), which can do this proxying +out of the box: + +```shell +npx http-server --proxy http://localhost:3000 -S -C $PATH_TO_CERT_PEM -K $PATH_TO_KEY_PEM -p 8044 -d false +``` + +### Update `bundler_url` setting in GitLab + +We need to update our `application_setting_implementation.rb` to point to the server that hosts the +Codesandbox `sandpack` assets. For instance, if these assets are hosted by a server at `https://sandpack.local:8044`: + +```patch +diff --git a/app/models/application_setting_implementation.rb b/app/models/application_setting_implementation.rb +index 6eed627b502..1824669e881 100644 +--- a/app/models/application_setting_implementation.rb ++++ b/app/models/application_setting_implementation.rb +@@ -391,7 +391,7 @@ def static_objects_external_storage_enabled? + # This will eventually be configurable + # https://gitlab.com/gitlab-org/gitlab/issues/208161 + def web_ide_clientside_preview_bundler_url +- 'https://sandbox-prod.gitlab-static.net' ++ 'https://sandpack.local:8044' + end + + private + +``` + +NOTE: +You can apply this patch by copying it to your clipboard and running `pbpaste | git apply`. + +You'll might want to restart the GitLab Rails server after making this change: + +```shell +gdk restart rails-web +``` diff --git a/doc/development/integrations/img/copy_cookies.png b/doc/development/integrations/img/copy_cookies.png Binary files differnew file mode 100644 index 00000000000..21575987173 --- /dev/null +++ b/doc/development/integrations/img/copy_cookies.png diff --git a/doc/development/integrations/img/copy_curl.png b/doc/development/integrations/img/copy_curl.png Binary files differnew file mode 100644 index 00000000000..9fa871efbd5 --- /dev/null +++ b/doc/development/integrations/img/copy_curl.png diff --git a/doc/development/integrations/example_vuln.png b/doc/development/integrations/img/example_vuln.png Binary files differindex bbfeb3c805f..bbfeb3c805f 100644 --- a/doc/development/integrations/example_vuln.png +++ b/doc/development/integrations/img/example_vuln.png diff --git a/doc/development/integrations/jenkins.md b/doc/development/integrations/jenkins.md index d81dee94f17..c87b15e192a 100644 --- a/doc/development/integrations/jenkins.md +++ b/doc/development/integrations/jenkins.md @@ -1,7 +1,7 @@ --- stage: Create group: Ecosystem -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # How to run Jenkins in development environment (on macOS) **(STARTER)** @@ -56,7 +56,8 @@ For more details, see [GitLab documentation about Jenkins CI](../../integration/ ## Configure Jenkins Project -Set up the Jenkins project you're going to run your build on. A **Freestyle** project is the easiest option because the Jenkins plugin will update the build status on GitLab. In a **Pipeline** project, updating the status on GitLab needs to be configured in a script. +Set up the Jenkins project to run your build on. A **Freestyle** project is the easiest +option because the Jenkins plugin updates the build status on GitLab. In a **Pipeline** project, updating the status on GitLab needs to be configured in a script. 1. On your Jenkins instance, go to **New Item**. 1. Pick a name, choose **Freestyle** or **Pipeline** and click **ok**. @@ -97,4 +98,5 @@ To activate the Jenkins service you must have a Starter subscription or higher. ## Test your setup -Make a change in your repository and open an MR. In your Jenkins project it should have triggered a new build and on your MR, there should be a widget saying **Pipeline #NUMBER passed**. It will also include a link to your Jenkins build. +Make a change in your repository and open an MR. In your Jenkins project it should have triggered a new build and on your MR, there should be a widget saying **Pipeline #NUMBER passed**. +It should also include a link to your Jenkins build. diff --git a/doc/development/integrations/jira_connect.md b/doc/development/integrations/jira_connect.md index 66a93f8c947..408b0e6068e 100644 --- a/doc/development/integrations/jira_connect.md +++ b/doc/development/integrations/jira_connect.md @@ -1,7 +1,7 @@ --- stage: Create group: Ecosystem -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Set up a development environment @@ -47,3 +47,37 @@ To install the app in Jira: You can also click **Getting Started** to open the configuration page rendered from your GitLab instance. _Note that any changes to the app descriptor requires you to uninstall then reinstall the app._ + +### Troubleshooting + +If the app install failed, you might need to delete `jira_connect_installations` from your database. + +1. Open the [database console](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/master/doc/howto/postgresql.md#access-postgresql). +1. Run `TRUNCATE TABLE jira_connect_installations CASCADE;`. + +## Add a namespace + +To add a [namespace](../../user/group/index.md#namespaces) to Jira: + +1. Make sure you are logged in on your GitLab development instance. +1. On the GitLab app page in Jira, click **Get started**. +1. Open your browser's developer tools and navigate to the **Network** tab. +1. Try to add the namespace in Jira. +1. If the request fails with 401 "not authorized", copy the request as a cURL command + and paste it in your terminal. + +  + +1. Go to your development instance (usually at: <http://localhost:3000>), open developer + tools, navigate to the Network tab and reload the page. +1. Copy all cookies from the first request. + +  + +1. Append the cookies to the cURL command in your terminal: + `--cookies "<cookies from the request>"`. +1. Submit the cURL request. +1. If the response is `{"success":true}`, the namespace was added. +1. Append the cookies to the cURL command in your terminal `--cookies "PASTE COOKIES HERE"`. +1. Submit the cURL request. +1. If the response is `{"success":true}` the namespace was added. diff --git a/doc/development/integrations/secure.md b/doc/development/integrations/secure.md index 9bb92709d54..fb9d894d203 100644 --- a/doc/development/integrations/secure.md +++ b/doc/development/integrations/secure.md @@ -1,7 +1,7 @@ --- stage: Protect group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Security scanner integration @@ -82,9 +82,9 @@ mysec_sast: sast: gl-sast-report.json ``` -Note that `gl-sast-report.json` is an example file path but any other file name can be used. See +Note that `gl-sast-report.json` is an example file path but any other filename can be used. See [the Output file section](#output-file) for more details. It's processed as a SAST report because -it's declared under the `reports:sast` key in the job definition, not because of the file name. +it's declared under the `reports:sast` key in the job definition, not because of the filename. ### Policies @@ -207,17 +207,17 @@ given by the `CI_PROJECT_DIR` environment variable. It is recommended to name the output file after the type of scanning, and to use `gl-` as a prefix. Since all Secure reports are JSON files, it is recommended to use `.json` as a file extension. -For instance, a suggested file name for a Dependency Scanning report is `gl-dependency-scanning.json`. +For instance, a suggested filename for a Dependency Scanning report is `gl-dependency-scanning.json`. The [`artifacts:reports`](../../ci/pipelines/job_artifacts.md#artifactsreports) keyword of the job definition must be consistent with the file path where the Security report is written. For instance, if a Dependency Scanning analyzer writes its report to the CI project directory, -and if this report file name is `depscan.json`, +and if this report filename is `depscan.json`, then `artifacts:reports:dependency_scanning` must be set to `depscan.json`. ### Exit code -Following the POSIX exit code standard, the scanner will exit with 0 for success and any number from 1 to 255 for anything else. +Following the POSIX exit code standard, the scanner exits with 0 for success and any number from 1 to 255 for anything else. Success also includes the case when vulnerabilities are found. When executing a scanning job using the [Docker-in-Docker privileged mode](../../user/application_security/sast/index.md#requirements), @@ -324,7 +324,7 @@ whereas the `message` may repeat the location. As a visual example, this screenshot highlights where these fields are used when viewing a vulnerability as part of a pipeline view. - + For instance, a `message` for a vulnerability reported by Dependency Scanning gives information on the vulnerable dependency, @@ -397,7 +397,9 @@ Not all vulnerabilities have CVEs, and a CVE can be identified multiple times. A isn't a stable identifier and you shouldn't assume it as such when tracking vulnerabilities. The maximum number of identifiers for a vulnerability is set as 20. If a vulnerability has more than 20 identifiers, -the system will save only the first 20 of them. +the system saves only the first 20 of them. Note that vulnerabilities in the [Pipeline +Security](../../user/application_security/security_dashboard/#pipeline-security) +tab do not enforce this limit and all identifiers present in the report artifact are displayed. ### Location diff --git a/doc/development/integrations/secure_partner_integration.md b/doc/development/integrations/secure_partner_integration.md index 52d10f0bd3c..80f632639ca 100644 --- a/doc/development/integrations/secure_partner_integration.md +++ b/doc/development/integrations/secure_partner_integration.md @@ -1,13 +1,13 @@ --- stage: Secure group: Static Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Secure Partner Integration - Onboarding Process If you want to integrate your product with the [Secure Stage](https://about.gitlab.com/direction/secure/), -this page will help you understand the developer workflow GitLab intends for +this page describes the developer workflow GitLab intends for our users to follow with regards to security results. These should be used as guidelines so you can build an integration that fits with the workflow GitLab users are already familiar with. @@ -19,7 +19,7 @@ integration as well as linking to more detailed resources for how to do so. ## Integration Tiers -GitLab's security offerings are designed for GitLab Gold and GitLab Ultimate users, and the +The security offerings in GitLab are designed for GitLab Gold and GitLab Ultimate users, and the [DevSecOps](https://about.gitlab.com/handbook/use-cases/#4-devsecops-shift-left-security) use case. All the features are in those tiers. This includes the APIs and standard reporting framework needed to provide a consistent experience for users to easily bring their preferred @@ -29,7 +29,7 @@ tiers so that we can provide the most value to our mutual customers. ## What is the GitLab Developer Workflow? This workflow is how GitLab users interact with our product and expect it to -function. Understanding how users use GitLab today will help you choose the +function. Understanding how users use GitLab today helps you choose the best place to integrate your own product and its results into GitLab. - Developers want to write code without using a new tool to consume results @@ -99,9 +99,9 @@ and complete an integration with the Secure stage. - In the [Security Dashboard](../../user/application_security/security_dashboard/index.md) ([Dashboard data flow](https://gitlab.com/snippets/1910005#project-and-group-dashboards)). 1. Optional: Provide a way to interact with results as Vulnerabilities: - Users can interact with the findings from your artifact within their workflow. They can dismiss the findings or accept them and create a backlog issue. - - To automatically create issues without user interaction, use the [issue API](../../api/issues.md). This will be replaced by [Standalone Vulnerabilities](https://gitlab.com/groups/gitlab-org/-/epics/634) in the future. + - To automatically create issues without user interaction, use the [issue API](../../api/issues.md). 1. Optional: Provide auto-remediation steps: - - If you specified `remediations` in your artifact, it is proposed through our [auto-remediation](../../user/application_security/index.md#automatic-remediation-for-vulnerabilities) + - If you specified `remediations` in your artifact, it is proposed through our [automatic remediation](../../user/application_security/index.md#automatic-remediation-for-vulnerabilities) interface. 1. Demo the integration to GitLab: - After you have tested and are ready to demo your integration please @@ -112,7 +112,7 @@ and complete an integration with the Secure stage. to support your go-to-market as appropriate. - Examples of supported marketing could include being listed on our [Security Partner page](https://about.gitlab.com/partners/#security), doing an [Unfiltered blog post](https://about.gitlab.com/handbook/marketing/blog/unfiltered/), - doing a co-branded webinar, or producing a co-branded whitepaper. + doing a co-branded webinar, or producing a co-branded white paper. We have a [video playlist](https://www.youtube.com/playlist?list=PL05JrBw4t0KpMqYxJiOLz-uBIr5w-yP4A) that may be helpful as part of this process. This covers various topics related to integrating your |
