| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
| |
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
|
| |
|
|\
| |
| |
| |
| |
| |
| | |
Fully migrate build stages again
Closes #38756
See merge request gitlab-org/gitlab-ce!15741
|
| |\
| | |
| | |
| | | |
# Conflicts:
# db/schema.rb
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| |\ \
| | | |
| | | |
| | | | |
Conflicts:
db/schema.rb
|
| | | | |
|
| | | |
| | | |
| | | |
| | | | |
[ci skip]
|
| | | | |
|
| |\ \ \
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
* master: (1480 commits)
Conflicts:
db/schema.rb
|
| | | | | |
|
| | | | | |
|
| | | | |
| | | | |
| | | | |
| | | | | |
For builds that are still missing `stage_id`.
|
|\ \ \ \ \
| | | | | |
| | | | | |
| | | | | | |
# Conflicts:
# db/schema.rb
|
| | |_|_|/
| |/| | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
This optimises searching for users when using queries consisting out of
one or two characters such as "ab". We optimise such cases by searching
for `LOWER(name)` and `LOWER(username)` instead of using `ILIKE`. Using
`LOWER` produces a _much_ better performing query.
For example, when searching for all users matching the term "a" we'd
produce the following plan:
Limit (cost=637.69..637.74 rows=20 width=805) (actual time=41.983..41.995 rows=20 loops=1)
Buffers: shared hit=8330
-> Sort (cost=637.69..638.61 rows=368 width=805) (actual time=41.982..41.990 rows=20 loops=1)
Sort Key: (CASE WHEN ((name)::text = 'a'::text) THEN 0 WHEN ((username)::text = 'a'::text) THEN 1 WHEN ((email)::text = 'a'::text) THEN 2 ELSE 3 END), name
Sort Method: top-N heapsort Memory: 35kB
Buffers: shared hit=8330
-> Bitmap Heap Scan on users (cost=75.47..627.89 rows=368 width=805) (actual time=9.452..41.305 rows=277 loops=1)
Recheck Cond: (((name)::text ~~* 'a'::text) OR ((username)::text ~~* 'a'::text) OR ((email)::text = 'a'::text))
Rows Removed by Index Recheck: 7601
Heap Blocks: exact=7636
Buffers: shared hit=8327
-> BitmapOr (cost=75.47..75.47 rows=368 width=0) (actual time=8.290..8.290 rows=0 loops=1)
Buffers: shared hit=691
-> Bitmap Index Scan on index_users_on_name_trigram (cost=0.00..38.85 rows=180 width=0) (actual time=4.369..4.369 rows=4071 loops=1)
Index Cond: ((name)::text ~~* 'a'::text)
Buffers: shared hit=360
-> Bitmap Index Scan on index_users_on_username_trigram (cost=0.00..34.41 rows=188 width=0) (actual time=3.896..3.896 rows=4140 loops=1)
Index Cond: ((username)::text ~~* 'a'::text)
Buffers: shared hit=328
-> Bitmap Index Scan on users_email_key (cost=0.00..1.94 rows=1 width=0) (actual time=0.022..0.022 rows=0 loops=1)
Index Cond: ((email)::text = 'a'::text)
Buffers: shared hit=3
Planning time: 3.912 ms
Execution time: 42.171 ms
With the changes in this commit we now produce the following plan
instead:
Limit (cost=13257.48..13257.53 rows=20 width=805) (actual time=1.567..1.579 rows=20 loops=1)
Buffers: shared hit=287
-> Sort (cost=13257.48..13280.93 rows=9379 width=805) (actual time=1.567..1.572 rows=20 loops=1)
Sort Key: (CASE WHEN ((name)::text = 'a'::text) THEN 0 WHEN ((username)::text = 'a'::text) THEN 1 WHEN ((email)::text = 'a'::text) THEN 2 ELSE 3 END), name
Sort Method: top-N heapsort Memory: 35kB
Buffers: shared hit=287
-> Bitmap Heap Scan on users (cost=135.66..13007.91 rows=9379 width=805) (actual time=0.194..1.107 rows=277 loops=1)
Recheck Cond: ((lower((name)::text) = 'a'::text) OR (lower((username)::text) = 'a'::text) OR ((email)::text = 'a'::text))
Heap Blocks: exact=277
Buffers: shared hit=287
-> BitmapOr (cost=135.66..135.66 rows=9379 width=0) (actual time=0.152..0.152 rows=0 loops=1)
Buffers: shared hit=10
-> Bitmap Index Scan on yorick_test_users (cost=0.00..124.75 rows=9377 width=0) (actual time=0.101..0.101 rows=277 loops=1)
Index Cond: (lower((name)::text) = 'a'::text)
Buffers: shared hit=4
-> Bitmap Index Scan on index_on_users_lower_username (cost=0.00..1.94 rows=1 width=0) (actual time=0.035..0.035 rows=1 loops=1)
Index Cond: (lower((username)::text) = 'a'::text)
Buffers: shared hit=3
-> Bitmap Index Scan on users_email_key (cost=0.00..1.94 rows=1 width=0) (actual time=0.014..0.014 rows=0 loops=1)
Index Cond: ((email)::text = 'a'::text)
Buffers: shared hit=3
Planning time: 0.303 ms
Execution time: 1.687 ms
Here we can see the new query is 25 times faster compared to the old
query.
|
|/ / / / |
|
| | | | |
|
| |_|/
|/| |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This helps with queries that get project ids based on the - comparably
rare - visibility levels 10 and 20. For these, postgres can now leverage
the partial index for a index-only scan to improve performance.
Example queries:
SELECT id FROM projects WHERE visibility_level IN (10,20)
SELECT id FROM projects WHERE visibility_level IN (10)
For MySQL, this results in a full index on id because MySQL omits the
WHERE clause. That is, the index is a duplicate of the primary key
basically.
|
| | |
| | |
| | |
| | | |
See https://gitlab.com/gitlab-org/gitlab-ce/issues/43271
|
|\ \ \
| | | |
| | | |
| | | |
| | | | |
Change SQL for expired artifacts to use new ci_job_artifacts.expire_at
See merge request gitlab-org/gitlab-ce!16578
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
migration efficiently
Artifacts are in the middle of being migrated from ci_builds to
ci_job_artifacts. The expiration date is currently visible in both of
these tables and the test for whether an expired artifact is present
for a job is complex as it requires checking both the of the tables.
Add two new indexes, one on ci_builds.artifacts_expire_at and one on
ci_job_artifacts.expire_at to enable finding expired artifacts
efficiently.
And until the migration is finished, replace the SQL for finding
expired and non-expired artifacts with a hand-crafted UNION ALL based
query instead of using OR. This overcomes a database optimizer
limitation that prevents it from using these indexes.
When the migration is finished the next version should remove this
query and replace it with a much simpler query on just
ci_job_artifacts. See
https://gitlab.com/gitlab-org/gitlab-ce/issues/42561 for followup.
|
|\ \ \ \
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Resolve "PrepareUntrackedUploads PostgreSQL syntax error"
Closes #42881
See merge request gitlab-org/gitlab-ce!17019
|
| | |_|/
| |/| |
| | | |
| | | |
| | | |
| | | | |
To finish migrating untracked files to uploads for installations that were affected by https://gitlab.com/gitlab-org/gitlab-ce/issues/42881
Or just to delete the temp table if it is empty and left behind.
|
|/ / /
| | |
| | |
| | |
| | |
| | | |
db/post_migrate/20170717111152_cleanup_move_system_upload_folder_symlink.rb where symlink already exists
Signed-off-by: Rémy Coutable <remy@rymai.me>
|
|/ /
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Since populating the fork networks was scheduled multiple times
because of bugs that needed to be fixed:
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15595/
Creating fork networks for projects that were missing the root of
the fork network.
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15709
The API allowed creating forked_project_links without
fork_network_members.
Scheduling this migration multiple times would case it to run
concurrently. Which in turn would try to insert the same data into
`fork_network_members` causing duplicate key errors.
This avoids running the migration multiple times.
|
| | |
|
|\ \
| | |
| | |
| | |
| | |
| | |
| | | |
Remove redundant pipeline stages from the database
Closes #41769
See merge request gitlab-org/gitlab-ce!16580
|
| | | |
|
| |\ \
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
fix/gb/fix-redundant-pipeline-stages
* commit '4457cf9d178dc9912fd9c16427ad81b389179d00': (76 commits)
Conflicts:
spec/services/ci/retry_build_service_spec.rb
|
| | | |
| | | |
| | | |
| | | |
| | | | |
This makes it possible to heavily optimize this migration, because we
need an outdated index to remove redundant stages faster.
|
| | | | |
|
| | | | |
|
| |\ \ \
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
* master: (441 commits)
Conflicts:
db/schema.rb
|
| | | | | |
|
| | | | | |
|
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Currently we still need to run EnsureStageService within a transaction,
because when it runs within in a transaction we are going to stick to
the primary database when using database load balancing. Extracting this
out of the transaction makes it possible to hit into problems with
replication lag in pipeline commit status API, which can cause a lot of
trouble.
|
| | | | | |
|
| | | | | |
|
| | | | | |
|
| | | | | |
|
| | | | | |
|
| | | | | |
|
| | | | | |
|
|\ \ \ \ \
| |_|_|/ /
|/| | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
into 'master'
Resolve "Add domain field to Auto DevOps application setting"
Closes #38175
See merge request gitlab-org/gitlab-ce!16604
|
| |\ \ \ \
| | | | | |
| | | | | |
| | | | | | |
38175-add-domain-field-to-auto-devops-application-setting
|
| |\ \ \ \ \
| | | | | | |
| | | | | | |
| | | | | | | |
38175-add-domain-field-to-auto-devops-application-setting
|
| | |_|/ / /
| |/| | | | |
|