| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |\
| |
| |
| |
| |
| |
| | |
Add a helper to rename a column using a background migration
Closes #47591
See merge request gitlab-org/gitlab-ce!20180
|
| | |
| |
| |
| |
| |
| |
| |
| | |
This works the same way as change_column_type_using_background_migration, but
for renaming a column. It takes a table, not a relation, to match its concurrent
counterpart.
Also, generalise the cleanup migrations to reduce code duplication.
|
| |/ |
|
| |
|
|
| |
Gitlab::Database::RenameReservedPathsMigration::V1::MigrationClasses::Namespace
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This commit does a number of things:
1. Reduces the number of queries needed by perform a single query to get all
the tuples for the relevant rows.
2. Uses a transaction to query the tuple counts to ensure that the data
is retrieved from the primary.
Closes #46742
|
| |
|
|
|
|
|
|
|
| |
Uses PostgreSQL tuple estimates to provide a much faster yet approximate
count. See https://wiki.postgresql.org/wiki/Slow_Counting for more details.
We only use this fast method if the table has been analyzed or vacuumed
within the last hour.
Closes #46255
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In Arel 7.0.0 (Arel 7.1.4 is used in Rails 5.0) the `engine` parameter
of `Arel::UpdateManager#initializer` was removed.
This commit makes the gitlab database helpers work both in rails 4 and
rails 5.
Fixes errors like this one:
```
1) Gitlab::Database::MigrationHelpers#update_column_in_batches when running outside of a transaction updates all the rows in a table
Failure/Error:
update_arel = Arel::UpdateManager.new(ActiveRecord::Base)
.table(table)
.set([[table[column], value]])
.where(table[:id].gteq(start_id))
ArgumentError:
wrong number of arguments (given 1, expected 0)
# ./lib/gitlab/database/migration_helpers.rb:317:in `new'
# ./lib/gitlab/database/migration_helpers.rb:317:in `block in update_column_in_batches'
# ./lib/gitlab/database/migration_helpers.rb:307:in `loop'
# ./lib/gitlab/database/migration_helpers.rb:307:in `update_column_in_batches'
# ./spec/lib/gitlab/database/migration_helpers_spec.rb:367:in `block (4 levels) in <top (required)>'
```
|
| |
|
|
|
|
|
|
|
|
|
| |
Direct disk access is done through Gitaly now, so the legacy path was
deprecated. This path was used in Gitlab::Shell however. This required
the refactoring in this commit.
Added is the removal of direct path access on the project model, as that
lookup wasn't needed anymore is most cases.
Closes https://gitlab.com/gitlab-org/gitaly/issues/1111
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
'security-10-6'
[10.6] Prevent notes on confidential issues from being sent to chat
See merge request gitlab/gitlabhq!2366
# Conflicts:
# app/helpers/services_helper.rb
|
| |
|
|
| |
This will use the same query as `\di` to find an index on postgresql.
|
| |\
| |
| |
| |
| | |
Route path lookups through legacy_disk_path
See merge request gitlab-org/gitlab-ce!17743
|
| | | |
|
| | |
| |
| |
| | |
Closes #43887.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Index creation does not have an effect if the index is present already.
Index removal does not have an affect if the index is not present.
This helps to avoid patterns like this in migrations:
```
if index_exists?(...)
remove_concurrent_index(...)
end
```
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The concurrency issue originates from inserts on
`user_interacted_projects` from the app while running the post-deploy
migration.
This change comes with a strategy to lock the table while removing
duplicates and creating the unique index (and similar for FK
constraints).
Also, we'll have a non-unique index until the post-deploy migration is
finished to speed up queries during that time.
Closes #44205.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
This fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/38634.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Prior to this commit we would essentially update all rows in a table,
even those where the source column (e.g. `issues.closed_at`) was NULL.
This in turn could lead to statement timeouts when using the default
batch size of 10 000 rows per job.
To work around this we don't schedule jobs for rows where the source
value is NULL. We also don't update rows where the source column is NULL
(as an extra precaution) or the target column already has a non-NULL
value. Using this approach it should be possible to update 10 000 rows
in the "issues" table in about 7.5 - 8 seconds.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/42158
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a minimum interval to BackgroundMigrationWorker, ensuring
background migrations of the same class only run once every 5 minutes.
This prevents a thundering herd problem where scheduled migrations all
run at once due to their delays having been expired (e.g. as the result
of a queue being paused for a long time).
If a job was recently executed it's rescheduled with a delay that equals
the remaining time of the job's lease. This means that if the lease
expires in two minutes we only need to wait two minutes, instead of
five.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/41624
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In a previous attempt (rolled back in
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/16021) we tried
to migrate `issues.closed_at` from timestamp to timestamptz using a
regular migration. This has a bad impact on GitLab.com and as such was
rolled back.
This commit re-implements the original migrations using generic
background migrations, allowing us to still migrate the data in a single
release but without a negative impact on availability.
To ensure the database schema is up to date the background migrations
are performed inline in development and test environments. We also make
sure to not migrate that that doesn't need migrating in the first place
or has already been migrated.
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* upstream/master: (671 commits)
Make rubocop happy
Use guard clause
Improve language
Prettify
Use temp branch
Pass info about who started the job and which job triggered it
Docs: add indexes for monitoring and performance monitoring
clearer-documentation-on-inline-diffs
Add docs for commit diff discussion in merge requests
sorting for tags api
Clear BatchLoader after each spec to prevent holding onto records longer than necessary
Include project in BatchLoader key to prevent returning blobs for the wrong project
moved lfs_blob_ids method into ExtractsPath module
Converted JS modules into exported modules
spec fixes
Bump gitlab-shell version to 5.10.3
Clear caches before updating MR diffs
Use new Ruby version 2.4 in GitLab QA images
moved lfs blob fetch from extractspath file
Update GitLab QA dependencies
...
|
| | | |
|
| |\ \
| |/
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* upstream/master: (126 commits)
Update VERSION to 10.3.0-pre
Update CHANGELOG.md for 10.2.0
default fill color for SVGs
ignore hashed repos (for now) when using `rake gitlab:cleanup:repos`
Use Redis cache for branch existence checks
Update CONTRIBUTING.md: Link definition of done to criteria
Use `make install` for Gitaly setups in non-test environments
FileUploader should check for hashed_storage?(:attachments) to use disk_path
Set the default gitlab-shell timeout to 3 hours
Update composite pipelines index to include "id"
Use arrays in Pipeline#latest_builds_with_artifacts
Fix blank states using old css
Skip confirmation user api
Custom issue tracker
Revert "check for `read_only?` first before seeing if request is disallowed"
add `#with_metadata` scope to remove a N+1 from the notes' API
Fix promoting milestone updating all issuables without milestone
Batchload blobs for diff generation
check for `read_only?` first before seeing if request is disallowed
use `Gitlab::Routing.url_helpers` instead of `Rails.application.routes.url_helpers`
...
|
| | |
| |
| |
| |
| |
| |
| |
| | |
When a project is using hashed storage, the repositories and
attachments wouldn't be saved on disk using the `full_path`. So the
migration would not do anything.
However: best to just skip moving when hashed storage is enabled.
|
| | | |
|
| | |
| |
| |
| |
| | |
And use .rubocop.yml to exclude paths we don't care,
rather than using the cop itself to exclude.
|
| | | |
|
| |\ \
| |/
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* upstream/master: (507 commits)
Add dropdowns documentation
Convert migration to populate latest merge request ID into a background migration
Set 0.69.0 instead of latest for codeclimate image
De-duplicate background migration matchers defined in spec/support/migrations_helpers.rb
Update database_debugging.md
Update database_debugging.md
Move installation of apps higher
Change to Google Kubernetes Cluster and add internal links
Add Ingress description from official docs
Add info on creating your own k8s cluster from the cluster page
Add info about the installed apps in the Cluster docs
Resolve "lock/confidential issuable sidebar custom svg icons iteration"
Update HA README.md to clarify GitLab support does not troubleshoot DRBD.
Update license_finder to 3.1.1
Make sure NotesActions#noteable returns a Noteable in the update action
Cache the number of user SSH keys
Adjust openid_connect_spec to use `raise_error`
Resolve "Clicking on GPG verification badge jumps to top of the page"
Add changelog for container repository path update
Update container repository path reference
...
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
This ensures we can check if the user has TRIGGER permissions without
querying restricted tables. Thanks to Steve Norman
(https://gitlab.com/stevenorman) for helping out with this merge
request.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/38372
|
| |/ |
|
| |\
| |
| |
| |
| |
| |
| | |
Clean up redirect routes that conflict with regular routes
Closes #36229
See merge request gitlab-org/gitlab-ce!13783
|
| | | |
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Prior to this commit running
Namespace#force_share_with_group_lock_on_descendants would result in
updating _all_ namespaces in the namespaces table, not just the
descendants. This is the result of ActiveRecord::Relation#update_all not
taking into account the CTE. To work around this we use the CTE query as
a sub-query instead of directly calling #update_all.
To prevent this from happening the relations returned by
Gitlab::GroupHierarchy are now marked as read-only, resulting in an
error being raised when methods such as #update_all are used.
Fortunately on GitLab.com our statement timeouts appear to have
prevented this query from actually doing any damage other than causing
a very large amount of dead tuples.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/37916
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a bunch of checks to migrations that may create or drop
triggers. Dropping triggers/functions is done using "IF EXISTS" so we
don't throw an error if the object in question has already been dropped.
We now also raise a custom error (message) when the user does not have
TRIGGER privileges. This should prevent the schema from entering an
inconsistent state while also providing the user with enough information
on how to solve the problem.
The recommendation of using SUPERUSER permissions is a bit extreme but
we require this anyway (Omnibus also configures users with this
permission).
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/36633
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Due to a missing `on_delete: :cascade`, users would hit the error that
looked like:
```
PG::ForeignKeyViolation: ERROR: update or delete on table "protected_tags"
violates foreign key constraint "fk_rails_f7dfda8c51" on table
"protected_tag_create_access_levels" DETAIL: Key (id)=(1385) is still
referenced from table "protected_tag_create_access_levels". : DELETE FROM
"protected_tags" WHERE "protected_tags"."id" = 1385
```
Closes #36013
|
| | |
|
| | |
|
| |
|
|
|
|
| |
These attributes are stored in binary in the database, but exposed as
strings. This allows one to query/create data using plain SHA1 hashes as
Strings, while storing them more efficiently as binary.
|
| | |
|
| | |
|
| | |
|
