| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
Previously, you could only access personal snippets in the API if you
had authored them. The documentation doesn't state that this is the
case, and it's quite surprising.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This completely rewrites the SnippetsFinder class from the ground up in
order to improve its performance. The old code was beyond salvaging. It
was complex, included various Rails 5 workarounds, comments that
shouldn't be necessary, and most important of all: it produced a really
poorly performing database query.
As a result, I opted for rewriting the finder from scratch, instead of
trying to patch the existing code. Instead of trying to reuse as many
existing methods as possible, I opted for defining new methods
specifically meant for the SnippetsFinder. This requires some extra code
here and there, but allows us to have much more control over the
resulting SQL queries. It is these changes that then allow us to produce
a _much_ more efficient query.
To illustrate how bad the old query was, we will use my own snippets as
an example. Currently I have 52 snippets, most of which are global ones.
To retrieve these, you would run the following Ruby code:
user = User.find_by(username: 'yorickpeterse')
SnippetsFinder.new(user, author: user).execute
On GitLab.com the resulting query will take between 10 and 15 seconds to
run, producing the query plan found at
https://explain.depesz.com/s/Y5IX. Apart from the long execution time,
the total number of buffers (the sum of all shared hits) is around 185
GB, though the real number is probably (hopefully) much lower as I doubt
simply summing these numbers produces the true total number of buffers
used.
The new query's plan can be found at https://explain.depesz.com/s/wHdN,
and this query takes between 10 and 100-ish milliseconds to run. The
total number of buffers used is only about 30 MB.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/52639
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit adds the module `FromUnion`, which provides the class method
`from_union`. This simplifies the process of selecting data from the
result of a UNION, and reduces the likelihood of making mistakes. As a
result, instead of this:
union = Gitlab::SQL::Union.new([foo, bar])
Foo.from("(#{union.to_sql}) #{Foo.table_name}")
We can now write this instead:
Foo.from_union([foo, bar])
This commit also includes some changes to make this new setup work
properly. For example, a bug in Rails 4
(https://github.com/rails/rails/issues/24193) would break the use of
`from("sub-query-here").includes(:relation)` in certain cases. There was
also a CI query which appeared to repeat a lot of conditions from an
outer query on an inner query, which isn't necessary.
Finally, we include a RuboCop cop to ensure developers use this new
module, instead of using Gitlab::SQL::Union directly.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/51307
|
| |\
| |
| |
| |
| | |
Enable frozen string in app/graphql + app/finders
See merge request gitlab-org/gitlab-ce!21681
|
| | |
| |
| |
| | |
Partially addresses #47424.
|
| |/
|
|
|
| |
This whitelists all existing offenses for the various CodeReuse cops, of
which most are triggered by the CodeReuse/ActiveRecord cop.
|
| |
|
|
|
|
|
|
|
|
|
| |
There is a bug https://github.com/rails/arel/issues/531 in Rails 5.0 and
5.1 in Arel/ActiveRecord.
This commit converts arel based queries to their raw counterparts to
make the finder work in Rails 5.0.
These changes should be reverted when on Rails 5.2 as since that version
such queries work well again. See the bug link.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This applies the changes introduced in `Project.public_or_visible_to_user`
to the snippets finder only.
We know that for `SnippetsFinder`, this change improves SQL timing from
5/23/25s to 0.7/2/4s (mean/p95/p99). At the same time, the scope was too
broad, (negatively) affecting SQL timings in various other places:
https://gitlab.com/gitlab-com/infrastructure/issues/3772
With this commit, the snippets dashboard stays usuable as we generally
don't run into statement timeouts. In contrast to the earlier change in
!17088, the scope of the change is limited to `SnippetsFinder` only,
thus not affecting other places.
|
| |
|
|
|
|
|
|
|
|
|
| |
These changes were introduced in MR
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17088. In
https://gitlab.com/gitlab-com/infrastructure/issues/3772 we discovered
these changes lead to a pretty drastic increase in SQL response timings.
We'll revert these changes so we can work on a better solution in the
mean time without GitLab.com (or other installations) experiecing
reduced performance in the mean time.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Immediately using #from here requires a lot of changes in
other finders (e.g. IssuableFinder, TodosFinder). In all places where we
use #merge, this goes completely the wrong way when passed in a relation
that was built with `#from(...)`: The original query's FROM part gets
completely replaced.
This avoids changing all other places and focuses on improving
SnippetFinder with the downside of two (small) codepaths to do the same
thing.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
'security-10-4-25223-snippets-finder-doesnt-obey-feature-visibility' into 'security-10-4'
[Port for security-10-4]: Makes SnippetFinder ensure feature visibility
|
| |
|
|
| |
Closes #40755.
|
| |
|
|
|
|
| |
Refactor snippets finder & dont return internal snippets for external users
See merge request !2094
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Resolve "Updated UI for Snippets pages"
## What does this MR do?
## Are there points in the code the reviewer needs to double check?
## Why was this MR needed?
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [Changelog entry](https://docs.gitlab.com/ce/development/changelog.html) added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] API support added
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
Closes #19990
See merge request !7861
|
| | | |
|
| | | |
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
| |
Adding the necessary API for the new /snippets Restful resource
added with this commit. Added a new Grape class `Snippets`, as
well as a `PersonalSnippet` entity.
Issue: #20042
Merge-Request: !6373
Signed-off-by: Guyzmo <guyzmo+gitlab+pub@m0g.net>
|
| | |
|
| |
|
|
| |
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/15580
|
| |
|
|
|
|
| |
This was removed from the interface in
https://github.com/gitlabhq/gitlabhq/pull/6027 but its implementation
lingered around for two years.
|
| | |
|
| | |
|
| | |
|
| |
|
