| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
| |
This is already handled for diffs, but not commits.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This version does not use transactions, but individual statements. As we have
unique constraints on the target tables for the inserts, we can just ignore
uniqueness violations there (as long as we always insert the same batch size, in
the same order).
This means the spec now must use truncation, not a transaction, as the
uniqueness violation means that the whole transaction for that spec would be
invalid, which isn't what we'd want. In real-world use, this isn't run in a
transaction anyway.
This commit also wraps unhandled exceptions, for easier finding in Sentry, and
logs with a consistent format, for easier searching.
|
| |
|
|
|
|
| |
We were hitting the statement timeout for very large MR diffs. Now we insert at
most 1,000 rows to `merge_request_diff_commits` in a single statement, or 100
rows to `merge_request_diff_files`.
|
| |
|
|
|
|
|
|
|
|
|
| |
Guess the modes based on the following:
1. If the file didn't exist, it's zero.
2. If the diff contains 'Subproject commit', it might be a submodule, so 0600.
3. Otherwise, it's 0644.
This isn't perfect, but it doesn't have to be - it won't change file modes in
the repository.
|
| |
|
|
|
| |
Not all diffs in st_diffs had the too_large attribute set at all, but the column
is non-nullable. Explicitly cast to boolean to avoid that problem.
|
|
|
Previously, we stored these as serialised fields - `st_{commits,diffs}` - on the
`merge_request_diffs` table. These now have their own tables -
`merge_request_diff_{commits,diffs}` - with a column for each attribute of the
serialised data.
Add a background migration to go through the existing MR diffs and migrate them
to the new format. Ignore any contents that cannot be displayed. Assuming that
we have 5 million rows to migrate, and each batch of 2,500 rows can be
completed in 5 minutes, this will take about 7 days to migrate everything.
|
