summaryrefslogtreecommitdiff
path: root/src/core
Commit message (Collapse)AuthorAgeFilesLines
* chore: Add comments for all statistics countersJoel Rosdahl2023-04-191-0/+120
| | | | As suggested in discussion #1271.
* chore: Update copyright yearsv4.8Joel Rosdahl2023-03-121-1/+1
|
* feat: Make it possible to disable ccache for a certain source code fileJoel Rosdahl2023-03-072-2/+4
|
* refactor: Use util::BitSet for hash_source_code_fileJoel Rosdahl2023-03-071-3/+3
|
* refactor: Use util::BitSet for core::SloppinessJoel Rosdahl2023-03-052-47/+7
|
* feat: Improve cache size presentation and specificationJoel Rosdahl2023-03-043-34/+75
| | | | | | | | | | | | | | | | | | Aligned how cache size is presented (in "ccache --show-stats", "ccache --show-compression", "ccache --recompress", debug logs, etc.) and specified (in configuration files, "ccache --max-size" and "ccache --trim-max-size"). The size units are now formatted according to the type of size unit prefix used for the max_size/CCACHE_MAXSIZE setting: a decimal size unit prefix (k/M/G/T with or without B for bytes) in max_size means using decimal size unit prefix for presented sizes, and similar for binary size unit prefixes (Ki/Mi/Gi/Ti with or without B for bytes). If no unit is specified, GiB is assumed, . For example, "ccache -M 10" means 10 GiB. Also aligned how cache sizes are calculated. Now all sizes are computed as "apparent size", i.e., rounded up to the disk block size. This means that the cache size in "--show-stats" and the sizes presented in "--show-compression" and "--recompress" now match.
* refactor: Move Util::parse_size to utilJoel Rosdahl2023-03-041-2/+2
|
* refactor: Move Util::format_{human,parsable}_* to utilJoel Rosdahl2023-03-041-8/+8
|
* chore: Enable and fix some more warningsJoel Rosdahl2023-03-043-10/+10
|
* feat: Add support for setting per-compilation config on command lineJoel Rosdahl2023-02-201-5/+9
| | | | Closes #1035.
* feat: Improve automatic cache cleanup mechanismJoel Rosdahl2023-01-172-3/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The cache cleanup mechanism has worked essentially the same ever since ccache was initially created in 2002: - The total number and size of all files in one of the 16 subdirectories (AKA level 1) are kept in the stats file in said subdirectory. - On a cache miss, the new compilation result file is written (based on the first digits of the hash) to a subdirectory of one of those 16 subdirectories, and the stats file is updated accordingly. - Automatic cleanup is triggered if the size of the level 1 subdirectory becomes larger than max_size / 16. - ccache then lists all files in the subdirectory recursively, stats them to check their size and mtime, sorts the file list on mtime and deletes the 20% oldest files. Some problems with the approach described above: - (A) If several concurrent ccache invocations result in a cache miss and write their results to the same subdirectory then all of them will start cleaning up the same subdirectory simultaneously, doing unnecessary work. - (B) The ccache invocation that resulted in a cache miss will perform cleanup and then exit, which means that an arbitrary ccache process that happens to trigger cleanup will take a long time to finish. - (C) Listing all files in a subdirectory of a large cache can be quite slow. - (D) stat-ing all files in a subdirectory of a large cache can be quite slow. - (E) Deleting many files can be quite slow. - (F) Since a cleanup by default removes 20% of the files in a subdirectory, the actual cache size will (once the cache limit is reached) on average hover around 90% of the configured maximum size, which can be confusing. This commit solves or improves on all of the listed problems: - Before starting automatic cleanup, a global "auto cleanup" lock is acquired (non-blocking) so that at most one process is performing cleanup at a time. This solves the potential "cache cleanup stampede" described in (A). - Automatic cleanup is now performed in just one of the 256 level 2 directories. This means that a single cleanup on average will be 16 times faster than before. This improves on (B), (C), (D) and (E) since the cleanup made by a single compilation will not have to access a large part of the cache. On the other hand, cleanups will be triggered 16 times more often, but the cleanup duty will be more evenly spread out during a build. - The total cache size is calculated and compared with the configured maximum size before starting automatic cleanup. This, in combination with performing cleanup on level 2, means that the actual cache size will stay very close to the maximum size instead of about 90%. This solves (F). The limit_multiple configuration option has been removed since it is no longer used. Closes #417.
* feat: Consistently show cache size and max size with one decimalJoel Rosdahl2023-01-111-3/+3
|
* fix: Avoid sometimes too wide percent figure in --show-statsJoel Rosdahl2023-01-111-3/+6
| | | | | | | If the nominator is 99999 and the denominator is 100000, the percent function in Statistics.cpp would return "(100.00%)" instead of the wanted "(100.0%)". Fix this by using the alternate format string if the result string overflows its target size.
* enhance: Add offsetted get/set/increment methods for StatisticsCountersJoel Rosdahl2023-01-112-7/+33
|
* feat: Activate logging for command mode optionsJoel Rosdahl2023-01-111-0/+2
| | | | | This makes it possible to check ordinary log messages when debugging "ccache -c" and similar options.
* fix: Fix conditions for --show-statsJoel Rosdahl2022-12-131-2/+4
|
* chore: Improve description of --set-configJoel Rosdahl2022-12-131-1/+1
|
* refactor: Improve FileRecompressor to take a statJoel Rosdahl2022-11-273-34/+30
| | | | This avoids extra stats in some scenarios.
* enhance: Only keep atime if neededJoel Rosdahl2022-11-273-4/+12
| | | | | | | - For the --recompress case, only reset timestamps if mtime has changed since local cache LRU cleanup always uses mtime. - For the --trim-dir/--trim-recompress case, always reset timestamps since atime may be used for LRU cleanup.
* feat: Add --trim-recompress and --trim-recompress-threadsJoel Rosdahl2022-11-272-32/+116
| | | | | | Note: Reading the header to check the current compression level affects atime, so we need to restore atime even when recompression is not performed.
* refactor: Extract file recompression code to a classJoel Rosdahl2022-11-273-0/+138
|
* feat: Add --recompress-threads optionJoel Rosdahl2022-11-271-1/+16
|
* fix: Avoid duplicate magic header in --inspectJoel Rosdahl2022-11-101-1/+1
|
* chore: Sort file listJoel Rosdahl2022-11-101-1/+1
|
* chore: Fix grammar in commentJoel Rosdahl2022-11-051-2/+2
|
* chore: Improve logging of file read failuresJoel Rosdahl2022-11-021-1/+1
|
* fix: Use configured umask for command line operations like --zero-statsJoel Rosdahl2022-10-281-0/+4
| | | | Closes #1197.
* fix: Handle -MD/-MMD when compiling assembler fileJoel Rosdahl2022-10-193-6/+11
| | | | | | | | | | When compiling an assembler file, -MD and -MMD don't produce any dependency file, so don't expect one. Also, make sure to fall back to running the real compiler in case an expected output file is missing instead of exiting with an error. Fixes #1189.
* refactor: Avoid an extra data copy when rewriting stdoutJoel Rosdahl2022-10-161-3/+3
|
* refactor: Rename ShowIncludesParser to MsvcShowIncludesOutputJoel Rosdahl2022-10-164-11/+11
| | | | | | | I think that this is more in line with what the namespace represents. I also renamed ShowIncludesParser::tokenize to MsvcShowIncludesOutput::get_includes since it's not returning generic tokens but specifically includes files.
* doc: Tweak manual and comments related to /showIncludesJoel Rosdahl2022-10-161-3/+3
|
* build: Add headers to CMake project files (#1178)Orgad Shaneh2022-10-151-0/+3
| | | | Useful for listing them in the IDE project tree, for IDEs that use CMake file api (like Qt Creator).
* feat: Support auto depend mode for MSVC without /showIncludes (#1176)Orgad Shaneh2022-10-153-1/+33
| | | | If MSVC is executed *without* /showIncludes, and ccache is configured with depend mode, add /showIncludes and strip the extra output from stdout.
* feat: Support depend mode for MSVC (#992)Orgad Shaneh2022-10-123-0/+84
| | | | | | | | | Based on -showIncludes, which prints included files on stdout with a certain text prefix. Otherwise pretty similar to depend mode handling for GCC. This makes MSVC building way faster. Co-authored-by: Luboš Luňák <l.lunak@centrum.cz>
* refactor: Store compiler output as bytesJoel Rosdahl2022-10-052-3/+3
| | | | As discussed in #1173.
* Revert "feat: Support auto depend mode for MSVC without /showIncludes (#1158)"Joel Rosdahl2022-10-054-117/+1
| | | | | | This reverts commit 8b65880b5ad817156b58c58b5133aafc99b0a264. See <https://github.com/ccache/ccache/pull/1158#issuecomment-1268748557>.
* feat: Support auto depend mode for MSVC without /showIncludes (#1158)Orgad Shaneh2022-10-054-1/+117
| | | Co-authored-by: Luboš Luňák <l.lunak@centrum.cz>
* feat: Improve handling of -frandom-seed and description of sloppinessJoel Rosdahl2022-10-041-1/+1
| | | | | | I should not be necessary to distinguish between existence and non-existence of -frandom-seed if random_seed sloppiness is requested, so don't hash the "-frandom-seed=" part either.
* feat: Add sloppiness for -frandom-seed (#1168)Raihaan Shouhell2022-10-041-0/+2
|
* feat: Improve statistics for remote hits/missesJoel Rosdahl2022-10-042-12/+39
| | | | | | | | | | | | | | | | | | | | ccache collects statistics about local and remote storage layer get/put operations and describes them as "local/remote hits/misses" in the output of "ccache -s". However, since "hits" and "misses" mean "result hit/miss" in the "cacheable calls" section, it's easy to think that they measure the same thing. This commit improves the situation by: - Adding new "local/remote hits/misses" counters that mean "result hit/miss". These are shown if remote storage is used (since they otherwise are redundant and equal to the normal "hits/misses" counters) or if the -v/--verbose option is given. - Presenting the previous "local/remote hits/misses" counters as "local/remote reads". Only shown in verbose mode. - Adding "local/remote writes" counters. Only shown in verbose mode. Closes #1016.
* refactor: Sort k_statistics_fieldsJoel Rosdahl2022-10-021-2/+2
|
* chore: Rename primary/secondary storage to local/remote storageJoel Rosdahl2022-09-278-53/+55
| | | | | | | | | | | | | | | | | There is a feature request to be able not to use a local cache at all, only a network cache. With such a feature, the names "primary storage" and "secondary storage" make less sense since ccache would be operating in "secondary only" mode, but then that storage would of course become the primary (and only). Let's rename "primary storage" to "local storage" and "secondary storage" to "remote storage" – operating in "remote only" mode then makes sense. One of the original motivations to call networked storage "secondary" is that the file storage can be used for local file systems as well, making such storage "not quite remote", but in practice I guess the file storage backend used primarily for network file systems.
* chore: Rename primary/secondary config to config/system configJoel Rosdahl2022-09-272-6/+5
|
* feat: Use subsecond resolution timestamps in manifest filesJoel Rosdahl2022-09-211-10/+22
| | | | This improves accuracy in with "file_stat_matches" sloppiness.
* refactor: Use util::TimePoint for timestampsJoel Rosdahl2022-09-217-45/+51
|
* feat: Merge local manifest with fetched remote manifestJoel Rosdahl2022-09-211-6/+29
| | | | | | | | | | | | | | | | | | | | With read-only secondary storage, it can happen that primary storage has a manifest named M with a result entry R1, while secondary storage also has a manifest M but with result R2. On a compilation that matches R2, ccache will first succeed to look up M in primary storage, fail to find R2 and then get M from secondary storage where R2 can be found. Since M already exists locally, ccache will simply return the cache hit but not store knowledge of R2 locally. On a rebuild of R2, ccache therefore needs to fetch from secondary storage again. The improvement brought by this commit is that ccache now merges the manifests from primary and secondary storage and stores the merged version in primary storage. In other words, ccache setups with read-only secondary storage will be able to accumulate local header file combinations and seamlessly combine them with changes from secondary storage. Closes #1049.
* refactor: Extract timestamp logic from ManifestJoel Rosdahl2022-09-212-49/+20
|
* refactor(storage): Pass cache entries via memory instead of filesJoel Rosdahl2022-09-212-6/+7
|
* feat: Print result format version and no of files in --inspectJoel Rosdahl2022-09-214-11/+40
|
* chore: Simplify cache entry reading and writingJoel Rosdahl2022-09-2122-920/+623
| | | | | | | | | | | | | | | | Cache entries are now fully read into memory before (de)compressing, checksumming and parsing, instead of streaming data like before. While this increases memory usage when working with large object files, it also simplifies the code a lot. Another motivation for this change is that cache entry data is not streamed from secondary storage anyway, and it makes sense to keep the architecture simple and similar for primary and secondary storage code paths. The cache entry format has modified so that the checksum covers the potentially compressed payload (plus the header), not the uncompressed payload (plus the header) like before. The checksum is now also stored in an uncompressed epilogue. Since the cache entry format has been changed, the input hash has been changed as well.
View Lorry Controller Status