summaryrefslogtreecommitdiff
path: root/include
Commit message (Collapse)AuthorAgeFilesLines
* Support Zstd compression level in Leveldbleveldb Team2023-04-201-0/+4
| | | | PiperOrigin-RevId: 520556840
* Add support for Zstd-based compression in LevelDB.leveldb Team2023-03-281-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change implements support for Zstd-based compression in LevelDB. Building up from the Snappy compression (which has been supported since inception), this change adds Zstd as an alternate compression algorithm. We are implementing this to provide alternative options for users who might have different performance and efficiency requirements. For instance, the Zstandard website (https://facebook.github.io/zstd/) claims that the Zstd algorithm can achieve around 30% higher compression ratios than Snappy, with relatively smaller (~10%) slowdowns in de/compression speeds. Benchmarking results: $ blaze-bin/third_party/leveldb/db_bench LevelDB: version 1.23 Date: Thu Feb 2 18:50:06 2023 CPU: 56 * Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz CPUCache: 35840 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillseq : 2.613 micros/op; 42.3 MB/s fillsync : 3924.432 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.609 micros/op; 30.7 MB/s overwrite : 4.508 micros/op; 24.5 MB/s readrandom : 6.136 micros/op; (864322 of 1000000 found) readrandom : 5.446 micros/op; (864083 of 1000000 found) readseq : 0.180 micros/op; 613.3 MB/s readreverse : 0.321 micros/op; 344.7 MB/s compact : 827043.000 micros/op; readrandom : 4.603 micros/op; (864105 of 1000000 found) readseq : 0.169 micros/op; 656.3 MB/s readreverse : 0.315 micros/op; 350.8 MB/s fill100K : 854.009 micros/op; 111.7 MB/s (1000 ops) crc32c : 1.227 micros/op; 3184.0 MB/s (4K per op) snappycomp : 3.610 micros/op; 1081.9 MB/s (output: 55.2%) snappyuncomp : 0.691 micros/op; 5656.3 MB/s zstdcomp : 15.731 micros/op; 248.3 MB/s (output: 44.1%) zstduncomp : 4.218 micros/op; 926.2 MB/s PiperOrigin-RevId: 509957778
* Fixes for C++20 support.Peter Kasting2022-05-051-2/+0
| | | | | | | Structs with user-declared constructors are no longer considered aggregates. Just remove the constructor declaration where applicable. Bug: chromium:1284275
* remove useless code in cache.hxindubawukong2022-01-061-8/+0
|
* Don't include C++ headers in extern CPaul Beusterien2021-03-031-4/+4
|
* Change version to 1.23.1.23Chris Mumford2021-02-231-1/+1
| | | | PiperOrigin-RevId: 359111035
* Add some std:: qualifiers to types and functions.Victor Costan2020-04-291-1/+1
| | | | PiperOrigin-RevId: 309110431
* Switch from C headers to C++ headers.Victor Costan2020-04-297-13/+11
| | | | | | | | | | | | | | | This CL makes the following substitutions. * assert.h -> cassert * math.h -> cmath * stdarg.h -> cstdarg * stddef.h -> cstddef * stdint.h -> cstdint * stdio.h -> cstdio * stdlib.h -> cstdlib * string.h -> cstring PiperOrigin-RevId: 309080151
* Add Env::Remove{File,Dir} which obsolete Env::Delete{File,Dir}.Victor Costan2020-01-091-21/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The "DeleteFile" method name causes pain for Windows developers, because <windows.h> #defines a DeleteFile macro to DeleteFileW or DeleteFileA. Current code uses workarounds, like #undefining DeleteFile everywhere an Env is declared, implemented, or used. This CL removes the need for workarounds by renaming Env::DeleteFile to Env::RemoveFile. For consistency, Env::DeleteDir is also renamed to Env::RemoveDir. A few internal methods are also renamed for consistency. Software that supports Windows is expected to migrate any Env implementations and usage to Remove{File,Dir}, and never use the name Env::Delete{File,Dir} in its code. The renaming is done in a backwards-compatible way, at the risk of making it slightly more difficult to build a new correct Env implementation. The backwards compatibility is achieved using the following hacks: 1) Env::Remove{File,Dir} methods are added, with a default implementation that calls into Env::Delete{File,Dir}. This makes old Env implementations compatible with code that calls into the updated API. 2) The Env::Delete{File,Dir} methods are no longer pure virtuals. Instead, they gain a default implementation that calls into Env::Remove{File,Dir}. This makes updated Env implementations compatible with code that calls into the old API. The cost of this approach is that it's possible to write an Env without overriding either Rename{File,Dir} or Delete{File,Dir}, without getting a compiler warning. However, attempting to run the test suite will immediately fail with an infinite call stack ending in {Remove,Delete}{File,Dir}, making developers aware of the problem. PiperOrigin-RevId: 288710907
* unsigned char -> uint8_tVictor Costan2019-05-281-10/+10
| | | | PiperOrigin-RevId: 250309603
* Add argument definition for void c functions.allangj2019-05-051-7/+7
| | | | | Allow the use c.h on projects with -Wstrict-prototypes Modify CMakelist to include -Wstrict-prototypes
* Style cleanup.Victor Costan2019-05-041-1/+1
| | | | | | | | | 1) Convert iterator-based for loops to C++11 foreach loops. 2) Convert "void operator=" to "T& operator=". 3) Switch from copy operators from private to public deleted. 4) Switch from empty ctors / dtors to "= default" where appropriate. PiperOrigin-RevId: 246679195
* Bump the version number from 1.21 to 1.22.1.22Chris Mumford2019-05-031-1/+1
| | | | PiperOrigin-RevId: 246558281
* Correct class/structure declaration order.Chris Mumford2019-05-037-34/+37
| | | | | | | | | | | | | 1. Correct the class/struct declaration order to be IAW the Google C++ style guide[1]. 2. For non-copyable classes, switched from non-implemented private methods to explicitly deleted[2] methods. 3. Minor const and member initialization fixes. [1] https://google.github.io/styleguide/cppguide.html#Declaration_Order [2] http://eel.is/c++draft/dcl.fct.def.delete PiperOrigin-RevId: 246521844
* Format all files IAW the Google C++ Style Guide.Chris Mumford2019-05-0214-74/+79
| | | | | | Use clang-format to correct formatting to be in agreement with the [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html). Doing this simplifies the process of accepting changes. Also fixed a few warnings flagged by clang-tidy. PiperOrigin-RevId: 246350737
* [leveldb] Specifically export the WriteBatch::Handler inner class for ↵leveldb Team2019-04-231-1/+1
| | | | | | | | | Windows link Windows linking visibility in shared libraries requires that inner classes are specifically exported as visible, even if the containing class is exported. PiperOrigin-RevId: 244886019
* Merge pull request #472 from zhoudayang:patch-1Chris Mumford2019-04-121-1/+1
|\ | | | | | | PiperOrigin-RevId: 243314507
| * fix comment周炀2017-05-221-1/+1
| |
* | Update the version to 1.21 in preparation for a new release.1.21Chris Mumford2019-03-291-2/+2
| | | | | | | | PiperOrigin-RevId: 241053616
* | Inline defaults in options.costan2019-03-201-48/+20
| | | | | | | | | | | | | | | | | | | | This CL moves default values for leveldb::{Options,ReadOptions,WriteOptions} from constructors to member declarations, and removes now-redundant comments stating the defaults. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=239271242
* | Added native support for Windows.cmumford2019-03-011-0/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds a native Windows port (port_windows.h) and a Windows Env (WindowsEnv). Note1: "small" is defined when including <Windows.h> so some parameters were renamed to avoid conflict. Note2: leveldb::Env defines the method: "DeleteFile" which is also a constant defined when including <Windows.h>. The solution was to ensure this macro is defined in env.h which forces the function, when compiled, to be either DeleteFileA or DeleteFileW when building for MBCS or UNICODE respectively. This resolves #519 on GitHub. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=236364778
* | leveldb: Make WriteBatch::ApproximateSize() const.costan2019-01-151-1/+1
| | | | | | | | | | | | ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=229395810
* | Clarify comments for leveldb::Env file reading methods.costan2018-09-041-2/+2
| | | | | | | | | | | | | | | | | | | | "Create a brand new [adjective] file" seems like the description for a method that will create a new file, but is used for methods that open existing files for read access. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=211468002
* | Expose WriteBatch::Append in the C API.costan2018-08-191-1/+3
| | | | | | | | | | | | ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=209345072
* | Expose WriteBatch::Append().costan2018-08-141-0/+7
| | | | | | | | | | | | | | | | | | WriteBatchInternal has a method for efficiently concatenating two WriteBatches. This commit exposes the method to the public API. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=208724311
* | Clean up Iterator.costan2018-06-041-4/+12
| | | | | | | | | | | | | | | | | | | | | | This CL renames the private struct Iterator::Cleanup -> Iterator::CleanupNode, to better reflect that it's a linked list node, and extracts duplicated code from its user in IsEmpty() and Run() methods. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=199175058
* | Add move constructor to Status.costan2018-04-231-11/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will result in smaller code generation when Status instances are passed around. Benchmarks don't indicate a significant change either way. CPU: 48 * Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz CPUCache: 30720 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Baseline: fillseq : 3.589 micros/op; 30.8 MB/s fillsync : 4165.299 micros/op; 0.0 MB/s (1000 ops) fillrandom : 5.864 micros/op; 18.9 MB/s overwrite : 7.830 micros/op; 14.1 MB/s readrandom : 5.534 micros/op; (1000000 of 1000000 found) readrandom : 4.292 micros/op; (1000000 of 1000000 found) readseq : 0.312 micros/op; 354.1 MB/s readreverse : 0.501 micros/op; 220.8 MB/s compact : 886211.000 micros/op; readrandom : 3.518 micros/op; (1000000 of 1000000 found) readseq : 0.251 micros/op; 441.2 MB/s readreverse : 0.456 micros/op; 242.4 MB/s fill100K : 1329.723 micros/op; 71.7 MB/s (1000 ops) crc32c : 1.976 micros/op; 1976.7 MB/s (4K per op) snappycomp : 4.705 micros/op; 830.2 MB/s (output: 55.1%) snappyuncomp : 0.958 micros/op; 4079.1 MB/s acquireload : 0.727 micros/op; (each op is 1000 loads) New: fillseq : 3.129 micros/op; 35.4 MB/s fillsync : 2748.099 micros/op; 0.0 MB/s (1000 ops) fillrandom : 5.394 micros/op; 20.5 MB/s overwrite : 7.253 micros/op; 15.3 MB/s readrandom : 5.655 micros/op; (1000000 of 1000000 found) readrandom : 4.425 micros/op; (1000000 of 1000000 found) readseq : 0.298 micros/op; 371.3 MB/s readreverse : 0.508 micros/op; 217.9 MB/s compact : 885842.000 micros/op; readrandom : 3.545 micros/op; (1000000 of 1000000 found) readseq : 0.252 micros/op; 438.2 MB/s readreverse : 0.425 micros/op; 260.2 MB/s fill100K : 1418.347 micros/op; 67.2 MB/s (1000 ops) crc32c : 1.987 micros/op; 1966.0 MB/s (4K per op) snappycomp : 4.767 micros/op; 819.4 MB/s (output: 55.1%) snappyuncomp : 0.916 micros/op; 4264.9 MB/s acquireload : 0.665 micros/op; (each op is 1000 loads) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=194002392
* | Replace NULL with nullptr in C++ files.costan2018-04-106-30/+30
| | | | | | | | | | | | ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=192365747
* | leveldb: Require C++11.costan2018-03-219-86/+93
| | | | | | | | | | | | | | | | | | | | This CL switches the public headers to C++11 default and deleted constructors, and adds override to the relevant leveldb::EnvWrapper methods. This should be a good test for C++11 compiler support. Once this CL settles, the rest of the codebase can be safely modernized to C++11. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=189873212
* | Add CMake build support.costan2018-03-161-3/+3
| | | | | | | | | | | | | | | | Fixes https://github.com/google/leveldb/issues/466 ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=189425354
* | Remove extern from function declarations.costan2018-03-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | External linkage is the default for function declarations in C++. This also fixes ClangTidy errors generated by removing the "extern" keyword as described above. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=188730416
* | Expose Env::GetTempDirectory() for use in C test.costan2018-03-091-0/+3
| | | | | | | | | | | | | | | | This removes the use of the non-portable headers <sys/types.h> and <unistd.h> in c_test.c. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=188503102
* | Add leveldb_options_set_max_file_size to the C API.cmumford2017-11-031-0/+2
| | | | | | | | | | | | | | | | | | | | | | When the max file size option was added in CL 134391640 the C API was not modified to support this. This change was contributed by GitHub user @olt and fixes issue #439. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=173466388
* | leveldb::DestroyDB will now delete empty directories.cmumford2017-11-031-0/+3
| | | | | | | | | | | | | | | | | | | | Env's that filtered out dot files ("." and "..") would return an empty vector of children causing DestroyDB to do nothing. This fixes https://github.com/google/leveldb/issues/215 ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=172501335
* | Add LEVELDB_EXPORT macro to export public symbols.cmumford2017-10-0415-183/+206
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | gcc defaults to exporting all symbols, but other linkers do not. Adding the LEVELDB_EXPORT macro allows a project to set LEVELDB_SHARED_LIBRARY when building/linking with leveldb as a shared library. This is to allow leveldb to be created as a shared library on all platforms support by Chrome and enables a fix for https://bugs.chromium.org/p/chromium/issues/detail?id=764810. This also has the benefit of reducing the shared library size from 418863 to 380367 bytes (64-bit Linux). ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=171037148
* | leveldb: Report missing CURRENT manifest file as database corruption.costan2017-07-101-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | BTRFS reorders rename and write operations, so it is possible that a filesystem crash and recovery results in a situation where the file pointed to by CURRENT does not exist. DB::Open currently reports an I/O error in this case. Reporting database corruption is a better hint to the caller, which can attempt to recover the database or erase it and start over. This issue is not merely theoretical. It was reported as having showed up in the wild at https://github.com/google/leveldb/issues/195 and at https://crbug.com/738961. Also, asides from the BTRFS case described above, incorrect data in CURRENT seems like a possible corruption case that should be handled gracefully. The Env API changes here can be considered backwards compatible, because an implementation that returns Status::IOError instead of Status::NotFound will still get the same functionality as before. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=161432630
* | LevelDB: Add WriteBatch::ApproximateSize().costan2017-07-101-0/+6
|/ | | | | | | | This can be used to report metrics on LevelDB usage. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=156934930
* Increase leveldb version to 1.20.v1.20costan2017-03-011-1/+1
| | | | | | ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=148937577
* Add option for max file size. The currend hard-coded value of 2M is ↵corrado2016-09-281-0/+12
| | | | | | | | inefficient in colossus. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=134391640
* Increase leveldb version to 1.19.v1.19cmumford2016-08-111-1/+1
| | | | | | ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=129930720
* Add "approximate-memory-usage" property to leveldb::DB::GetPropertyssid2015-12-092-0/+6
| | | | | | | | | The approximate RAM usage of the database is calculated from the memory allocated for write buffers and the block cache. This is to give an estimate of memory usage to leveldb clients. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=104222307
* Add leveldb::Cache::Prunetzik2015-12-091-0/+7
| | | | | | | | Prune() drops on-memory read cache of the database, so that the client can relief its memory shortage. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=101335710
* Added leveldb::Status::IsInvalidArgument() method.cmumford2015-12-091-0/+3
| | | | | | | | All other Status::Code enum values have an Is**() method with the one exception of InvalidArgument. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=97166441
* LevelDB now attempts to reuse the preceding MANIFEST and log file when ↵Sanjay Ghemawat2015-08-113-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | re-opened. (Based on a suggestion by cmumford.) "open" benchmark on my workstation speeds up significantly since we can now avoid three fdatasync calls and a compaction per open: Before: ~80000 microseconds After: ~130 microseconds Details: (1) Added Options::reuse_logs (currently defaults to false) to control new behavior. The intention is to change the default to true after some baking. (2) Added Env::NewAppendableFile() whose default implementation returns a not-supported error. (3) VersionSet::Recovery attempts to reuse the MANIFEST from which it is recovering. (4) DBImpl recovery attempts to reuse the last log file and memtable. (5) db_test.cc now tests a new configuration that sets reuse_logs to true. (6) fault_injection_test also tests a reuse_logs==true config. (7) Added a new recovery_test.
* Fixed incorrect comment wording for Iterator::Seek.Chris Mumford2014-12-111-1/+1
|
* Release 1.18v1.18Chris Mumford2014-09-166-5/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
* Release LevelDB 1.17v1.171.17Chris Mumford2014-05-012-2/+2
| | | | | | | | | | | | | | | | | - Cleanup: delete unused IntSetToString It was added in http://cr/19491949 (and was referenced at the time). The last reference was removed in http://cr/19507363. This fixes warning/error with pre-release crosstoolv18: 'std::string leveldb::{anonymous}::IntSetToString(const std::set<long unsigned int>&)' defined but not used [-Werror=unused-function] - Added arm64 and and armv7s to IOS build as suggested on leveldb mailing list. - Changed local variable type from int to size_t This eliminates compiler warning/error and resolves https://code.google.com/p/leveldb/issues/detail?id=140
* Release LevelDB 1.16v1.16David Grogan2014-02-102-2/+1
| | | | | | | - Make Log::Reader not report a corruption when the last record in a log file is truncated. - Fix issue 224: variable created but not utilized. - Remove comment that referenced a removed feature.
* Release LevelDB 1.15v1.15David Grogan2013-12-102-2/+2
| | | | | | | | | | | | | | | | | | | - switched from mmap based writing to simpler stdio based writing. Has a minor impact (0.5 microseconds) on microbenchmarks for asynchronous writes. Synchronous writes speed up from 30ms to 10ms on linux/ext4. Should be much more reliable on diverse platforms. - compaction errors now immediately put the database into a read-only mode (until it is re-opened). As a downside, a disk going out of space and then space being created will require a re-open to recover from, whereas previously that would happen automatically. On the plus side, many corruption possibilities go away. - force the DB to enter an error-state so that all future writes fail when a synchronous log write succeeds but the sync fails. - repair now regenerates sstables that exhibit problems - fix issue 218 - Use native memory barriers on OSX - fix issue 212 - QNX build is broken - fix build on iOS with xcode 5 - make tests compile and pass on windows
* Release LevelDB 1.14v1.14David Grogan2013-09-191-1/+1
| | | | | | | | | | | Fix issues 200, 201 Also, * Fix link to bigtable paper in docs. * New sstables will have the file extension .ldb. .sst files will continue to be recognized. * When building for iOS, use xcrun to execute the compiler. This may affect issue 177.
View Lorry Controller Status