summaryrefslogtreecommitdiff
path: root/port
Commit message (Collapse)AuthorAgeFilesLines
* Support Zstd compression level in Leveldbleveldb Team2023-04-202-2/+8
| | | | PiperOrigin-RevId: 520556840
* Add support for Zstd-based compression in LevelDB.leveldb Team2023-03-283-2/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change implements support for Zstd-based compression in LevelDB. Building up from the Snappy compression (which has been supported since inception), this change adds Zstd as an alternate compression algorithm. We are implementing this to provide alternative options for users who might have different performance and efficiency requirements. For instance, the Zstandard website (https://facebook.github.io/zstd/) claims that the Zstd algorithm can achieve around 30% higher compression ratios than Snappy, with relatively smaller (~10%) slowdowns in de/compression speeds. Benchmarking results: $ blaze-bin/third_party/leveldb/db_bench LevelDB: version 1.23 Date: Thu Feb 2 18:50:06 2023 CPU: 56 * Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz CPUCache: 35840 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillseq : 2.613 micros/op; 42.3 MB/s fillsync : 3924.432 micros/op; 0.0 MB/s (1000 ops) fillrandom : 3.609 micros/op; 30.7 MB/s overwrite : 4.508 micros/op; 24.5 MB/s readrandom : 6.136 micros/op; (864322 of 1000000 found) readrandom : 5.446 micros/op; (864083 of 1000000 found) readseq : 0.180 micros/op; 613.3 MB/s readreverse : 0.321 micros/op; 344.7 MB/s compact : 827043.000 micros/op; readrandom : 4.603 micros/op; (864105 of 1000000 found) readseq : 0.169 micros/op; 656.3 MB/s readreverse : 0.315 micros/op; 350.8 MB/s fill100K : 854.009 micros/op; 111.7 MB/s (1000 ops) crc32c : 1.227 micros/op; 3184.0 MB/s (4K per op) snappycomp : 3.610 micros/op; 1081.9 MB/s (output: 55.2%) snappyuncomp : 0.691 micros/op; 5656.3 MB/s zstdcomp : 15.731 micros/op; 248.3 MB/s (output: 44.1%) zstduncomp : 4.218 micros/op; 926.2 MB/s PiperOrigin-RevId: 509957778
* fix typo in port_example.hJayice2021-03-281-1/+1
|
* Remove leveldb::port::kLittleEndian.Victor Costan2020-04-143-12/+0
| | | | | | | | | Clang 10 includes the optimizations described in https://bugs.llvm.org/show_bug.cgi?id=41761. This means that the platform-independent implementations of {Decode,Encode}Fixed{32,64}() compile to one instruction on the most recent Clang and GCC. PiperOrigin-RevId: 306330166
* Merge pull request #624 from adam-azarchs:masterChris Mumford2019-05-061-0/+5
|\ | | | | | | PiperOrigin-RevId: 246903086
| * Add O_CLOEXEC to open calls.Adam Azarchs2019-02-221-0/+5
| | | | | | | | | | | | | | | | | | | | | | This prevents file descriptors from leaking to child processes. When compiled for older (pre-2.6.23) kernels which lack support for O_CLOEXEC there is no change in behavior. With newer kernels, child processes will no longer inherit leveldb's file handles, which reduces the changes of accidentally corrupting the database. Fixes https://github.com/google/leveldb/issues/623
* | Moved port/README to port/README.md.Chris Mumford2019-05-021-0/+0
| | | | | | | | | | | | Easier to read on sites supporting Markdown (i.e. GitHub). PiperOrigin-RevId: 246385089
* | Format all files IAW the Google C++ Style Guide.Chris Mumford2019-05-023-18/+25
| | | | | | | | | | | | Use clang-format to correct formatting to be in agreement with the [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html). Doing this simplifies the process of accepting changes. Also fixed a few warnings flagged by clang-tidy. PiperOrigin-RevId: 246350737
* | leveldb: Minor cleanup in ports.costan2019-03-291-3/+4
| | | | | | | | | | | | ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=240619768
* | leveldb: Silence unused argument warnings in MSVC.costan2019-03-291-0/+11
| | | | | | | | | | | | | | | | This CL uses a well-known workaround for silencing arguments that may be unused, depending on the build configuration. The silenced warnings were responsible for a large amount of noise in the MSVC build on Windows. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=240357359
* | leveldb: Replace AtomicPointer with std::atomic.costan2019-03-113-201/+0
| | | | | | | | | | | | | | | | | | | | | | | | This CL removes AtomicPointer from leveldb's port interface. Its usage is replaced with std::atomic<> from the C++11 standard library. AtomicPointer was used to wrap flags, numbers, and pointers, so its instances are replaced with std::atomic<bool>, std::atomic<int>, std::atomic<size_t> and std::atomic<Node*>. This CL does not revise the memory ordering. AtomicPointer's methods are replaced mechanically with their std::atomic equivalents, even when the underlying usage is incorrect. (Example: DBImpl::has_imm_ is written using release stores, even though it is always read using relaxed ordering.) Revising the memory ordering is left for future CLs. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=237865146
* | leveldb: Remove unused file port/win/stdint.h.costan2019-03-111-24/+0
| | | | | | | | | | | | ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=237832823
* | Added native support for Windows.cmumford2019-03-012-5/+1
|/ | | | | | | | | | | | | | | | | | | | This change adds a native Windows port (port_windows.h) and a Windows Env (WindowsEnv). Note1: "small" is defined when including <Windows.h> so some parameters were renamed to avoid conflict. Note2: leveldb::Env defines the method: "DeleteFile" which is also a constant defined when including <Windows.h>. The solution was to ensure this macro is defined in env.h which forces the function, when compiled, to be either DeleteFileA or DeleteFileW when building for MBCS or UNICODE respectively. This resolves #519 on GitHub. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=236364778
* leveldb: Fix PosixWritableFile::Sync() on Apple systems.costan2019-01-091-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Apple doesn't follow POSIX specifications for fsync(). Instead, fsync() guarantees to flush the buffer cache to the device, which means the data will survive kernel panics, but may not survive power outages. Applications that need stronger guarantees (like databases) need to use fcntl(F_FULLFSYNC). This CL switches PosixWritableFile::Sync() to get the stronger guarantees on Apple systems. The improved implementation follows the same principles as SQLite [1] and node.js [2]. Research for the fcntl() to fsync() fallback strategy: Apple's released source code at https://opensource.apple.com/ shows at least three different error codes being returned when a filesystem does not support F_FULLFSYNC. fcntl() is implemented in xnu-4903.221.2 in bsd/kern/kern_descrip.c, where it delegates to fcntl_nocancel(). The documentation for fcntl_nocancel() mentions error codes for some operations, but does not include F_FULLFSYNC. The F_FULLSYNC branch in fcntl_nocancel() calls VNOP_IOCTL(_, F_FULLSYNC, NULL, 0, _), whose return value sets the error code. VNOP_IOCTL() is implemented in bsd/vfs/kpi_vfs.c and calls the ioctl function in the vnode's operation vector. The per-filesystem function names follow the pattern _vnop_ioctl() for all the instances in opensource code: {hfs,msdosfs,nfs,ntfs,smbfs,webdav,zfs}_vnop_ioctl(). hfs-407.30.1, msdosfs-229.200.3, and nfs in xnu-4903.221.2 handle F_FULLFSYNC. ntfs-94.200.1 and smb-759.40.1 do not handle F_FULLFSYNC, and the default branch returns ENOSUP. webdav-380.200.1 also does not handle F_FULLFSYNC, but the default branch returns EINVAL. zfs-59 also does not handle F_FULLSYNC, and its default branch returns ENOTTY. From a different angle, Apple's ntfs-94.200.1 includes utility code that uses fcntl(F_FULLFSYNC) and falls back to fsync() just like we do, supporting the hypothesis that there is no good way to detect lack of F_FULLFSYNC support. Also, Apple's fcntl() man page [3] does not mention a way to detect lack of F_FULLFSYNC support. [1] https://www.sqlite.org/src/doc/trunk/src/os_unix.c [2] https://github.com/libuv/libuv/blob/master/src/unix/fs.c [3] https://developer.apple.com/library/archive/documentatiVon/System/Conceptual/ManPages_iPhoneOS/man2/fcntl.2.html Tested: https://travis-ci.org/pwnall/leveldb/builds/477318498 TAP global presubmit ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=228593729
* Fix fdatasync() feature detection in opensource build.costan2019-01-081-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | The CMake feature-detection code used check_symbol_exists(), which invokes the C compiler. However, some glibc versions don't expose the fdatasync() declaration when compiled with -std=c11, but do expose it when compiled with -std=c++11. This most likely comes down to how _POSIX_SOURCE is defined -- it needs to be >= 201112L for <unistd.h> to expose fdatasync(). This CL switches to check_cxx_symbol_exists(), which uses the C++ compiler. Asides from fixing the problem above, this is the right thing to do, because we use <unistd.h> in env_posix.cc, which is compiled with the C++ compiler. This CL also fixes a previously introduced inconsistency, where the macro indicating the fdatasync() feature detection result was referred to as HAVE_FDATASYNC and HAVE_FUNC_FDATASYNC. The former appears to be used in other libraries, so this CL switches all our references to HAVE_FDATASYNC. Fixes https://github.com/google/leveldb/issues/629 ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=228392612
* Remove InitOnce from the port API.costan2018-09-102-18/+0
| | | | | | | | | | | | | | | | | | | | | | | | This is not an API-breaking change, because it reduces the API that the leveldb embedder must implement. The project will build just fine against ports that still implement InitOnce. C++11 guarantees thread-safe initialization of static variables inside functions. This is a more restricted form of std::call_once or pthread_once_t (e.g., single call site), so the compiler might be able to generate better code [1]. Equally important, having less code in port_example.h makes it easier to port to other platforms. Due to the change above, this CL introduces a new approach for storing the singleton BytewiseComparatorImpl instance returned by BytewiseComparator(). The new approach avoids a dynamic memory allocation, which eliminates the false positive from LeakSanitizer reported in https://github.com/google/leveldb/issues/200 [1] https://stackoverflow.com/a/27206650/ ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=212348004
* Replace port_posix with port_stdcxx.costan2018-04-174-87/+44
| | | | | | | | | | | | | | | The porting layer implements threading primitives: atomic pointers, condition variables, mutexes, thread-safe initialization. These are all specified in C++11, so the reference open source port implementation can become platform-independent. The porting layer will remain in place to allow the use of other implementations with more features, such as the built-in deadlock detection in abseil's Mutex. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=193245934
* Replace NULL with nullptr in C++ files.costan2018-04-101-2/+2
| | | | | | ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=192365747
* Remove PLATFORM_IS_LITTLE_ENDIAN from port/posix.h.costan2018-04-101-4/+0
| | | | | | | | | This is an accidental leftover from the CMake migration. The macro has been replaced with LEVELDB_IS_BIG_ENDIAN. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=192364918
* Take <atomic> for granted in port/atomic_pointer.h.costan2018-03-212-82/+5
| | | | | | | | | C++11 requires <atomic>. This lets us remove the header detection (LEVELDB_ATOMIC_PRESENT) and simplify port/atomic_pointer.h. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=189919098
* Add CMake build support.costan2018-03-162-42/+55
| | | | | | | | Fixes https://github.com/google/leveldb/issues/466 ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=189425354
* Remove extern from function declarations.costan2018-03-122-9/+9
| | | | | | | | | | | External linkage is the default for function declarations in C++. This also fixes ClangTidy errors generated by removing the "extern" keyword as described above. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=188730416
* Bypass OSMemoryBarrier() warning on Mac.costan2018-03-091-3/+6
| | | | | | | | | | | This is a stopgap for removing warnings on Mac builds, so -Werror can be turned on. C++11 will be required in the nearby future, which guarantees <atomic> support. Once that happens, the simplified version of this will match https://github.com/google/leveldb/pull/503 ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=188553251
* Switch HAVE_ library detection macros to 0/1.costan2018-03-091-12/+12
| | | | | | ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=188488298
* Enable thread safety annotations in open source version.costan2018-02-133-23/+77
| | | | | | | | The thread safety annotations used by leveldb got opensourced in Abseil [1]. This CL replaces leveldb's stubs with the relevant definitions from [1], and adds annotations to the Mutex classes in the POSIX port. [1] https://github.com/abseil/abseil-cpp/blob/master/absl/base/thread_annotations.h
* Replace SSE-optimized CRC32C in POSIX port with external library.costan2017-10-102-136/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Maintaining a hardware-accelerated CRC32C implementation tailored for all modern platforms deserves a repository of its own. We extracted the implementation here into https://github.com/google/crc32c and improved it in that repository. This CL removes the SSE-optimized implementation from this codebase, and adds the ability to use the google/crc32c library, if it is present on the system. The benchmarks below show the performance impact of the change. In summary, open source builds that use the google/crc32c library can expect a 3x improvement in CRC32C throughput, whereas builds that do not use the library will see a 50% drop in CRC32C throughput. This translates in much smaller changes in overall leveldb performance. Baseline, MacBookPro13,3 with Core i7 6920HQ: LevelDB: version 1.20 Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillseq : 3.064 micros/op; 36.1 MB/s fillsync : 57.861 micros/op; 1.9 MB/s (1000 ops) fillrandom : 3.887 micros/op; 28.5 MB/s overwrite : 4.140 micros/op; 26.7 MB/s readrandom : 7.433 micros/op; (1000000 of 1000000 found) readrandom : 6.825 micros/op; (1000000 of 1000000 found) readseq : 0.244 micros/op; 453.4 MB/s readreverse : 0.387 micros/op; 285.8 MB/s compact : 449707.000 micros/op; readrandom : 4.196 micros/op; (1000000 of 1000000 found) readseq : 0.228 micros/op; 485.8 MB/s readreverse : 0.320 micros/op; 345.2 MB/s fill100K : 562.556 micros/op; 169.6 MB/s (1000 ops) crc32c : 0.768 micros/op; 5085.0 MB/s (4K per op) snappycomp : 4.220 micros/op; 925.7 MB/s (output: 55.1%) snappyuncomp : 0.635 micros/op; 6155.7 MB/s acquireload : 13.054 micros/op; (each op is 1000 loads) New with crc32c, MacBookPro13,3 with Core i7 6920HQ: LevelDB: version 1.20 Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillseq : 2.820 micros/op; 39.2 MB/s fillsync : 51.988 micros/op; 2.1 MB/s (1000 ops) fillrandom : 3.747 micros/op; 29.5 MB/s overwrite : 4.047 micros/op; 27.3 MB/s readrandom : 7.287 micros/op; (1000000 of 1000000 found) readrandom : 6.927 micros/op; (1000000 of 1000000 found) readseq : 0.253 micros/op; 437.5 MB/s readreverse : 0.411 micros/op; 269.2 MB/s compact : 440405.000 micros/op; readrandom : 4.159 micros/op; (1000000 of 1000000 found) readseq : 0.230 micros/op; 481.1 MB/s readreverse : 0.320 micros/op; 345.9 MB/s fill100K : 558.222 micros/op; 170.9 MB/s (1000 ops) crc32c : 0.214 micros/op; 18263.5 MB/s (4K per op) snappycomp : 4.471 micros/op; 873.7 MB/s (output: 55.1%) snappyuncomp : 0.833 micros/op; 4688.5 MB/s acquireload : 13.289 micros/op; (each op is 1000 loads) New without crc32c, MacBookPro13,3 with Core i7 6920HQ LevelDB: version 1.20 Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) ------------------------------------------------ fillseq : 3.094 micros/op; 35.8 MB/s fillsync : 52.160 micros/op; 2.1 MB/s (1000 ops) fillrandom : 4.090 micros/op; 27.0 MB/s overwrite : 4.006 micros/op; 27.6 MB/s readrandom : 6.584 micros/op; (1000000 of 1000000 found) readrandom : 6.676 micros/op; (1000000 of 1000000 found) readseq : 0.280 micros/op; 395.2 MB/s readreverse : 0.391 micros/op; 283.2 MB/s compact : 433911.000 micros/op; readrandom : 4.261 micros/op; (1000000 of 1000000 found) readseq : 0.251 micros/op; 440.5 MB/s readreverse : 0.356 micros/op; 310.9 MB/s fill100K : 584.023 micros/op; 163.3 MB/s (1000 ops) crc32c : 1.384 micros/op; 2822.3 MB/s (4K per op) snappycomp : 4.763 micros/op; 820.1 MB/s (output: 55.1%) snappyuncomp : 0.766 micros/op; 5098.6 MB/s acquireload : 12.931 micros/op; (each op is 1000 loads) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=171667771
* leveldb: Rename SNAPPY to HAVE_SNAPPY.costan2017-10-051-8/+8
| | | | | | | | | This follows the general naming convention for preprocessor macros used to detect feature (library / header file / symbol) presence. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=171184641
* leveldb: Remove *_unlocked feature detection from POSIX port.costan2017-10-051-9/+0
| | | | | | | | | | CL 170738066 removed all instances of fread_unlocked, fwrite_unlocked and fflush_unlocked calls from the codebase, so the feature detection can be removed as well. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=171154269
* leveldb: Fix alignment code in SSE4.2-optimized CRC32C.costan2017-08-241-2/+6
| | | | | | | | | | | | When faced with a pointer that is misaligned by K bytes (pointer % 8 == K), the code previously moved forward by K bytes. In order to end up with an aligned pointer, the code must move by 8 - K bytes. This lands https://github.com/google/leveldb/pull/488 ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=166295921
* Use __APPLE__ instead of OS_MACOS. The former is compiler-provided.scrubbed2017-08-242-5/+5
| | | | | | | | | | Use __APPLE__ instead of OS_MACOS when testing for the Apple platform and remove the latter symbol from the BUILD file. This fixes incompatibility issues when using the library on an Apple device. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=162958094
* leveldb: Fix compilation warnings in port_posix_sse.cc on x86 (32-bit).costan2017-03-011-0/+4
| | | | | | | | LE_LOAD64 is only used when _mm_crc32_u64 is available, on 64-bit x86 processors. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=148906169
* Implement support for Intel crc32 instruction (SSE 4.2)costan2017-02-283-0/+133
| | | | | | | | | | | | | | | | | | | | | This change authored by vadimskipin and submitted via: https://github.com/google/leveldb/pull/309 Changes made to support iOS builds and other architectures without support for SSE 4.2. db_bench reports original crc32 speed at: crc32c : 3.610 micros/op; 1082.0 MB/s (4K per op) with this change performance has increased to: crc32c : 0.843 micros/op; 4633.6 MB/s (4K per op) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=148694935
* Merge pull request #272 from vapier/masterChris Mumford2016-01-121-0/+9
|\ | | | | Fix Android/MIPS build.
| * Fix Android/MIPS build.David Turner2014-12-171-0/+9
| | | | | | | | | | port/atomic_pointer.h was missing an implementation for MemoryBarrier() for this platform.
* | Including atomic_pointer.h in port_posixcmumford2015-12-091-1/+0
|/ | | | | | | | | | | | | | | | A recent CL (104348226) created the port_posix library, but omitted: port/atomic_pointer.h. And when: [] test third_party/leveldb:all was run this error was reported: //third_party/leveldb:port_posix does not depend on a module exporting 'third_party/leveldb/port/atomic_pointer.h' ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=105243399
* Add arm64 support to leveldb.Chris Mumford2014-12-111-0/+10
|
* Release 1.18v1.18Chris Mumford2014-09-163-19/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changes are: * Update version number to 1.18 * Replace the basic fprintf call with a call to fwrite in order to work around the apparent compiler optimization/rewrite failure that we are seeing with the new toolchain/iOS SDKs provided with Xcode6 and iOS8. * Fix ALL the header guards. * Createed a README.md with the LevelDB project description. * A new CONTRIBUTING file. * Don't implicitly convert uint64_t to size_t or int. Either preserve it as uint64_t, or explicitly cast. This fixes MSVC warnings about possible value truncation when compiling this code in Chromium. * Added a DumpFile() library function that encapsulates the guts of the "leveldbutil dump" command. This will allow clients to dump data to their log files instead of stdout. It will also allow clients to supply their own environment. * leveldb: Remove unused function 'ConsumeChar'. * leveldbutil: Remove unused member variables from WriteBatchItemPrinter. * OpenBSD, NetBSD and DragonflyBSD have _LITTLE_ENDIAN, so define PLATFORM_IS_LITTLE_ENDIAN like on FreeBSD. This fixes: * issue #143 * issue #198 * issue #249 * Switch from <cstdatomic> to <atomic>. The former never made it into the standard and doesn't exist in modern gcc versions at all. The later contains everything that leveldb was using from the former. This problem was noticed when porting to Portable Native Client where no memory barrier is defined. The fact that <cstdatomic> is missing normally goes unnoticed since memory barriers are defined for most architectures. * Make Hash() treat its input as unsigned. Before this change LevelDB files from platforms with different signedness of char were not compatible. This change fixes: issue #243 * Verify checksums of index/meta/filter blocks when paranoid_checks set. * Invoke all tools for iOS with xcrun. (This was causing problems with the new XCode 5.1.1 image on pulse.) * include <sys/stat.h> only once, and fix the following linter warning: "Found C system header after C++ system header" * When encountering a corrupted table file, return Status::Corruption instead of Status::InvalidArgument. * Support cygwin as build platform, patch is from https://code.google.com/p/leveldb/issues/detail?id=188 * Fix typo, merge patch from https://code.google.com/p/leveldb/issues/detail?id=159 * Fix typos and comments, and address the following two issues: * issue #166 * issue #241 * Add missing db synchronize after "fillseq" in the benchmark. * Removed unused variable in SeekRandom: value (issue #201)
* Release LevelDB 1.15v1.15David Grogan2013-12-101-7/+7
| | | | | | | | | | | | | | | | | | | - switched from mmap based writing to simpler stdio based writing. Has a minor impact (0.5 microseconds) on microbenchmarks for asynchronous writes. Synchronous writes speed up from 30ms to 10ms on linux/ext4. Should be much more reliable on diverse platforms. - compaction errors now immediately put the database into a read-only mode (until it is re-opened). As a downside, a disk going out of space and then space being created will require a re-open to recover from, whereas previously that would happen automatically. On the plus side, many corruption possibilities go away. - force the DB to enter an error-state so that all future writes fail when a synchronous log write succeeds but the sync fails. - repair now regenerates sstables that exhibit problems - fix issue 218 - Use native memory barriers on OSX - fix issue 212 - QNX build is broken - fix build on iOS with xcode 5 - make tests compile and pass on windows
* added utility to dump leveldb filesv1.8Sanjay Ghemawat2012-12-271-1/+7
|
* Update to leveldb 1.6v1.6David Grogan2012-10-123-2/+140
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Highlights ---------- Mmap at most 1000 files on Posix to improve performance for large databases. Support for more architectures (thanks to Alexander K.) Building and porting -------------------- HP/UX support (issue 126) AtomicPointer for ia64 (issue 123) Sparc v9 support (issue 124) Atomic ops for powerpc Use -fno-builtin-memcmp only when using g++ Simplify IOS build rules (issue 114) Use CXXFLAGS instead of CFLAGS when invoking C++ compiler (issue 118) Fix snappy shared library problem (issue 94) Fix shared library installation path regression Endian-ness detection tweak for FreeBSD Bug fixes --------- Stop ignoring FLAGS_open_files in db_bench Make bloom test behavior agnostic to endian-ness Performance ----------- Limit number of mmapped files to 1000 to improve perf for large dbs Do not delay for 1 second on shutdown path (issue 125) Misc ---- Make InMemoryEnv return a no-op logger C binding now has a wrapper for free (issue 117) Add thread-safety annotations Added an in-process lock table (issue 120) Make RandomAccessFile and SequentialFile non-copyable
* remove obsolete android port filesSanjay Ghemawat2012-05-302-223/+0
|
* Remove static initializer; fix endian-ness detection; fix build onSanjay Ghemawat2012-05-305-16/+51
| | | | | | | | | | | | | | | | | | | | | | various platforms; improve android port speed. Avoid static initializer by using a new portability interface for thread-safe lazy initialization. Custom ports will need to be extended to implement InitOnce/OnceType/LEVELDB_ONCE_INIT. Fix endian-ness detection (fixes Powerpc builds). Build related fixes: - Support platforms that have unversioned shared libraries. - Fix IOS build rules. Android improvements - Speed up atomic pointers - Share more code with port_posix. Do not spin in a tight loop attempting compactions if the file system is inaccessible (e.g., if kerberos tickets have expired or if it is out of space).
* Added bloom filter support.v1.4Sanjay Ghemawat2012-04-171-0/+3
| | | | | | | | | | | | | | | | | | | | In particular, we add a new FilterPolicy class. An instance of this class can be supplied in Options when opening a database. If supplied, the instance is used to generate summaries of keys (e.g., a bloom filter) which are placed in sstables. These summaries are consulted by DB::Get() so we can avoid reading sstable blocks that are guaranteed to not contain the key we are looking for. This change provides one implementation of FilterPolicy based on bloom filters. Other changes: - Updated version number to 1.4. - Some build tweaks. - C binding for CompactRange. - A few more benchmarks: deleteseq, deleterandom, readmissing, seekrandom. - Minor .gitignore update.
* Build fixes and cleanups:Sanjay Ghemawat2012-03-211-1/+1
| | | | | | (1) Separate out C++ and CC flags (fixes c_test compilation) (2) Move snappy/perftools detection to script (3) Fix db_bench_sqlite3 and db_bench_tree_db build rules
* add .gitignore; support for building on a few BSD variantsSanjay Ghemawat2012-03-051-3/+11
|
* A number of fixes:Hans Wennborg2011-10-315-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Replace raw slice comparison with a call to user comparator. Added test for custom comparators. - Fix end of namespace comments. - Fixed bug in picking inputs for a level-0 compaction. When finding overlapping files, the covered range may expand as files are added to the input set. We now correctly expand the range when this happens instead of continuing to use the old range. For example, suppose L0 contains files with the following ranges: F1: a .. d F2: c .. g F3: f .. j and the initial compaction target is F3. We used to search for range f..j which yielded {F2,F3}. However we now expand the range as soon as another file is added. In this case, when F2 is added, we expand the range to c..j and restart the search. That picks up file F1 as well. This change fixes a bug related to deleted keys showing up incorrectly after a compaction as described in Issue 44. (Sync with upstream @25072954)
* Sync with upstream @23860137.Hans Wennborg2011-09-123-362/+0
| | | | | | | | | | | | | Fix GCC -Wshadow warnings in LevelDB's public header files, reported by Dustin. Add in-memory Env implementation (helpers/memenv/*). This enables users to create LevelDB databases in-memory. Initialize ShardedLRUCache::last_id_ to zero. This fixes a Valgrind warning. (Also delete port/sha1_* which were removed upstream some time ago.)
* Adding FreeBSD support, removing Chromium files, adding benchmark.gabor@google.com2011-07-273-186/+3
| | | | | | | | | | | | | | | | | | | - LevelDB patch for FreeBSD. This resolves Issue 22. Contributed by dforsythe (thanks!). - Removing Chromium-specific files. They are now going to live in the Chromium repository. - Adding a benchmark page comparing LevelDB performance to SQLite and Kyoto Cabinet's TreeDB, along with code to generate the benchmarks. Thanks to Kevin Tseng for compiling the benchmarks, and Scott Hess and Mikio Hirabayashi for their help and advice. git-svn-id: https://leveldb.googlecode.com/svn/trunk@40 62dab493-f737-651d-591e-8d6aee1b9529
* Speed up Snappy uncompression, new Logger interface.gabor@google.com2011-07-216-24/+58
| | | | | | | | | | | | | | | | | | | | - Removed one copy of an uncompressed block contents changing the signature of Snappy_Uncompress() so it uncompresses into a flat array instead of a std::string. Speeds up readrandom ~10%. - Instead of a combination of Env/WritableFile, we now have a Logger interface that can be easily overridden applications that want to supply their own logging. - Separated out the gcc and Sun Studio parts of atomic_pointer.h so we can use 'asm', 'volatile' keywords for Sun Studio. git-svn-id: https://leveldb.googlecode.com/svn/trunk@39 62dab493-f737-651d-591e-8d6aee1b9529
* Sun Studio support, and fix for test related memory fixes.gabor@google.com2011-07-191-2/+3
| | | | | | | | | | | | - LevelDB patch for Sun Studio Based on a patch submitted by Theo Schlossnagle - thanks! This fixes Issue 17. - Fix a couple of test related memory leaks. git-svn-id: https://leveldb.googlecode.com/svn/trunk@38 62dab493-f737-651d-591e-8d6aee1b9529
View Lorry Controller Status