| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
| |
* Verbose messages for lookaside activity are generated once-per-checkpoint.
|
| |
|
|
|
|
|
|
|
| |
loop (#3378)
Traversing lists to close handles could turn into an infinite loop on error, the underlying close functions could return without unlinking the cursor from its linked list.
Add a macro pair, `WT_TAILQ_SAFE_REMOVE_BEGIN/END`, that include a test that if we see the same element twice on a linked list, we remove it so we don't loop forever.
Clean up various loops that remove elements from lists to either use the standard `TAILQ_FOREACH_SAFE`, the new macro or a pattern where the `TAILQ_REMOVE` is explicit in the loop.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* WT-3136 bug fix: WiredTiger doesn't check sprintf calls for error return
Make a pass through the source base to check sprintf, snprintf, vsprintf
and vsnprintf calls for errors.
* A WiredTiger key is a uint64_t.
Use sizeof(), don't hard-wire buffer sizes into the code.
* More (u_int) vs. (uint64_t) fixes.
* Use CONFIG_APPEND instead of FORMAT_APPEND, it makes more sense.
* revert part of 4475ae9, there's an explicit allocation of the size of
the buffer.
* MVSC complaints:
test\format\config.c(765): warning C4018: '<': signed/unsigned mismatch
test\format\config.c(765): warning C4018: '>': signed/unsigned mismatch
* Change Windows testing shim to correctly use __wt_snprintf
* Change Windows test shim to use the __wt_XXX functions
* MSDN's _vscprintf API returns the number of characters excluding the
termininating nul byte, return that value.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
extension (#3261)
This commit represents fixes for Coverity errors, LeakSanitizer errors, and additional cleanup:
* pread/pwrite return value is -1 on error, but the error is in errno.
* Convert size_t and off_t to uintmax_t/PRIuMAX, not uint64_t/PRIu64.
* Coverity ID 1369085 (#1 of 1): Extra sizeof expression (SIZEOF_MISMATCH)
suspicious_pointer_arithmetic: Adding allocated * 8UL /* sizeof (char
*) */ to pointer entries of type char ** is suspicious because adding
an integral value to this pointer automatically scales that value by the
size, 8 bytes, of the pointed-to type, char *. Most likely, the
multiplication by sizeof (char *) in this expression is extraneous and
should be eliminated.
* CID 1369084 (#1 of 1): Resource leak (RESOURCE_LEAK) 9. overwrite_var:
Overwriting handle ret in ret = 12 leaks the handle.
* CID 1369083 (#1 of 1): Logically dead code (DEADCODE) dead_error_line: Execution cannot reach this statement: while (count > 0U) null:
At condition entries != NULL, the value of entries must be NULL. dead_error_condition: The condition entries != NULL cannot be true.
* Custom filesystems have to configure early-load, otherwise we'll have already configured a default filesystem by the time the extension is loaded.
* Add early-load configuration to the wt3120_filesys test.
* Add code to WiredTiger that fails if a custom filesystem is configured after we've already configured a default filesystem.
|
| |
|
| |
loading extensions, custom filesystems (for example) needs to know the database home.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
* Default to an initial 250 hazard slots and grow from there.
* Make hazard_max undocumented, add an internal limit of 1000 eviction walks.
* If we grow the hazard pointer array, schedule the original to be freed when the database is closed.
* Update test_bug011 back to stress eviction with the hard-coded limit of 1000 active trees. Only run during "long" tests.
|
| | |
|
| |
|
| |
While in the area, fix sending "config={values}" to extensions: just the values should be passed in.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
* WT-3018 lint
clang version 3.4.1
random-abort.c:37:6: error: no previous extern declaration for
non-static variable 'inmem' [-Werror,-Wmissing-variable-declarations]
* Clean up WT_UNUSED() macro uses, where we do use the variable.
* Back out part of 6028ca3, the #ifdef'd code has variable declarations
and ISO C90 forbids mixed declarations and code.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix a bug found by inspection in LZ4 code: we're going to offset the destination buffer by sizeof(LZ4_PREFIX), so we need to offset the destination buffer length by the same amount.
* Prettiness pass through the snappy driver so it and the zstd driver look the same, minor cleanup in zlib.
* Add the compression_level configuration option to the zstd extension initialization function so it's possible to set the compression level from an application.
* Fix a bug in zlib raw compression: the destination buffer size (dst_len), can be smaller than the target page size (page_max), use the smaller of the two values to set the target compression size.
* The zlib raw compression function could return without calling deflateEnd on its z_stream structures, potentially leaking memory and scratch buffers.
* If the default reserved bytes for zlib raw compression isn't sufficient, we fail on compression of what might be very large blocks. We don't have information on how many bytes need to be reserved in order to know the final deflate() will succeed, and the default value was experimentally derived, for all we know there are workloads where the default will fail
a lot. Add a fallback before failing hard, try with 2x the default reserved space.
|
| |
|
| |
Exposed via a new 'cache_walk' statistics configuration option.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
high level locks (#3086)
* WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks
Sort the statistics categories so it's easier to find stuff, no real change.
* Add counters and usec wait times to long-term locks (currently the
checkpoint, handle-list, metadata, schema and table locks).
* mutex.i:295:26: error: conversion to int64_t {aka long int} from long
unsigned int may change the sign of the result [-Werror=sign-conversion]
[t->slot_usecs] += WT_TIMEDIFF_US(leave, enter);
* Rename the lock statistics so they group together.
Split lock wait times between internal and application threads.
* Separate the connection's dummy session initialization out into its own
function, that way it's clear what we're doing.
* The session's connection statistics are fixed when the session ID is
allocated, so we can cache a pointer to them and avoid u_int divisions
(which are currently about the slowest thing you can do on a modern
architecture).
* A slightly different change: instead of caching a reference to the
connection statistics, cache the offset into the array of statistics
pointers, that way we can avoid the integer division when updating
almost all statistics.
* Review comments:
Add comments describing the use of statistics array offsets in lock
tracking.
Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET.
Whitespace cleanup.
* __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly,
if the connection open fails at any point, __wt_cache_destroy() will be
called as part of that cleanup.
* Append the suffix "_off" to the spinlock structure statistics field
names, clarify they're offsets into the statistics structures.
|
| |
|
| |
Don't unlock the spin lock unless we've locked it.
|
| |
|
|
|
|
|
| |
Having _FAST_ macros gives an impression that when we use them,
we are collecting fast statistics only, which is not true.
Except when statistics=none is set, we collect all the stats.
This change removes _FAST_ macros and modifies the basic macros
to only collect stats when statistics=none is not set.
|
| |
|
| |
Some functions return an error code even though they don't need to. That adds complexity to our code. Switch to returning a void.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix bugs related to reconfiguring eviction settings.
The code to support reconfiguring eviction worker threads had
several bugs. Implemented an abstracted utility thread group API
and switch eviction workers over to using the new abstraction.
* Remove unused function from new thread group API.
* Commit auto-generated files.
* Be more careful cleaning up on error in thread group code.
* Fix uninitialized variable.
* Ensure thread group structures are cleared after destruction.
This is necessary since the structure is re-used for eviction workers
when recovery is run.
* Remove util and worker as notions from thread group as per review feedback.
Implement other feedback review as well.
* Fix a bug where application threads could attempt to help with eviction
before the server is setup.
Happens if using a shared cache so the cache size starts out at 0.
* Remove _util_ prefix from thread group functions.
* Add session name to thread group. Fix some comments and whitespace.
* Restore error return path.
|
| |
|
|
|
|
|
|
|
|
| |
* The pthread mutex implementation of spinlocks lock/unlock functions didn't check the underlying pthread_mutex functions for failure. Panic if pthreads fails.
* Change condition mutex functions to not return errors.
* Change __wt_verbose() to not return errors.
* Make a final panic check before writing anything.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
API to make durable the default (#2867)
Change the default remove/rename calls to flush the enclosing directory.
Simplify the pluggable file system API by replacing the directory-sync method
with "durable" boolean argument to the remove, rename and open-file methods.
* Add "durable" arguments to relevant functions so that each remove or rename
call specifies its durability requirements.
* Switch the WT_FILE_SYSTEM::fs_open_file type enum from WT_OPEN_FILE_TYPE,
with WT_OPEN_XXX names, to the WT_FS_OPEN_FILE_TYPE, with WT_FS_OPEN_XXX
names.
Switch the WT_FILE_SYSTEM::fs_open_file flags from WT_OPEN_XXX names to
WT_FS_OPEN_XXX names.
* Replace the "bool durable" argument to WT_FILE_SYSTEM.fs_remove and
WT_FILE_SYSTEM.fs_rename with a "uint32_t flags" argument, and the
WT_FS_DURABLE flag.
* Remove a stray bracket.
|
| |
|
|
|
|
|
|
|
|
| |
KNF, remove space after "sizeof" keyword.
Info 790: Suspicious truncation, integral to float
strlen returns a size_t, cast before comparing against a wt_off_t.
size_t is 8B, more size_t cleanups.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Build a Windows-to-POSIX/ANSI error translation layer.
Replace the read-only error mapping to WT_NOTFOUND and WT_PERM_DENIED
with EACCES and ENOENT.
Windows no longer needs its own version of __wt_strerror(), move the POSIX implementation from os_posix/os_errno.c to os_common/os_errno.c. Rename os_win/os_errno.c to os_win/os_winerr.c to avoid a collision.
Windows now has DWORD types in prototypes, split the Windows/POSIX extern.h files. (This actually cleans up some noise, previously we had to sort the OS prototypes to remove duplicates, which wasn't trivial.)
Add the WT_EXTENSION_API.map_windows_error method to map Windows system codes to POSIX/ANSI system codes.
|
| |
|
|
|
| |
gcc 4.7, clang 3.4 and clang 3.8 all complain about different things;
cast the arguments to our new wrapper functions, hopefully that makes
all of the complaints go away.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Cast arguments for ctype functions to avoid sign extension errors.
Create __wt_* versions of all ctype functions, and use them whenever
wt_internal.h is available. Add check to prevent direct use of ctype functions
from core source.
* Change wrappers to use u_char arguments, and return bool.
Remove unused wrappers.
* Use u_char in preference to unsigned char.
* Examples do not have u_char defined on Windows.
|
| |
|
|
|
| |
Remove the need for a hash and name comparison to identify the metadata file, set a flag in the data handle when the file is opened.
Move the code to insert the data handle into the connection list from the find function to the allocate function for clarity.
|
| |
|
|
|
|
|
| |
Add examples of passing in per-device configuration information to the demo file system initialization function.
Use the WiredTiger extension API to handle errors in functions called by WiredTiger.
Set the default session's strerror method. If one of the extensions being loaded reports an error via the WT_EXTENSION_API strerror method, but doesn't supply that method a WT_SESSION handle, we'll use the WT_CONNECTION_IMPL's default session and its strerror method.
|
| |
|
|
|
| |
* WT-2630 Rename pluggable filesystem methods to avoid reserved names.
Use "fstr" consistently as the variable name and prefix for WT_FSTREAM.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* WT-2552 Add public API for pluggable filesystems
Not yet compiling. The main parts of this change should be here,
but it involved extensive parameter re-organization. There are also
a number of layering violations between our existing file system
implementations and the WT_FH, that aren't possible with the new
structure.
There are a number of specific todo comments in the code. One of the main
issues is that the in-memory file system had a special close semantic
that relied on WiredTiger handle tracking. The in-memory file-system should
do it's own tracking of file handles, I've gone part way down that road by
adding a queue for closed handles. Need to also add in live handles, and
manage the queue as appropriate.
I haven't created an example application that uses the new API yet.
* WT-2552 Add public API for pluggable filesystems
I always forget you have to remove the already-built html files when
changing PREDEFINED, add a reminder to the complaint.
* WT-2552 Add public API for pluggable filesystems
You have to remove the .js files, too.
* WT-2552 Add public API for pluggable filesystems
Make dist/s_all run cleanly.
* WT-2552 Add public API for pluggable filesystems
Whitespace.
* WT-2552 Add public API for pluggable filesystems
Make it compile/build/lint.
* WT-2552 Add public API for pluggable filesystems
block_write.c: In function '__wt_block_extend':
block_write.c:130:71: error: missing terminating ' character [-Werror]
* WT-2552 Add public API for pluggable filesystems
os_fs_inmemory.c: In function '__im_file_truncate':
os_fs_inmemory.c:344:10: error: 'session' is used uninitialized in this
function [-Werror=uninitialized]
* WT-2552 Add public API for pluggable filesystems
os_fs.c: In function '__posix_directory_sync':
os_fs.c:92:10: error: 'session' is used uninitialized in this function
[-Werror=uninitialized]
* WT-2552 Add public API for pluggable filesystems
Go back to using bool types in the file-system API, this requires we add
<stdbool.h> to the "standard" wiredtiger.h includes.
Consistently use wt_session to represent a WT_SESSION, we were using
"wtsession" in some places.
Make a pass over the Windows code, but I'm sure it doesn't compile yet.
* WT-2552 Add public API for pluggable filesystems
Fix up another couple of bool types.
* WT-2552 Add public API for pluggable filesystems
Move the file naming work out of the underlying filesystem functions,
the calls to __wt_filename are now in the upper-level code,n os_fs.i;
that means the filesystem code is no longer responsible for figuring out
paths. This is cleaner, although the directory-sync call is a bit of a
kluge, and I've commimtted us to handling NULL filesystem methods.
With this set of changes, in-memory runs again.
More Windows naming fixes.
* WT-2552 Add public API for pluggable filesystems
os_fs.c: In function '__posix_directory_sync':
os_fs.c:96:3: error: label 'err' used but not defined
* WT-2552 Add public API for pluggable filesystems
Pull out another call to __wt_filename() from the filesystem-dependent
code.
* WT-2552 Add public API for pluggable filesystems
Consistently check for missing file-system methods when doing
file-system calls.
Other minor lint & cleanup.
* WT-2552 Add public API for pluggable filesystems
Change the in-memory code to maintain a complete list of the files it
has ever opened, and depend on that list instead of reaching up into the
common layer for the WT_FH handle list.
This means __wt_handle_search is only used by the common WT_FH handle
code, simplify it, and add a __wt_handle_is_open function that can be
called for diagnostic purposes (to check for open files that are being
renamed or removed, for example).
* Fix comiler warning and ignore the file system API in Java
* Flesh out the example file system implementation.
* Add in some plumbing for set_file_system in wiredtiger_open.
* WT-2552 Add public API for pluggable filesystems
Whitespace.
* WT-2552 Add public API for pluggable filesystems
WT_CONFIG_ITEM.val isn't a boolean, don't use boolean types in
equal/not-equal comparisons.
* WT-2552 Add public API for pluggable filesystems
Remove unused #includes.
Increment/decrement the DEMO_FILE_SYSTEM.{opened,closed}_file_count.
Allocate demo structures, they're larger than the underlying structures.
Swap the number/size calloc arguments, number comes first.
Fix a couple of statics.
* WT-2552 Add public API for pluggable filesystems
Use %u instead of casting to %d.
* WT-2552 Add public API for pluggable filesystems
Add ex_file_system.c to the list of example programs.
* WT-2552 Add public API for pluggable filesystems
Change ex_file_system.c to not require <wt_internal.h>: strip down a
copy of FreeBSD's <queue.h> for local inclusion, rewrite a few other
minor pieces of code.
* WT-2552 Add public API for pluggable filesystems
Update spell check info
* WT-2552 Add public API for pluggable filesystems
__conn_load_extensions() shouldn't set the "early" boolean to true.
* WT-2552 Add public API for pluggable filesystems
Don't indirect through a NULL pointer if "local" was set and no path was
specified, always set the name to something useful.
* WT-2552 Add public API for pluggable filesystems
Don't indirect through a NULL pointer if "local" was set and no path was
specified, always set the name to something useful.
* WT-2552 Add public API for pluggable filesystems
wt_off_t vs. size_t conversion lint.
* WT-2552 Add public API for pluggable filesystems
Add -rdynamic to the load for ex_file_system, the main executable
symbols are not exported by default.
* WT-2552 Add public API for pluggable filesystems
The underlying handle name includes the enclosing directory,
compare against the WT_FH.name field instead.
* WT-2552 Add public API for pluggable filesystems
demo_fs_rename should return 0 if successful, simplify error handling
Don't bother casting arguments to free(), it's not necessary.
* WT-2552 Add public API for pluggable filesystems
General WT_FILE_SYSTEM cleanup.
Move OS initialization into the wiredtiger_open() code (the
os_common/os_init.c file is no longer needed).
Allow early-load extensions to be part of the environment settings,
matching the "in-memory" and "readonly" configurations.
Syntax check the set of a file-system, remove tests for NULL methods in
the file-system structure unless it's legal for them to be NULL.
Windows, POSIX and in-memory file systems now set WT_FILE_SYSTEM.terminate,
call that function to cleanup when discarding a WT_CONNECTION.
Export file-type and open-flags constants for WT_FILE_SYSTEM.open_file,
sort the WT_FILE_SYSTEM methods, do an editing pass.
Change the WT_FILE_HANDLE type from (const char *) to (char *), it's
"owned" by the underlying layer, and it's simpler that way.
Minor (untested) cleanup of the Windows WT_FILE_SYSTEM.open-file method.
* WT-2552 Add public API for pluggable filesystems
Export the advise argument #defines for the WT_FILE_HANDLE.fadvise method.
Sort the WT_FILE_HANDLE methods.
* WT-2552 Add public API for pluggable filesystems
Clean up and simplify WT_FILE_SYSTEM/WT_FILE_HANDLE documentation's
description of the handles.
* WT-2552 Add public API for pluggable filesystems
WT_FILE_HANDLE.close is a required function (at the least, it
has to free the memory).
WT_FILE_HANDLE.fadvise isn't a required function, if it's not
configured, don't call it.
* WT-2552 Add public API for pluggable filesystems
The WT_FILE_HANDLE.lock function is required.
Change the __wt_open() signature to match WT_FILE_SYSTEM.open_file().
* WT-2552 Add public API for pluggable filesystems
Rework all of the WT_FILE_HANDLE mapped region methods to be optional.
* WT-2552 Add public API for pluggable filesystems
The WT_FILE_HANDLE.{read,size} methods are required.
The WT_FILE_HANDLE.sync method is not required.
Split the WT_FILE_HANDLE.sync method into .sync and .sync_nowait versions,
it makes the upper-level code simpler (Windows supports .sync but doesn't
support .sync_nowait).
* WT-2552 Add public API for pluggable filesystems
The WT_FILE_HANDLE.{truncate,write} methods are required IFF the file
is not readonly.
* WT-2552 Add public API for pluggable filesystems
POSIX shouldn't declare a no-sync handle function unless the
sync_file_range system call is available.
* WT-2552 Add public API for pluggable filesystems
Typo, missing semi-colon.
* Fix a bug in ex_file_system.c
* Fix a memory leak in posix file handle implementation
* WT-2552 Use the correct flags when opening backup file.
* WT-2552 Add public API for pluggable filesystems
Simplify open-file error handling by calling the close function on the
handle, that way we won't forget to free all of the applicable memory
allocations.
* WT-2552 Add public API for pluggable filesystems
Simplify the directory-list method, don't pass in an include/exclude
file, if prefix is non-NULL, it implies we only want files matching
the prefix.
* WT-2552 Add public API for pluggable filesystems
Replace WT_FILE_HANDLE_POSIX.fallocate_{available,requires_locking} wiht
WT_FILE_HANDLE.fallocate and WT_FILE_HANDLE.fallocate_nolock.
Example code doesn't need to set WT_FILE_HANDLE methods to NULL, the
allocation does that.
Free the I/O buffer if open-handle allocation fails in the example code.
Remove snippets for WT_FILE_SYSTEM and WT_FILE_HANDLE methods, we're
not going to provide example code for them.
* WT-2552 Add public API for pluggable filesystems
Document we expect either ENOTSUP or EBUSY from optionally supported
APIs. Review/cleanups ENOTSUP/EBUSY returns from optionally supported
APIs.
Make WT_FILE_HANDLE.lock optional.
Don't configure or call the POSIX fadvise function on files configured
for direct I/O.
Rename __wt_filesize_name to __wt_size for consistency.
Update the spelling list.
* WT-2552 Add public API for pluggable filesystems
WT_FILE_HANDLE.truncate requires locking in all known implementations,
document it is not called concurrently with other operations.
* WT-2552 Add public API for pluggable filesystems
Don't terminate the filesystem unless we've actually configured one.
* WT-2552 Add public API for pluggable filesystems
Remove WT_FILE_SYSTEM and WT_FILE_HANDLE from SWIG so the test suite
can pass again.
* WT-2552 Add public API for pluggable filesystems
Merge __conn_load_early_extensions() and __conn_load_extensions().
Fix a problem where I moved the early extensions load to where it could
include the WiredTiger environment variable, but I didn't pass the built
cfg into the function.
* WT-2552 Add public API for pluggable filesystems
Linux build typo.
* WT-2552 Add public API for pluggable filesystems
Get rid of the "bool silent" argument to WT_FILE_SYSTEM.size by testing
for the file's existence before requesting the size (an extra system
call, but guaranteed to hit in the buffer cache at least).
* WT-2552 Add public API for pluggable filesystems
Naming consistency pass over the WT_FILE_SYSTEM functions.
* WT-2552 Add public API for pluggable filesystems
Fix a spin lock mismatch.
* WT-2552 Add public API for pluggable filesystems
Another spinlock mismatch.
* Update example pluggable file system.
Add a directory list implementation to the example, which uncovered
an issue with the API. The directory list API allocates memory that
is freed by WiredTiger, which I don't think is kosher.
* Change file-directory-sync to use reguar fsync.
The distinction in os_fs.i doesn't work with the filesystem API.
Also add directory_sync application to the example application.
* WT-2552 Add public API for pluggable filesystems
Whitespace.
* WT-2552 Add public API for pluggable filesystems
Rewrite __wt_free to not evaluate macro arguments multiple times.
* WT-2552 Add public API for pluggable filesystems
Simplify the directory-list functions: __wt_realloc_def() already
handles scaling the size of the allocations, there's no need to
involve a separate constant that increments the allocation size.
* WT-2552 Add public API for pluggable filesystems
Fix a grouping problem in a realloc call, we need to multiple the size
times the previously allocated slots + 10.
Fix buffer overrun, if "count" has already been incremented, the memset
would skip clearing the first slot and clear one slot past the end of
the buffer.
Remove a comment, realloc requires clearing allocated memory, it's not
paranoia.
* WT-2552 Add public API for pluggable filesystems
Add the mapping-cookie argument to the map-preload and map-discard
functions.
Change page-discard to stop reaching down through the block manager,
instead, provide a block-manager map-discard function that does the
work.
* WT-2552 Add public API for pluggable filesystems
Require a directory-list function.
Implement a directory-list function for the in-memory filesystem.
Consistency pass, make all the directory-list functions look the same.
* WT-2552 Add public API for pluggable filesystems
The WT_FILE_SYSTEM.{directory_sync, remove, rename} methods are not
required for read-only systems.
* WT-2552 Add public API for pluggable filesystems
Change the WT_FILE_SYSTEM.open_file file_type argument from a set of
constants to an enum.
This requires changing how we store connection direct I/O configuration
(the constants used to be flags stored in the WT_CONNECTION_IMPL), and
requiring all callers of __wt_open() do their own work to figure out if
WT_OPEN_DIRECTIO should be specified.
* WT-2552 Add public API for pluggable filesystems
Make no guarantees WT_FILE_SYSTEM and WT_FILE_HANDLE methods are
not called concurrently (except for WT_FILE_HANDLE::fallocate and
WT_FILE_HANDLE::fallocate_nolock).
Rewrite the in-memory FS code to lock across all methods (for example,
WT_FILE_HANDLE.close), that means including a reference to the enclosing
WT_FILE_SYSTEM in the WT_FILE_HANDLE structure so we can find a lock
without using the WT_CONNECTION_IMPL structure.
* WT-2552 Add public API for pluggable filesystems
Remove __wt_directory_sync_fh, it's no longer useful.
* WT-2552 Add public API for pluggable filesystems
Rename WT_INMEMORY_FILE_SYSTEM to WT_FILE_SYSTEM_INMEM, matching
WT_FILE_HANDLE_INMEM.
* WT-2552 Add public API for pluggable filesystems
Add WT_FILE_SYSTEM.directory_list_free, to free memory allocated
by WT_FILE_SYSTEM.direct_list.
Fix a memory leak in __log_archive_once (if __wt_readlock failed,
we leaked the directory-list memory).
* WT-2552 Add public API for pluggable filesystems
Typo, check WT_DIRECT_IO_LOG, not WT_DIRECT_IO_CHECKPOINT.
* WT-2552 Add public API for pluggable filesystems
Typo, unreachable code.
* WT-2552 Add public API for pluggable filesystems
We don't require WT_FILE_SYSTEM.{remove,rename} if the system is
read-only.
* Fix Windows build with pluggable file system.
Involved removing u_int from the public API.
* Fix line wrapping.
* Fix Windows terminate function.
* Forgot something in my last commit.
* Fix Windows munmap bug.
* Add new example to Windows build. Extend example to be more complete.
* Fix example loading on Windows
* Update documentation
* Add missing spell words
* Remove old comment.
|
| |
|
|
| |
Periods followed by colons looks bad, remove periods from error messages.
|
| |\ |
|
| | |
| |
| |
| | |
It was a remnant from early development.
|
| |/
|
|
| |
Initial commit, doesn't yet handle buffering.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Change WiredTiger to always open handles in read/write mode unless the
database itself is configured read-only. The WT_OPEN_READONLY flag is
now OS-specific, and not used above that level.
Requires the creation of a new open flag, WT_OPEN_DIRECTIO: this flag
is set in the OS layer, based on how the connection is configured. It
is also set in the block-manager layer, because that's the only place
we know we're opening a read-only data file (required for support of
the "direct_io=checkpoint" configuration).
|
| |
|
|
|
| |
Do verbose configuration early, with error configuration, so we output
all the messages the application wants.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The __wt_open() "dio_type" has become a general file-type, it should
always be specified. It remains a flag, because it shares definitions
with WT_CONNECTION_IMPL fields, holding flags set during configuration.
Remove the set of the file-type to WT_FILE_TYPE_CHECKPOINT in the block
manager code; instead, test for a read-only data file in the open code.
This means WT_FILE_TYPE_CHECKPOINT is purely a configuration flag, it's
not used inside WiredTiger to specify the type of a file being opened.
Review __wt_open calls, add file-types where not specified, add readonly
where appopriate.
Sue's review comment, change u_int flags to __wt_open to uint32_t.
|
| |
|
|
|
| |
Add a new verbose option, "handleops", add verbose file and handle
operation messages back into the top-level functions.
|
| |
|
|
| |
Make sure the cfg array is always NULL-terminated.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Move file based APIs into the WT_CONNECTION_IMPL structure, move
file-handle based APIs into the WT_FH structure.
Create real WT_FH structures for stdout/stderr instead of using fake
pointer values to flag them, give them their own functions set up when
the WT_CONNECTION_IMPL structure is first created.
Move file based and file-handle based API stubs into misc.i for now,
not sure where they'll end up, or if we'll push WT_FH methods up into
the WiredTiger code.
|
| |
|
|
|
|
| |
Check the wiredtiger_open() config string and the WIREDTIGER_CONFIG
environment variable for read-only and in-memory configuration before
looking at anything else.
|
| |\ |
|
| | | |
|
| | |
| |
| |
| | |
Will allow us to deprecate the incomplete pack/unpack there now.
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Create a pluggable OS layer, add in-memory version so in-memory
configurations don't create on-disk files.
Notes:
The WiredTiger library no longer uses FILE * objects, the file descriptor
and stream functions now use the same open and close functions, and both
have a WT_FH handle.
The sync/async versions of __wt_fsync have been merged, there's now a
boolean flag that indicates wait/no-wait on the flush.
The transaction log's print-log function no longer supports unspecified
handles, output is always to stdout (which the wt utility redirects into
another file as needed).
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
