summaryrefslogtreecommitdiff
path: root/src/support
Commit message (Collapse)AuthorAgeFilesLines
* WT-283 Add WT_SESSION::alter method (#3177)sueloverso2016-12-151-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * WT-283 Add WT_SESSION:alter method * Add new method to ex_all.c * Typo * Add testing and fixes for LSM, column groups and indexes for alter. * Fix WITH_*_LOCK macro usage. * Whitespace * Change table alter to not walk down into column groups and indexes. * Whitespace * Use base file meta config and compare strings. * Add skip stat * Add alter support to wt command. * Add util_alter.c to SConstruct * Fix subsection in doc. * Add alter thread to test/format * Swap the value we send into alter in test/format. * Add data-source alter support * Only alter access pattern hint in test/format. * Make LSM alter not exclusive. * Whitespace * Use access_pattern_hint in example. * Include base lsm metadata configuration. * Fix text/comment. * Minor changes from review comments. * Unused variable warning.
* SERVER-26545 Remove fixed-size limitation on WiredTiger hazard pointers (#3187)Keith Bostic2016-12-151-84/+197
| | | | | | | | | | * Default to an initial 250 hazard slots and grow from there. * Make hazard_max undocumented, add an internal limit of 1000 eviction walks. * If we grow the hazard pointer array, schedule the original to be freed when the database is closed. * Update test_bug011 back to stress eviction with the hard-coded limit of 1000 active trees. Only run during "long" tests.
* WT-3057 WiredTiger hazard pointers should use the WT_REF, not the WT_PAGE. ↵Keith Bostic2016-12-081-31/+26
| | | | (#3181)
* WT-3053 Revert: Python and Java support use internal WiredTiger functions ↵Alex Gorrod2016-12-061-0/+1
| | | | (#3178)
* WT-3053 Python and Java support use internal WiredTiger functions (#3174)Keith Bostic2016-12-051-1/+0
| | | | | | | | | | | | * WT-3053 Python and Java support use internal WiredTiger functions Remove calls to internal WiredTiger functions from the Java and Python APIs, meaning we no longer need to expose a few more internal WiredTiger functions. * Try using SWIGWIN instead of _MSC_VER for the swig windows #ifdef. * Another try at a Windows #ifdef in the SWIG code.
* WT-3014 Add GCC/clang support for ELF symbol visibility. (#3138)Keith Bostic2016-12-052-0/+7
| | | | | * Rewrite the demo file system example to not require internal WiredTiger functions. * Replace calls to __wt_calloc/__wt_free with C library calloc/free/calls, avoid using internal WiredTiger functions.
* WT-3051 Remove external __wt_hex symbol. (#3172)Keith Bostic2016-12-051-6/+4
|
* WT-3017 Don't set NULL hazard pointers. (#3141)Michael Cahill2016-11-141-4/+18
|
* SERVER-26753 Add a call to spin on rwlocks with no active writers (#3126)Michael Cahill2016-11-041-12/+32
|
* SERVER-26753 Don't spin on a read-lock in a tight loop. (#3113)Michael Cahill2016-11-021-6/+12
| | | | | | | | | | | | * SERVER-26753 Don't spin on a read-lock in a tight loop. We could be starving a thread that we are waiting on of CPU. * Switch to yielding on rwlocks rather than sleeping. * Revert the attempt to be clever when spinning on rwlocks. * Spin getting a read lock while it is available to readers.
* WT-2998 add error messages to error returns that might be confusing. (#3110)Keith Bostic2016-11-013-5/+17
|
* WT-2975 Only allocate space for statistics when we open a data source (#3107)Keith Bostic2016-11-011-4/+28
|
* WT-2976 Add a statistic tracking how long app threads spend reading and ↵Alex Gorrod2016-10-181-0/+13
| | | | writing (#3096)
* WT-1592 Add per-dhandle current cache usage statistics (#3062)Alex Gorrod2016-10-141-0/+90
| | | Exposed via a new 'cache_walk' statistics configuration option.
* WT-2955 Add statistics tracking the amount of time threads spend waiting for ↵Keith Bostic2016-10-131-0/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | high level locks (#3086) * WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks Sort the statistics categories so it's easier to find stuff, no real change. * Add counters and usec wait times to long-term locks (currently the checkpoint, handle-list, metadata, schema and table locks). * mutex.i:295:26: error: conversion to int64_t {aka long int} from long unsigned int may change the sign of the result [-Werror=sign-conversion] [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter); * Rename the lock statistics so they group together. Split lock wait times between internal and application threads. * Separate the connection's dummy session initialization out into its own function, that way it's clear what we're doing. * The session's connection statistics are fixed when the session ID is allocated, so we can cache a pointer to them and avoid u_int divisions (which are currently about the slowest thing you can do on a modern architecture). * A slightly different change: instead of caching a reference to the connection statistics, cache the offset into the array of statistics pointers, that way we can avoid the integer division when updating almost all statistics. * Review comments: Add comments describing the use of statistics array offsets in lock tracking. Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET. Whitespace cleanup. * __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly, if the connection open fails at any point, __wt_cache_destroy() will be called as part of that cleanup. * Append the suffix "_off" to the spinlock structure statistics field names, clarify they're offsets into the statistics structures.
* WT-2948 simplify error handling by making __wt_epoch return never fail (#3080)Keith Bostic2016-10-072-30/+15
| | | If a system call to retrieve a timestamp fails it will now result in a panic. We couldn't find any case where that's a real possibility.
* WT-2831 Skip checkpointing if there have been no modifications (#3067)Sulabh Mahajan2016-10-061-0/+4
| | | This saves locking and scanning the list of active handles for read-only workloads, which can be time consuming when there are many tables.
* WT-2920 New eviction statistics (#3076)Keith Bostic2016-10-041-0/+8
| | | | | | | | | * WT-2920 New eviction statistics Add "application_eviction" statistic, the usecs application threads spent doing eviction. * Add a finer grained timer around eviction as well.
* WT-2942 Verbose messages should not have newlines. (#3074)sueloverso2016-09-301-5/+5
| | | | | | | * WT-2942 Verbose messages should not have newlines. * Fix another set of ERR/RET macros and __wt_verbose calls including newline characters.
* WT-2921 Reset the count when the last hazard pointer is cleared. (#3058)Michael Cahill2016-09-261-2/+3
|
* WT-2859 Restructure statistics gathering macros (#3016)Sulabh Mahajan2016-09-142-6/+6
| | | | | | | Having _FAST_ macros gives an impression that when we use them, we are collecting fast statistics only, which is not true. Except when statistics=none is set, we collect all the stats. This change removes _FAST_ macros and modifies the basic macros to only collect stats when statistics=none is not set.
* WT-2902 Track per-tree the efficiency of eviction walks. (#3036)Michael Cahill2016-09-141-0/+8
| | | | | | | | | Some workloads create "deserts" within a tree where no good eviction candidates can be found (e.g., zipfian distributions, append-only workloads, etc.). Rather than walking all the way through such regions, give up and try in the opposite direction next time. Also add a statistic for the eviction empty score.
* WT-2901 Make checkpoint scrubbing configurable. (#3029)Michael Cahill2016-09-091-4/+0
| | | | Also avoid eviction of dirty pages entirely in the eviction server if worker threads are configured.
* WT-2873 Refactor CRC32 code (#3000)Keith Bostic2016-09-081-1/+1
| | | | | | | | | | | | * Add --enable-crc32-hardware configuration option (configured on by default), which allows CRC32 hardware support to be turned off. * Move the CRC32 implementation in software from the x86-specific code to software/checksum.c, leave the x86-specific code in x86/crc32-x86.c. * Move the pointer to the checksum function from a file static to the WT_PROCESS.cksum field, change the __wt_cksum_init() function to set that field. * WT-2882 Create CRC32 Hardware implementation for ARM8 * Change "cksum" to "checksum" everywhere (hopefully) no semantic changes.
* WT-2888 Switch functions to return void where possible (#3019)Keith Bostic2016-09-061-3/+1
| | | Some functions return an error code even though they don't need to. That adds complexity to our code. Switch to returning a void.
* WT-2881 Add -Wpedantic to clang compiler warning flags (#3010)Keith Bostic2016-09-051-5/+5
| | | | | The only thing that stops us from turning on the clang -Wpedantic flag is passing pointers of various types to printf(3) functions, using the %p format -- cast the arguments.
* WT-2878 Verbose changes affected performance (#3020)Keith Bostic2016-09-031-0/+1
| | | Remove the GCC "cold" attribute from __wt_verbose(), it includes the flag test and so has to always be called. Instead, set the GCC cold attribute for the __wt_eventv() function which __wt_verbose() calls, so we get good branch prediction, the verbose flag test will almost always fail.
* SERVER-25846 Coverity analysis defect 99861: Dereference after null check ↵Keith Bostic2016-08-291-8/+4
| | | | | | | | | | | | | | | | | (#2993) * SERVER-25846 Coverity analysis defect 99861: Dereference after null check Now the __wt_cond_alloc, __wt_cond_wait_signal and __wt_cond_signal functions can panic on failure, they potentially indirect through the WT_SESSION handle to panic the WT_CONNECTION structure. Fortunately, they should not be called with a NULL WT_SESSION handle, remove the tests for that condition, making Coverity happy. * Missed a call to __wt_cond_wait that passed an explicit NULL WT_SESSION; replace the NULL with the existing session handle. * minor whitespace cleanup during condition variable review.
* Fix a bug in thread group shrink where it could leak memory. (#2987)Alex Gorrod2016-08-261-26/+19
|
* WT-2846 Fixes for new thread group code (#2985)Alex Gorrod2016-08-261-3/+0
| | | | | | * It isn't safe to assert that a lock isn't held before acquiring it. * Signal the eviction server to speed up connection close.
* WT-2816 General improvements to WiredTiger eviction performance (#2949)Michael Cahill2016-08-261-6/+5
| | | | | | | | | | A set of changes to the eviction algorithm including: * Fix a bug in how many items can be added to the urgent queue * Have the eviction server sleep less so it recovers from disruptions faster. * Only have application threads evict dirty pages if they are blocked on the dirty trigger. * Swap eviction queues when one becomes empty. * Have the eviction server populate the "other" queue whenever it notices that it isn't full.
* WT-2846 Ensure that all resources are released when destroying a thread ↵sueloverso2016-08-261-15/+27
| | | | group. (#2983)
* WT-2846 Fix bugs related to reconfiguring eviction settings. (#2973)Alex Gorrod2016-08-251-0/+338
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix bugs related to reconfiguring eviction settings. The code to support reconfiguring eviction worker threads had several bugs. Implemented an abstracted utility thread group API and switch eviction workers over to using the new abstraction. * Remove unused function from new thread group API. * Commit auto-generated files. * Be more careful cleaning up on error in thread group code. * Fix uninitialized variable. * Ensure thread group structures are cleared after destruction. This is necessary since the structure is re-used for eviction workers when recovery is run. * Remove util and worker as notions from thread group as per review feedback. Implement other feedback review as well. * Fix a bug where application threads could attempt to help with eviction before the server is setup. Happens if using a shared cache so the cache size starts out at 0. * Remove _util_ prefix from thread group functions. * Add session name to thread group. Fix some comments and whitespace. * Restore error return path.
* WT-2853 Don't force eviction if multiple cursors are pinning the page. (#2974)Michael Cahill2016-08-251-0/+19
| | | | | | | | In some patterns of index lookups, we don't want the second cursor in a session to attempt forced eviction: it will just stall itself to no purpose. Expand test case added testutil_assert for failure condition and reduced test run length. Also cosmetic changes: remove unneeded vars, var declaration reordering, var renaming.
* WT-2847 Merge fair locks into read/write locks. (#2966)Keith Bostic2016-08-171-13/+31
| | | | | | | | | | | | | | | | * WT-2847 Merge fair locks into read/write locks. Merge the fair-lock implementation into the read/write lock implementation (fair locks were the same as the read/write lock's write lock), add a new read/write "is locked?" function for diagnostic tests. * Rename the WT_RWLOCK "users" field to be "next", it makes the code easier to read (and we already depend on "next" not being a keyword, WT_CURSOR.next). * Extend a comment. * aspell does not know amongst!
* WT-2822 panic mutex and other functions that cannot fail (#2952)Keith Bostic2016-08-162-41/+17
| | | | | | | | | | * The pthread mutex implementation of spinlocks lock/unlock functions didn't check the underlying pthread_mutex functions for failure. Panic if pthreads fails. * Change condition mutex functions to not return errors. * Change __wt_verbose() to not return errors. * Make a final panic check before writing anything.
* WT-2820 add gcc warn_unused_result attribute (#2938)Keith Bostic2016-08-122-7/+19
| | | | | | | * Add the gcc "cold" attribute to the verbose, message, and assorted error functions (informs the compiler that the function is unlikely to be executed). * Add the WT_IGNORE_RET() macro that ignores return values in the handfull of places we want to ignore return values. * Replace calls to fprintf() in the utility with calls to util_err(), replace local variable names in error messages with class names.
* WT-2764 Replace write leaves phase of checkpoints. (#2876)Michael Cahill2016-08-031-9/+13
| | | | | | | | | | | | | | | | | | | | | | | | Instead, get multi-threaded writes by dialling down the eviction dirty trigger. Change eviction_dirty_* defaults (to 20/5). Make sure all files are available for eviction before starting fsync. Take more care excluding clean files from checkpoints. Only use scrubbing mode when there is minimal cache pressure. Avoid penalizing read-only operations with dirty eviction targets. Enable scrubbing when eviction is keeping space available in cache. Add stats for checkpoint scrubbing phase. Improve cache scrubbing with big caches and slow I/O. Use the rate of bytes written from cache to decide how long to wait for the dirty bytes to come down.
* WT-2770 Add statistics tracking schema operations (#2925)Sulabh Mahajan2016-08-031-0/+64
| | | | Schema operations for this purpose are table level methods on WT_SESSION handles.
* WT-2805 Avoid infinite recursion on error stream failure. (#2921)Michael Cahill2016-08-021-0/+6
| | | | Previously, if both a custom and the default error streams failed to write it caused infinite recursion.
* WT-2793 Enhance statistics related to overflow values (#2912)sueloverso2016-07-281-3/+6
| | | | | | | | * WT-2793 Remove very long running config. Rename the one we run. * Fix overflow stats. Make 130K overflow test use btree. * Add line to upgrading doc stating stat field removed.
* WT-2746 Add a new statistic tracking I/O for checkpoints (#2853)Keith Bostic2016-07-081-0/+4
| | | Separate cache write statistics into checkpoint and non-checkpoint I/O.
* WT-2743 Fixup new I/O thread count statistics. (#2850)Alex Gorrod2016-07-061-6/+6
| | | | The increment and decrement were backward, so they never reported positive values. While I was there make the statistics macros consistent.
* WT-2665 Limit allocator fragmentation from the WiredTiger cache (#2799)Michael Cahill2016-06-301-0/+6
| | | | | | | | | | | | | | | | | | | | | | * Add a statistic for the amount of cache used for page images vs other objects * Reset the "maximum page size" statistic each checkpoint * Add a separate queue for urgent eviction, replaces existing WOULD_BLOCK eviction mode Until now, the eviction server has had to walk the cache to find pages that would otherwise block application threads. This change allows threads to put those pages directly on a queue that is checked before ordinary LRU eviction. * Don't queue pages for urgent eviction when eviction is disabled in a tree. Decouple evict_queue_lock and the individual queue's evict_lock in __evict_get_ref. Don't block the eviction server if there are many application threads waiting on a queue. * Take care with the urgent queue. Entries in the urgent queue cannot be skipped or they will not be considered again.
* WT-2729 Focus eviction on the largest trees in cache. (#2832)Michael Cahill2016-06-281-0/+4
| | | | | | | | | | | | | | | | | | Randomize visits to trees that use a tiny fraction of the cache. Other general eviction optimizations. Now that we are queuing more entries (potentially), make sure enough of them become candidates. Previously, a skewed distribution of read generations could mean that only 10% of queue entries were considered. Improve the efficiency of sorting the queue by calculating the score once when pages are added to the queue. Add a workload to exercise differential eviction from trees. In order to do that, add range partitioning to wtperf. Don't override icount in wtperf workloads with random_range set.
* SERVER-24580 Add more eviction stats to track efficiency. (#2830)Michael Cahill2016-06-231-0/+24
|
* WT-2408 Windows error translation layer (#2819)Keith Bostic2016-06-231-1/+1
| | | | | | | | | | | | Build a Windows-to-POSIX/ANSI error translation layer. Replace the read-only error mapping to WT_NOTFOUND and WT_PERM_DENIED with EACCES and ENOENT. Windows no longer needs its own version of __wt_strerror(), move the POSIX implementation from os_posix/os_errno.c to os_common/os_errno.c. Rename os_win/os_errno.c to os_win/os_winerr.c to avoid a collision. Windows now has DWORD types in prototypes, split the Windows/POSIX extern.h files. (This actually cleans up some noise, previously we had to sort the OS prototypes to remove duplicates, which wasn't trivial.) Add the WT_EXTENSION_API.map_windows_error method to map Windows system codes to POSIX/ANSI system codes.
* WT-2691 Use wrappers for ctype functions to avoid sign extension errors (#2824)Keith Bostic2016-06-221-1/+1
| | | | | gcc 4.7, clang 3.4 and clang 3.8 all complain about different things; cast the arguments to our new wrapper functions, hopefully that makes all of the complaints go away.
* WT-2691 Use wrappers for ctype functions to avoid sign extension errors (#2818)Don Anderson2016-06-211-1/+1
| | | | | | | | | | | | | | * Cast arguments for ctype functions to avoid sign extension errors. Create __wt_* versions of all ctype functions, and use them whenever wt_internal.h is available. Add check to prevent direct use of ctype functions from core source. * Change wrappers to use u_char arguments, and return bool. Remove unused wrappers. * Use u_char in preference to unsigned char. * Examples do not have u_char defined on Windows.
* WT-2541 Add stats showing active readers and writers (#2793)David Hows2016-06-201-0/+12
| | | | | | | | | | | | | | | | | | | | * WT-2541 Add stats showing active readers and writers * Fix use of RET_MSG instead of ERR_MSG * Change where we increment the active r/w stats * Add "active fsync" stat * Add new "Thread State" stat section, add fsync_io stat * KNF * Make thread stats not backwards breaking. Add atomic stat functions * Add ThreadState stats to system and evict groups * KNF
View Lorry Controller Status