summaryrefslogtreecommitdiff
path: root/src/include
Commit message (Collapse)AuthorAgeFilesLines
* WT-2771 Add a statistic to track per-btree dirty cache usage. (#3207)Michael Cahill2016-12-204-68/+96
|
* WT-2833 Add projections to wt dump utility (#3192)Don Anderson2016-12-201-1/+1
| | | | | | | | | | | | | | | When dumping with a projection, the dumped metadata associated with the table is modified so that the list of columns matches the list from the projection, and the value format corresponds to projected format. We use an open cursor using the projection to obtain the value format to show, that requires a slight reordering of when cursors are opened in the dump utility. Also fix a problem in the JSON dump cursor which did not handle projections. Added tests for dump with projections, including tables and indices using the normal dump formats, and tables dumped using JSON dump format. Tables are also dumped with projections and then loaded into the modified table format to be checked.
* WT-283 Add WT_SESSION::alter method (#3177)sueloverso2016-12-154-82/+130
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * WT-283 Add WT_SESSION:alter method * Add new method to ex_all.c * Typo * Add testing and fixes for LSM, column groups and indexes for alter. * Fix WITH_*_LOCK macro usage. * Whitespace * Change table alter to not walk down into column groups and indexes. * Whitespace * Use base file meta config and compare strings. * Add skip stat * Add alter support to wt command. * Add util_alter.c to SConstruct * Fix subsection in doc. * Add alter thread to test/format * Swap the value we send into alter in test/format. * Add data-source alter support * Only alter access pattern hint in test/format. * Make LSM alter not exclusive. * Whitespace * Use access_pattern_hint in example. * Include base lsm metadata configuration. * Fix text/comment. * Minor changes from review comments. * Unused variable warning.
* WT-2402 Pad structures to avoid cache line sharing. (#3191)Michael Cahill2016-12-159-25/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * WT-2402 Pad structures to avoid cache line sharing. We previously used compiler alignment to avoid cache line sharing, which is problematic because we don't allocate memory with the necessary alignment, so some behavior is undefined. Instead, this change merely pads structs to ensure they won't share a cache line when allocated in an array. * Add some #if barriers around the new union's * Change where CACHE_LINE_PAD defines live and rename _START to _BEGIN * Add ax_check_compile_flag to aclocal * Lint * Allocate the TINFO structures separately to minimize false sharing. * Comment cleanup to make spell happy. * Pad the read-write locks to avoid false sharing. * Add a comment as to why we're adding -std=c11 to the compile. * whitespace * Add asserts that we're padding the structures we intended to pad. Minor cleanup, don't use #defines outside of the WiredTiger namespace.
* SERVER-26545 Remove fixed-size limitation on WiredTiger hazard pointers (#3187)Keith Bostic2016-12-156-58/+11
| | | | | | | | | | * Default to an initial 250 hazard slots and grow from there. * Make hazard_max undocumented, add an internal limit of 1000 eviction walks. * If we grow the hazard pointer array, schedule the original to be freed when the database is closed. * Update test_bug011 back to stress eviction with the hard-coded limit of 1000 active trees. Only run during "long" tests.
* WT-3066 lint (#3189)Keith Bostic2016-12-121-2/+1
|
* WT-3057 WiredTiger hazard pointers should use the WT_REF, not the WT_PAGE. ↵Keith Bostic2016-12-084-9/+6
| | | | (#3181)
* WT-3052 Improve search if index hint is wrong (#3166)Michael Cahill2016-12-071-1/+1
| | | | | * After a split, child references in an internal page can end up in a different location in the index array. WiredTiger maintains a hint of the expected offset, and searches if the expected reference is not found at the location of the hint. We now search in both directions starting at the hint: the old hint can be wrong in either direction but usually not by much. * Mark the index hint volatile: it would be incorrect if the compiler read it multiple times rather than using a single copy.
* WT-2960 Reduce likelihood of using the lookaside file, especially when ↵Keith Bostic2016-12-071-1/+1
| | | | | | | | | | | | | | | inserting multi-megabyte values (#3171) Don't configure the lookaside table as soon as eviction is stuck, only configure the lookaside table if update/restore reconciliation fails and there's reason to believe the lookaside table will be effective, based on the updates that were skipped during the update/restore reconciliation. The evaluation of whether an update/restore reconciliation would make progress was a check for a single block rewrite with no updates to restore or at least one update chain without a skipped entry. Check more deeply, if there's any block without updates to be restored or at least 10% of the update chains didn't have skipped entries, assume we're making progress.
* WT-2969 Possible snapshot corruption during compaction (#3160)Keith Bostic2016-12-0611-47/+67
| | | | | | | | | | | | | | | * Change compaction to use database-wide checkpoints: rather than processing each file separately and checkpointing after each time we compact a file, do a compaction pass over all of the files then database-wide checkpoints. * Save/restore the WT_SESSION_CAN_WAIT, WT_SESSION_NO_EVICTION flags set during checkpoint, the session handle may be used for other tasks in the future. * There's no need to hold the checkpoint lock while opening the metadata cursor. * When adc1cfb went in (WT-2394, gather handles for compaction first, before doing the actual compaction), we broke data-source support for compaction. Add back in data-source support and simplify the __wt_schema_worker() code, it no longer needs to know about LSM or data-source compaction. * Disallow LSM compaction in an existing transaction (LSM didn't check, but there's no reason to special case LSM compaction so it can run in an existing transaction, and it's potentially confusing, or fragile if LSM compaction some day requires checkpoints). * Do a checkpoint after removing the key/value pairs, otherwise we might not find anything to work with in our compaction pass. * Replace WT_SESSION_LOCK_NO_WAIT with per-lock operation flags. If we set WT_SESSION_LOCK_NO_WAIT and the lock is acquired, then the underlying operation eventually needs to acquire its own locks, having the WT_SESSION_LOCK_NO_WAIT flag set in the session may not be correct.
* WT-3053 Revert: Python and Java support use internal WiredTiger functions ↵Alex Gorrod2016-12-062-4/+4
| | | | (#3178)
* WT-3053 Python and Java support use internal WiredTiger functions (#3174)Keith Bostic2016-12-052-4/+4
| | | | | | | | | | | | * WT-3053 Python and Java support use internal WiredTiger functions Remove calls to internal WiredTiger functions from the Java and Python APIs, meaning we no longer need to expose a few more internal WiredTiger functions. * Try using SWIGWIN instead of _MSC_VER for the swig windows #ifdef. * Another try at a Windows #ifdef in the SWIG code.
* WT-2670 Add access_pattern_hint configuration for tables (#3155)sueloverso2016-12-051-6/+17
|
* WT-3014 Add GCC/clang support for ELF symbol visibility. (#3138)Keith Bostic2016-12-053-798/+798
| | | | | * Rewrite the demo file system example to not require internal WiredTiger functions. * Replace calls to __wt_calloc/__wt_free with C library calloc/free/calls, avoid using internal WiredTiger functions.
* WT-3051 Remove external __wt_hex symbol. (#3172)Keith Bostic2016-12-052-3/+10
|
* WT-3034 Add support for named snapshots including updates. (#3161)Michael Cahill2016-12-052-1/+5
| | | This supports a model where one session performs updates in a transaction after creating a named snapshot and other sessions can use that snapshot and read the updates. In other words, they see exactly what the updating session sees.
* WT-3048 WiredTiger maximum size warning uses the wrong format. (#3164)Keith Bostic2016-12-021-1/+1
| | | | | | Also force the type of WT_BTREE_MAX_OBJECT_SIZE, there's no guarantee all systems will consistently declare UINT32_MAX.
* WT-3028 Don't check for blocked eviction with in-memory workloads. (#3159)Michael Cahill2016-11-292-24/+19
| | | | | | | * Don't check for blocked eviction with in-memory workloads. The cache can become full of clean pages so eviction can't make progress, but application threads are not blocked. We shouldn't fail in this case. * Directly track if reconciliation makes progress. Give up on page rewrites that don't use any updates.
* WT-3024 Fix a hang on close caused by leaving a transaction ID pinned. (#3152)Michael Cahill2016-11-161-0/+3
|
* WT-3023 Don't treat splits as eviction making progress. (#3151)Michael Cahill2016-11-162-19/+24
| | | | | | * We still want to get aggressive and/or "stuck" if all we do is rewrite pages over and over. * Reset the eviction skip count in tiny trees: we may never have enough pages to hit the target.
* WT-3022 Change lsm_tree flags to fields to prevent race conditions (#3148)Sulabh Mahajan2016-11-151-10/+14
|
* WT-3020 Always make LSM chunks evictable when they are switched out. (#3147)Michael Cahill2016-11-151-12/+10
|
* WT-2984 Keep sufficient history in the metadata for queries. (#3144)Michael Cahill2016-11-143-4/+19
| | | | | | | | | | | | * WT-2984 Keep sufficient history in the metadata for queries. Since we treat the checkpoint transaction specially, we also have to track the amount of history required for the metadata specially. Previously, there was a window where a query started while a checkpoint was running could fail to see the results of the checkpoint when it queried the metadata. * Remove the unused "session" argument to WT_IS_METADATA.
* WT-2962 Allow configuration of builtin extensions. (#3137)Michael Cahill2016-11-143-8/+37
| | | While in the area, fix sending "config={values}" to extensions: just the values should be passed in.
* WT-3012 Check a btree is LSM primary before switching. (#3143)Michael Cahill2016-11-141-12/+21
|
* WT-3012 Don't track the LSM Primary as part of dirty bytes in cache (#3136)David Hows2016-11-112-14/+59
|
* WT-3004 lint: declare functions that don't return a value as void (#3119)Keith Bostic2016-11-083-7/+5
| | | | | | | | | | * WT-3004 lint: declare functions that don't return a value as void __wt_txn_get_snapshot() no longer returns a value, which means __wt_txn_cursor_op() and __curds_txn_enter() no longer return values. * Run s_void by default, enhance the output to make it clear where to add false positives.
* WT-3002 Allow applications to exempt threads from eviction. (#3116)Keith Bostic2016-11-081-0/+10
| | | | | | | | | | | | | | * WT-3002 Allow applications to exempt threads from eviction. * Update wiredtiger.in. * Rename config key, make WT_SESSION::reconfigure only apply to passed-in values. Add a basic smoke test of WT_SESSION::reconfigure. * whitespace * Update.
* SERVER-26753 Add a call to spin on rwlocks with no active writers (#3126)Michael Cahill2016-11-042-1/+2
|
* WT-3003 Fix doxygen comment blocks. (#3124)Michael Cahill2016-11-031-6/+8
|
* WT-3003 Don't generate log record and op types. (#3121)sueloverso2016-11-031-37/+34
|
* WT-2998 add error messages to error returns that might be confusing. (#3110)Keith Bostic2016-11-013-4/+7
|
* WT-2975 Only allocate space for statistics when we open a data source (#3107)Keith Bostic2016-11-015-10/+14
|
* WT-2968 Don't open file handles on backup. (#3101)sueloverso2016-10-264-8/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | * WT-2968 Don't open file handles on backup. * whitespace, KNF * Add timing to test and populate with thousands of tables. * Remove macro and meta function. Add schema backup check function. * Refactor new function per review. * Remove cursor backup entry data structure. Simplify test a bit. * Schema ops walk file list in memory instead of on-disk file. * Set connection list entry to NULL before freeing memory. * Remove call to backup_stop on error. Errors are handled above. * Add comment * Fix test description. * Fix new test to use new DataSet code.
* WT-2945 Occasional hang running reconfigure fuzz test (#3098)Keith Bostic2016-10-261-1/+0
| | | | | * Add a write barrier in front of __wt_cond_signal() to ensure the caller's flags meant to cause a thread to exit are seen by the thread. * Make the LSM start/stop worker thread loops look the same.
* WT-2415 Add support for joins to return false positives from the Bloom ↵Don Anderson2016-10-252-3/+8
| | | | filter. (#3099)
* WT-2954 Enable fast appends with small in-memory pages. (#3094)Michael Cahill2016-10-241-0/+2
| | | Eviction tuning to help improve the workload
* WT-2880 Add Zstandard compression support (#3075)Keith Bostic2016-10-211-8/+8
| | | | | | | | | | | | | | * Fix a bug found by inspection in LZ4 code: we're going to offset the destination buffer by sizeof(LZ4_PREFIX), so we need to offset the destination buffer length by the same amount. * Prettiness pass through the snappy driver so it and the zstd driver look the same, minor cleanup in zlib. * Add the compression_level configuration option to the zstd extension initialization function so it's possible to set the compression level from an application. * Fix a bug in zlib raw compression: the destination buffer size (dst_len), can be smaller than the target page size (page_max), use the smaller of the two values to set the target compression size. * The zlib raw compression function could return without calling deflateEnd on its z_stream structures, potentially leaking memory and scratch buffers. * If the default reserved bytes for zlib raw compression isn't sufficient, we fail on compression of what might be very large blocks. We don't have information on how many bytes need to be reserved in order to know the final deflate() will succeed, and the default value was experimentally derived, for all we know there are workloads where the default will fail a lot. Add a fallback before failing hard, try with 2x the default reserved space.
* WT-2985 checkpoint core dump (#3100)Keith Bostic2016-10-201-23/+34
| | | | | | * Checkpoint must not skip writing a leaf page that's never been written before. * Split out the code to make a tree dirty: checkpoint needs it, and it's relatively expensive to dirty a page.
* WT-2976 Add a statistic tracking how long app threads spend reading and ↵Alex Gorrod2016-10-182-201/+213
| | | | writing (#3096)
* WT-2974 lint (#3095)Keith Bostic2016-10-171-1/+1
| | | | | | | | | | | | | | | | __evict_stat_walk(), __wt_curstat_cache_walk(), query_docs() don't return error values, switch to void declarations. __evict_stat_walk() unused variable, bytes_inmem. WT_CURSOR.{get_key,get_value} and pthread_join can all return errors, assert they don't fail. KNF, whitespace Update spelling list. Add a new test utility macro, testutil_assertfmt(), so it's possible to assert with more expansive error messages.
* WT-1592 Add per-dhandle current cache usage statistics (#3062)Alex Gorrod2016-10-148-66/+191
| | | Exposed via a new 'cache_walk' statistics configuration option.
* WT-2403 Add next_random to LSM cursors (#2887)David Hows2016-10-143-9/+18
|
* WT-2955 Add statistics tracking the amount of time threads spend waiting for ↵Keith Bostic2016-10-136-122/+263
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | high level locks (#3086) * WT-2955 Add statistics tracking the amount of time threads spend waiting for high level locks Sort the statistics categories so it's easier to find stuff, no real change. * Add counters and usec wait times to long-term locks (currently the checkpoint, handle-list, metadata, schema and table locks). * mutex.i:295:26: error: conversion to int64_t {aka long int} from long unsigned int may change the sign of the result [-Werror=sign-conversion] [t->slot_usecs] += WT_TIMEDIFF_US(leave, enter); * Rename the lock statistics so they group together. Split lock wait times between internal and application threads. * Separate the connection's dummy session initialization out into its own function, that way it's clear what we're doing. * The session's connection statistics are fixed when the session ID is allocated, so we can cache a pointer to them and avoid u_int divisions (which are currently about the slowest thing you can do on a modern architecture). * A slightly different change: instead of caching a reference to the connection statistics, cache the offset into the array of statistics pointers, that way we can avoid the integer division when updating almost all statistics. * Review comments: Add comments describing the use of statistics array offsets in lock tracking. Rename WT_STATS_FIELD_TO_SLOT to WT_STATS_FIELD_TO_OFFSET. Whitespace cleanup. * __wt_cache_create() doesn't need to call __wt_cache_destroy() explicitly, if the connection open fails at any point, __wt_cache_destroy() will be called as part of that cleanup. * Append the suffix "_off" to the spinlock structure statistics field names, clarify they're offsets into the statistics structures.
* WT-2959 Ensure WT_SESSION_IMPL is never used before it's initialized (#3087)Keith Bostic2016-10-111-1/+1
| | | This is not a bug fix, it's a change to make the code safer against future misuse.
* WT-2954 Evict pages that exceed memory_page_max even if the transaction ID ↵Alex Gorrod2016-10-101-9/+33
| | | | | | | | | | | is stuck (#3088) Try to evict pages that exceed memory_page_max even if the transaction ID isn't increasing. Workloads that insert large values checking only for the transaction ID moving forward can lead to very large in-memory pages. Allow the page to exceed the split size, but not the configured maximum. Skipping such pages is a performance optimization - allow the optimization to work as long as pages are between split size and maximum size. Also don't require so many entries on a leaf page to trigger a split when there are large values.
* WT-2932 Add a configuration option allowing tables to ignore cache limits ↵Alex Gorrod2016-10-073-13/+21
| | | | | | | (#3069) Only available for in-memory configurations, designed to be used for applications that maintain a small amount of content that is required for correctness.
* WT-2948 simplify error handling by making __wt_epoch return never fail (#3080)Keith Bostic2016-10-074-7/+5
| | | If a system call to retrieve a timestamp fails it will now result in a panic. We couldn't find any case where that's a real possibility.
* WT-2831 Skip checkpointing if there have been no modifications (#3067)Sulabh Mahajan2016-10-065-12/+32
| | | This saves locking and scanning the list of active handles for read-only workloads, which can be time consuming when there are many tables.
* WT-2920 New eviction statistics (#3076)Keith Bostic2016-10-042-26/+32
| | | | | | | | | * WT-2920 New eviction statistics Add "application_eviction" statistic, the usecs application threads spent doing eviction. * Add a finer grained timer around eviction as well.
View Lorry Controller Status