summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* PGLog: maintain writeout_from and trimmedwip-dumpling-perf2Samuel Just2013-08-272-14/+42
| | | | | | | This way, we can avoid omap_rmkeyrange in the common append and trim cases. Signed-off-by: Samuel Just <sam.just@inktank.com>
* PGLog: don't maintain log_keys_debug if the config is disabledSamuel Just2013-08-272-11/+14
| | | | Signed-off-by: Samuel Just <sam.just@inktank.com>
* WBThrottle: use fdatasync instead of fsyncSamuel Just2013-08-261-1/+1
| | | | Signed-off-by: Samuel Just <sam.just@inktank.com>
* FileStore: add config option to disable the wbthrottleSamuel Just2013-08-262-1/+3
| | | | Signed-off-by: Samuel Just <sam.just@inktank.com>
* PGLog: move the log size check after the early returnwip-dumpling-log-assertSamuel Just2013-08-261-1/+1
| | | | | | | | Amazingly, some versions of g++ (like the one on my ubuntu 12.04 machine) appear to have a list::size() which is linear in the size of the list. That assert, therefore, is quite expensive! Signed-off-by: Samuel Just <sam.just@inktank.com>
* ceph-disk: partprobe after creating journal partitionSage Weil2013-08-261-1/+13
| | | | | | | | | | | | At least one user reports that a partprobe is needed after creating the journal partition. It is not clear why sgdisk is not doing it, but this fixes ceph-disk for them, and should be harmless for other users. Fixes: #5599 Tested-by: lurbs in #ceph Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit 2af59d5e81c5e3e3d7cfc50d9330d7364659c5eb) (cherry picked from commit 3e42df221315679605d68b2875aab6c7eb6b3cc4)
* mon/Paxos: always refresh after any store_stateSage Weil2013-08-261-1/+7
| | | | | | | | | | | | | | | | | | | | | If we store any new state, we need to refresh the services, even if we are still in the midst of Paxos recovery. This is because the subscription path will share any committed state even when paxos is still recovering. This prevents a race like: - we have maps 10..20 - we drop out of quorum - we are elected leader, paxos recovery starts - we get one LAST with committed states that trim maps 10..15 - we get a subscribe for map 10..20 - we crash because 10 is no longer on disk because the PaxosService is out of sync with the on-disk state. Fixes: #6045 Backport: dumpling Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com> (cherry picked from commit 981eda9f7787c83dc457f061452685f499e7dd27)
* mon/Paxos: return whether store_state stored anythingSage Weil2013-08-262-2/+7
| | | | | | Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com> (cherry picked from commit 7e0848d8f88f156a05eef47a9f730b772b64fbf2)
* mon/Paxos: cleanup: use do_refresh from handle_commitSage Weil2013-08-261-9/+3
| | | | | | | | | This avoid duplicated code by using the helper created exactly for this purpose. Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com> (cherry picked from commit b9dee2285d9fe8533fa98c940d5af7b0b81f3d33)
* osdc/ObjectCacher: do not merge rx buffersSage Weil2013-08-261-0/+4
| | | | | | | | | | We do not try to merge rx buffers currently. Make that explicit and documented in the code that it is not supported. (Otherwise the last_read_tid values will get lost and read results won't get applied to the cache properly.) Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit 1c50c446152ab0e571ae5508edb4ad7c7614c310)
* osdc/ObjectCacher: match reads with their original rx buffersSage Weil2013-08-262-12/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Consider a sequence like: 1- start read on 100~200 100~200 state rx 2- truncate to 200 100~100 state rx 3- start read on 200~200 100~100 state rx 200~200 state rx 4- get 100~200 read result Currently this makes us crash on osdc/ObjectCacher.cc: 738: FAILED assert(bh->length() <= start+(loff_t)length-opos) when processing the second 200~200 bufferhead (it is too big). The larger issue, though, is that we should not be looking at this data at all; it has been truncated away. Fix this by marking each rx buffer with the read request that is sent to fill it, and only fill it from that read request. Then the first reply will fill the first 100~100 extend but not touch the other extent; the second read will do that. Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit b59f930ae147767eb4c9ff18c3821f6936a83227)
* mon/Paxos: fix another uncommitted value corner caseSage Weil2013-08-261-1/+10
| | | | | | | | | | | | | | | | | | | | It is possible that we begin the paxos recovery with an uncommitted value for, say, commit 100. During last/collect we discover 100 has been committed already. But also, another node provides an uncommitted value for 101 with the same pn. Currently, we refuse to learn it, because the pn is not strictly > than our current uncommitted pn... even though it is the next last_committed+1 value that we need. There are two possible fixes here: - make this a >= as we can accept newer values from the same pn. - discard our uncommitted value metadata when we commit the value. Let's do both! Fixes: #6090 Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit fe5010380a3a18ca85f39403e8032de1dddbe905)
* os: make readdir_r buffers largerSage Weil2013-08-262-4/+5
| | | | | | | | PATH_MAX isn't quite big enough. Backport: dumpling, cuttlefish, bobtail Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit 99a2ff7da99f8cf70976f05d4fe7aa28dd7afae5)
* os: fix readdir_r buffer sizeSage Weil2013-08-261-3/+5
| | | | | | | | | | | | The buffer needs to be big or else we're walk all over the stack. Backport: dumpling, cuttlefish, bobtail Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit 2df66d9fa214e90eb5141df4d5755b57e8ba9413) Conflicts: src/os/BtrfsFileStoreBackend.cc
* rgw: fix crash when creating new zone on initYehuda Sadeh2013-08-231-8/+8
| | | | | | | | | Moving the watch/notify init before the zone init, as we might need to send a notification. Reviewed-by: Sage Weil <sage@inktank.com> Signed-off-by: Yehuda Sadeh <yehuda@inktank.com> (cherry picked from commit 3d55534268de7124d29bd365ea65da8d2f63e501)
* rgw: change cache / watch-notify init sequenceYehuda Sadeh2013-08-233-5/+16
| | | | | | | | | | | | | | Fixes: #6046 We were initializing the watch-notify (through the cache init) before reading the zone info which was much too early, as we didn't have the control pool name yet. Now simplifying init/cleanup a bit, cache doesn't call watch/notify init and cleanup directly, but rather states its need through a virtual callback. Signed-off-by: Yehuda Sadeh <yehuda@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit d26ba3ab0374e77847c742dd00cb3bc9301214c2)
* enable mds rejoin with active inodes' old parent xattrsAlexandre Oliva2013-08-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the parent xattrs of active inodes that the mds attempts to open during rejoin lack pool info (struct_v < 5), this field will be filled in with -1, causing the mds to retry fetching a backtrace with a pool number that matches the expected value, which fails and causes the err==-ENOENT branch to be taken and retry pool 1, which succeeds, but with pool -1, and so keeps on bouncing between the two retry cases forever. This patch arranges for the mds to go along with pool -1 instead of insisting that it be refetched, enabling it to complete recovery instead of eating cpu, network bandwidth and metadata osd's resources like there's no tomorrow, in what AFAICT is an infinite and very busy loop. This is not a new problem: I've had it even before upgrading from Cuttlefish to Dumpling, I'd just never managed to track it down, and force-unmounting the filesystem and then restarting the mds was an easier (if inconvenient) work-around, particularly because it always hit when the filesystem was under active, heavy-ish use (or there wouldn't be much reason for caps recovery ;-) There are two issues not addressed in this patch, however. One is that nothing seems to proactively update the parent xattr when it is found to be outdated, so it remains out of date forever. Not even renaming top-level directories causes the xattrs to be recursively rewritten. AFAICT that's a bug. The other is that inodes that don't have a parent xattr (created by even older versions of ceph) are reported as non-existing in the mds rejoin message, because the absence of the parent xattr is signaled as a missing inode (?failed to reconnect caps for missing inodes?). I suppose this may cause more serious recovery problems. I suppose a global pass over the filesystem tree updating parent xattrs that are out-of-date would be desirable, if we find any parent xattrs still lacking current information; it might make sense to activate it as a background thread from the backtrace decoding function, when it finds a parent xattr that's too out-of-date, or as a separate client (ceph-fsck?). Backport: dumpling, cuttlefish Signed-off-by: Alexandre Oliva <oliva@gnu.org> Reviewed-by: Zheng, Yan <zheng.z.yan@intel.com> (cherry picked from commit 617dc36d477fd83b2d45034fe6311413aa1866df)
* mds: remove waiting lock before merging with neighboursDavid Disseldorp2013-08-231-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | CephFS currently deadlocks under CTDB's ping_pong POSIX locking test when run concurrently on multiple nodes. The deadlock is caused by failed removal of a waiting_locks entry when the waiting lock is merged with an existing lock, e.g: Initial MDS state (two clients, same file): held_locks -- start: 0, length: 1, client: 4116, pid: 7899, type: 2 start: 2, length: 1, client: 4110, pid: 40767, type: 2 waiting_locks -- start: 1, length: 1, client: 4116, pid: 7899, type: 2 Waiting lock entry 4116@1:1 fires: handle_client_file_setlock: start: 1, length: 1, client: 4116, pid: 7899, type: 2 MDS state after lock is obtained: held_locks -- start: 0, length: 2, client: 4116, pid: 7899, type: 2 start: 2, length: 1, client: 4110, pid: 40767, type: 2 waiting_locks -- start: 1, length: 1, client: 4116, pid: 7899, type: 2 Note that the waiting 4116@1:1 lock entry is merged with the existing 4116@0:1 held lock to become a 4116@0:2 held lock. However, the now handled 4116@1:1 waiting_locks entry remains. When handling a lock request, the MDS calls adjust_locks() to merge the new lock with available neighbours. If the new lock is merged, then the waiting_locks entry is not located in the subsequent remove_waiting() call because adjust_locks changed the new lock to include the old locks. This fix ensures that the waiting_locks entry is removed prior to modification during merge. Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Greg Farnum <greg@inktank.com> (cherry picked from commit 476e4902907dfadb3709ba820453299ececf990b)
* mon/PGMap: OSD byte counts 4x too large (conversion to bytes overzealous)Dan Mick2013-08-231-3/+3
| | | | | | Fixes: #6049 Signed-off-by: Dan Mick <dan.mick@inktank.com> (cherry picked from commit eca53bbf583027397f0d5e050a76498585ecb059)
* ceph-disk: specify the filetype when mountingAlfredo Deza2013-08-231-0/+1
| | | | | | Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit f040020fb2a7801ebbed23439159755ff8a3edbd)
* v0.67.2v0.67.2Gary Lowell2013-08-222-1/+7
|
* .gitignore: ignore test-driverSage Weil2013-08-221-0/+1
| | | | | Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit edf2c3449ec96d91d3d7ad01c50f7a79b7b2f7cc)
* fuse: fix warning when compiled against old fuse versionsSage Weil2013-08-221-1/+1
| | | | | | | | client/fuse_ll.cc: In function 'void invalidate_cb(void*, vinodeno_t, int64_t, int64_t)': warning: client/fuse_ll.cc:540: unused variable 'fino' Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit 9833e9dabe010e538cb98c51d79b6df58ce28f9e)
* json_spirit: remove unused typedefSage Weil2013-08-221-2/+0
| | | | | | | | | | | | In file included from json_spirit/json_spirit_writer.cpp:7:0: json_spirit/json_spirit_writer_template.h: In function 'String_type json_spirit::non_printable_to_string(unsigned int)': json_spirit/json_spirit_writer_template.h:37:50: warning: typedef 'Char_type' locally defined but not used [-Wunused-local-typedefs] typedef typename String_type::value_type Char_type; (Also, ha ha, this file uses \r\n.) Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit 6abae35a3952e5b513895267711fea63ff3bad09)
* gtest: add build-aux/test-driver to .gitignoreSage Weil2013-08-221-0/+1
| | | | | Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit c9cdd19d1cd88b84e8a867f5ab85cb51fdc6f8e4)
* objecter: resend unfinished lingers when osdmap is no longer pausedJosh Durgin2013-08-211-2/+12
| | | | | | | | | | | | | | | | Plain Ops that haven't finished yet need to be resent if the osdmap transitions from full or paused to unpaused. If these Ops are triggered by LingerOps, they will be cancelled instead (since should_resend = false), but the LingerOps that triggered them will not be resent. Fix this by checking the registered flag for all linger ops, and resending any of them that aren't paused anymore. Fixes: #6070 Signed-off-by: Josh Durgin <josh.durgin@inktank.com> Reviewed-by: Sage Weil <sage.weil@inktank.com> (cherry picked from commit 38a0ca66a79af4b541e6322467ae3a8a4483cc72)
* pybind: fix Rados.conf_parse_env testSage Weil2013-08-201-6/+2
| | | | | | | | | | This happens after we connect, which means we get ENOSYS always. Instead, parse_env inside the normal setup method, which had the added benefit of being able to debug these tests. Backport: dumpling Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit 6ef1970340c57d6e02f947348fb38882b51d131c)
* librados: fix MWatchNotify leakSage Weil2013-08-201-9/+6
| | | | | | | | | | | Do not leak the message if the watcher is not registered. Also, simplify this block. Fixes (part of): #5949 Backport: dumpling, cuttlefish Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Yehuda Sadeh <yehuda@inktank.com> (cherry picked from commit 6f5d8036f3e70c5e30edf7e36fb8ff9a56197f60)
* PG: remove old log when we upgrade log versionSamuel Just2013-08-192-0/+9
| | | | | | | | | | Otherwise the log_oid will be non-empty and the next boot will cause us to try to upgrade again. Fixes: #6057 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit 1f851cb2489a95526de932ec6734ebf413e43490)
* PGLog: add a config to disable PGLog::check()Samuel Just2013-08-193-3/+12
| | | | | | | | | This is a debug check which may be causing excessive cpu usage. Reviewed-by: Sage Weil <sage@inktank.com> Signed-off-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit 00080d785f6695b800f71317a3048a21064e61cb)
* ceph: parse CEPH_ARGS environment variableSage Weil2013-08-191-0/+1
| | | | | | | Fixes: #6052 Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Dan Mick <dan.mick@inktank.com> (cherry picked from commit 67a95b9880c9bc6e858150352318d68d64ed74ad)
* rados pybind: add conf_parse_env()Sage Weil2013-08-192-0/+19
| | | | | | Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Dan Mick <dan.mick@inktank.com> (cherry picked from commit eef7cacdb19313907a9367187b742db5382ee584)
* rgw: drain requests before exitingYehuda Sadeh2013-08-181-0/+1
| | | | | | | | Fixes: #5953 Signed-off-by: Yehuda Sadeh <yehuda@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit 3cbf6a7b031c2ce8072733c5c0b7ceb53fdcb090)
* rgw: do not leak handler in get_handler() error pathSage Weil2013-08-181-1/+3
| | | | | | | | | If we fail to initialize, delete the handler. Fixes (part of): #5949 Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Yehuda Sadeh <yehuda@inktank.com> (cherry picked from commit 810c52de36719c3ee6cf2bdf59d5cde8840bbe55)
* rgw: fix leak of RGWDataChangesLog::renew_threadSage Weil2013-08-181-0/+1
| | | | | Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit 25948319c4d256c4aeb0137eb88947e54d14cc79)
* rgw: free resolver on shutdownSage Weil2013-08-183-1/+8
| | | | | Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit a31356338b8ae55df59d829d9080ffad70b97d10)
* rgw: fix up signal handlingSage Weil2013-08-181-20/+35
| | | | | | | | | | | | OMG libfcgi is annoying with shutdown and signals. You need to close the fd *and* resend a signal to ensure that you kick the accept loop hard enough to make it shut down. Document this, and switch to the async signal handlers. Put them tightly around the runtime loop as we do with other daemons. Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit 59b13cebee600dad2551d2c7dc3482b05eaf8b22)
* v0.67.1v0.67.1Gary Lowell2013-08-162-1/+7
|
* ceph.in: --admin-daemon was not returning EINVAL on bad commandDan Mick2013-08-151-18/+19
| | | | | | | | Fix by restructuring code to hoist common code and have only one place where admin_socket is actually called. Signed-off-by: Dan Mick <dan.mick@inktank.com> (cherry picked from commit 266460e97ec9ef9711e9eaa4bd954f3188d8da69)
* mon: use str_join instead of std::copySage Weil2013-08-152-12/+4
| | | | | | | | The std::copy method leaves a trailing separator. Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Dan Mick <dan.mick@inktank.com> (cherry picked from commit 35565ee64e41d7fddc7849c6006692c78227132c)
* config: fix stringification of config valuesSage Weil2013-08-151-4/+4
| | | | | | | | | | | The std::copy construct leaves a trailing separator character, which breaks parsing for booleans (among other things) and probably mangles everything else too. Backport: dumpling Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit fc23cfe3fe567b30413d8af0c614a32fec238939)
* common: add str_join helperSage Weil2013-08-151-1/+14
| | | | | | Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Dan Mick <dan.mick@inktank.com> (cherry picked from commit ce3a0944d9b47f7b178fe7775c9d105305b238e0)
* rados.py: fix Rados() unicode checkingJosh Durgin2013-08-142-1/+15
| | | | | | | | | Check new parameters and check that rados_id is not None again to catch the empty string. Signed-off-by: Josh Durgin <josh.durgin@inktank.com> Reviewed-by: Sage Weil <sage.weil@inktank.com> (cherry picked from commit 4422f21a6586467a63ce6841552d0f60aa849cf1)
* rados.py: fix Rados() backwards compatibilityJosh Durgin2013-08-142-2/+27
| | | | | | | | | | | | | | | | | Previously it had no name parameter, so the default will be used by old clients. However, if an old client set rados_id, a new check that both rados_id and name are set would result in an error. Fix this by only applying the default names after the check, and add tests of this behavior. This was introduced in 783b7ec847c7f987ac1814c9c41c91921cac4eba, so it does not affect cuttlefish. Fixes: #5970 Reported-by: Michael Morgan <mmorgan@dca.net> Signed-off-by: Josh Durgin <josh.durgin@inktank.com> Reviewed-by: Sage Weil <sage.weil@inktank.com> (cherry picked from commit 34da9cbc33205623cf64aee1989f53dfb2c5bddd)
* librados: fix async aio completion wakeupSage Weil2013-08-141-12/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | For aio flush, we register a wait on the most recent write. The write completion code, however, was *only* waking the waiter if they were waiting on that write, without regard to previous writes (completed or not). For example, we might have 6 and 7 outstanding and wait on 7. If they finish in order all is well, but if 7 finishes first we do the flush completion early. Similarly, if we - start 6 - start 7 - finish 7 - flush; wait on 7 - finish 6 we can hang forever. Fix by doing any completions that are prior to the oldest pending write in the aio write completion handler. Refs: #5919 Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Josh Durgin <josh.durgin@inktank.com> Tested-by: Oliver Francke <Oliver.Francke@filoo.de> (cherry picked from commit 16ed0b9af8bc08c7dabead1c1a7c1a22b1fb02fb)
* librados: fix locking for AioCompletionImpl refcountingJosh Durgin2013-08-141-5/+9
| | | | | | | | | | | | | | | | | | | Add an already-locked helper so that C_Aio{Safe,Complete} can increment the reference count when their caller holds the lock. C_AioCompleteAndSafe's caller is not holding the lock, so call regular get() to ensure no racing updates can occur. This eliminates all direct manipulations of AioCompletionImpl->ref, and makes the necessary locking clear. The only place C_AioCompleteAndSafe is used is in handling aio_flush_async(). This could cause a missing completion. Refs: #5919 Signed-off-by: Josh Durgin <josh.durgin@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> Tested-by: Oliver Francke <Oliver.Francke@filoo.de> (cherry picked from commit 7a52e2ff5025754f3040eff3fc52d4893cafc389)
* v0.67v0.67Gary Lowell2013-08-131-0/+6
|
* rgw: fix multi deleteYehuda Sadeh2013-08-121-1/+1
| | | | | | | | | | | | Fixes: #5931 Backport: bobtail, cuttlefish Fix a bad check, where we compare the wrong field. Instead of comparing the ret code to 0, we compare the string value to 0 which generates implicit casting, hence the crash. Signed-off-by: Yehuda Sadeh <yehuda@inktank.com> Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
* ceph-disk: fix mount options passed to move_mountSage Weil2013-08-091-0/+6
| | | | | | | | | | Commit 6cbe0f021f62b3ebd5f68fcc01a12fde6f08cff5 added a mount_options but in certain cases it may be blank. Fill in with the defaults, just as we do in mount(). Backport: cuttlefish Reviewed-by: Dan Mick <dan.mick@inktank.com> Signed-off-by: Sage Weil <sage@inktank.com>
* config_opts.h: reduce osd_recovery_max_active and osd_recovery_max_single_startSamuel Just2013-08-091-2/+2
| | | | | Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: David Zafman <david.zafman@inktank.com>
View Lorry Controller Status