summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
* all: add broken pipe detection while waiting for inputCarl Edquist2023-02-283-0/+128
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a program's output becomes a broken pipe, future attempts to write to that ouput will fail (SIGPIPE/EPIPE). Once it is known that all future write attepts will fail (due to broken pipes), in many cases it becomes pointless to wait for further input for slow devices like ttys. Ideally, a program could use this information to exit early once it is known that future writes will fail. Introduce iopoll() to wait on a pair of fds (input & output) for input to become ready or output to become a broken pipe. This is relevant when input is intermittent (a tty, pipe, or socket); but if input is always ready (a regular file or block device), then a read() will not block, and write failures for a broken pipe will happen normally. Introduce iopoll_input_ok() to check whether an input fd is relevant for iopoll(). Experimentally, broken pipes are only detectable immediately for pipes, but not sockets. Errors for other file types will be detected in the usual way, on write failure. Introduce iopoll_output_ok() to check whether an output fd is suitable for iopoll() -- namely, whether it is a pipe. iopoll() is best implemented with a native poll(2) where possible, but fall back to a select(2)-based implementation platforms where there are portability issues. See also discussion in tail.c. In general, adding a call to iopoll() before a read() in filter programs also allows broken pipes to "propagate" backwards in a shell pipeline. * src/iopoll.c, src/iopoll.h (iopoll): New function implementing broken pipe detection on output while waiting for input. (IOPOLL_BROKEN_OUTPUT, IOPOLL_ERROR): Return codes for iopoll(). (IOPOLL_USES_POLL): Macro for poll() vs select() implementation. (iopoll_input_ok): New function to check whether an input fd is relevant for iopoll(). (iopoll_output_ok): New function to check whether an input fd is suitable for iopoll(). * src/local.mk (noinst_HEADERS): add src/iopoll.h.
* build: update to latest gnulibPádraig Brady2023-02-272-0/+4
| | | | | | | | | * NEWS: Mention the fts fix to avoid the following assert in rm on mem pressure: Program terminated with signal SIGSEGV, Segmentation fault. at ../lib/cycle-check.c:60 assure (state->magic == CC_MAGIC); * gnulib: Update to the latest to pick up fts commit f17d3977.
* tests: avoid hang in new testPádraig Brady2023-02-261-8/+8
| | | | | | | * tests/rm/empty-inacc.sh: Ensure we're not reading from stdin when we're relying on no prompt to proceed. Also change the file being tested so that a failure in one test doesn't impact following tests causing a framework failure.
* tests: avoid gdb on macOSPádraig Brady2023-02-263-0/+11
| | | | | | | | | | | | gdb was seen to hang intermittently on macOS 12. Also gdb requires signing on newer macOS systems: https://sourceware.org/gdb/wiki/PermissionsDarwin So restrict its use on macOS systems for now. * tests/rm/r-root.sh: Skip on darwin systems. * tests/tail-2/inotify-race.sh: Restrict the test to inotify capable systems to avoid the hang with some gdbs. * tests/tail-2/inotify-race.sh: Likewise.
* cp,install,mv: --debug: output debug info when reflinkingPádraig Brady2023-02-241-5/+5
| | | | | * src/copy.c (copy_reg): Always check whether to output debug info. (emit_debug): Restrict output with `cp --attributes-only`.
* tests: determine if SEEK_HOLE is enabledPádraig Brady2023-02-245-4/+16
| | | | | | | | | | | | | | Upcomming gnulib changes may disable SEEK_HOLE even if the system supports it, so dynamically check if we've SEEK_HOLE enabled. * init.cfg (seek_data_capable_): SEEK_DATA may be disabled in the build if the system support is deemed insufficient, so also use `cp --debug` to determine if it's enabled. * tests/cp/sparse-2.sh: Adjust to a more general diagnostic. * tests/cp/sparse-extents-2.sh: Likewise. * tests/cp/sparse-extents.sh: Likewise. * tests/cp/sparse-perf.sh: Likewise.
* cp,install,mv: add --debug to explain how a file is copiedPádraig Brady2023-02-249-4/+172
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | How a file is copied is dependent on the sparseness of the file, what file system it is on, what file system the destination is on, the attributes of the file, and whether they're being copied or not. Also the --reflink and --sparse options directly impact the operation. Given it's hard to reason about the combination of all of the above, the --debug option is useful for users to directly identify if copy offloading, reflinking, or sparse detection are being used. It will also be useful for tests to directly query if these operations are supported. The new output looks as follows: $ src/cp --debug src/cp file.sparse 'src/cp' -> 'file.sparse' copy offload: yes, reflink: unsupported, sparse detection: no $ truncate -s+1M file.sparse $ src/cp --debug file.sparse file.sparse.cp 'file.sparse' -> 'file.sparse.cp' copy offload: yes, reflink: unsupported, sparse detection: SEEK_HOLE $ src/cp --reflink=never --debug file.sparse file.sparse.cp 'file.sparse' -> 'file.sparse.cp' copy offload: avoided, reflink: no, sparse detection: SEEK_HOLE * doc/coreutils.texi (cp invocation): Describe the --debug option. (mv invocation): Likewise. (install invocation): Likewise. * src/copy.h: Add a new DEBUG member to cp_options, to control whether to output debug info or not. * src/copy.c (copy_debug): A new global structure to unconditionally store debug into from the last copy_reg operations. (copy_debug_string, emit_debug): New functions to print debug info. * src/cp.c: if ("--debug") x->debug=true; * src/install.c: Likewise. * src/mv.c: Likewise. * tests/cp/debug.sh: Add a new test. * tests/local.mk: Reference the new test. * NEWS: Mention the new feature.
* build: update gnulib submodule to latestPaul Eggert2023-02-231-0/+0
|
* doc: chgrp,chmod,chown: state --reference always dereferencesPádraig Brady2023-02-233-5/+6
| | | | | | | | * src/chgrp.c (usage): State that --reference always dereferences symbolic links. * src/chmod.c (usage): Likewise. * src/chown.c (usage): Likewise. Fixes https://bugs.gnu.org/61720
* doc: fix some spelling mistakesChuanGang Jiang2023-02-214-8/+8
| | | | | | | | | | * doc/coreutils.texi: s/functionalty/functionality/, s/sychronize/synchronize/, s/millsecond/millisecond/ s/paramter/parameters/ * init.cfg: s/parmeters/parameters/ * scripts/build-older-versions/README.older-versions: s/vesion/version/ * tests/misc/env-S-script.sh: s/paramaters/parameters/ Fixes https://bugs.gnu.org/61681
* maint: avoid -Wmaybe-uninitialized warning from GCC13Pádraig Brady2023-02-211-1/+3
| | | | | * src/copy.c (infer_scantype): Always set scan_inference.ext_start, as per commit 6c03e8fb which was inadvertently reverted by d374d32c.
* tests: initialize a variableJim Meyering2023-02-211-0/+1
| | | | | * tests/rm/interactive-once.sh (write_prot_msg1): Initialize it, so an envvar setting cannot perturb the test.
* rm: --dir (-d): fix bugs in handling of empty, inaccessible directoriesJim Meyering2023-02-213-7/+49
| | | | | | | | | | | | | | | * src/remove.c (prompt, rm_fts): In the dir-handling code of both of these functions, relax a "get_dir_status (...) == DS_EMPTY" condition to instead test only "get_dir_status (...) != 0", enabling flow control to reach the prompt function also for unreadable directories. However, that function itself also needed special handling for this case: (prompt): Handle empty, inaccessible directories properly, deleting them with -d (--dir), and prompting about whether to delete with -i (--interactive). * tests/rm/empty-inacc.sh: Add tests for the new code. Reported by наб <nabijaczleweli@nabijaczleweli.xyz> in bugs.debian.org/1015273 * NEWS (Bug fixes): Mention this.
* tests: port chmod/setgid.sh to macOS 12Paul Eggert2023-02-181-8/+7
| | | | | | * tests/chmod/setgid.sh: Try all the groups you’re a member of, in case id -g returns 4294967295 (nogroup) which is special and does not let you chgrp a file to it.
* tests: port better to macOS group numbersPaul Eggert2023-02-181-1/+13
| | | | | | * init.cfg (groups): Port better to macOS 12, where group 4294967295 (nogroup) is special: you can be a member without being able to chgrp files to the group.
* build: update gnulib submodule to latestPaul Eggert2023-02-181-0/+0
|
* cp: fclonefileat security fix + CLONE_ACL + fixupsPaul Eggert2023-02-162-13/+107
| | | | | | | | | | | | | | | * src/copy.c: Some changes if HAVE_FCLONEFILEAT && !USE_XATTR. (fd_has_acl): New function. (CLONE_ACL): Default to 0. (copy_reg): Use CLONE_NOFOLLOW to avoid races like CVE-2021-30995 <https://www.trendmicro.com/en_us/research/22/a/ analyzing-an-old-bug-and-discovering-cve-2021-30995-.html>. Use CLONE_ACL if available and working, falling back to cloning without it if it fails due to EINVAL. If the only problem with fclonefileat is that it would create the file with the wrong timestamp, or with too few permissions, do that but fix the timestamp and permissions afterwards, rather than falling back on a traditional copy.
* maint: fix some typos in commentsChuanGang Jiang2023-02-105-5/+5
| | | | | | | | | * NEWS: s/commmand/command/ * cfg.mk: Adjust old_NEWS_hash with `make update-NEWS-hash`. * src/expand-common.c: s/specifed/specified/ * src/pr.c: s/e.g/e.g./ * tests/misc/comm.pl: s/ouput/output/ Fixes https://bugs.gnu.org/61405
* cp: simplify infer_scantypePaul Eggert2023-02-091-10/+8
| | | | | | | | | * src/copy.c (infer_scantype): Do not set *SCAN_INFERENCE when returning a value other than LSEEK_SCANTYPE. This is just minor refactoring; it simplifies the code a bit. Callers are uneffected. doc: document --preserve=mode better
* tail: improve --follow=name with single non regular filesPádraig Brady2023-02-063-2/+17
| | | | | | | | * src/tail (tail_forever): Attempt to read() from non blocking single non regular file, which shouldn't block, but also read data even when the mtime doesn't change. * NEWS: Mention the improvement. * THANKS.in: Thanks for detailed testing.
* tail: fix support for -F with non seekable filesPádraig Brady2023-02-062-1/+6
| | | | | | | | | | This was seen to be an issue when following a symlink that was being updated to point to different underlying devices. * src/tail.c (recheck): Guard the lseek() call to only be performed for regular files. * NEWS: Mention the bug fix.
* cksum: add --raw option to output a binary digestPádraig Brady2023-02-069-10/+144
| | | | | | | | | | | | | | | | | | | | | | | | | --raw output is the most composable format, and also is a robust way to discard the file name without parsing (escaped) output. Examples: $ cksum --raw -a crc "$afile" | basenc --base16 4ACFC4F0 $ cksum --raw -a crc "$afile" | basenc --base2msbf 01001010110011111100010011110000 $ cksum --raw -a sha256 "$bfile" | basenc --base32 AAAAAAAADHLGRHAILLQWLAY6SNH7OY5OI2RKNQLSWPY3MCUM4JXQ==== * doc/coreutils.texi (cksum invocation): Describe the new feature. * src/digest.c (output_file): Inspect the new RAW_DIGEST global, and output the bytes directly if set. * src/cksum.c (output_crc): Likewise. * src/sum.c (output_bsd, output_sysv): Likewise. * tests/misc/cksum-raw.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the new feature.
* build: uptime: avoid issues on systems without utmp.hPádraig Brady2023-02-052-5/+4
| | | | | | | * src/uptime.c (print_uptime): Following gnulib commit 9041103 HAVE_UTMP_H will always be defined. Therefore key on whether the utmp.ut_type member is present. * boottime.m4 (GNULIB_BOOT_TIME): Assume utmp.h is present.
* maint: use alignasof, not stdalignPaul Eggert2023-02-043-3/+2
| | | | | | | * .gitignore: Update accordingly. * bootstrap.conf (gnulib_modules): Replace obsolescent stdalign with alignasof. * gl/modules/randread (Depends-on): Depend on alignasof, not stdalign.
* maint: prefer https: to git:Paul Eggert2023-02-043-4/+4
| | | | The idea is to defend against some adversary-in-the-middle attacks.
* build: update gnulib submodule to latestPaul Eggert2023-02-041-0/+0
|
* maint: avoid line length syntax check failurePádraig Brady2023-02-031-2/+2
| | | | * src/cp.c: Adjust source line to be <= 80 chars.
* tests: fix exit status check in cp -u testPádraig Brady2023-02-031-1/+1
| | | | | * tests/cp/preserve-link.sh: This should have been part of commit v9.1-134-g01503ce73.
* cksum: accept new option: --base64 (-b)Jim Meyering2023-01-316-53/+261
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * src/digest.c [HASH_ALGO_CKSUM]: Include "base64.h" [HASH_ALGO_CKSUM] (base64_digest): New global. [HASH_ALGO_CKSUM] (enum BASE64_DIGEST_OPTION): New enum. [HASH_ALGO_CKSUM] (long_options): Add "base64". (valid_digits): Rename from hex_digits, now taking an input length argument. Adjust callers. (bsd_split_3): Rename arg from hex_digits to digest. Add new *d_len parameter for length of extracted digest. Move "i" declaration down to first use. (split_3): Rename arg from hex_digits to digest. Add new *d_len parameter for length of extracted digest. Instead of relying on "known" length of digest to find the following must-be-whitespace byte, search for the first whitespace byte. [HASH_ALGO_CKSUM] (output_file): Handle base64_digest. [HASH_ALGO_CKSUM] (main): Set base64_digest. [HASH_ALGO_CKSUM] (b64_equal): New function. (hex_equal): New function, factored out of digest_check. (digest_check) Factored part into b64_equal and hex_equal. Rename local hex_digest to digest. * tests/misc/cksum-base64.pl: Add tests. * tests/local.mk (all_tests): Add to the list. * cfg.mk (_cksum): Define. (exclude_file_name_regexp--sc_prohibit_test_backticks): Exempt new test. (exclude_file_name_regexp--sc_long_lines): Likewise. * doc/coreutils.texi (cksum invocation): Document it. (md5sum invocation) [--check]: Mention digest encoding auto-detect. * NEWS (New Features): Mention this.
* doc: document --preserve=mode betterPaul Eggert2023-01-311-1/+8
| | | | * doc/coreutils.texi: Spruce up cp --preserve=mode doc.
* cp: improve --preserve usage docPaul Eggert2023-01-311-5/+9
| | | | * src/cp.c (usage): Improve description of --preserve.
* cp,mv: skipping due to -u is success, not failurePaul Eggert2023-01-314-8/+8
| | | | | | | | | | This reverts the previous change, so that when a file is skipped due to -u, this is not considered a failure. * doc/coreutils.texi: Document this. * src/copy.c (copy_internal): If --update says to skip, treat this as success instead of failure. * tests/mv/update.sh, tests/cp/slink-2-slink.sh: Revert previous change, to match reverted behavior.
* cp,ln,mv: when skipping exit with nonzero statusPaul Eggert2023-01-3111-30/+38
| | | | | | | | | | | * NEWS, doc/coreutils.texi: Document this. * src/copy.c (copy_internal): * src/ln.c (do_link): Return false when skipping action due to --interactive or --no-clobber. * tests/cp/cp-i.sh, tests/cp/preserve-link.sh: * tests/cp/slink-2-slink.sh, tests/mv/i-1.pl, tests/mv/i-5.sh: * tests/mv/mv-n.sh, tests/mv/update.sh: Adjust expectations of exit status to match revised behavior.
* tests: cksum: fix test to include more casesPádraig Brady2023-01-301-1/+1
| | | | | * tests/misc/cksum-c.sh: Fix typo which caused part of the test data to be ignored.
* digest.c: remove a duplicate variableJim Meyering2023-01-301-4/+1
| | | | | | * src/digest.c (digest_check): Locals n_misformatted_lines and n_improperly_formatted_lines were declared and set/incremented identically. Remove declaration of the latter. Use the other instead.
* build: avoid spurious failures due to lack of EGREP definitionJim Meyering2023-01-302-1/+2
| | | | | | * configure.ac: Use AC_PROG_EGREP, since many doc and test rules use $EGREP. * cfg.mk (sc_tests_list_consistency): Use grep -E, not $(EGREP) here.
* tests: ensure we fail if mv --no-copy crashesPádraig Brady2023-01-271-2/+2
| | | | | * tests/mv/no-copy.sh: Honor `make syntax` check and use the `returns_ 1 ...` pattern.
* mv: new option --no-copyPaul Eggert2023-01-277-17/+63
| | | | | | | | | | | Wishlist item from Mike Frysinger (Bug#61050). * src/copy.c (copy_internal): Do not fall back on copying if x->no_copy. * src/copy.h (struct cp_options): New member no_copy. * src/mv.c (NO_COPY_OPTION): New constant. (long_options, usage, main): Support --no-copy. * tests/mv/no-copy.sh: New test. * tests/local.mk (all_tests): Add it.
* maint: add lib/error.h to .gitignorePádraig Brady2023-01-171-0/+1
| | | | | * .gitignore: Add /lib/error.h as suggested by sc_gitignore_missing as a result of gnulib change 2886cca8.
* doc: csplit: more accurate --elide-empty-files helpPádraig Brady2023-01-171-1/+1
| | | | | | | | | * src/csplit.c (usage): Use "suppress" rather than "remove" when describing -z so it's more apparent that the effect is a particular numbered file is not created, rather than being removed later. I.e., don't suggest -z may induce gaps in file numbering. Reported at https://bugs.debian.org/1029103
* build: update gnulib submodule to latestPaul Eggert2023-01-132-43/+0
| | | | | * m4/xattr.m4: Remove. This file is now autogenerated by 'bootstrap', since it's now in Gnulib. (I did this part by hand.)
* copy: copy_file_range: handle ENOENT for CIFSPádraig Brady2023-01-082-0/+9
| | | | | | | | | * src/copy.c (sparse_copy): Fallback to standard copy upon ENOENT, which was seen intermittently across CIFS file systems. * NEWS: Mention the bug fix, though qualify it as an "issue" rather than a bug, as coreutils is likely only highlighting a CIFS bug in this case. Fixes https://bugs.gnu.org/60455
* maint: update .gitignorePaul Eggert2023-01-071-0/+2
| | | | | * .gitignore: Add confdefs*, conftest* (temporaries built by ‘configure’).
* maint: adjust to Gnulib macro renamingPaul Eggert2023-01-071-20/+20
| | | | | | | | | * src/local.mk (LDADD, copy_ldadd, remove_ldadd, src_sort_LDADD) (src_test_LDADD, copy_ldadd, src_date_LDADD, src_ginstall_LDADD) (src_ln_LDADD, src_ls_LDADD, src_mktemp_LDADD, src_pr_LDADD) (src_tac_LDADD, src_touch_LDADD, src_dd_LDADD, src_sleep_LDADD) (src_sort_LDADD, src_tail_LDADD, src_sort_LDADD, LDADD): Adjust to recent Gnulib changes.
* build: update gnulib submodule to latestPaul Eggert2023-01-071-0/+0
|
* copy: immediately fail with transient reflink errorsPádraig Brady2023-01-063-25/+80
| | | | | | | | | | | | | | | | * src/copy.c (handle_clone_fail): A new function refactored from copy_reg() to handle failures from FICLONE or fclonefileat(). Fail with all errors from FICLONE, unless they're from the set indicating the file system or file do not support the clone operation. Also fail with errors from fclonefileat() (dest_dest < 0) if they're from the set indicating a transient failure for the file. (copy_ref): Call handle_clone_fail() after fclonefileat() and FICLONE. (sparse_copy): Call the refactored is_CLONENOTSUP() which is now also used by the new handle_clone_fail() function. * NEWS: Mention the bug fix. Also mention explicitly the older --reflink=auto default change to aid searching. * cfg.mk: Adjust old_NEWS_hash with `make update-NEWS-hash`. Fixes https://bugs.gnu.org/60489
* all: further adjustments for new Ronna, Quetta SI prefixesPádraig Brady2023-01-069-13/+21
| | | | | | | | | | | | | | * src/dd.c (parse_integer): Support Q,R suffixes. * src/od.c (main): Likewise. * src/split.c (main): Likewise. * src/stdbuf.c (parse_size): Likewise. * src/truncate.c (main): Likewise. * src/sort.c (specify_size_size): Likewise. Also line length syntax check fix. * tests/misc/numfmt.pl: Adust top end large number checks to the new largest values. * doc/coreutils.texi (numfmt invocation): Add a numfmt example. * NEWS: Tweak to aid searchability.
* numfmt: add support for new SI prefixesPaul Eggert2023-01-0514-77/+126
| | | | | | | | | | | | | | | | | | | | | * src/dd, src/head.c, src/od.c, src/sort.c, src/stdbuf.c, src/tail.c: (usage): * src/system.h (emit_size_note): Mention new SI prefixes. * src/du.c (main): * src/head.c (head_file): * src/numfmt.c (suffix_power, suffix_power_char, prepare_padded_number): * src/shred.c (main): * src/sort.c (unit_order): * src/tail.c (parse_options): Support new SI prefixes. * src/numfmt.c (MAX_ACCEPTABLE_DIGITS): Increase to 33. (zero_and_valid_suffixes, valid_suffixes): New constants, with new SI prefixes. (valid_suffix, unit_to_umax): Use them. (prepare_padded_number): Diagnose "999Q" instead of "999Y". * tests/misc/numfmt.pl, tests/misc/sort.pl: Adjust tests to match new max.
* ls: adjust to Gnulib renamingPaul Eggert2023-01-051-1/+1
| | | | | * src/local.mk (src_ls_LDADD): In Gnulib, LIB_HAS_ACL was renamed to FILE_HAS_ACL_LIB.
* build: update gnulib submodule to latestPaul Eggert2023-01-051-0/+0
|
View Lorry Controller Status