summaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* split: advise the kernel of sequential access patternPádraig Brady2023-05-081-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | As split is often dealing with large files, ensure we indicate to the kernel our sequential access pattern. This was seen to operate 5% faster when reading from SSD, as tested with: dd bs=1M count=2K if=/dev/urandom of=big.in for split in split.orig split; do # Ensure big file is not cached dd of=big.in oflag=nocache conv=notrunc,fdatasync count=0 status=none # Test read efficiency CWD=$PWD; (cd /dev/shm && time $CWD/src/$split -n2 $CWD/big.in) done real 0m9.039s user 0m0.055s sys 0m3.510s real 0m8.568s user 0m0.056s sys 0m3.752s * src/split.c (main): Use fdadvise to help the kernel choose a more appropriate readahead buffer. * NEWS: Mention the improvement.
* pr: fix parsing of empty argumentsPádraig Brady2023-05-061-0/+6
| | | | | | | | | | | | | | | | | Before: $ pr --expand-tabs= pr: '-e' extra characters or invalid number in the argument: ‘SHELL=/bin/bash’: Value too large for defined data type After: $ pr --expand-tabs= pr: '-e': Invalid argument: ‘’ * src/pr.c (getoptarg): Ensure we don't parse beyond the end of an empty argument, thus outputting arbitrary stack info in subsequent error messages. Addresses https://bugs.debian.org/1035596
* cp: -p --parents: minor cleanup of previous patchPaul Eggert2023-05-051-25/+30
| | | | | | | This doesn’t change behavior; it just clarifies the code a bit. * src/cp.c (re_protect): New arg DST_SRC_NAME, for clarity, and so that we need to skip '/'s only once. Caller changed. Rename a couple of local variables to try to make things clearer.
* cp: -p --parents: fix failure to preserve permissions for absolute pathsPádraig Brady2023-05-031-5/+11
| | | | | | | * src/cp.c (re_protect): Ensure copy_acl() is passed an absolute path. * tests/cp/cp-parents.sh: Add a test case. * NEWS: Mention the bug. Fixes https://bugs.gnu.org/63245
* maint: remove redundant exit status handlingPádraig Brady2023-04-301-2/+1
| | | | | * src/numfmt.c: Remove redundant / confusing use of TIMEOUT_FAILURE.
* maint: suppress GCC 13 false alarmsPaul Eggert2023-04-264-0/+22
| | | | | | * src/csplit.c, src/fmt.c, src/make-prime-list.c, src/nohup.c: Add pragmas to pacify GCC 13 when coreutils is configured with --enable-gcc-warnings='expensive'.
* chmod: pacify GCC 13Paul Eggert2023-04-261-9/+8
| | | | | | | * src/chmod.c (main): Use xpalloc instead of X2REALLOC, and make the corresponding variables signed instead of unsigned. When reallocating the buffer, this grows it by a factor of 1.5, not 2. This also pacifies gcc -Wanalyzer-null-dereference.
* csplit: pacify GCC 13Paul Eggert2023-04-261-26/+15
| | | | | | * src/csplit.c (load_buffer): Refactor for clarity. This also xpacifies gcc -Wanalyzer-use-of-uninitialized-value. When reallocating the buffer, grow it by a factor of 1.5, not 2.
* uniq: be more specific when diagnosing read errorsPádraig Brady2023-04-261-1/+1
| | | | * src/uniq.c (check_file): Use the errno when diagnosing read errors.
* pr: fix infinite loop when double spacingPádraig Brady2023-04-251-1/+1
| | | | | | | | | | * src/pr.c (init_parameters): Ensure we avoid a 0 lines_per_body which was possible when adjusting for double spacing. That caused print_page() to always return true, causing an infinite loop. * tests/pr/pr-tests.pl: Add a test case. * NEWS: Mention the fix. Fixes https://bugs.debian.org/1034808
* copy: reduce verbosity of -i and -u with --verbosePádraig Brady2023-04-251-3/+3
| | | | | | | | | | | | | Since skipping of files is central to the operation of -i and -u, and with -u one may be updating few files out of many, reinstate the verbosity of this functionality as it was before 9.3. * src/copy.c (copy_internal): Only output "skipped" message with --debug. Also adjust so message never changes with --debug. * tests/cp/cp-i.sh: Adjust accordingly. * tests/mv/mv-n.sh: Likewise. * tests/cp/debug.sh: Add explicit test case for message. * NEWS: Mention the change in behavior.
* factor: diagnose errors reading the inputPádraig Brady2023-04-241-1/+6
| | | | | * src/factor.c (do_stdin): Exit with failure upon read errors. * NEWS: Mention the bug fix.
* numfmt: diagnose errors reading the inputPádraig Brady2023-04-241-1/+1
| | | | | * src/numfmt.c (main): Exit with failure upon read errors. * NEWS: Mention the bug fix.
* tsort: diagnose errors reading the inputPádraig Brady2023-04-241-1/+5
| | | | | * src/tsort.c (tsort): Check for errors after readtoken(). * NEWS: Mention the bug fix.
* cksum: fix failure to diagnose read errors with crc32Pádraig Brady2023-04-242-14/+2
| | | | | | | | The default crc32 mode fails to diagnose read errors. * src/cksum.c (cksum_slice8): Fix the check for read errors. (cksum_pclmul): Likewise. * NEWS: Mention the bug fix.
* install: support stripping files with a leading hyphenPádraig Brady2023-04-211-2/+7
| | | | | | | * src/install.c (strip): Prepend "./" to file names with a leading "-". * tests/install/strip-program.sh: Add a test case. * NEWS: Mention the bug fix. Reported in https://bugs.debian.org/1034429
* copy: --debug: indicate if NUL detection is used with SEEK_HOLEPádraig Brady2023-04-131-0/+4
| | | | | * src/copy.c (sparse_copy): With --sparse=always we also detect NULs in extents we're copying, so indicate this with --debug.
* build: fix _Noreturn compilation failurePádraig Brady2023-04-102-2/+4
| | | | | | | | | Fix a build failure seen on gcc 3.4 on Solaris 10 at least. * src/crctab.c: Ensure we include config.h for all compilation units. This is now required for new _Noreturn usage in gnulib for stdint.h. * src/cksum.c: Update generation code to ensure config.h included. * cfg.mk: Remove crctab.c exclusion from the config.h check.
* wc: ensure we update file offsetPádraig Brady2023-04-081-1/+4
| | | | | | | | * src/wc.c (wc): Update the offset when not reading, and do read if we can't update the offset. * tests/misc/wc-proc.sh: Add a test case. * NEWS: Mention the bug fix. Fixes https://bugs.gnu.org/61300
* cp,mv: issue "skipped" messages when skipping filesPádraig Brady2023-04-081-4/+23
| | | | | | | | | * NEWS: Mention the change in behavior to issue a "not replaced" error diagnostic with -n, and the "skipped" message with -v. * src/copy.c (copy_internal): Adjust to output the "skipped" messages depending on -i, -n, -u. * tests/cp/cp-i.sh: Adjust accordingly. * tests/mv/mv-n.sh: Likewise.
* cp,mv: add --update=none to always skip existing filesPádraig Brady2023-04-085-18/+131
| | | | | | | | | | | | | | | | | | | | | | | | | | Add --update=none which is equivalent to the --no-clobber behavior from before coreutils 9.2. I.e. existing files are unconditionally skipped, and them not being replaced does not affect the exit status. * src/copy.h [enum Update_type]: A new type to support parameters to the --update command line option. [enum Interactive]: Add I_ALWAYS_SKIP. * src/copy.c: Treat I_ALWAYS_SKIP like I_ALWAYS_NO (-n), except that we don't fail when skipping. * src/system.h (emit_update_parameters_note): A new function to output the description of the new --update parameters. * src/cp.c (main): Parse --update arguments, ensuring that -n takes precedence if specified. (usage): Describe the new option. Also allude that -u is related in the -n description. * src/mv.c: Accept the new --update parameters and update usage() accordingly. * doc/coreutils.texi (cp invocation): Describe the new --update parameters. Also reference --update from the --no-clobber description. (mv invocation): Likewise. * tests/mv/update.sh: Test the new parameters. * NEWS: Mention the new feature. Addresses https://bugs.gnu.org/62572
* wc: diagnose overflow of total countsPádraig Brady2023-03-311-7/+44
| | | | | | | | | * src/wc.c (wc): Use INT_ADD_WRAPV() to detect overflow. (main): Upon overflow, saturate the total, print a diagnostic, and set exit status. * tests/misc/wc-total.sh: Add a test case, which operates on BTRFS and 64 bit systems at least. Reported at https://bugs.debian.org/1027100
* dircolors: diagnose read errorsPádraig Brady2023-03-281-0/+5
| | | | | | | * NEWS: Mention the fix. * src/dircolors.c: Fail upon read error from getline(). * tests/misc/dircolors.sh: Add a new test. * tests/local.mk: Reference the new test.
* date: diagnose -f read errorsPaul Eggert2023-03-281-1/+3
| | | | * src/date.c (batch_convert): Diagnose read errors, fixing Bug#62497.
* cp: clarify commentaryPaul Eggert2023-03-251-11/+11
| | | | * src/copy.c: Make comments a bit clearer.
* copy: fix --reflink=auto to fallback in more casesPádraig Brady2023-03-241-27/+35
| | | | | | | | | | | | | | | | | On restricted systems like android or some containers, FICLONE could return EPERM, EACCES, or ENOTTY, which would have induced the command to fail to copy rather than falling back to a more standard copy. * src/copy.c (is_terminal_failure): A new function refactored from handle_clone_fail(). (is_CLONENOTSUP): Merge in the handling of EACCES, ENOTTY, EPERM as they also pertain to determination of whether cloning is supported if we ever use this function in that context. (handle_clone_fail): Use is_terminal_failure() in all cases, so that we assume a terminal failure in less errno cases. * NEWS: Mention the bug fix. Addresses https://bugs.gnu.org/62404
* cksum: fix reporting of failed checksPádraig Brady2023-03-231-2/+2
| | | | | | | | | | | | This applies to all checksumming utilities, where we incorrectly report all subsequent files as checking 'OK' once any file has passed a digest check. The exit status was not impacted, only the printed status. * src/digest.c (digest_check): Use the correct state variable to determine if the _current_ file has passed or not. * tests/misc/md5sum.pl: Add a test case. Fixes https://bugs.gnu.org/62403
* stty: ensure arbitrary data is not displayedPádraig Brady2023-03-191-2/+6
| | | | | | | | | | * src/stty.c (main): Use static structures to ensure they're initialized (to zero), so that random data is not displayed, or compared resulting in a inaccurate failure reported to users. This was seen on musl libc where some parts of the termios c_cc array were not initialized by tcgetattr(). Reported by Bruno Haible.
* doc: uniq: revert previous man page adjustmentPádraig Brady2023-03-181-3/+4
| | | | | * src/uniq.c (usage): -D doesn't take an argument so should be separated in the description.
* doc: uniq: use more standard description of -D long optionPádraig Brady2023-03-181-4/+3
| | | | | | * src/uniq.c: Describe -D and --all-repeated together, as with all other long options. Fixes https://bugs.gnu.org/62249
* maint: avoid discarded-qualifiers warnings with SELinux 3.5Bernhard Voelker2023-03-171-2/+2
| | | | | | | | | | | | | | | | | | | | Since SELinux version 3.5, the return value of context_str(3) is declared as const; see: https://github.com/SELinuxProject/selinux/commit/dd98fa322766 Therefore, GCC complains (here with -Werror): src/selinux.c: In function 'defaultcon': src/selinux.c:152:16: error: assignment discards 'const' qualifier \ from pointer target type [-Werror=discarded-qualifiers] 152 | if (!(constr = context_str (tcontext))) | ^ src/selinux.c: In function 'restorecon_private': src/selinux.c:252:16: error: assignment discards 'const' qualifier \ from pointer target type [-Werror=discarded-qualifiers] 252 | if (!(constr = context_str (tcontext))) | ^ * src/selinux.c (defaultcon): Define CONSTR as const. (restorecon_private): Likewise.
* cksum: fix --raw on 64 bit big endian systemsPádraig Brady2023-03-151-2/+4
| | | | | | | | | | * src/sum.c (output_bsd): On sparc64 for example, a crc of 0 was output due to casting an int variable to uint16_t and thus operating on the wrong end of the variable. Instead use explicit assignment to the narrower type to ensure we get the appropriate data. (output_sysv): Likewise. Reported by Bruno Haible.
* build: avoid -Wsometimes-uninitialized on macOS 12Bruno Haible2023-03-152-2/+2
| | | | | * src/pr.c (integer_overflow): Mark as _Noreturn. * src/test.c (beyond): Likewise.
* tee: avoid undefined behavior after fclose()Pádraig Brady2023-03-133-9/+11
| | | | | | | * iopoll.c (fclose_wait): Rename from confusing fclose_nonblock name. Also adjust to do no operations on the stream after fclose() as this is undefined. Instead use fflush() to determine EAGAIN status. (fwrite_wait): Renamed from confusing fwrite_nonblock name.
* doc: dircolors: better separate the sections in the databasePádraig Brady2023-03-131-0/+12
| | | | | | * src/dircolors.hin: Make the separate sections of the self documenting dircolors database more apparent, by adding heading comments, and appropriate separation.
* ls: --color: honor separate sequences for extension casesPádraig Brady2023-03-132-8/+66
| | | | | | | | | | | | | | | | | | | Following on from commit v8.29-45-g24053fbd8 which unconditionally used case insensitive extension matching, support selective case sensitive matching when there are separate extension cases defined with different display sequences. * src/dircolors.hin: Document how file name suffixes are matched. Note this is displayed with `dircolors --print-database` which the texi info recommends to use for details. * src/ls.c (parse_ls_color): Postprocess the list to mark entries for case sensitive matching, and also adjust so that unmatchable entries are more quickly ignored. (get_color_indicator): Use exact matching rather than case insensitive matching if so marked. * tests/ls/color-ext.sh: Add test cases. * NEWS: Mention the change in behavior. Addresses https://bugs.gnu.org/33123
* tee: support non blocking outputsPádraig Brady2023-03-083-24/+119
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Non blocking outputs can be seen for example when piping telnet through tee to a terminal. In that case telnet sets its input to nonblocking mode, which results in tee's output being nonblocking, in which case in may receive an EAGAIN error upon write(). The same issue was seen with mpirun. The following can be used to reproduce this locally at a terminal (in most invocations): $ { dd iflag=nonblock count=0 status=none; dd bs=10K count=10 if=/dev/zero status=none; } | tee || echo fail >/dev/tty * src/iopoll.c (iopoll_internal): A new function refactored from iopoll(), to also support a mode where we check the output descriptor is writeable. (iopoll): Now refactored to just call iopoll_internal(). (fwait_for_nonblocking_write): A new internal function which uses iopoll_internal() to wait for writeable output if an EAGAIN or EWOULDBLOCK was received. (fwrite_nonblock): An fwrite() wrapper which uses fwait_for_nonblocking_write() to handle EAGAIN. (fclose_nonblock): Likewise. src/iopoll.h: Add fclose_nonblock, fwrite_nonblock. src/tee.c: Call fclose_nonblock() and fwrite_nonblock wrappers, instead of the standard functions. * tests/misc/tee.sh: Add a test case. * NEWS: Mention the improvement. The idea was suggested by Kamil Dudka in https://bugzilla.redhat.com/1615467
* split: support split -n on larger pipe inputPaul Eggert2023-03-071-35/+54
| | | | | | | | | * bootstrap.conf (gnulib_modules): Add free-posix, tmpfile. * src/split.c (copy_to_tmpfile): New function. (input_file_size): Use it to split larger files when sizes cannot easily be determined via fstat or lseek. See Bug#61386#235. * tests/split/l-chunk.sh: Mark tests of /dev/zero as very expensive since they exhaust /tmp.
* tee: fix a crash with unwriteable filesPádraig Brady2023-03-071-3/+5
| | | | | | | | This was introduced recently with commit v9.1-166-g6b12e62d9 * src/tee.c (tee_files): Check the return from fopen() before passing to fileno() etc. * tests/misc/tee.sh: Add a test case.
* maint: pacify ‘make syntax-check’Paul Eggert2023-03-061-3/+3
| | | | | | | | Problem reported by Pádraig Brady (Bug#61386#226). * src/split.c (parse_chunk): Use die instead of error. (main): Quote a string. * tests/local.mk (all_root_tests): Move du/apparent.sh from here ... (all_tests): ... to here.
* tail,tee: avoid issues with many files on systems without pollPádraig Brady2023-03-061-0/+6
| | | | | * src/iopoll.c (iopoll): Protect the call to select against passing in a descriptor larger than FD_SETSIZE.
* du: --apparent counts only symlinks and regularPaul Eggert2023-03-041-4/+4
| | | | | | | | Problem reported by Christoph Anton Mitterer (Bug#61884). * src/du.c (process_file): When counting apparent sizes, count only usable st_size members. * tests/du/apparent.sh: New file. * tests/local.mk (all_root_tests): Add it.
* split: tune for when creating output filesPaul Eggert2023-03-041-3/+9
| | | | | | | | * src/split.c (create): Avoid fstat + ftruncate in the usual case where the output file does not already exist, by trying to create it with O_EXCL first. This costs a failed open in the unusual case where the output file already exists, but that’s OK.
* split: style fixPaul Eggert2023-03-041-1/+3
| | | | * src/split.c (ofile_open): Avoid ‘if (! (a = b))’ style.
* split: prefer signed integers to size_tPaul Eggert2023-03-041-53/+52
| | | | | | | | | | | | This allows for better runtime checking with gcc -fsanitize=undefined. * src/split.c: Include idx.h. (open_pipes_alloc, n_open_pipes, suffix_length) (set_suffix_length, input_file_size, sufindex, outbase_length) (outfile_length, addsuf_length, create, cwrite, bytes_split) (lines_split, line_bytes_split, lines_chunk_split) (bytes_chunk_extract, ofile_open, lines_rr, main): Prefer signed integers (typically idx_t) to size_t.
* split: handle large numbers betterPaul Eggert2023-03-041-63/+73
| | | | | | | | | | | | | | | | | | | | | | | Prefer signed types to uintmax_t, as this allows for better runtime checking with gcc -fsanitize=undefined. Also, when an integer overflows just use the maximal value when the code will do the right thing anyway. * src/split.c (set_suffix_length, bytes_split, lines_split) (line_bytes_split, lines_chunk_split, bytes_chunk_extract) (lines_rr, parse_chunk, main): Prefer a signed type (typically intmax_t) to uintmax_t. (strtoint_die): New function. (OVERFLOW_OK): New macro. Use it elsewhere, where we now allow LONGINT_OVERFLOW because the code then does the right thing on all practical platforms (they have int wide enough so that it cannot be practically exhausted). We can do this now that we can safely assume intmax_t has at least 64 bits. (parse_n_units): New function. (parse_chunk, main): Use it. (main): Do not worry about integer overflow when the code will do the right thing anyway with the extreme value. Just use the extreme value. * tests/split/fail.sh: Adjust to match new behavior.
* split: prefer ssize_t for read resultPaul Eggert2023-03-041-14/+15
| | | | | | | * src/split.c (bytes_split, lines_chunk_split) (bytes_chunk_extract, main): Prefer ssize_t to size_t when representing the return value of ‘read’. Use a negative value instead of SIZE_MAX to indicate a missing value.
* split: be more careful about buffer sizesPaul Eggert2023-03-041-28/+31
| | | | | | | | | * src/split.c: Include sys-limits.h, not safe-read.h. (input_file_size, bytes_split, lines_split, line_bytes_split) (lines_chunk_split, bytes_chunk_extract, lines_rr): Call read, not safe_read, since safe_read no longer buys us anything. (main): Reject outlandish buffer sizes right away, rather than allocating huge buffers and never using them.
* split: minor -1 / 0 refactorPaul Eggert2023-03-041-3/+3
| | | | | * src/split.c (create, bytes_split, ofile_open): Prefer comparing to 0 to comparing to -1.
* split: don’t worry about ECHILDPaul Eggert2023-03-041-3/+3
| | | | | | * src/split.c (closeout): There should be no need for a special case for ECHILD, since we never wait for the same child twice. Simplify with this in mind.
View Lorry Controller Status