summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2015-11-25doc: provide an example for prefixing all tail output linesPádraig Brady
* doc/coreutils.texi (tail invocation): Provide an example using awk to convert tail ==> file <== headers to file: prefixes on each line. Suggested by Stephen Shirley.
2015-11-24copy: fix copying of extents beyond the apparent file sizeDmitry Monakhov
fallocate can allocate extents beyond EOF via FALLOC_FL_KEEP_SIZE. Where there is a gap (hole) between the extents, and EOF is within that gap, the final hole wasn't reproduced, resulting in silent data corruption in the copied file (size too small). * src/copy.c (extent_copy): Ensure we don't process extents beyond the apparent file size, since processing and allocating those is not currently supported. * tests/cp/fiemap-extents.sh: Renamed from tests/cp/fiemap-empty.sh and re-enable parts checking the extents at and beyond EOF. * tests/local.mk: Reference the renamed test. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/21790
2015-11-23md5sum,sha*sum: ensure --ignore-missing fails when no file verifiedPádraig Brady
* src/md5sum.c (digest_check): Update a matched_checksums bool upon matched checksum, and fail (loudly unless --status is specified) if there were no matches. Also change properly_formatted_lines to a bool while at it since we don't need to track the plurality. * tests/misc/md5sum.pl: Add a test case. Suggested by Jim Meyering.
2015-11-23md5sum,sha*sum: add --ignore-missing for checking a subset of filesLuther Thompson
* doc/coreutils.texi (md5sum invocation): Document the new option. * src/md5sum.c (digest_file): Return an empty digest to indicate a missing file. (digest_check): Don't fail or output status given an empty checksum. (usage): Document the new option. (main): Process and validate the new option. * tests/misc/md5sum.pl: Add new test cases. * NEWS: Mention the new feature. Fixes http://bugs.gnu.org/15604
2015-11-23maint: allow 'sha*sum:' tag in commit summariesPádraig Brady
To support "md5sum,sha*sum:" Suggested by Bernhard Voelker.
2015-11-21doc: give a tee example for combining process substitution outputsPádraig Brady
This can be useful if you want to further process data from process substitutions. For example: datagen | tee >(md5sum --tag) > >(sha256sum --tag) | sort * doc/coreutils.texi (tee invocation): Mention that -p is useful with pipes that may not consume all data. Add an example, similar to the one above. * THANKS.in: Add Jirka Hladky.
2015-11-21tests: avoid false failure on older NFS implementationsPádraig Brady
* tests/ls/stat-failed.sh: Skip the test if 'd' is returned as the type, and document where this was seen. Also flag failure to write small temp files during the test as an error rather than a failure. Fixes http://bugs.gnu.org/21130
2015-11-19build: fix VPATH build with --disable-dependency-trackingPádraig Brady
* src/local.mk: Run `mkdir -p src` in all our explicit rules, as in a VPATH build the src/ dir is only created at configure time as a side effect of dependency tracking generation.
2015-11-17scripts: update versions of gettext and libtoolJim Meyering
2015-11-17maint: remove unmaintained file, c99-to-c89.diffJim Meyering
* src/c99-to-c89.diff: Remove file. * src/local.mk (EXTRA_DIST): Remove it from this list. * README (Pre-C99 build failure): Update section.
2015-11-12csplit: check and report fwrite errors with errnoAssaf Gordon
discussed in: http://lists.gnu.org/archive/html/coreutils/2015-10/msg00091.html * src/csplit.c: (save_line_to_file): check fwrite failures, report and exit immediately instead of deferring to 'close_output'. * tests/misc/csplit-io-err.sh: test fwrite failure using LD_PRELOAD. * tests/local.mk: add new test.
2015-11-11stat: improve support for new Linux pseudo FS and ACFSPádraig Brady
* src/stat.c (human_fstype): Add new file system ID definitions. * NEWS: Mention the improvement.
2015-11-10tests: ensure programs are built before testingPádraig Brady
programs may not be built due to missing system dependencies, or any program can be excluded at configure time with --enable-no-install-program. So ensure we're not testing the system versions in these cases. * init.cfg (print_ver_): Call require_built_ first. * tests/misc/tty-eof.pl: Skip programs not built. * tests/Coreutils.pm (run_tests): Likewise. * tests/misc/ls-misc.pl: Use 'env test' rather than abs path. * tests/misc/test-diag.pl: Likewise. * tests/local.mk: Adjust include order for dependencies. * tests/misc/arch.sh: Remove redundant calls to require_built_. * tests/misc/chroot-fail.sh: Likewise. * tests/misc/groups-dash.sh: Likewise. * tests/misc/groups-version.sh: Likewise. * tests/misc/stdbuf.sh: Likewise. * tests/cp/acl.sh: Remove problematic call to print_ver_ [gs]etfacl. * tests/mv/acl.sh: Likewise. * cfg.mk (sc_env_test_dependencies): A new syntax check to enforce specifying dependencies with print_ver_ for programs specified through the env command. * du/bigtime.sh: Add new print_ver_ dependencies. * du/max-depth.sh: Likewise. * dd/ascii.sh: Likewise. * tests/ls/capability.sh: Likewise. * tests/ls/root-rel-symlink-color.sh: Likewise. * tests/misc/chroot-fail.sh: Likewise. * tests/misc/readlink-fp-loop.sh: Likewise. * tests/misc/sort-debug-keys.sh: Likewise. * tests/readlink/can-e.sh: Likewise. * tests/readlink/can-f.sh: Likewise. * tests/readlink/can-m.sh: Likewise. * tests/tail-2/inotify-race.sh: Likewise. * tests/tail-2/inotify-race2.sh: Likewise. * tests/touch/no-create-missing.sh: Likewise. * tests/touch/no-dereference.sh: Likewise. * tests/misc/printenv.sh: Tweak to avoid syntax check trigger. * tests/misc/help-version.sh: Likewise. * tests/misc/yes.sh: Likewise. * tests/misc/printf-quote.sh: Use previously unused $prog. * configure.ac (EXTRA_MANS): Add $gl_no_install_prog to the list so that check-x-vs-1 syntax check is satisfied.
2015-11-10maint: use standard spacing in shebang line in testsPádraig Brady
It's better to be consistent even though spacing is insignificant: http://www.in-ulm.de/~mascheck/various/shebang/#blankrequired
2015-11-10tests: cleanup trapping of signal namesPádraig Brady
A side effect of this cleanup is we no longer depend on our own kill command being built. * init.cfg (require_trap_signame_): A new function to verify that the shell supports specifying traps by signal name. (require_kill_group_): A new function to ensure the shell supports sending a signal to a group. * tests/du/move-dir-while-traversing.sh: Ensure trap supports signal names. * tests/misc/stty-invalid.sh: Likewise. * tests/misc/stty-pairs.sh: Likewise. * tests/misc/stty-row-col.sh: Likewise. * tests/misc/stty.sh: Likewise. * tests/misc/sort-compress.sh: Likewise. Also simplify trap call. * tests/install/trap.sh: Likewise. * tests/misc/timeout.sh: Likewise. * tests/dd/stats.sh: Likewise. Also use default kill command. * tests/misc/timeout-group.sh: Likewise.
2015-11-06tests: fix dirent d_type support verificationPádraig Brady
* init.cfg (require_dirent_d_type_): Don't use df -x to exclude XFS, since this depends on a correct mtab which is brittle and often not correct within chroots. * tests/d_type-check: Check also the d_type of files, which excludes XFS appropriately. Specify all argument and return types to avoid truncated pointers being passed, which skipped the test due to crashes on x86_64 at least. Simplify the C library lookup by reusing the interpreter's. chroot issue reported at https://bugzilla.redhat.com/1263341
2015-11-04maint: add a syntax check to avoid unstyled quotingPádraig Brady
* src/paste.c (main): Use our styled wrapper for quotearg_colon(). * cfg.mk (sc_prohibit-quotearg): A new syntax check to avoid future uses of unstyled quotearg to one of the internal slots, and thus destined for diagnostic output.
2015-11-04paste: avoid confusing backslash quoting in diagnosticPádraig Brady
* src/paste.c (main): Setting the quoting style to "escape" went against the intent of the comment about presenting doubled backslashes to the user. Instead use "c-maybe" which is the only mode which avoids doubled backslashes, and provides protection against arbitrary control characters. * tests/misc/paste.pl: Adjust accordingly.
2015-11-04ls: default to --quoting=shell-escape for output to terminalPádraig Brady
* src/ls.c (decode_switches): Set "shell-escape" if isatty(). * doc/coreutils.texi (ls invocation): Update the defaults description. * NEWS: Mention the change in behavior. It should not have backwards compat issues, but mentioning here just in case.
2015-11-04printf: support the %q format to quote for shellPádraig Brady
* src/printf.c (usage): Mention the new format. (print_formatted): Handle the quoting by calling out to the quotearg module with "shell-escape" mode. * doc/coreutils.texi (printf invocation): Document %q. * tests/misc/printf-quote.sh: New test. * tests/local.mk: Reference new test. * NEWS: Mention the new feature.
2015-11-04ls: avoid redundant processing when already escapingPádraig Brady
This is mainly noticeable when the multi-byte code within ls.c is triggered by multi-byte quotes. $ seq 200000 | xargs touch $ time ls-old -U --quoting=locale --hide-control-chars >/dev/null real 0m0.483s $ time ls-new -U --quoting=locale --hide-control-chars >/dev/null real 0m0.430s * src/ls.c (quote_name): Avoid rescanning the output looking for unprintable chars when we know the quoting mode already escapes them. * tests/misc/ls-misc.pl: Add tests for all quoting modes, with and without -q, to verify this assumption.
2015-11-04ls: document and test new shell-escape quotingPádraig Brady
* doc/coreutils.texi (ls invocation): Describe the new 'shell-escape' and 'shell-escape-always' quoting options. * src/ls.c (usage): Mention the new quoting options. * tests/misc/ls-misc.pl: Add a test for 'shell-escape'
2015-11-04test: use consistent quotingPádraig Brady
* src/test.c (test_syntax_error): Reuse verror() rather than open coding the error output format. (term): Don't hardcode '' quoting. (main): Likewise.
2015-11-04all: avoid quoting file names when possiblePádraig Brady
Quote file names using the "shell-escape" or "shell-escape-always" methods, which quote as appropriate for most shells, and better support copy and paste of presented names. The "always" variant is used when the file name is embedded in an error message with surrounding spaces. * cfg.mk (sc_error_shell_quotes): A new syntax check rule to suggest quotef() where appropriate. (sc_error_shell_always_quotes): Likewise for quoteaf(). * src/system.h (quotef): A new define to apply shell quoting when needed. I.E. when shell character or ':' is present. (quoteaf): Likewise, but always quote. * src/*.c: Use quotef() and quoteaf() rather than quote() where appropriate. * tests/: Adjust accordingly.
2015-11-04build: update gnulib submodule to latestPádraig Brady
Includes support for "shell-escape" from quotearg
2015-11-04md5sum: ensure a single status line per filePádraig Brady
* src/md5sum.c: Use the same file name escaping method used when generating and checking checksums. I.E. ensure a single line per file by starting the line with '\' for any file name containing '\n' and replacing those with "\\n". * NEWS: Move the item from changes in behavior to improvements, since this is no longer a backwards incompat change when processing stdout status messages. * tests/misc/md5sum.pl: Remove quotes from expected status output. * tests/misc/sha1sum.pl: Likewise.
2015-11-04all: replace most uses of quotearg_colon() with quote()Pádraig Brady
Related to commit v8.24-61-g6796698 this provides more consistent quoting, as quotearg_colon() defaults to "literal" quoting by default, while quote() provides appropriate quoting for diagnostics by default. * gl/modules/randread: Depend on quote module rather than quotearg. * gl/lib/randread.c: Used quote() not quotearg_colon(). * src/: Likewise. * src/shred.c: Likewise. Also avoid unnecessary quoting introducing overhead when wiping names. * cfg.mk: Relax the matching expression to allow "qname" variables as used in shred.c to satisfy the check. * tests/: Adjust accordingly.
2015-11-02doc: fix texinfo for short options taking a parameterPádraig Brady
* doc/coreutils.texi: (tail invocation): Add missing -s, along with the existing --sleep-interval description. (csplit invocation): s/--suffix/--suffix-format/. (head invocation): Use same variable (COUNT) for -n and --head-count. (seq invocation): Add opindex items for all options. (ptx invocation): Likewise. Fix typo s/--flac-truncation/--flag-truncation/. (touch invocation): State explicitly that -d takes a parameter, which also indicates that an '=' is not to be used for the short option syntax. (ls invocation): Likewise for the -w option. Fixes http://bugs.gnu.org/21809
2015-10-29maint: allow 'all:' tag in commit summariesPádraig Brady
To tag changes that are user visible and affect all (or many) commands.
2015-10-28doc: clarify iso-8601 formats used by ls and duPádraig Brady
* doc/coreutils.texi (du invocation): Remove 'like' from mentions of ISO-8601 as the components are individually conformant. (ls invocation): Likewise, except for --time-style=iso for recent files, where the MM-DD component is not a valid ISO-8601 timestamp.
2015-10-27date: use extended format timezone for --iso-8601Pádraig Brady
* src/date.c (main): Use %:z rather than %z with --iso-8601 as the standard states to consistently use extended format. Note either format can be parsed by date. * tests/misc/date.pl: Adjust accordingly. * doc/coreutils.texi (du invocation): Clarify that "iso" time styles are only similar to ISO-8601. (ls invocation): Likewise. (date invocation): Adjust the comment stating that only --rfc-3339 output can be parsed by date(1). * NEWS: Mention the change in behavior. Reported at http://bugs.debian.org/799479
2015-10-27doc: reference related commands from users(1) and groups(1)Pádraig Brady
* man/users.x: See also getent, who * man/groups.x: See also getent Addresses http://bugs.gnu.org/21735
2015-10-27copy,dd: simplify and optimize NUL bytes detectionPádraig Brady
* src/factor.c: Move LIKELY() definition to... * src/system.h: ...here. (is_nul): Reimplement with a version that doesn't require a sentinel after the buffer, and which calls down to (the system optimized) memcmp. Performance analyzed at http://rusty.ozlabs.org/?p=560 * src/dd.c (alloc_obuf): Simplify the is_nul() call by not needing to write the sentinel. * src/copy.c (sparse_copy): Likewise. (copy_reg): Simplify the buffer allocation by avoiding consideration of the sentinel in the buffer size calculation.
2015-10-27all: quote string arguments in error messagesPádraig Brady
These strings are often file names or other user specified parameters, which can give confusing errors in the presence of unexpected characters for example. * cfg.mk (sc_error_quotes): A new syntax check rule. * src/*.c: Wrap error() string arguments with quote(). * tests/: Adjust accordingly. * NEWS: Mention the improvement.
2015-10-27md5sum: quote all printed file namesPádraig Brady
This is especially significant when using --check with files generated on a windows system, where the \r characters produce corrupted and confusing error messages. This also ensures status messages are output on a single line. * src/md5sum.c: Use quote() for printed file names. * tests/misc/md5sum.pl: Adjust accordingly. * NEWS: Mention the change in behavior. Fixes http://bugs.gnu.org/21757
2015-10-21ls: fix off by one error when determining max display columnsPádraig Brady
* src/ls.c (main): Account for the first column not including a separator when calculating max_idx. * tests/ls/w-option.sh: Add a test case. * NEWS: Mention the bug fix.
2015-10-21ls: allow -w0 to mean no limit on line lengthPádraig Brady
* src/ls.c (print_with_separator): Renamed from print_with_commas, and parameterized to accept the separator to print. Also fix an edge case where '\n' not printed when the POS variable overflows SIZE_MAX. (print_current_files): Degenerate -x and -C to using the cheaper print_with_separator() in the -w0 case. * doc/coreutils.texi (ls invocation): Document the new feature. * tests/ls/w-option.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/21325
2015-10-19ls: detect terminal color support using glob patternsPádraig Brady
* src/ls.c (know_term_type): Corresponding to commit v8.24-48-gc249a5a, use fnmatch to inspect the dircolors database. Noticed due to failing tests/ls/color-{dtype-dir,term}.sh tests.
2015-10-19tests: adjust recent change to csplit VM limitPádraig Brady
* tests/misc/csplit-heap.sh: A little more memory is required for the full run case. Noticed with --enable-single-binary.
2015-10-19maint: fixes to support improved sc_tight_scopePádraig Brady
The gnulib provided sc_tight_scope target was ineffective, as it was checking against an invalid blank regular expression, and thus ignoring any extern function issues. This is now fixed up in gnulib, and so we need to fix our scoping issues before the next gnulib update. * cfg.mk: Setup and document the tight_scope config variables appropriately. * src/selinux.h: Since declared in *_SOURCES, use the two line form for the extern function declarations. * src/set-fields.h: Add the extern declarations, and since declared in noinst_HEADERS use the single line form.
2015-10-19tests: avoid failure when auto selecting factor testsPádraig Brady
* tests/factor/run.sh: If this template is found through `grep -El "print_ver_.* factor"` for example, then just skip it.
2015-10-19factor: remove unreachable SQUFOF code at compile timePádraig Brady
It was a little confusing as to whether the SQUFOF algorithm was enabled, and in fact there were no options available to enable it. Therefore clarify the 3 configurable behaviors for the code to 3 defines at the top of the program, and only include the SQUFOF code if enabled at compile time. $ size src/factor-before text data bss 93997 1412 2504 $ size src/factor-after text data bss 87885 1404 2504 * src/factor.c: Only include the SQUFOF factor code when enabled via the USE_SQUFOF define. * doc/coreutils.texi (factor invocation): Update note about factor limits, as we can factor 128 bit numbers without GMP.
2015-10-19doc: reference shuf(1) from the sort (-R) man pagePádraig Brady
* man/sort.x: Cross reference with shuf(1). * src/sort.c (usage): Mention shuf(1) with -R option. Suggested in http://bugs.debian.org/641166
2015-10-18maint: avoid duplicate test runsBernhard Voelker
* tests/local.mk (all_tests): Remove the tests 'tests/id/setgid.sh' and 'tests/mkdir/smack-root.sh' because they are mentioned in the 'all_root_tests' list; these tests are skipped anyway during a non-root run because flagged with 'require_root_'.
2015-10-16doc: remove obsolete performance commentPaul Eggert
sha512sum can be faster than sha256sum. E.g., ‘dd if=/dev/zero bs=1024k count=1024 | time sha256sum’ reports 8.16 user CPU seconds on my host, whereas sha512sum consumes 5.45 seconds (Fedora x86-64 on an AMD Phenom II X4 910e). Although sha512sum is still considerably slower on x86, a good chunk of uses are on 64-bit hosts and anyway there’s little point to scaring people away from sha512sum nowadays. * doc/coreutils.texi (sha2 utilities): Remove obsolete comment.
2015-10-16maint: avoid uniq.c warning from bleeding-edge gcc's -Wstrict-overflowJim Meyering
* src/uniq.c (main): Make the type of "nfiles" unsigned, to avoid a brand new warning from a gcc I built from today's sources (gcc version 6.0.0 20151015 (experimental) (GCC)): src/uniq.c:523:14: error: assuming signed overflow does not occur \ when simplifying conditional to constant [-Werror=strict-overflow] if (nfiles == 2) ^
2015-10-16dircolors: support globbing of TERM entriesPádraig Brady
* src/dircolors.c (dc_parse_stream): Support globbing of TERM entries, to allow entries like "TERM *256color*" for example. * src/dircolors.hin: Reduce the internal list with globbing. * tests/misc/dircolors.pl: New test cases. * NEWS: Mention the improvement.
2015-10-15tests: avoid false failure in rm/r-root.sh with gdb warningsPádraig Brady
* tests/rm/r-root.sh: Skip the test if there are gdb warnings that will impact further stderr checks. For example some buggy gdb versions may report "Got object file from memory but can't read symbols: File truncated". Also fix an incorrect stderr check from the previous change. Reported by Bernhard Voelker.
2015-10-15tests: avoid false failure in rm/r-root.sh under loadPádraig Brady
* tests/rm/r-root.sh: Use gdb rather than timeout(1) as the last resort protection against unlinkat() calls. The timeout of 2s was susceptible to false positives under load, and gdb is stronger protection in any case. We remove the "expensive" tag on this test also since it should be robust. Reported by Jim Meyering.
2015-10-15tests: avoid false failure in a tail test under loadPádraig Brady
* tests/tail-2/follow-stdin.sh: Use the standard tail testing framework to avoid the race seen under very high load, and also test the non inotify case. Reported by Jim Meyering