summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2014-05-25build: libstdbuf.so: avoid new OS X link failureJim Meyering
* src/local.mk (src_libstdbuf_so_LDADD): Add $(LIBINTL), to avoid link failure on OS X.
2014-05-25cat,cp,split: use a larger buffer for copyingJim Meyering
* src/ioblksize.h (IO_BUFSIZE): Double the size, to 128KB. Add four more columns to the blksize-vs-bandwidth table
2014-05-24doc: use nicer quotesPaul Eggert
* doc/coreutils.texi: Add "@documentencoding UTF-8".
2014-05-23stat,tail: improve support for LogFS and ConfigFSPádraig Brady
* src/stat.c (human_fstype): Add new file system ID definitions. * NEWS: Mention the improvement.
2014-05-22maint: enforce consistent width and case of file system constantsPádraig Brady
* src/stat.c (human_fstype): Adjust a couple of existing constants to be a consistent width and capitalization so that the src/fs-magic-compare target works without reporting false positives. * cfg.mk (sc_fs-magic-compare): A new syntax check to enforce this. Improved by: Jim Meyering
2014-05-21stdbuf: support compilers other than __GNUC__Pádraig Brady
__SUNPRO_C >= 0x590 /*12.0*/ supports __attribute((constructor)) required by stdbuf, so use a more direct check for this. Note ensure that --libexecdir is set to the appropriate install location for libstdbuf.so so that stdbuf works when installed on the system like it does when running tests in the build directory. * configure.ac (stdbuf_supported): Use a test prog to determine support. * src/libstdbuf.c (stdbuf): Define appropriately for non GCC compilers, and provide early feedback (compilation warning) if trying to compile libstdbuf without the necessary support. * src/stdbuf.c (set_LD_PRELOAD): Add a note on having stdbuf look for libstdbuf.so in the default lib search path. * cfg.mk (sc_prohibit-gl-attributes): Adjust so we can exclude libstdbuf.so from prohibiting '__attribute', since we want this form to avoid silently eliding this required attribute on non GCC. Reported and tested by Rich Burridge.
2014-05-21tests: use chroot --user rather than internal setuidgid toolPádraig Brady
* init.cfg (require_root_): Adjust to use chroot, and make `require_built_ chroot` implicit when chroot used in the test. * po/POTFILES.in: Remove reference to setuidgid tool. * src/.gitignore: Likewise. * src/local.mk: Likewise. * src/setuidgid.c: Remove. * tests/cp/preserve-gid.sh: s/setuidgid/chroot --user/. * tests/cp/special-bits.sh: Likewise. * tests/id/setgid.sh: Likewise. * tests/misc/truncate-owned-by-other.sh * tests/mv/sticky-to-xpart.sh: Likewise. * tests/rm/fail-2eperm.sh: Likewise. * tests/rm/no-give-up.sh: Likewise. * tests/touch/now-owned-by-other.sh: Likewise. * tests/misc/chroot-fail.sh: Skip if chroot not built.
2014-05-21chroot: make changing root check more robustPádraig Brady
* src/chroot.c (is_root): A new helper function to determine if the passed argument is the root directory based on inode comparison. (main): Use the new helper rather than comparing strings. * tests/misc/chroot-fail.sh: Add cases for alternative root paths.
2014-05-21chroot: exit immediately upon failurePádraig Brady
* src/chroot.c (main): Consistently exit with failure status immediately upon hitting a terminal issue, rather than diagnosing multiple issues lest users think previous failing actions are optional.
2014-05-21chroot: with --userspec clear root's supplemental groupsPádraig Brady
It's dangerous and confusing to leave root's supplemental groups in place when specifying other users with --userspec. In the edge case that that is desired one can explicitly specify --groups. Also we implicitly set the system defined supplemental groups for a user. The existing mechanism where supplemental groups needed to be explicitly specified is confusing and not general when the lookup needs to be done within the chroot. Also we extend the --groups syntax slightly to allow clearing the set of supplementary groups using --groups=''. * src/chroot.c (setgroups): On systems without supplemental groups, clearing then is a noop and so should return success. (main): Lookup the primary GID with getpwuid() when just a numeric uid is specified, and also infer the USERNAME from this call, needed when we're later looking up the supplemental groups for a user. Support clearing supplemental groups, either implicitly for unknown users, or explicitly when --groups='' is specified. * tests/misc/chroot-credentials.sh: Various new test cases * doc/coreutils.texi (chroot invocation): Adjust for the new behavior. * NEWS: Mention the change in behavior.
2014-05-16chroot: don't chdir() if not changing rootPádraig Brady
This allows chroot to be used as a light weight tool to change user identification for a command, while not changing the current working directory. It also makes `chroot / true` consistently succeed on all platforms for non root users. * src/chroot.c (main): If the same root is specified. i.e. '/' then don't change the current working directory, and avoid the overhead of the other redundant calls. * tests/misc/chroot-fail.sh: Remove failure guard previously needed on some systems. Also add an explicit case to ensure we don't change directory. * NEWS: Mention the change in behavior.
2014-05-13maint: avoid clang -Wtautological-constant-out-of-range-compare warningPádraig Brady
* src/df.c (decode_output_arg): Use only enum constants to avoid clang "warning: comparison of constant -1 with expression of type 'display_field_t' is always false"
2014-05-13df: ignore non file system entries in /proc/mountsPádraig Brady
Linux with network namespaces contains entries in /proc/mounts like: proc net:[4026532464] proc rw,nosuid,nodev,noexec,relatime 0 0 resulting in a failure to stat 'net:[...]', inducing a warning and an exit with failure status. * src/df.c (get_dev): Ignore all relative mount points. * tests/df/skip-duplicates.sh: Add an entry to test relative dirs.
2014-05-13df: fix handling of symlinks in mount listPádraig Brady
The symlink handling in commit v8.21-172-g33660b4 was incomplete in the case where there were symlinks in the mount list itself. For example, in the case where /dev/mapper/fedora-home was in the mount list and that in turn was a symlink to /dev/dm-2, we have: before> df --out=source /dev/mapper/fedora-home devtmpfs after > df --out=source /dev/mapper/fedora-home /dev/mapper/fedora-home * src/df.c (get_disk): Compare canonicalized device names from the mount list. Note we still display the non canonicalized name, even if longer, as we assume that is the most representative. * tests/df/df-symlink.sh: This could theoretically fail on some systems depending on the content of the mount list, but adjust to fail on any system where symlinks are present in the mount list for the current dir.
2014-05-13df: also deduplicate virtual file systemsPádraig Brady
* src/df.c (filter_mountlist): Remove the constraint that a '/' needs to be in the device name for a mount entry to be considered for deduplication. Virtual file systems also have storage associated with them (like tmpfs for example), and thus need to be deduplicated since they will be shown in the default df output and subject to --total processing also. * test/df/skip-duplicates.sh: Add a test to ensure we deduplicate all entries, even for virtual file systems. Also avoid possible length operations on many remote file systems in the initial check of df operation. Also avoid the assumption that "/root" is on the same file system as "/". * NEWS: Mention the change in behavior.
2014-05-13tests: fix spurious failure with leading spaces in file namesPádraig Brady
* tests/ls/stat-vs-dirent.sh: This test lists all parent directories, and would spuriously fail if any of those had a file name with a leading space as the first entry. There is only ever a single space between the right aligned inode number and the file name, so process accordingly.
2014-05-12build: avoid bootstrap error with gettext 0.18.3.1Assaf Gordon
* bootstrap: Create critical bootstrap files for autopoint, before gnulib re-generates them, avoiding the issue. See: http://lists.gnu.org/archive/html/coreutils/2013-11/msg00038.html http://savannah.gnu.org/bugs/?40083 https://pad.lv/1311895
2014-05-10shred: don't infloop upon negative sizeJim Meyering
* src/shred.c (main): With the preceding change, shred -s-2 FILE would write 64KB blocks forever -- or until disk full. This change makes shred reject a negative size. * tests/misc/shred-negative.sh: New file. * tests/local.mk (all_tests): Add it.
2014-05-10shred: fix overflow checking of command-line optionsPaul Eggert
* src/shred.c (main): Limit -n (number of passes) value to ULONG_MAX, not to UINT32_MAX, since the vars are unsigned long. Limit the -s (file size) value to OFF_T_MAX.
2014-05-07tests: improve new ascii test of ddBernhard Voelker
* tests/dd/ascii.sh: Avoid unnecessary subshells. Catch dd's exit code. Remove testing artifact. In the case of a comparison failure, show the differences in octal format in addition to "binary files differ". Simplify the creation of the 'in' file.
2014-05-06dd: fix conv=ascii, conv=ebcdic, conv=ibm to match POSIXPaul Eggert
Problem reported by Don Baggett in <http:/bugs.gnu.org/17422>. * NEWS: * doc/coreutils.texi (dd invocation): Document this. * src/dd.c (conversions): conv=ascii implies conv=unblock. conv=ebcdic and conv=ibm imply conv=block. (ascii_to_ebcdic, ebcdic_to_ascii): Correct to match POSIX 1003.1-2013. * tests/dd/ascii.sh: New file. * tests/local.mk (all_tests): Add it.
2014-05-07tests: improve diagnostics when asserting empty filesPádraig Brady
* tests/chmod/c-option.sh: Use `compare /dev/null ... || fail=1` rather than `test -s ... && fail=1`, so that the file contents are output, thus improving diagnostics for failing tests. * tests/cp/acl.sh: Likewise. * tests/cp/cp-a-selinux.sh: Likewise. * tests/cp/cp-mv-enotsup-xattr.sh: Likewise. * tests/cp/reflink-perm.sh: Likewise. * tests/dd/misc.sh: Likewise. * tests/misc/env-null.sh: Likewise. * tests/misc/env.sh: Likewise. * tests/misc/nice.sh: Likewise. * tests/misc/nohup.sh: Likewise. * tests/misc/printenv.sh: Likewise. * tests/misc/xattr.sh: Likewise. * tests/mv/update.sh: Likewise. * tests/rm/deep-2.sh: Likewise. * tests/rm/read-only.sh: Likewise. * tests/split/r-chunk.sh: Likewise. * tests/tail-2/follow-stdin.sh: Likewise. * tests/tail-2/inotify-race.sh: Likewise. * tests/tail-2/wait.sh: Likewise. * tests/touch/no-dereference.sh: Likewise. * cfg.mk (sc_prohibit_test_empty:): New syntax-check. * tests/cp/proc-zero-len.sh: Adjust to avoid false syntax-check failure. * tests/cp/proc-zero-len.sh: Likewise. * tests/mv/part-symlink.sh: Likewise. * tests/tail-2/infloop-1.sh: Likewise.
2014-05-06mv,cp: preserve symlink xattrs when copying across file systemsPádraig Brady
* src/copy.c (copy_internal): Include the copy_attr() call for symlinks. This should not dereference symlinks, since llistxattr() is used in attr_copy_file() in libattr, and so should copy all but the filtered extended attributes. Note we don't just move the copy_attr() call before the set_owner() call, as that would break capabilities for non symlinks. * tests/cp/cp-mv-enotsup-xattr.sh: Add a test case. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/16131
2014-05-05tests: initial SMACK testsJarkko Sakkinen
* init.cfg (require_smack_): New function. * local.mk: Referenced new tests. * tests/id/smack.sh: SMACK tests (new file). * tests/mkdir/smack-no-root.sh: SMACK tests (new file). * tests/mkdir/smack-root.sh: SMACK tests (new file).
2014-05-04maint: autotools-install: update tool version numbers to latestJim Meyering
* scripts/autotools-install (tarballs): Update to latest.
2014-05-03maint: don't let envvar setting break ChangeLog generationJim Meyering
* Makefile.am (gen-ChangeLog): Clear amend_git_log when we don't set it, so that an envvar setting cannot cause trouble.
2014-05-02numfmt: improve processing throughput by 800%Pádraig Brady
The devmsg() calls that took quote_n() arguments, didn't normally output anything, but still incurred the overhead of those quote_n() calls. * src/numfmt.c (devmsg): Move the inline function with _internal_ enablement check to... * src/system.h: ...here as a variadic macro, with the enablement check at the outer level. * src/factor.c: As per numfmt.c but there is no performance change in this case. * NEWS: Mention the significant performance improvement.
2014-05-02numfmt: support zero padding using --format="%010f"Pádraig Brady
* src/numfmt.c (setup_padding_buffer): Simplify the code by not explicitly dealing with heap exhaustion. (parse_format_string): Likewise. Handle multiple grouping modifiers as does the standard printf. Handle the new leading zero --format modifier. (double_to_human): Use more defensive coding against overwriting stack buffers. Honor the leading zeros width. (usage): Mention the leading zero --format modifier. (main): Allow --padding in combo with a --format (width), as the number of leading zeros are useful independent of the main field width. * doc/coreutils.texi (numfmt invocation): Likewise. * tests/misc/numfmt.pl: Add new test cases. * NEWS: Mention the improvement.
2014-04-29doc: clarify the pr --page-width descriptionsPádraig Brady
* doc/coreutils.texi (pr invocation): Clarify that -w or -W will be rounded down so that each column has the same width. Adjust the wording for -W, to avoid the implication that the width of -S is insignificant to the page width. * src/pr.c (usage): Add a period to avoid ambiguity in the man page output.
2014-04-29ptx: fix whitespace trimming with multiple filesPádraig Brady
This issue was identified by running the test suite with http://code.google.com/p/address-sanitizer/ which is included in GCC 4.8 and enabled with -fsanitize=address This was checked on Fedora 20 with GCC 4.8 as follows: $ yum install libasan # http://bugzilla.redhat.com/991003 $ rm -f src/ptx.o $ make check AM_CFLAGS='-fsanitize=address' SUBDIRS=. VERBOSE=yes $ failure identified in tests/test-suite.log To see this particular failure triggered with multiple files: $ src/ptx <(echo a) <(echo a) 2>&1 | asan_symbolize.py -d ================================================================= ==32178==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000e74f at pc 0x435442 bp 0x7fffe8a1b290 sp 0x7fffe8a1b288 READ of size 1 at 0x60200000e74f thread T0 #0 0x435441 in define_all_fields coreutils/src/ptx.c:1425 #1 0x7fa206d31d64 in __libc_start_main ??:? #2 0x42f77c in _start ??:? 0x60200000e74f is located 1 bytes to the left of 3-byte region [0x60200000e750,0x60200000e753) allocated by thread T0 here: #0 0x421809 in realloc ??:? #1 0x439b4e in fread_file coreutils/lib/read-file.c:97 Shadow bytes around the buggy address: 0x0c047fff9c90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9ca0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cb0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fd fd =>0x0c047fff9ce0: fa fa 03 fa fa fa fd fd fa[fa]03 fa fa fa 00 00 0x0c047fff9cf0: fa fa 04 fa fa fa 04 fa fa fa fd fa fa fa fd fa 0x0c047fff9d00: fa fa 00 fa fa fa fd fa fa fa 00 fa fa fa 00 fa 0x0c047fff9d10: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa 0x0c047fff9d20: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa 0x0c047fff9d30: fa fa fd fa fa fa 00 fa fa fa 00 fa fa fa 00 fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Heap right redzone: fb Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack partial redzone: f4 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 ASan internal: fe ==32178==ABORTING The initial report and high level analysis were from Jim Meyering... "The underlying problem is that swallow_file_in_memory() is setting the contents of the global text_buffer for the first file, then updating it (clobbering old value) for the second file. Yet, some pointers to the initial buffer have been squirreled away and later, one of them (keyafter) is presumed to point into the new "text_buffer", which it does not. The subsequent SKIP_WHITE_BACKWARDS use backs up "cursor" and goes out of bounds." * src/ptx.c (text_buffers): Maintain references for the limits of each buffer corresponding to each file, rather than just the last processed. (struct OCCURS): Add a member to map back to the corresponding file. Note normally this could be computed from the "reference" member rather than needing the extra storage, however this is not possible when in --references mode. (find_occurs_in_text): Reference the array rather than a single entry. (define_all_fields): Likewise. Also avoid computing the file index since this is now stored directly. (main): Update text_buffers[] array rather than a single text_buffer. * tests/misc/ptx-overrun.sh: Even though this issue is already triggered with AddressSanitizer, add a new case to demonstrate the whitespace trimming issue, and to trigger without AddressSanitizer. Fixes https://bugs.gnu.org/16171
2014-04-25maint: make ChangeLog generation more robustPádraig Brady
* Makefile.am (gen-ChangeLog): Sync changes from GNU hello, to ensure exit status is propagated, and to support an optional git-log-fix file.
2014-04-25maint: remove trailing comma from enum to make gl/ c89 compatEdgars Irmejs
* gl/lib/fadvise.c (fadvice_t): This might go to gnulib some day so remove the trailing comma. Fixes http://bugs.gnu.org/17329
2014-04-22doc: clarify meaning of stat's mtime and ctime formatsAssaf Gordon
* src/stat.c (usage): s/modification time/data modification time/; s/change time/status change time/ * doc/coreutils.texi: Ditto.
2014-04-18tests: make ls tests independent of COLORTERM envPádraig Brady
Since the recent commit v8.22-68-g08783f1, ls coloring is now dependent on the COLORTERM environment variable. * tests/envvar-check: Unset COLORTERM from test environment. * tests/ls/color-dtype-dir.sh: Ensure coloring is used. * tests/misc/ls-misc.pl: Likewise. Prompted by the continuous integration build failure at: http://hydra.nixos.org/build/10397646
2014-04-17doc: disambiguate chmod man page reference to (a) usersPádraig Brady
* man/chmod.x: Don't rely on the bold markup for 'a' to distinguish it as that's not done in all cases.
2014-04-17dircolors: add hurd and mach-gnu-color terminal typesDavid Michael
mach-color was replaced by mach-gnu-color in Hurd in 2012. mach-color is left for compatibility and corresponding definitions for mach-color are still found in ncurses. * src/dircolors.hin: Add hurd and mach-gnu-color.
2014-04-17ls: don't output colors with unknown TERM env variableGuilherme de Almeida Suckevicz
--colors controls whether to output colors depending on whether we're connected to a terminal or not, while this change gives control over which terminals we output colors to. * NEWS: Mention the change in behavior. * src/ls.c (known_term_type): A new function to search the static list from dircolors.h (parse_ls_colors): Honor the TERM when both LS_COLORS and COLORTERM are non empty. * tests/ls/color-term.sh: A new test. * tests/local.mk: Reference the new test. Fixes http://bugs.gnu.org/15992
2014-04-16doc: improve remove prompt translator commentBenno Schulenberg
* src/remove.c (prompt): Explain where the difficulty with translating these two strings resides, and suggest an alternative: the one that Paul Eggert first proposed back in 2002, which seems fully resistant.
2014-04-16maint: df: avoid sizeof(char**) static analysis warningBernhard Voelker
* src/df.c (alloc_table_row): Use the size of char** to enlarge the table. Spotted by Coverity.
2014-04-11tests: fix false dd conv=sparse failure on newer XFSPádraig Brady
* tests/dd/sparse.sh: When testing that a hole is created, use an existing sparse destination file, so that we're not write extending the file size, and thus avoiding speculative preallocation which can result in smaller holes than requested. Workaround suggested by Brian Foster
2014-04-10maint: avoid static analysis warnings in ln.cPádraig Brady
* src/ln.c (do_link): It's not obvious that record_file() is a noop in the symlink case (in that case dest_set is NULL and so ignored). So to make it obvious, and to avoid false positives seen in coverity, add the explicit condition here.
2014-04-09maint: document timeout --preserve-status in v8.21 NEWSPádraig Brady
* NEWS: Add the omitted new feature. * cfk.mk (old_NEWS_hash): Adjust accordingly.
2014-04-09doc: clarify in --help that -Z doesn't take an argumentPádraig Brady
* NEWS: Fix a confusing old entry. * cfg.mk (old_NEWS_hash): Adjust accordingly. * src/cp.c (usage): Separate the -Z and --context descriptions. * src/install.c: Likewise. * src/mkdir.c: Likewise. * src/mkfifo.c: Likewise. * src/mknod.c: Likewise. Fixes http://bugs.gnu.org/17220
2014-04-05maint: various cleanupsPádraig Brady
* tests/misc/numfmt.pl: Fix comment misspelling. * src/cut.c: Likewise. * src/tsort.c (detect_loop): Replace an fprintf() with error().
2014-04-05shred: overwrite inode storage used by some file systemsPádraig Brady
* doc/coreutils.texi (shred invocation): Mention some reasons why clearing slack space might be useful. * src/shred.c (do_wipefd): Add initial writes for each pass for small regular files in case the storage for those is in the inode, and thus a larger write up to a block size would bypass that. Move the direct I/O control to... (dopass): ... here so we can avoid enabling it for these small initial writes. It's better to retry direct I/O for each pass anyway to handle the case where direct I/O is disabled for only the last portion of a file when the size is not a multiple of the block size. Note we don't avoid the sync for the initial write as it will be small but more importantly could be on a different part of the disk and so worth doing independently to ensure the write is not discarded. * tests/misc/shred-exact.sh: Check some more direct I/O cases. * NEWS: Mention the improvements. The inode storage issue was mentioned by Paul Eggert.
2014-04-04shred: avoid a data pass on empty filesPádraig Brady
* src/shred.c (do_wipefd): Don't increase the size written for an empty file up to a full block. Also increase the size to OFF_T_MAX in the edge case where we do overflow. * NEWS: Mention the shred improvements from recent changes. * tests/misc/shred-passes.sh: Adjust as we no longer write a BLKSIZE of data for empty files.
2014-04-03cp: don't reserve a device numberPaul Eggert
* src/copy.c (copy_internal): Replace dev_t arg DEVICE with struct stat pointer arg PARENT. All callers changed. This removes an unwarranted assumption that dev_t values of 0 cannot occur in file systems. See: http://bugs.gnu.org/17179
2014-04-03shred: shred one block even for empty filesPaul Eggert
* src/shred.c (do_wipefd): Shred one block of empty regular files. This reverts an unintended part of the previous change.
2014-04-02shred: port to GNU/Linux behavior with tape drivesPaul Eggert
See: http://bugs.gnu.org/17149 * src/shred.c [__linux__]: Include <sys/mtio.h>. (dorewind): New function, which works around the lseek problem with tape drives on GNU/Linux, the same way that dd does. (dopass): Use it. New arg ST, needed for dorewind. All uses changed. (do_wipefd): Don't rely on undefined behavior on integer overflow of file sizes. Use INT_ADD_OVERFLOW instead.
2014-04-02head: port to Darwin and use simpler seeksPaul Eggert
This removes an unportable assumption that if lseek succeeds, the file is capable of seeking. See: http://bugs.gnu.org/17145 * src/head.c (elseek): New function, for consistency in reporting lseek failures. (elide_tail_bytes_file, elide_tail_lines_seekable) (elide_tail_lines_file, head_lines, head): Use it. (elide_tail_bytes_file, elide_tail_lines_file): New args CURRENT_POS and SIZE. All uses changed. Don't bother invoking lseek, since we know the file's pos and size now. (elide_tail_bytes_file): Change a local from uintmax_t to off_t, since it fits. (head): Use lseek only on regular files, since its behavior on unseekable devices is implementation-defined. * NEWS: Document this.