summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2014-12-30build: update to latest gnulibPádraig Brady
Included in this are gnulib changes 3ea43e02 2768ceb7 which make the device IDs from /proc/self/mountinfo available to df. This can be leveraged by a subsequent change to df to present a more accurate list of file systems. * bootstrap: Merge from gnulib. * src/ls.c (dev_ino_pop): s/obstack_blank/obstack_blank_fast/ as this API/ABI has changed, giving memory exhausted errors if negative (large positive) numbers are passed to obstack_blank(). * tests/df/skip-duplicates.sh: Adjust as the new gnulib code requires a non NULL mnt_opts even when mnt_type is not "none".
2014-12-29maint: remove duplicate names from THANKSPádraig Brady
* .mailmap: Adjust so that there is only a single entry per name in the generated THANKS.
2014-12-26doc: update the URL referencing SI prefixesPádraig Brady
Identified at https://www.gnu.org/software/gnun/linc/linc.html * doc/coreutils.texi (Block size): Fix the stale link.
2014-12-25tests: fix error message check on some systemsPádraig Brady
http://hydra.nixos.org/build/18129583 identified (on OS X) an incorrect test assumption in the previous commit. * gl/lib/xdectoint.c (__xnumtoint): Suppress the EINVAL error message as it's redundant in this context. * tests/misc/tail.pl: Suppress _optionally_ appended strerror messages. * tests/fmt/base.pl: Likewise. * tests/pr/pr-tests.pl: Likewise. * tests/split/l-chunk.sh: Likewise.
2014-12-19diagnose too-large numbers betterPádraig Brady
Following on from commit v8.23-82-gaddae94, consistently diagnose numbers that are too large, so as to distinguish from other errors, and make the limits obvious. * gl/modules/xdectoint: A new module implementing xdecto[iu]max(), which handles the common case of parsing a bounded integer and exiting with a diagnostic on error. * gl/lib/xdectoimax.c: The signed variant. * gl/lib/xdectoint.c: The parameterized implementation. * gl/lib/xdectoint.h: The interface. * gl/lib/xdectoumax.c: The unsigned variant. * bootstrap.conf: Reference the new module. * cfg.mk (exclude_file_name_regexp--sc_require_config_h_first): Exclude the parameterized templates. * src/csplit.c: Output EOVERFLOW or ERANGE errors if appropriate. * src/fmt.c: Likewise. * src/fold.c: Likewise. * src/head.c: Likewise. * src/ls.c: Likewise. * src/nl.c: Likewise. * src/nproc.c: Likewise. * src/shred.c: Likewise. * src/shuf.c: Likewise. * src/stdbuf.c: Likewise. * src/stty.c: Likewise. * src/tail.c: Likewise. * src/truncate.c: Likewise. * src/split.c: Likewise. * src/pr.c: Likewise. * tests/pr/pr-tests.pl: Adjust to avoid matching errno diagnostic. * tests/fmt/base.pl: Likewise. * tests/split/l-chunk.sh: Likewise. * tests/misc/shred-negative.sh: Likewise. * tests/misc/tail.pl: Likewise. Also remove the redundant existing ERR_SUBST from test err-6. * tests/ls/hex-option.sh: Check HEX/OCT options. * tests/misc/shred-size.sh: Likewise. * tests/misc/stty-row-col.sh: Likewise.
2014-12-19build: add $(EXEEXT) suffix to man page make targetsKO Myung-Hun
* man/local.mk: Add $(EXEEXT) suffix to the executables, which is significant on OS/2 for example.
2014-12-19build: don't call OS/2 routines on all systemsKO Myung-Hun
* src/system.h: Add a missing __OS2__ ifdef guard. Also adjust spacing around () to avoid a syntax-check failure.
2014-12-18build: expand a response file and a wildcard on OS/2KO Myung-Hun
OS/2 traditional shells(cmd) do not expand a response file(@file) or a wildcard. Expand them in each utility itself. * src/system.h (initialize_main): Define on OS/2. Expand a response file and a wildcard.
2014-12-16maint: fix how to request changes for THANKS.inBernhard Voelker
* THANKS.in: Change the comment at the top to send change requests regarding this file to the main mailing list rather than referring to cp's --help output for the mailing list's address - which does not include that information anymore.
2014-12-16dd: fix typo in previous changePaul Eggert
Reported by Bernhard Voelker in: http://lists.gnu.org/archive/html/bug-gnulib/2014-12/msg00191.html * src/dd.c (scanargs): s/IN/OUT/.
2014-12-15dd: diagnose too-large numbers betterPaul Eggert
Reported by Isabella Parakiss in: http://lists.gnu.org/archive/html/bug-gnulib/2014-12/msg00184.html * src/dd.c (parse_integer): Return strtol_error code, not bool. All callers changed. (scanargs): Improve quality of diagnostic when a number is too large.
2014-12-10build: use the system printf routines in most casesPádraig Brady
glibc <= 2.5 would crash when passed invalid long double values, therefore internal gnulib routines were used, essentially only by od, to output such invalid values. Later glibc versions don't crash, as per https://sourceware.org/bugzilla/show_bug.cgi?id=4586 and subsequently od was adjusted to use the system printf routines through the use of the ftoastr module with commit v8.7-22-ga71c22f. Consequently our testing of this feature was moot, and use of the gnulib printf replacement for printf(1), od(1) and error(3) etc. was redundant. * configure.ac (gl_printf_safe): Unset so that we don't check that "nan" is output for these long double values. * tests/misc/od-float.sh: Adjust all existing checks to fail if od exits with failure status (like crashing for example). Add a new case for one of the problematic invalid long double values for x86_64. We only check that od exits successfully at present, which may change if https://sourceware.org/bugzilla/show_bug.cgi?id=17661 is resolved.
2014-12-03maint: avoid -fsanitize=undefined warning in rand-isaacPádraig Brady
* gl/lib/rand-isaac.c (isaac_refill): readisaac() purposefully passes unaligned pointers to avoid memory copies. This is only done on platforms where this is defined, so avoid the associated runtime warning generated with -fsanitize=undefined, which is: lib/rand-isaac.c:125:182: runtime error: store to misaligned address 0x63100003d7fd for type 'isaac_word', which requires 8 byte alignment 0x63100003d7fd: note: pointer points here 47 ce ed a4 be be be 00 00 00 00 00 00 00 00 ... ^
2014-12-02maint: avoid signed overflow warning with -O3Pádraig Brady
Prompted by the implicit -O3 added by american-fuzzy-lop, seen with GCC 4.9.2 on x86_64. src/pr.c: In function 'print_files.part.5': src/pr.c:1781:6: error: assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow] if (cols_ready_to_print () == 0) This happens because cols_ready_to_print() is inlined thus reducing the comparison to the N variable in print_page(). Now this can't overflow due to the protection when parsing the specified column, but use an unsigned type to avoid the apparent signed overflow. * src/pr.c (cols_ready_to_print): Increment an unsigned type to avoid the subsequent signed overflow warning.
2014-12-02du: handle sub-bind-mount cycles gracefullyBoris Ranto
This patch fixes the handling of sub-bind-mount cycles which are incorrectly detected as the file system errors. If you bind mount the directory 'a' to its subdirectory 'a/b/c' and then run 'du a/b' you will get the circular dependency warning even though nothing is wrong with the file system. This happens because the first directory that is traversed twice in this case is not a bind mount but a child of bind mount. The solution is to traverse all the directories in the cycle that fts detected and check whether they are not a (bind) mount. * src/du.c (mount_point_in_fts_cycle): New function that checks whether any of the directories in the cycle that fts detected is a mount point. * src/du.c (process_file): Update the function to use the new function that looks up all the directories in the fts cycle instead of only the last one. * tests/du/bind-mount-dir-cycle-v2.sh: New test case that exhibits the described behavior. * tests/local.mk: Reference the new root test. * NEWS: Mention the bug fix.
2014-11-29tests: avoid hardlink to symlink tests where not supportedPádraig Brady
These checks weren't correctly avoided in commit v8.23-66-g222d7ac * tests/cp/same-file.sh: Avoid all hardlink to symlink tests on platforms where that's not supported. Identified by http://hydra.nixos.org/build/17636446
2014-11-28build: fix missing casts from recent changePádraig Brady
* src/dd.c (alloc_[io]buf): I committed a stale patch that omitted the casts needed on 32 bit. Identified by http://hydra.nixos.org/build/17610188
2014-11-27rm: fix prompted number of arguments to remove on some platformsPádraig Brady
"zu" was output on solaris 8 for example rather than the number, since coreutils-8.22. * cfg.mk: Disallow %z, since we don't currently use the gnulib fprintf module, so any usage with it is non portable. Also our usage with error() currently works only through an ancillary dependency on the vfprintf gnulib module. * src/rm.c (main): Use %PRIuMAX rather than %zu for portability. * src/dd.c (alloc_[io]buf): Likewise for consistency. * src/od.c (main): Likewise. * src/split.c (set_suffix_length): Likewise. * NEWS: Mention the rm bug fix. Reported in http://bugs.gnu.org/19184
2014-11-26tests: fix portability issue in dd/ascii testPádraig Brady
Solaris 8 was seen to issue this error: "printf: `&': illegal format character" * test/dd/ascii.sh: Use the coreutils printf in this test rather than the system one, to avoid portability issues.
2014-11-25build: port new rule for coreutils.h to old BashPaul Eggert
Reported by Ted Carr in: http://bugs.gnu.org/19184 * src/local.mk (src/coreutils.h): Don't assume single_binary_progs is nonempty.
2014-11-24paste: fix possible truncated output with large filesTobias Stoeckmann
If '\n' was present at the size_t boundary of a file, then that and subsequent data would be discarded. * src/paste.c (paste_parallel): Avoid the overflow issue by changing the flag to a boolean rather than a count. * NEWS: Mention the bug fix.
2014-11-24df: only suppress remote mounts of separate exports with --totalPádraig Brady
* src/df.c (filter_mount_list): Separate remote locations are generally explicitly mounted, so list each even if they share the same remote device and thus storage. However with --total keep the suppression to give a more accurate value for the total storage available. (usage): Expand on the new implications of --total and move it in the options list according to alphabetic order. doc/coreutils.texi (df invocation): Mention that --total impacts on deduplication of remote file systems and also move location according to alphabetic order. * tests/df/skip-duplicates.sh: Add remote test cases. * NEWS: Mention the change in behavior. Reported in http://bugs.debian.org/737399 Reported in http://bugzilla.redhat.com/920806 Reported in http://bugzilla.opensuse.org/866010 Reported in http://bugzilla.opensuse.org/901905
2014-11-23df: ensure -a shows all remote file system entriesPádraig Brady
commit v8.22-125-g9d736f8 printed placeholder "-" values for device names that didn't match the preferred device name for a particular mount point. However that was seen to erroneously suppress values for aliased host names or exports, common with remote file systems. * src/df.c (me_for_dev): Rename from devname_for_dev() so that we can determine the remoteness as well as the name for the preferred mount entry. (get_dev): Don't output place holder values when both current and preferred mount entries are remote. Reported in http://bugs.debian.org/737399
2014-11-22tests: add a case verifying mv on case insensitive file systemsPádraig Brady
* NEWS: Update the recent entry to also mention the avoidance of incorrectly unlinking a multi-hardlinked "source" file when presented with source and dest that only differ in case. * src/copy.c (same_file_ok): Mention the case issue with same_name(). * tests/mv/hardlink-case.sh: Test the issue on HFS+. * tests/local.mk: Reference the new test case. * tests/mv/vfat: Remove an old related but unused test case.
2014-11-21doc: mention how to avoid newlines impacting ls -1Pádraig Brady
* src/ls.c (usage): Mention the -b and -q options in the -1 description. * doc/coreutils.texi (ls invocation): Likewise.
2014-11-21tests: chcon: avoid false failure with newer selinuxPádraig Brady
file_t is now mapped to unlabeled_t as per: http://danwalsh.livejournal.com/68189.html Therefore use the latter to ensure we match correctly. This is needed on >= Fedora 21 for example, while it also works on earlier releases.
2014-11-21mv: fail when moving a file to a hardlinkBoris Ranto
We may run into a race condition if we treat hard links to the same file as distinct files. If we do 'mv a b' and 'mv b a' in parallel, both a and b can disappear from the file system. The reason is that in this case the unlink on src is called and the system calls can end up being run in the order where unlink(a) and unlink(b) are the last two system calls. Therefore exit with an error code so that we avoid the potential data loss. * src/copy.c (same_file_ok): Don't set unlink_src that was used by mv, and return false for two hardlinks to a file in move_mode. *src/copy.c (copy_internal): No longer honor the unlink_src option, used only by mv. NEWS: Mention the change in behavior. * tests/cp/same-file.sh: Augment to cover the `cp -a hlsl1 sl1` case. * tests/mv/hard-verbose.sh: Remove no longer needed test. * tests/local.mk: Remove the reference to hard-verbose.sh. * tests/mv/hard-4.sh: Adjust so we fail in this case. * tests/mv/i-4.sh: Likewise. * tests/mv/symlink-onto-hardlink-to-self.sh: Likewise.
2014-11-18maint: run strftime syntax check on newer systemsPádraig Brady
* cfg.mk (sc_strftime_check): Adjust regex to handle newer glibc info formatting with different indentation and quoting.
2014-10-30maint: tests: fix comments about retry_delay_Pádraig Brady
* tests/tail-2/F-vs-missing.sh: Comment with the correct total delay. * tests/tail-2/F-vs-rename.shi: Likewise. * tests/tail-2/flush-initial.sh: Likewise. * tests/tail-2/inotify-hash-abuse.sh: Likewise. * tests/tail-2/pipe-f2.sh: Likewise. * tests/misc/chroot-fail.sh: Initialize can_chroot_root in all cases.
2014-10-30tests: make inotify-rotate more robust and efficientPádraig Brady
* tests/tail-2/inotify-rotate.sh: Use retry_delay_ to employ an exponential backoff with a total delay of up to 25.5s. The 15s delay was seen to trigger a false failure in http://hydra.nixos.org/build/16546517 Also remove the .1s sleep in each of the 50 iterations to reduce the running time of the test and thus the expensive_ tag on this test was removed. Also ensure that we use the standard exit procedure upon failure to avoid any erroneous diagnostics due to persistent files on NFS.
2014-10-29tests: make du/move-dir-while-traversing more robustBernhard Voelker
* tests/du/move-dir-while-traversing.sh: Catch failure of retry_delay_ when waiting for the watcher to get ready.
2014-10-27doc: mention that df -a includes duplicate file systemsPádraig Brady
* src/df.c (usage): Mention that duplicate file systems are shown with this option, not just dummy file systems. * doc/coreutils.texi (df invocation): For the --all option, expand on the class of normally suppressed mount entries that it includes. Reported in http://bugs.debian.org/737399
2014-10-23maint: improve sc_long_lines syntax-check speedPádraig Brady
sc_long_lines was the slowest syntax check before$ time make sc_long_lines long_lines real 0m2.740s after $ time make sc_long_lines long_lines real 0m0.677s * src/cfg.mk (sc_dd_max_sym_length): s/--max-line-length/-L/ for compat with BSDs. (sc_long_lines): Prefilter with wc -L to only identify lines in files that have lines longer than 80 characters.
2014-10-23tests: d_type-check: don't hardcode the C library nameMike Frysinger
* tests/d_type-check: The hardcoded name doesn't hold true for all Linux/glibc platforms, let alone Linux/non-glibc. Use ctypes.util.find_library() instead to search for the library.
2014-10-17tests: avoid false failure when comparing /proc filesBernhard Voelker
At least the MHz number in /proc/cpuinfo may change, thus leading to a false positive failure when comparing the expected against the actual output file. Use an invariant file instead: /proc/version. * tests/misc/head-c.sh: s/cpuinfo/version/
2014-10-16tests: fix test hang with unstable inodes in /procPádraig Brady
* cp/proc-zero-len.sh: Search the 'err' file for the error to ignore, not stdin.
2014-10-16chroot: call chroot() unconditionally to handle bind mounted "/"Pádraig Brady
* src/chroot.c (is_root): Adjust to compare canonicalized paths rather than inodes, to handle (return false in) the case where we have a tree that is constructed by first bind mounting "/" (thus having the same inode). (main): Unconditionally call chroot() because it's safer and of minimal performance benefit to avoid in this case. This will cause inconsistency with some platforms not allowing `chroot / true` for non root users. * tests/misc/chroot-fail.sh: Adjust appropriately. * NEWS: Mention the bug fixes. Fixes http://bugs.gnu.org/18736
2014-10-15copy: avoid an extraneous error when reporting errorsPádraig Brady
* src/copy.c (copy_reg): If sparse_copy() failed, then an erroneous error about failing to extend the file would be reported.
2014-10-15cp: read sparse files more efficiently with non regular destinationPádraig Brady
* src.copy.c (copy_reg): Use fiemap to read sparse files, even if the output is not to a regular file. * NEWS: Mention the improvement.
2014-10-15cp: avoid speculative preallocation with --sparse=alwaysPádraig Brady
With --sparse=always use fallocate(...PUNCH_HOLE...) to avoid any permanent allocation due to speculative preallocation employed by file systems such as XFS. * m4/jm-macros.m4: Check for <linux/falloc.h> and fallocate(). * src/copy.c (punch_hole): A new function to try and punch a hole at the specified offset if supported. (create_hole): Call punch_hole() after requesting a hole. (extent_copy): Likewise. * NEWS: Mention the improvement.
2014-10-15copy: detect smaller holes than the copy buffer sizePádraig Brady
Previously cp would not detect runs of NULs that were smaller than the buffer size used for I/O (currently 128KiB). * src/copy.c (copy_reg): Use an independent hole_size, set to st_blksize, to increase the chances of detecting a representable hole, in a run of NULs read from the input. (create_hole): A new function refactored from sparse_copy() and extent_copy() so we have a single place to handle holes. (sparse_copy): Adjust to loop over the larger input buffer in chunks of the passed hole size. Also adjust to only call lseek once per hole, rather than at least once per input buffer. * tests/cp/sparse.sh: Add test cases for various sparse chunk sizes. * NEWS: Mention the improvement.
2014-10-09doc: clarify that timeout limits are not 2038 secondsWieland Hoffmann
* man/timeout.x: The 2038 that the sentence is referring to is the year 2038, not 2038 seconds (the default unit used for timeouts).
2014-10-08maint: avoid new signed overflow warning on 32 bitPádraig Brady
Prompted by http://hydra.nixos.org/build/15682577 with GCC 4.8.3 on i686 src/tac.c:557:6: error: assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow] if (bytes_copied < 0) This happens because copy_to_temp() is inlined in tac_nonseekable(), thus reducing the comparison to the bytes_copied variable in copy_to_temp. Now this can't overflow on either 32 or 64 bit due to the protection of the preceding fwrite(). We could use a guard like "if (bytes_copied <= OFF_T_MAX - bytes_read)" to avoid the warning, but rather than a runtime branch, just use an unsigned type to avoid apparent signed overflow on systems where the accumulation is not promoted to unsigned (32 bit size_t, 64 bit off_t). * src/tac.c (copy_to_temp): Increment an unsigned type to avoid the subsequent signed overflow warning.
2014-10-07tests: split/b-chunk.sh: avoid spurious fail on non-LinuxJim Meyering
* tests/split/b-chunk.sh: Skip each file that does not exist. Some systems lack /proc/version or /sys/kernel/profiling
2014-10-08maint: avoid syntax-check failures in previous patchPádraig Brady
* tests/misc/od-j.sh: Non standard comparison order. * tests/split/b-chunk.sh: Confusing input file name. * tests/tail-2/tail-c.sh: Redundant require ulimit.
2014-10-07wc: don't miscount /sys and similar file systemsPaul Eggert
Fix similar problems in head, od, split, tac, and tail. Reported by George Shuklin in: http://bugs.gnu.org/18621 * NEWS: Document this. * src/head.c (elseek): Move up. (elide_tail_bytes_pipe, elide_tail_lines_pipe): New arg CURRENT_POS. All uses changed. (elide_tail_bytes_file, elide_tail_lines_file): New arg ST and remove arg SIZE. All uses changed. * src/head.c (elide_tail_bytes_file): * src/od.c (skip): Avoid optimization for /sys files, where st_size is bogus and st_size == st_blksize. Don't report error at EOF when not optimizing. * src/head.c, src/od.c, src/tail.c: Include "stat-size.h". * src/split.c (input_file_size): New function. (bytes_split, lines_chunk_split, bytes_chunk_extract): New arg INITIAL_READ. All uses changed. Use it to double-check st_size. * src/tac.c (tac_seekable): New arg FILE_POS. All uses changed. (copy_to_temp): Return size of temp file. All uses changed. * src/tac.c (tac_seekable): * src/tail.c (tail_bytes): * src/wc.c (wc): Don't trust st_size; double-check by reading. * src/wc.c (wc): New arg CURRENT_POS. All uses changed. * tests/local.mk (all_tests): Add tests/misc/wc-proc.sh, tests/misc/od-j.sh, tests/tail-2/tail-c.sh. * tests/misc/head-c.sh: * tests/misc/tac-2-nonseekable.sh: * tests/split/b-chunk.sh: Add tests for problems with /proc and /sys files. * tests/misc/od-j.sh, tests/misc/wc-proc.sh, tests/tail-2/tail-c.sh: New files.
2014-10-04doc: document stat's output with the --terse optionBernhard Voelker
* doc/coreutils.texi (stat invocation): Add a paragraph documenting stat's output format when the --terse option is specified, both in normal and in --file-system mode. Reported by Dan Jacobson <jidanni@jidanni.org> in http://bugs.gnu.org/18624
2014-10-02tests: fix false failure for test referencing libdlNick Alcock
* init.cfg (gcc_shared_): -ldl has to be positioned after the object files that may rely upon it. This fixes tests/cp/nfs-removal-race.sh which references dlsym() from libdl.
2014-10-02maint: avoid double semicolon syntax check failureBernhard Voelker
A syntax-check recently added to gnulib would trigger a failure (once gnulib gets updated here) for a statement introduced with commit v8.23-43-gaf2a4ed: src/dd.c:806: char const *time_fmt = _(", %g s, %s/s\n");; maint.mk: Double semicolon detected make: *** [sc_prohibit_double_semicolon] Error 1 * src/dd.c (print_xfer_stats): s/;;/;/
2014-09-30dd: new status=progress level to print stats periodicallyFederico Simoncelli
* src/dd.c: Report the transfer progress every second when the new status=progress level is used. Adjust the handling and description of the status= option so that they're treated as mutually exclusive levels, rather than flags with implicit precedence. * doc/coreutils.texi (dd invocation): Document the new progress status level. Reference the new level in the description of SIGUSR1. * tests/dd/stats.sh: Add new test for status=progress. * tests/dd/misc.sh: Change so status=none only takes precedence if it's the last level specified. * NEWS: Mention the feature.