summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2014-05-04maint: autotools-install: update tool version numbers to latestJim Meyering
* scripts/autotools-install (tarballs): Update to latest.
2014-05-03maint: don't let envvar setting break ChangeLog generationJim Meyering
* Makefile.am (gen-ChangeLog): Clear amend_git_log when we don't set it, so that an envvar setting cannot cause trouble.
2014-05-02numfmt: improve processing throughput by 800%Pádraig Brady
The devmsg() calls that took quote_n() arguments, didn't normally output anything, but still incurred the overhead of those quote_n() calls. * src/numfmt.c (devmsg): Move the inline function with _internal_ enablement check to... * src/system.h: ...here as a variadic macro, with the enablement check at the outer level. * src/factor.c: As per numfmt.c but there is no performance change in this case. * NEWS: Mention the significant performance improvement.
2014-05-02numfmt: support zero padding using --format="%010f"Pádraig Brady
* src/numfmt.c (setup_padding_buffer): Simplify the code by not explicitly dealing with heap exhaustion. (parse_format_string): Likewise. Handle multiple grouping modifiers as does the standard printf. Handle the new leading zero --format modifier. (double_to_human): Use more defensive coding against overwriting stack buffers. Honor the leading zeros width. (usage): Mention the leading zero --format modifier. (main): Allow --padding in combo with a --format (width), as the number of leading zeros are useful independent of the main field width. * doc/coreutils.texi (numfmt invocation): Likewise. * tests/misc/numfmt.pl: Add new test cases. * NEWS: Mention the improvement.
2014-04-29doc: clarify the pr --page-width descriptionsPádraig Brady
* doc/coreutils.texi (pr invocation): Clarify that -w or -W will be rounded down so that each column has the same width. Adjust the wording for -W, to avoid the implication that the width of -S is insignificant to the page width. * src/pr.c (usage): Add a period to avoid ambiguity in the man page output.
2014-04-29ptx: fix whitespace trimming with multiple filesPádraig Brady
This issue was identified by running the test suite with http://code.google.com/p/address-sanitizer/ which is included in GCC 4.8 and enabled with -fsanitize=address This was checked on Fedora 20 with GCC 4.8 as follows: $ yum install libasan # http://bugzilla.redhat.com/991003 $ rm -f src/ptx.o $ make check AM_CFLAGS='-fsanitize=address' SUBDIRS=. VERBOSE=yes $ failure identified in tests/test-suite.log To see this particular failure triggered with multiple files: $ src/ptx <(echo a) <(echo a) 2>&1 | asan_symbolize.py -d ================================================================= ==32178==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000e74f at pc 0x435442 bp 0x7fffe8a1b290 sp 0x7fffe8a1b288 READ of size 1 at 0x60200000e74f thread T0 #0 0x435441 in define_all_fields coreutils/src/ptx.c:1425 #1 0x7fa206d31d64 in __libc_start_main ??:? #2 0x42f77c in _start ??:? 0x60200000e74f is located 1 bytes to the left of 3-byte region [0x60200000e750,0x60200000e753) allocated by thread T0 here: #0 0x421809 in realloc ??:? #1 0x439b4e in fread_file coreutils/lib/read-file.c:97 Shadow bytes around the buggy address: 0x0c047fff9c90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9ca0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cb0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fd fd =>0x0c047fff9ce0: fa fa 03 fa fa fa fd fd fa[fa]03 fa fa fa 00 00 0x0c047fff9cf0: fa fa 04 fa fa fa 04 fa fa fa fd fa fa fa fd fa 0x0c047fff9d00: fa fa 00 fa fa fa fd fa fa fa 00 fa fa fa 00 fa 0x0c047fff9d10: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa 0x0c047fff9d20: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa 0x0c047fff9d30: fa fa fd fa fa fa 00 fa fa fa 00 fa fa fa 00 fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Heap right redzone: fb Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack partial redzone: f4 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 ASan internal: fe ==32178==ABORTING The initial report and high level analysis were from Jim Meyering... "The underlying problem is that swallow_file_in_memory() is setting the contents of the global text_buffer for the first file, then updating it (clobbering old value) for the second file. Yet, some pointers to the initial buffer have been squirreled away and later, one of them (keyafter) is presumed to point into the new "text_buffer", which it does not. The subsequent SKIP_WHITE_BACKWARDS use backs up "cursor" and goes out of bounds." * src/ptx.c (text_buffers): Maintain references for the limits of each buffer corresponding to each file, rather than just the last processed. (struct OCCURS): Add a member to map back to the corresponding file. Note normally this could be computed from the "reference" member rather than needing the extra storage, however this is not possible when in --references mode. (find_occurs_in_text): Reference the array rather than a single entry. (define_all_fields): Likewise. Also avoid computing the file index since this is now stored directly. (main): Update text_buffers[] array rather than a single text_buffer. * tests/misc/ptx-overrun.sh: Even though this issue is already triggered with AddressSanitizer, add a new case to demonstrate the whitespace trimming issue, and to trigger without AddressSanitizer. Fixes https://bugs.gnu.org/16171
2014-04-25maint: make ChangeLog generation more robustPádraig Brady
* Makefile.am (gen-ChangeLog): Sync changes from GNU hello, to ensure exit status is propagated, and to support an optional git-log-fix file.
2014-04-25maint: remove trailing comma from enum to make gl/ c89 compatEdgars Irmejs
* gl/lib/fadvise.c (fadvice_t): This might go to gnulib some day so remove the trailing comma. Fixes http://bugs.gnu.org/17329
2014-04-22doc: clarify meaning of stat's mtime and ctime formatsAssaf Gordon
* src/stat.c (usage): s/modification time/data modification time/; s/change time/status change time/ * doc/coreutils.texi: Ditto.
2014-04-18tests: make ls tests independent of COLORTERM envPádraig Brady
Since the recent commit v8.22-68-g08783f1, ls coloring is now dependent on the COLORTERM environment variable. * tests/envvar-check: Unset COLORTERM from test environment. * tests/ls/color-dtype-dir.sh: Ensure coloring is used. * tests/misc/ls-misc.pl: Likewise. Prompted by the continuous integration build failure at: http://hydra.nixos.org/build/10397646
2014-04-17doc: disambiguate chmod man page reference to (a) usersPádraig Brady
* man/chmod.x: Don't rely on the bold markup for 'a' to distinguish it as that's not done in all cases.
2014-04-17dircolors: add hurd and mach-gnu-color terminal typesDavid Michael
mach-color was replaced by mach-gnu-color in Hurd in 2012. mach-color is left for compatibility and corresponding definitions for mach-color are still found in ncurses. * src/dircolors.hin: Add hurd and mach-gnu-color.
2014-04-17ls: don't output colors with unknown TERM env variableGuilherme de Almeida Suckevicz
--colors controls whether to output colors depending on whether we're connected to a terminal or not, while this change gives control over which terminals we output colors to. * NEWS: Mention the change in behavior. * src/ls.c (known_term_type): A new function to search the static list from dircolors.h (parse_ls_colors): Honor the TERM when both LS_COLORS and COLORTERM are non empty. * tests/ls/color-term.sh: A new test. * tests/local.mk: Reference the new test. Fixes http://bugs.gnu.org/15992
2014-04-16doc: improve remove prompt translator commentBenno Schulenberg
* src/remove.c (prompt): Explain where the difficulty with translating these two strings resides, and suggest an alternative: the one that Paul Eggert first proposed back in 2002, which seems fully resistant.
2014-04-16maint: df: avoid sizeof(char**) static analysis warningBernhard Voelker
* src/df.c (alloc_table_row): Use the size of char** to enlarge the table. Spotted by Coverity.
2014-04-11tests: fix false dd conv=sparse failure on newer XFSPádraig Brady
* tests/dd/sparse.sh: When testing that a hole is created, use an existing sparse destination file, so that we're not write extending the file size, and thus avoiding speculative preallocation which can result in smaller holes than requested. Workaround suggested by Brian Foster
2014-04-10maint: avoid static analysis warnings in ln.cPádraig Brady
* src/ln.c (do_link): It's not obvious that record_file() is a noop in the symlink case (in that case dest_set is NULL and so ignored). So to make it obvious, and to avoid false positives seen in coverity, add the explicit condition here.
2014-04-09maint: document timeout --preserve-status in v8.21 NEWSPádraig Brady
* NEWS: Add the omitted new feature. * cfk.mk (old_NEWS_hash): Adjust accordingly.
2014-04-09doc: clarify in --help that -Z doesn't take an argumentPádraig Brady
* NEWS: Fix a confusing old entry. * cfg.mk (old_NEWS_hash): Adjust accordingly. * src/cp.c (usage): Separate the -Z and --context descriptions. * src/install.c: Likewise. * src/mkdir.c: Likewise. * src/mkfifo.c: Likewise. * src/mknod.c: Likewise. Fixes http://bugs.gnu.org/17220
2014-04-05maint: various cleanupsPádraig Brady
* tests/misc/numfmt.pl: Fix comment misspelling. * src/cut.c: Likewise. * src/tsort.c (detect_loop): Replace an fprintf() with error().
2014-04-05shred: overwrite inode storage used by some file systemsPádraig Brady
* doc/coreutils.texi (shred invocation): Mention some reasons why clearing slack space might be useful. * src/shred.c (do_wipefd): Add initial writes for each pass for small regular files in case the storage for those is in the inode, and thus a larger write up to a block size would bypass that. Move the direct I/O control to... (dopass): ... here so we can avoid enabling it for these small initial writes. It's better to retry direct I/O for each pass anyway to handle the case where direct I/O is disabled for only the last portion of a file when the size is not a multiple of the block size. Note we don't avoid the sync for the initial write as it will be small but more importantly could be on a different part of the disk and so worth doing independently to ensure the write is not discarded. * tests/misc/shred-exact.sh: Check some more direct I/O cases. * NEWS: Mention the improvements. The inode storage issue was mentioned by Paul Eggert.
2014-04-04shred: avoid a data pass on empty filesPádraig Brady
* src/shred.c (do_wipefd): Don't increase the size written for an empty file up to a full block. Also increase the size to OFF_T_MAX in the edge case where we do overflow. * NEWS: Mention the shred improvements from recent changes. * tests/misc/shred-passes.sh: Adjust as we no longer write a BLKSIZE of data for empty files.
2014-04-03cp: don't reserve a device numberPaul Eggert
* src/copy.c (copy_internal): Replace dev_t arg DEVICE with struct stat pointer arg PARENT. All callers changed. This removes an unwarranted assumption that dev_t values of 0 cannot occur in file systems. See: http://bugs.gnu.org/17179
2014-04-03shred: shred one block even for empty filesPaul Eggert
* src/shred.c (do_wipefd): Shred one block of empty regular files. This reverts an unintended part of the previous change.
2014-04-02shred: port to GNU/Linux behavior with tape drivesPaul Eggert
See: http://bugs.gnu.org/17149 * src/shred.c [__linux__]: Include <sys/mtio.h>. (dorewind): New function, which works around the lseek problem with tape drives on GNU/Linux, the same way that dd does. (dopass): Use it. New arg ST, needed for dorewind. All uses changed. (do_wipefd): Don't rely on undefined behavior on integer overflow of file sizes. Use INT_ADD_OVERFLOW instead.
2014-04-02head: port to Darwin and use simpler seeksPaul Eggert
This removes an unportable assumption that if lseek succeeds, the file is capable of seeking. See: http://bugs.gnu.org/17145 * src/head.c (elseek): New function, for consistency in reporting lseek failures. (elide_tail_bytes_file, elide_tail_lines_seekable) (elide_tail_lines_file, head_lines, head): Use it. (elide_tail_bytes_file, elide_tail_lines_file): New args CURRENT_POS and SIZE. All uses changed. Don't bother invoking lseek, since we know the file's pos and size now. (elide_tail_bytes_file): Change a local from uintmax_t to off_t, since it fits. (head): Use lseek only on regular files, since its behavior on unseekable devices is implementation-defined. * NEWS: Document this.
2014-03-31head: fix bug with head -c-N when stdin is not at startPaul Eggert
* src/head.c (elide_tail_bytes_file): Fix typo in lseek invocation. * tests/misc/head-c.sh: Add test for this bug. * NEWS: Document this.
2014-03-26mv: fix issues with previous prompt adjustmentsPádraig Brady
* src/copy.c (overwrite_ok): Fix the gettext calls so that the second string is tagged for translation. Display the correct "replace ..." prompt when in move_mode. * tests/mv/i-3.sh: Display the output on failure to ease debugging.
2014-03-26ptx: avoid --format long option falling through into --helpBernhard Voelker
* src/ptx.c (main): Add a 'break' after the --format handling case. Otherwise it would fall through into the usage case. * tests/misc/ptx.pl: Add test cases for --format=tex and --format=roff. * NEWS (Bug fixes): Mention the fix. Bug introduced in 1999-04-04 commit, SH-UTILS-1_16f-269-gd815c15. Spotted by coverity (MISSING_BREAK).
2014-03-26basename: mention that the -s option implies -a in usageBernhard Voelker
* src/basename.c (usage): Mention that -s implies -a. (main): Add "fall through" comment to case 's'. Spotted by coverity: MISSING_BREAK.
2014-03-24cp: simplify overwrite logicPaul Eggert
* src/copy.c (overwrite_ok): Rename from overwrite_prompt. Invoke yesno instead of having the caller do it; that's cleaner. Return bool, not void. All callers changed.
2014-03-24cp: improve quality of overwrite promptPaul Eggert
* src/copy.c (overwrite_prompt): New arg X. All callers changed. Use X to improve the quality of the prompt (Bug#17087). * tests/mv/i-2.sh, tests/mv/i-3.sh: Change test to match new prompt.
2014-03-24split: avoid unnecessary input bufferingPádraig Brady
Input buffering is best avoided because it introduces delayed processing of output for intermittent input, especially when the output size is less than that of the input buffer. This is significant when output is being further processed which could happen if split is writing to precreated fifos, or through --filter. If input is arriving quickly from a pipe then this will already be buffered before we read it, so fast arriving input shouldn't be a performance issue. * src/split.c (lines_split, lines_bytes_split, bytes_split, lines_chunk_split, bytes_chunk_extract): s/full_read/safe_read/. * THANKS.in: Mention the reporter. * NEWS: Mention the improvement.
2014-03-19doc: clarify the operation of uniq -uDPádraig Brady
The description of -u was inaccurate when combined with -D: $ printf '%s\n' '1 a' '2 a' | uniq -uD -f1 1 a * doc/coreutils.texi (uniq invocation): Clarify that it's the last repeated line that is suppressed from the output. Fixes http://bugs.gnu.org/17022
2014-03-19doc: avoid regex ambiguity in chmod mode descriptionEdward Welbourne
* man/chmod.x: s/[+-=]/[-+=]/ * doc/perm.texi (Symbolic Modes): Likewise. * THANKS.in: Remove patch owner. Fixes http://bugs.gnu.org/17041
2014-03-19chroot: always diagnose failure to set supplemental groupsPádraig Brady
* src/chroot.c (setgroups): Change this replacement to fail when called so that platforms like Interix without support for supplemental groups don't silently ignore a --groups option.
2014-03-19chmod: fix erroneous warnings with -R --changesDylan Simon
For files with "special" bits set, we would stat the relative file name in the wrong directory, giving an erroneous ENOENT diagnostic. This issue was introduced with commit v5.92-653-gc1994c1 which changed fts to not change directory on traversal. * src/chmod.c (mode_changed): Use fts->fts_cwd_fd with fstatat rather than stat. All callers changed. * tests/chmod/c-option.sh: Add a test case. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/17035
2014-03-19maint: fix typo in comment in fadvise.cBernhard Voelker
* gl/lib/fadvise.c: s/the the/the/, indroduced in commit v8.22-40-g4f21182. Promted by sc_prohibit_doubled_word. While at it, also s/be candidate/be a candidate/.
2014-03-18stat: port birthtime to Solaris 11Paul Eggert
Problem reported by Rich Burridge. * src/stat.c [HAVE_GETATTRAT]: Include <attr.h>, <sys/nvpair.h>. (print_statfs, print_stat, print_it): Pass fd, too, for the benefit of get_birthtime. All uses changed. (get_birthtime): New function, for porting to Solaris 11. (print_stat): Use it. * configure.ac (getattrat, LIB_NVPAIR): New checks. * src/local.mk (src_stat_LDADD): Add $(LIB_NVPAIR).
2014-03-13ln: with -sr, don't segfault for a TARGET of ''Jim Meyering
Prior to this change, "ln -sr '' F" would segfault, attempting to read path2[1] in relpath.c's path_common_prefix function. This problem arises whenever canonicalize_filename_mode returns NULL. * src/ln.c (convert_abs_rel): Call relpath only when both canonicalize_filename_mode calls return non-NULL. * tests/ln/relative.sh: Add a test to trigger this failure. * THANKS.in: List reporter's name/address. * NEWS (Bug fixes): Mention it. Reported by Erik Bernstein in 739752@bugs.debian.org. Fixes http://bugs.gnu.org/17010.
2014-03-13maint: avoid "attribute-const"-suggesting warning from gccJim Meyering
* gl/lib/fadvise.c: Use a pragma to turn off this warning option: -Wsuggest-attribute=const. Without this change, building with --enable-gcc-warnings would evoke this error: lib/fadvise.c:25:1: error: function might be candidate for\ attribute 'const' [-Werror=suggest-attribute=const]
2014-03-13scripts: autotools-install: updateJim Meyering
* scripts/autotools-install: Update version numbers of latest automake and gettext packages.
2014-03-13chroot: improve --userspec and --groups look-upPádraig Brady
- Support arbitrary numbers in --groups, consistent with what is already done for --userspec - Avoid look-ups entirely for --groups items with a leading '+' - Support names that are actually numbers in --groups - Ignore an empty --groups="" option for consistency with --userspec - Look up both inside and outside the chroot with inside taking precedence. The look-up outside may load required libraries to complete the look-up inside the chroot. This can happen for example with a 32 bit chroot on a 64 bit system, where the 32 bit NSS plugins within the chroot fail to load. * src/chroot.c (parse_additional_groups): A new function refactored from set_addition_groups(), to just do the parsing. The actual setgroups() call is separated out for calling from the chroot later. (main): Call parse_user_spec() and parse_additional_groups() both outside and inside the chroot for the reasons outlined above. * tests/misc/chroot-credentials.sh: Ensure arbitrary numeric IDs can be specified without causing look-up errors. * NEWS: Mention the improvements. * THANKS.in: Add Norihiro Kamae who initially reported the issue with a proposed patch. Also thanks to Dmitry V. Levin for his diagnosis and sample patch.
2014-03-09doc: fix typo in texinfo docsAleksej Serdjukov
* doc/coreutils.texi (Operating on characters): s/This/These/. Fixes http://bug.gnu.org/16973
2014-03-07doc: fix readpath typo in texinfo contentsPádraig Brady
* docs/coreutils.texi: s/readpath/realpath/ Fixes http://bugs.gnu.org/16964
2014-03-05tests: avoid the :> construct which can hide errorsPádraig Brady
On most shells `:>file || framework_failure_` will not evaluate the framework_failure_ even if there was an error writing the file. shells which do evaluate the failure are ksh 93u+ and bash 4.2, while shells wich don't include bash 4.3, solaris, freebsd, dash. Furthermore this construct is problematic on Solaris 10 sh, which will try to optimize away a `:' command in a loop after the first iteration, even if it is redirected. * tests/cp/link-deref.sh: Remove the leading colon on redirections. * tests/cp/reflink-perm.sh: Likewise. * tests/id/zero.sh: Likewise. * tests/install/install-C.sh: Likewise. * tests/misc/env.sh: Likewise. * tests/misc/md5sum-bsd.sh: Likewise. * tests/misc/runcon-no-reorder.sh: Likewise. * tests/mv/partition-perm.sh: Likewise. * tests/rm/r-root.sh: Likewise. * tests/split/l-chunk.sh: Likewise. * tests/split/line-bytes.sh: Likewise. * tests/tail-2/inotify-rotate.sh: Likewise. * tests/tail-2/retry.sh: Likewise. * tests/tail-2/symlink.sh: Likewise. * tests/tail-2/wait.sh: Likewise. * tests/touch/read-only.sh: Likewise. + cfg.mk (sc_prohibit_colon_redirection): A new syntax check to avoid further instances of this creeping in.
2014-03-05tests: fix false failure in nohup.sh in non tty buildsPádraig Brady
* tests/misc/nohup.sh: When running tests without a controlling tty, an exec failure is triggered in a subshell, which causes POSIX shells to immediately exit the subshell. This was brought to notice by the newly conforming bash 4.3. Fixes http:/bugs.gnu.org/16940
2014-03-03doc: improve df --human and --si, help and man pagePádraig Brady
* src/df.c (usage): Adjust the --human and --si descriptions to not depend on each other. Also include an example that is illustrative of the rounding, suffix, width, and localized fractions. * src/system.h (emit_size_note). Adjust so that it's obvious the description is pertaining to the input SIZE argument, and not to any sizes that might be output by df for example. Fixes http://bugs.gnu.org/16922
2014-02-27date: fix crash or infinite loop when parsing a malformed TZ=""Pádraig Brady
* NEWS: Mention the fix. * gnulib: Update to incorporate the fix. This is the only change in this gnulib update. * tests/misc/date.pl: Add a test for this case. Fixes http://bugs.gnu.org/16872
2014-02-27cp: copy files by inode only if that facility is availablePaul Eggert
* src/copy.c (copy_dir): Use the new SAVEDIR_SORT_FASTREAD, not SAVEDIR_SORT_INODE. Problem reported by Bernhard Voelker in: http://lists.gnu.org/archive/html/coreutils/2014-02/msg00037.html