summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2016-11-27shred,sort: ensure faster unaligned access to rand modulePádraig Brady
glibc has changed the public define from _STRING_ARCH_unaligned to _STRING_INLINE_unaligned as per https://sourceware.org/bugzilla/show_bug.cgi?id=19462 * gl/lib/rand-isaac.c: Cater for both defines. * gl/lib/randread.c: Likewise. * src/system.h: Update commented out code.
2016-11-26shuf: test input-closed bugPaul Eggert
Problem reported by Alex Ryan (Bug#25029). * tests/misc/shuf.sh: Test for shuffling with stdin closed.
2016-11-26build: update gnulib submodule to latestPaul Eggert
2016-11-25numfmt: pacify Sun C 5.14Paul Eggert
* src/numfmt.c (main): Don't implicitly coerce pointer in bool initializer. Although it's portable C99 code, it's confusing.
2016-11-25tests: fix false failure on new ls testPádraig Brady
* tests/ls/quote-align.sh: Remove "total" line which can vary per file system depending on allocation. Reported by Assaf Gordon on OpenSolaris (5.11/5.10).
2016-11-25pr: fix read from invalid memory with tabs in separatorPádraig Brady
This was detected with: echo a > a; pr "-S$(printf "\t\t\t")" a -m a > /dev/null Resulting in ASAN triggering: ==================================================== ERROR: AddressSanitizer: global-buffer-overflow READ of size 1 at 0x00000041b622 thread T0 #0 0x40506a in print_sep_string ../src/pr.c:2241 #1 0x407ec4 in read_line ../src/pr.c:2493 #2 0x40985c in print_page ../src/pr.c:1802 #3 0x40985c in print_files ../src/pr.c:1618 #4 0x4036e0 in main ../src/pr.c:1136 * src/pr.c (init_parameters): Ensure we only override the specified separator when it's a single tab, thus matching the calculated separator length. * tests/pr/pr-tests.pl: Add a test case. * NEWS: Mention the fix.
2016-11-24ptx: fix an invalid heap reference with short --widthPádraig Brady
* src/ptx.c (fix_output_parameters): Ensure line_width doesn't go negative, which can happen when the --width is less than the --gap-size. * tests/misc/ptx-overrun.sh: Add a test case that triggers with ASAN. (Note the longer filename is needed to trigger). Fixes http://bugs.gnu.org/25011
2016-11-24doc: clarify that readlink's --quiet option is on by defaultBernhard Voelker
* src/readlink.c (usage): Mark the --quiet/--silent option as active by default. * doc/coreutils.texi (readlink invocation): Likewise.
2016-11-24tests: fix false failure with ASAN in rm-readdir-failPádraig Brady
* tests/rm/rm-readdir-fail.sh: ASAN correctly indicated that fts was writing to freed memory. This was because we reused a single dirent in our readdir() test wrapper. Since fts was deallocating those dirents, we now get a new dirent for each call to our readdir wrapper.
2016-11-24split: fix memory corruption during chunk extractionPádraig Brady
ASAN reported this error for: split -n2/3 /dev/null ERROR: AddressSanitizer: negative-size-param: (size=-1) #0 0x7f0d4c36951d in __asan_memmove (/lib64/libasan.so.2+0x8d51d) #1 0x404e06 in memmove /usr/include/bits/string3.h:59 #2 0x404e06 in bytes_chunk_extract src/split.c:988 #3 0x404e06 in main src/split.c:1626 Specifically there would be invalid memory access and subsequent processing if the chunk to be extracted was beyond the initial amount read from file (which is currently capped at 128KiB). This issue is not in a released version, only being introduced in commit v8.25-4-g62e7af0 * src/split.c (bytes_chunk_extract): The initial_read != SIZE_MAX should have been combined with && rather than ||, but also this condition is always true in this function so remove entirely. * tests/split/b-chunk.sh: Add a test case. Fixes http://bugs.gnu.org/25003
2016-11-24tail: fix checking of remoteness when not using inotifyPádraig Brady
In recent commit v8.25-93-g7fc7206 we used the f->remote flag which wasn't set in all cases. This was detected with ASAN giving this error when reading f->remote; runtime error: load of value 190, which is not a valid value for type '_Bool' * src/tail.c (fremote): Query the system even without inotify. (recheck): Always set f->fremote for valid files.
2016-11-23build: avoid racy failure of "make -jN install"Jim Meyering
Installing with -j2 or greater could result in a failure like /bin/sh: line 29: /P/bin/install: Permission denied when /P/bin/install specifies your PATH-selected install program. This would arise because we're using "install" to install all man/*.1 files, and that command would run concurrently with the one that installs "/P/bin/install" itself. We would run this command: "src/ginstall src/ginstall /P/bin/install", and it would result in intervals during which the destination file does not exist, is empty or incomplete and not executable. We addressed this problem long ago for installation of actual binaries by telling the installation rules to use our just-built bin/ginstall (only when not cross-compiling) rather than the PATH-resolved "install" program. This change is to do the same for those .1 files. * src/local.mk (INSTALL): Override automake's default of something like "INSTALL = /P/bin/install -c". (INSTALL_PROGRAM): Now that we set INSTALL, there is no longer any need to set this derived variable. Its default definition, "INSTALL_PROGRAM = ${INSTALL}" does what we require. Improved by Eric Blake.
2016-11-23maint: fix 'syntax-check' targets for VPATH buildsBernhard Voelker
* cfg.mk (sc_gitignore_missing): Add $(srcdir) to .gitignore filename. (sc_gitignore_redundant): Likewise.
2016-11-23maint: add build-aux/ar-lib to .gitignoreBernhard Voelker
* .gitignore: Add entry for the above file which is created by './bootstrap'.
2016-11-23pr: fix integer overflow in buffer size calcsPaul Eggert
Problem reported by Marcel Böhme (Bug#24996). * configure.ac (WERROR_CFLAGS): Avoid -Wtype-limits. * src/pr.c (col_sep_string): Now a const pointer. All uses changed. (integer_overflow): New function. (separator_string, main, init_parameters, init_store_cols): Check for integer overflow. (align_column, read_line, print_stored): Avoid integer overflow.
2016-11-22comm: add --total optionBernhard Voelker
* src/comm.c (total_option): Add bool variable for the new option. (TOTAL_OPTION): Add enum value. (long_options): Add array element for the new option. (usage): Document the new option here. (compare_files): Count the lines in total[3], and output the summary at the end. (main): Accept the new option. * doc/coreutils.texi (comm invocation): Document it. * tests/misc/comm.pl: Test it. While at it, improve the test data to have 1 unique line in the first file, 2 unique lines in the second file, and 3 common lines. * NEWS (New Features): Mention the new option. Fixes http://bugs.gnu.org/24929
2016-11-22all: update gnulib submodule to latestPádraig Brady
Also sync these copies with gnulib: * bootstrap: Sync missed part of AIX support. * tests/init.sh: Support compare_ on AIX.
2016-11-22tests: fix false failure with FreeBSD 11 multibyte quotingPádraig Brady
* tests/misc/printf-quote.sh: FreeBSD 11 was seen to treat \u0378 as a printable character. Therefore change to using the \u0081 C1 control character. We use the UTF-8 representation because our printf implementation explicitly disallows \u0081 as input. Reported by Assaf Gordon
2016-11-22tests: fix false fails due to passing env vars to returns_Pádraig Brady
On BSD /bin/sh it was seen that unexported env vars passed to returns_() would not be propagated to the wrapped command. * cfg.mk (sc_prohibit_env_returns): Add a syntax check to disallow. * tests/misc/csplit-io-err.sh: Rearrange to export vars in a subshell. * tests/rm/rm-readdir-fail.sh: Likewise. * tests/misc/nohup.sh: Export and unset vars around returns_. * tests/misc/printenv.sh: Likewise. Reported by Assaf Gordon
2016-11-22tests: fix recent tail-2 test regressionsPádraig Brady
* tests/tail-2/pipe-f.sh: Avoid issue with readable directories on BSD systems. * tests/tail-2/retry.sh: Likewise. Reported by Assaf Gordon
2016-11-22ls: improve alignment of quoted namesPádraig Brady
This provides better alignment when some names are quoted, which also provides better indication that quotes are not part of the name. * src/ls.c (align_variable_outer_quotes): A new variable set when ls is aligning columns (not using -m, non-zero -w), and has a variable quoting style (shell, shell-escape, c-maybe). (quote_name_buf): Writes to buffer rather than FILE, taking care to avoid data copying if possible. Refactored from... (quote_name): ...here. This now manages the buffer passed to quote_name_buf() and outputs the padding, colors and name in the appropriate order, while managing the --dired offsets. (get_color_indicator): A new function to return the color sequence, refactored from... (print_color_indicator): ...here. This now simply outputs. (print_dir): Refactor common parts to quote_name(). (clear_files): Reset the flag indicating at least one file is quoted in the current directory. (needs_quoting): A new function to indicate at the scan stage whether a name needs quoting. Called from... (gobble_file): ...here, until we find the first quoted file. (print_name_with_quoting): Mostly refactored to quote_name(). * tests/ls/quote-align.sh: A new test for various output formats. * tests/local.mk: Reference the new test. * NEWS: Mention the improvement.
2016-11-22b2sum: a new checksum utility with md5sum like interfacePádraig Brady
Note we don't support the --algorithm option of the b2sum command in the external BLAKE2 project, as that was deemed too confusing for users. "BLAKE2b" was chosen as the default algorithm to use, which is single threaded but performs well on 64 bit. * src/blake2: CC0 source copied from external project. * cfg.mk[VC_LIST_ALWAYS_EXCLUDE_REGEX]: Exclude blake2/ from syntax checks, make update-copyright, etc. * src/local.mk: Reference the sources for b2sum, and set the compilation flags. * doc/coreutils.texi (b2sum invocation): Reference the md5sum invocation node, and add descriptions of -l. * tests/misc/b2sum.sh: Add new test. * tests/local.mk: Reference new test. * AUTHORS: Add new binary. * README: Likewise. * build-aux/gen-lists-of-programs.sh: Likewise. * man/.gitignore: Likewise. * scripts/git-hooks/commit-msg: Likewise. * man/b2sum.x: New man page template. * man/local.mk: Reference new template. * src/.gitignore: Ignore new binaries. * src/blake2/.gitignore: Ignore new build atrifacts. * src/md5sum.c (usage): Describe the new -l option. * NEWS: Mention the new program.
2016-11-22maint: avoid "make distcheck" failure due to excess .deps directoriesJim Meyering
* Makefile.am (my-distcheck): Remove all .deps directories before the recursive diff that searches for left-behind files. Otherwise, with automake master (some time after v1.15), "make distcheck" would fail due to those directories being left behind after "make distclean".
2016-11-19cp: improve status message when omitting directoriesPádraig Brady
* src/copy.h (cp_options): Add a new flag for install(1). * src/copy.c (copy_internal): For cp, also output remediation advice which also indicates why directories aren't copied by default. The message is unchanged for install(1). * src/cp.c (cp_option_init): Init install_mode to false. * src/mv.c (cp_option_init): Likewise. * src/install.c (cp_option_init): Init install_mode to true. * tests/install/basic-1.sh: Add a test case. * tests/cp/link-deref.sh: Adjust test case. Fixes http://bugs.gnu.org/24958
2016-11-16tests: fix non portable date in recent testPádraig Brady
* tests/misc/date.pl: Years < 1900 fail on solaris at least. Reported by Assaf Gordon
2016-11-15tests: fix false failure on OS XPádraig Brady
* tests/misc/sort-h-thousands-sep.sh: The sv_SE locale may be available while sv_SE.utf8 is not. Therefore test and use the same locale, now definfed in a variable. Reported by Jim Meyering.
2016-11-15dircolors: highlight Motion JPEG multimedia filesAntonio Ospite
* src/dircolors.hin: Add .mjpg and .mjpeg multimedia files. This resolves http://bugs.gnu.org/24436.
2016-11-13tests: use "returns_" rather than explicit comparison with "$?"Jim Meyering
The previous "returns_"-using change failed to convert many uses of "$?". Convert all but two of the remaining ones. * tests/ls/stat-vs-dirent.sh: Likewise. * tests/misc/head-write-error.sh: Likewise. * tests/misc/nice.sh: Likewise. * tests/misc/nohup.sh: Likewise. * tests/misc/stdbuf.sh: Likewise. * tests/misc/sync.sh: Likewise. * tests/tail-2/pid.sh: Likewise. * tests/tail-2/wait.sh: Likewise. Thanks to Bernhard Volker for spotting this.
2016-11-14all: reduce usage of proper_name_utf8()Pádraig Brady
This reduces a standard coreutils install size by about 160K. * src/cat.c: Change to proper_name() which removes about 18K text. * src/cp.c: Likewise. * src/df.c: Likewise. * src/du.c: Likewise. * src/getlimits.c: Likewise. * src/realpath.c: Likewise. * src/split.c: Likewise. * src/stdbuf.c: Likewise. * src/timeout.c: Likewise. * src/truncate.c: Likewise. * src/local.mk: Remove -llibiconv from the above programs. * cfg.mk (sc_check-AUTHORS): Adjust to use factor(1). * AUTHORS: Adjust to use ASCII to satisfy sc_check-AUTHORS.
2016-11-13maint: add 'returns_' to exclude list in sc_prohibit_and_fail_1Bernhard Voelker
The previous commit v8.25-96-g22063c8 lets 'make syntax-check' fail, because the above check falsely detects a case where 'returns_' is already in use. * cfg.mk (sc_prohibit_and_fail_1): Add 'returns_' to exclude list.
2016-11-13tests: use "returns_" rather than explicit comparison with "$?"Jim Meyering
* tests/chmod/no-x.sh: Likewise. * tests/cp/no-deref-link1.sh: Likewise. * tests/cp/no-deref-link2.sh: Likewise. * tests/cp/no-deref-link3.sh: Likewise. * tests/du/move-dir-while-traversing.sh: Likewise. * tests/ls/infloop.sh: Likewise. * tests/ls/stat-failed.sh: Likewise. * tests/ls/time-style-diag.sh: Likewise. * tests/misc/chroot-fail.sh: Likewise. * tests/misc/env-null.sh: Likewise. * tests/misc/env.sh: Likewise. * tests/misc/nice-fail.sh: Likewise. * tests/misc/printenv.sh: Likewise. * tests/misc/timeout.sh: Likewise. * tests/tail-2/follow-name.sh: Likewise.
2016-11-10tail: only retry file open if --retry specifedPádraig Brady
* src/tail.c (tail_file): On failure to open a file, set ignore=true when --retry is not specified. * tests/tail-2/assert-2.sh: Adjust to the new behavior. * tests/tail-2/retry.sh: Add a test case. Also change from `tail ... && fail=1` to the more robust `returns_ 1 ...` construct which detects segfaults etc. * NEWS: Document the fix.
2016-11-09tail: ensure -f --retry 'missing' handles truncationPádraig Brady
* src/tail.c (tail_forever): The BLOCKING optimization is only enabled for non regular files (which can't be truncated), so ensure we don't enable that unless we've a valid file descriptor. * tests/tail-2/retry.sh: Add a test case. * NEWS: Mention the bug fix.
2016-11-09tail: avoid outputting repeated data with remote filesPádraig Brady
* src/tail.c (tail_forever): Only read up to st_size on network file systems to avoid the issue with a stale attribute cache returning a smaller st_size than we have already read(). The was seen with glusterfs at least and caused the complete file to be repeatedly output due to assuming the file was truncated in this case. * NEWS: Mention the fix.
2016-11-08tail: terminate when following pipes and untailable non pipesPádraig Brady
* src/tail.c (ignore_pipe_or_fifo): Mark the descriptor as -1 for pipes so that any_live_files() detects correctly that the entry is no longer live. * tests/tail-2/pipe-f.sh: Add a test case. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/24903 which was detected using Symbolic Execution techniques developed in the course of the SYMBIOSYS research project at COMSYS, RWTH Aachen University.
2016-11-07maint: misc tweaks to recent changesPádraig Brady
* doc/coreutils.texi(cut invocation): Give a more accurate description of cut field handling limitations. * tests/misc/stat-fmt.sh: Fix the test header to be more general. * tests/tail-2/retry.sh: Spellings.
2016-11-07maint: refactor printing of backup suffix --helpPádraig Brady
* src/system.h (emit_backup_suffix_note): A new function to output the backup suffix info. The strings are unchanged, so translations are not impacted. * src/cp.c (usage): Use the new function. * src/ln.c (usage): Likewise. * src/mv.c (usage): Likewise. * src/install.c (usage): Likewise.
2016-11-07maint: simplify handling of backup --suffix in various toolsRishabh Dave
* src/cp.c (main): Avoid the getenv("SIMPLE_BACKUP_SUFFIX") call, which is now done if needed in the gnulib backupfile module. Also avoid the redundant strdup, as we don't modify this suffix. * src/install.c (main): Likewise. * src/ln.c (main): Likewise. * src/mv.c (main): Likewise. Fixes http://bugs.gnu.org/23153
2016-11-07date: add %q to output the quarter of the yearPádraig Brady
* doc/coreutils.texi (date invocation): Document %q. * src/date.c (usage): Likewise. * tests/misc/date.pl: Add a test case. * cfg.mk (sc_strftime_check): Adjust to allow %q. * NEWS: Mention the new feature.
2016-11-07all: update gnulib submodule to latestPádraig Brady
Incorporating these coreutils specific changes: backupfile: initialize default suffix within the implementation strftime,strptime: support %q to represent the quarter Also sync these copies with gnulib: * bootstrap: Now supports AIX. * gnulib: Update to the latest. * tests/init.sh: dash(1) is now disabled.
2016-11-05doc: clarify in texinfo that cut doesn't reorder fieldsPádraig Brady
* doc/coreutils.texi (cut invocation): Clarify the more sophisticated field processing supported by awk(1). Fixes http://bugs.gnu.org/24881
2016-11-05stat: make --format=%N honor the QUOTING_STYLE env varPádraig Brady
* doc/coreutils.texi (stat invocation): Describe the QUOTING_STYLE values now supported. * src/stat.c (getenv_quoting_style): A new function called from main, that sets the default quoting style for quotearg. (main): Call getenv_quoting_style() when %N specified. * tests/misc/stat-fmt.sh: Add a test case. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/23422
2016-11-05doc: improve the --help description of ls -NPádraig Brady
* src/ls.c (usage): The existing description was inaccurate as -N does not directly impact whether control characters are displayed or not.
2016-11-04dd: warn about counts specified with confusing 0x prefixPádraig Brady
* src/dd.c (parse_integer): Suggest to use "00x" instead of "0x", which is significant for the "count", "seek", and "skip" operands. * tests/dd/misc.sh: Add a test case. Fixes http://bugs.gnu.org/24874
2016-10-26md5sum,sha*sum: fix --ignore-missing with checksums starting with 00Pádraig Brady
* NEWS: Mention the fix. * src/md5sum.c (digest_file): Add a new MISSING parameter to return whether the file was missing, separately from the digest. * tests/misc/md5sum.pl: Add a test case. Fixes http://bugs.gnu.org/24795
2016-10-21doc: fix typo in stdbuf man pageBishop Bettini
* man/stdbuf.x: Correct spelling of immediately. Fixes http://bugs.gnu.org/24763
2016-10-16maint: we use die; remove now-unnecessary uses of abortJim Meyering
* src/expr.c (integer_overflow): Remove an abort-after-die. * src/paste.c (write_error): Likewise. * src/sort.c (badfieldspec): Likewise. * src/tr.c (string2_extend): Likewise. Also remove a few break statements, each after an abort.
2016-10-16all: use die() rather than error(EXIT_FAILURE)Pádraig Brady
die() has the advantage of being apparent to the compiler that it doesn't return, which will avoid warnings in some cases, and possibly generate better code. * cfg.mk (sc_die_EXIT_FAILURE): A new syntax check rule to catch any new uses of error (CONSTANT, ...);
2016-10-15build: also distribute new file, src/die.hJim Meyering
* src/local.mk (noinst_HEADERS): Add die.h.
2016-10-15build: add die.h; avoid new warnings from GCC 7Jim Meyering
* src/die.h (die): New file/function from grep. Note: we expect this file to migrate to gnulib. * src/csplit.c: Include die.h. (check_format_conv_type): Use die in place of error-nonzero;break; * src/install.c (strip): Likewise. * src/nl.c (proc_text): Likewise. This also suppresses a new warning from GCC 7's -Werror=strict-overflow. * src/tail.c (parse_options): Likewise. * src/basename.c (main): Adjust "fall through" comment so that GCC 7's -Wimplicit-fallthrough honors it. * src/cp.c (main): Add a "fall through" comment. * src/ls.c (gobble_file): Likewise. (get_funky_string): Adjust a "fall through" comment so it is recognized. * cfg.mk (exclude_file_name_regexp--sc_system_h_headers): Add die.h to this list of exempt src/*.h files.
generated by cgit v1.2.3-70-g09d2 (git 2.47.0) at 2024-11-14 17:28:12 +0000