summaryrefslogtreecommitdiff
path: root/tests/misc
AgeCommit message (Collapse)Author
2015-01-05shuf: do not mishandle 'shuf -i0-0 1'Paul Eggert
Problem reported by Daiki Ueno in: http://bugs.gnu.org/19520 * src/shuf.c (main): Avoid core dump if !input_range. * tests/misc/shuf.sh: Test for this bug.
2015-01-01maint: update all copyright year number rangesPádraig Brady
Run "make update-copyright" and then... * tests/sample-test: Adjust to use the single most recent year. * tests/du/bind-mount-dir-cycle-v2.sh: Fix case in copyright message, so that year is updated automatically in future.
2014-12-25tests: fix error message check on some systemsPádraig Brady
http://hydra.nixos.org/build/18129583 identified (on OS X) an incorrect test assumption in the previous commit. * gl/lib/xdectoint.c (__xnumtoint): Suppress the EINVAL error message as it's redundant in this context. * tests/misc/tail.pl: Suppress _optionally_ appended strerror messages. * tests/fmt/base.pl: Likewise. * tests/pr/pr-tests.pl: Likewise. * tests/split/l-chunk.sh: Likewise.
2014-12-19diagnose too-large numbers betterPádraig Brady
Following on from commit v8.23-82-gaddae94, consistently diagnose numbers that are too large, so as to distinguish from other errors, and make the limits obvious. * gl/modules/xdectoint: A new module implementing xdecto[iu]max(), which handles the common case of parsing a bounded integer and exiting with a diagnostic on error. * gl/lib/xdectoimax.c: The signed variant. * gl/lib/xdectoint.c: The parameterized implementation. * gl/lib/xdectoint.h: The interface. * gl/lib/xdectoumax.c: The unsigned variant. * bootstrap.conf: Reference the new module. * cfg.mk (exclude_file_name_regexp--sc_require_config_h_first): Exclude the parameterized templates. * src/csplit.c: Output EOVERFLOW or ERANGE errors if appropriate. * src/fmt.c: Likewise. * src/fold.c: Likewise. * src/head.c: Likewise. * src/ls.c: Likewise. * src/nl.c: Likewise. * src/nproc.c: Likewise. * src/shred.c: Likewise. * src/shuf.c: Likewise. * src/stdbuf.c: Likewise. * src/stty.c: Likewise. * src/tail.c: Likewise. * src/truncate.c: Likewise. * src/split.c: Likewise. * src/pr.c: Likewise. * tests/pr/pr-tests.pl: Adjust to avoid matching errno diagnostic. * tests/fmt/base.pl: Likewise. * tests/split/l-chunk.sh: Likewise. * tests/misc/shred-negative.sh: Likewise. * tests/misc/tail.pl: Likewise. Also remove the redundant existing ERR_SUBST from test err-6. * tests/ls/hex-option.sh: Check HEX/OCT options. * tests/misc/shred-size.sh: Likewise. * tests/misc/stty-row-col.sh: Likewise.
2014-12-10build: use the system printf routines in most casesPádraig Brady
glibc <= 2.5 would crash when passed invalid long double values, therefore internal gnulib routines were used, essentially only by od, to output such invalid values. Later glibc versions don't crash, as per https://sourceware.org/bugzilla/show_bug.cgi?id=4586 and subsequently od was adjusted to use the system printf routines through the use of the ftoastr module with commit v8.7-22-ga71c22f. Consequently our testing of this feature was moot, and use of the gnulib printf replacement for printf(1), od(1) and error(3) etc. was redundant. * configure.ac (gl_printf_safe): Unset so that we don't check that "nan" is output for these long double values. * tests/misc/od-float.sh: Adjust all existing checks to fail if od exits with failure status (like crashing for example). Add a new case for one of the problematic invalid long double values for x86_64. We only check that od exits successfully at present, which may change if https://sourceware.org/bugzilla/show_bug.cgi?id=17661 is resolved.
2014-11-21tests: chcon: avoid false failure with newer selinuxPádraig Brady
file_t is now mapped to unlabeled_t as per: http://danwalsh.livejournal.com/68189.html Therefore use the latter to ensure we match correctly. This is needed on >= Fedora 21 for example, while it also works on earlier releases.
2014-10-30maint: tests: fix comments about retry_delay_Pádraig Brady
* tests/tail-2/F-vs-missing.sh: Comment with the correct total delay. * tests/tail-2/F-vs-rename.shi: Likewise. * tests/tail-2/flush-initial.sh: Likewise. * tests/tail-2/inotify-hash-abuse.sh: Likewise. * tests/tail-2/pipe-f2.sh: Likewise. * tests/misc/chroot-fail.sh: Initialize can_chroot_root in all cases.
2014-10-17tests: avoid false failure when comparing /proc filesBernhard Voelker
At least the MHz number in /proc/cpuinfo may change, thus leading to a false positive failure when comparing the expected against the actual output file. Use an invariant file instead: /proc/version. * tests/misc/head-c.sh: s/cpuinfo/version/
2014-10-16chroot: call chroot() unconditionally to handle bind mounted "/"Pádraig Brady
* src/chroot.c (is_root): Adjust to compare canonicalized paths rather than inodes, to handle (return false in) the case where we have a tree that is constructed by first bind mounting "/" (thus having the same inode). (main): Unconditionally call chroot() because it's safer and of minimal performance benefit to avoid in this case. This will cause inconsistency with some platforms not allowing `chroot / true` for non root users. * tests/misc/chroot-fail.sh: Adjust appropriately. * NEWS: Mention the bug fixes. Fixes http://bugs.gnu.org/18736
2014-10-08maint: avoid syntax-check failures in previous patchPádraig Brady
* tests/misc/od-j.sh: Non standard comparison order. * tests/split/b-chunk.sh: Confusing input file name. * tests/tail-2/tail-c.sh: Redundant require ulimit.
2014-10-07wc: don't miscount /sys and similar file systemsPaul Eggert
Fix similar problems in head, od, split, tac, and tail. Reported by George Shuklin in: http://bugs.gnu.org/18621 * NEWS: Document this. * src/head.c (elseek): Move up. (elide_tail_bytes_pipe, elide_tail_lines_pipe): New arg CURRENT_POS. All uses changed. (elide_tail_bytes_file, elide_tail_lines_file): New arg ST and remove arg SIZE. All uses changed. * src/head.c (elide_tail_bytes_file): * src/od.c (skip): Avoid optimization for /sys files, where st_size is bogus and st_size == st_blksize. Don't report error at EOF when not optimizing. * src/head.c, src/od.c, src/tail.c: Include "stat-size.h". * src/split.c (input_file_size): New function. (bytes_split, lines_chunk_split, bytes_chunk_extract): New arg INITIAL_READ. All uses changed. Use it to double-check st_size. * src/tac.c (tac_seekable): New arg FILE_POS. All uses changed. (copy_to_temp): Return size of temp file. All uses changed. * src/tac.c (tac_seekable): * src/tail.c (tail_bytes): * src/wc.c (wc): Don't trust st_size; double-check by reading. * src/wc.c (wc): New arg CURRENT_POS. All uses changed. * tests/local.mk (all_tests): Add tests/misc/wc-proc.sh, tests/misc/od-j.sh, tests/tail-2/tail-c.sh. * tests/misc/head-c.sh: * tests/misc/tac-2-nonseekable.sh: * tests/split/b-chunk.sh: Add tests for problems with /proc and /sys files. * tests/misc/od-j.sh, tests/misc/wc-proc.sh, tests/tail-2/tail-c.sh: New files.
2014-09-24test: check for Fedora 20 sort key bugPaul Eggert
Problem reported by Göran Uddeborg in: http://bugs.gnu.org/18540 * tests/misc/sort.pl: New test 23.
2014-09-11cat: allow copying empty files to themselvesPaul Eggert
Problem reported by Vincent Lefevre in: http://bugs.gnu.org/18449 * src/cat.c (main): Allow copying an empty file to itself. * tests/misc/cat-self.sh: New test. * tests/local.mk (all_tests): Add it.
2014-08-01chroot: perform chdir("/") again unless new --skip-chdir is specifiedBernhard Voelker
Since commit v8.22-94-g99960ee, chroot(1) skips the chroot(2) syscall for "/" arguments (and synonyms). The problem is that it also skips the following chdir("/") call in that case. The latter breaks existing scripts which expect "/" to be the working directory inside the chroot. While the first part of the change - i.e., skipping chroot("/") - is okay for consistency with systems where it might succeed for a non-root user, the second part might be malicious, e.g. cd /home/user && chroot '/' bin/foo In the "best" case, chroot(1) could not execute 'bin/foo' with ENOENT, but in the worst case, chroot(1) would execute '/home/user/bin/foo' in the case that exists - instead of '/bin/foo'. Revert that second part of the patch, i.e., perform the chdir("/) in the common case again - unless the new --skip-chdir option is specified. Restrict this new option to the case of "/" arguments. * src/chroot.c (SKIP_CHDIR): Add enum. (long_opts): Add entry for the new --skip-chdir option. (usage): Add --skip-chdir option, and while at it, move the other to options into alphabetical order. (main): Accept the above new option, allowing it only in the case when NEWROOT is the old "/". Move down the chdir() call after the if-clause to ensure it is run in any case - unless --skip-chdir is specified. Add a 'newroot' variable for the new root directory as it is used in a couple of places now. * tests/misc/chroot-fail.sh: Invert the last tests which check the working directory of the execvp()ed program when a "/"-like argument was passed: now expect it to be "/" - unless --skip-chdir is given. * doc/coreutils.texi (chroot invocation): Document the new option. Document that chroot(1) usually calls chdir("/") unless the new --skip-chdir option is specified. Sort options. * NEWS (Changes in behavior): Mention the fix. (New features): Mention the new option. * init.cfg (nonroot_has_perm_): Add chroot's new --skip-chdir option. * tests/cp/preserve-gid.sh (t1): Likewise. * tests/cp/special-bits.sh: Likewise. * tests/id/setgid.sh: Likewise. * tests/misc/truncate-owned-by-other.sh: Likewise. * tests/mv/sticky-to-xpart.sh: Likewise. * tests/rm/fail-2eperm.sh: Likewise. * tests/rm/no-give-up.sh: Likewise. * tests/touch/now-owned-by-other.sh: Likewise. Reported by Andreas Schwab in http://bugs.gnu.org/18062
2014-07-18tests: fix false failure with spaces in the test directoryPádraig Brady
* tests/misc/env.sh: Skip if we can't execute the generated shebang, which would be the case if there are spaces in the directory hierarchy. This is triggered by `make distcheck`
2014-07-14tests: fix portabilitiy issue in new od-endian testPádraig Brady
* tests/misc/od-endian.sh: '\n' is not generally supported in the replacement, so use the more portable "\\$NL".
2014-07-13build: support building all tools in a single binaryAlex Deymo
Add the --enable-single-binary option to the configure file. When enabled, this option builds a single binary file containing the selected tools. Which tool gets executed depends on the value of argv[0] which can be set implicitly through symlinks to the single program. This setup reduces significantly the size of a complete coreutils install, since code from lib/libcoreutils.a is not duplicated in every one of the more than 100 binaries. Runtime overhead is increased due to more dynamic libraries being loaded, and extra initialization being performed for all utils. Also initially a larger binary is loaded from storage, though this is usually alleviated due to caching and lazy mmaping of unused blocks, and in fact the single binary should have better caching characteristics. Comparing the size of the individual versus single binary on x86_64: $ cd src $ size coreutils $ size -t $(../build-aux/gen-lists-of-programs.sh --list-progs | grep -Ev '(coreutils|libstdbuf)') | tail -n1 text data bss dec hex filename 1097416 5388 88432 1191236 122d44 src/coreutils 4901010 124964 163768 5189742 4f306e (TOTALS) Storage requirements are reduced similarly: $ cd src $ du -h coreutils $ du -ch $(../build-aux/gen-lists-of-programs.sh --list-progs | grep -Ev '(coreutils|libstdbuf)') | tail -n1 1.2M coreutils 5.3M total When installing, the makefile will create either symlinks or shebangs based on the --enable-single-binary setting, for each configured tool. In this way, all the tools are still callable individually, but they are all implemented by the same "coreutils" binary installed on the same directory. * .gitignore: Add new generated files. * Makefile.am: New rules to generate build-aux/gen-single-binary.sh and install symlinks. * NEWS: Mention the new feature. * README: Add "coreutils" to the list of utils. * bootstrap.conf: Regenerate src/single-binary.mk * build-aux/gen-lists-of-programs.sh: New --list-progs option. * build-aux/gen-single-binary.sh: Regenerate * configure.ac: New --enable-single-binary option and other variables. Disallow --enable-single-binary=symlinks with --program-prefix et. al. * man/coreutils.x: Manpage hook. * man/local.mk: Add manpage hook and fix dependencies. * src/coreutils.c: Multicall implementation. * src/local.mk: New rules for the single binary option. * tests/local.mk: Add $single_binary_progs to support require_built_() from init.cfg * tests/misc/env.sh: Avoid the use of symlink to echo. * tests/misc/help-version.sh: Add exception for coreutils. * tests/install/basic-1.sh: Really avoid using ginstall strip functionality if there is an issue with the independent strip command. * src/kill.c: Changes to call exit() in main. * src/readlink.c: Likewise. * src/shuf.c: Likewise. * src/timeout.c: Likewise. * src/truncate.c: Likewise.
2014-07-04maint: simplify parsing df's output in shell scriptsBernhard Voelker
Avoid complicated and error-prone parsing of df's output via sed(1), cut(1), etc., and instead use df's more modern --output option. * src/ioblksize.h (in a comment): Simplify the extraction of the device name of the mounted file system from df's output. * tests/dd/skip-seek-past-dev.sh: Likewise. * tests/du/2g.sh: Likewise for the 'avail' column here. Also avoid the deprecated use of "tail -NUM". * tests/misc/stat-mount.sh: While at it, remove the determination of the mount point of "." via df(1) plus sed(1) as it is unused since commit v8.5-159-gf57cb37 anyway. Instead, improve this test by verifying that the output of "stat -c%m ." at least starts with a slash '/'.
2014-07-02pwd: revert default mode back to -PPádraig Brady
Revert commit v8.22-131-g3e89d5b as even though POSIX states that the default mode should be -L, common practice for stand-alone pwd implementations is to default to -P. * src/pwd.c (usage): Retain mention of the default mode of operation. Suggested by Bob Proulx
2014-07-02tests: use "env builtin" in place of "$abs_top_builddir/src/builtin"Jim Meyering
This works for any program that might also be a shell built-in. It is more concise. In addition, it makes output more reproducible: some diagnostics include argv[0], which will now be just the program name, rather than the full absolute name of the executable. * tests/misc/sort-compress.sh: Use env kill, rather than absolute name. * tests/install/trap.sh: Likewise. * tests/misc/timeout.sh: Likewise. * tests/touch/no-dereference.sh: Do the same for two uses of "test". * tests/touch/no-create-missing.sh: Likewise.
2014-07-01tests: adapt tests to change in pwd's defaultJim Meyering
* tests/misc/pwd-option.sh (base): Initialize with -P, now that -L is the default, to accommodate an initial working directory with a symlink component. * tests/misc/readlink-fp-loop.sh: Use $(env pwd -P) to get the absolute working directory. Using "env" ensures we do not invoke any shell built-in, and PATH ensures we invoke the one from coreutils. * tests/readlink/can-e.sh: Likewise. * tests/readlink/can-f.sh: Likewise. * tests/readlink/can-m.sh: Likewise.
2014-06-30pwd: assume -L mode by default, as per POSIXPádraig Brady
* src/pwd.c (main): Adjust default mode to be "logical" and independent of the POSIXLY_CORRECT env var. (usage): Mention the default mode of operation. * doc/coreutils.texi (pwd invocation): Adjust accordingly. * tests/misc/pwd-option.sh: Likewise. * NEWS: Mention the change in behavior.
2014-06-26tests: use predetermined NON_ROOT_GIDPádraig Brady
* HACKING: GID is more useful in tests than group name, so rename input param from NON_ROOT_GROUP to NON_ROOT_GID to make it obvious that only a group ID is now acceptable, thus allowing GID lookups to be avoided throughout the tests. * init.cfg (require_root_): Likewise. * tests/misc/truncate-owned-by-other.sh: Avoid looking up the GID. * tests/touch/now-owned-by-other.sh: Likewise. * tests/misc/chroot-credentials.sh: Likewise. Also fix an instance of comparison against NON_ROOT_GROUP which would have given a false failure if a non numeric value was passed in. * tests/id/setgid.sh: Use previously looked up gid as a more accurate base for the subsequent adjustment, and move the uid lookup within chroot, rather than having the overhead of a separate `id` invocation.
2014-06-18seq: fix incorrect output with start or end of -0Pádraig Brady
* src/seq.c (main): Avoid seq_fast() with a start or end of -0. * tests/misc/seq.pl: Add test cases. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/17800
2014-06-03tests: consolidate tests for true and falsePádraig Brady
* src/true.c (main): Add a comment about the possibility of true returning EXIT_FAILURE due to write failure. * tests/misc/false-status.sh: Fix so we're testing the tool and not the shell builtin. Add a case for true(1). * tests/misc/help-version.sh: Skip /dev/full test for true as well as false since the exit status is tested separately. Also remove the iterations for different LC_MESSAGES, as this was only applied for false(1). Translations are not honored in the test dir and so would need separate handling in any case.
2014-06-01cut: restore special case handling of -f with -d$'\n'Pádraig Brady
commits v8.20-98-g51ce0bf and v8.20-99-gd302aed changed cut(1) to process each line independently and thus promptly output each line without buffering. As part of those changes we removed the special handling of --delimiter=$'\n' --fields=... which could be used to select arbitrary (ranges of) lines, so as to simplify and optimize the implementation while also matching the behavior of different cut(1) implementations. However that GNU behavior was in place for a long time, and could be useful in certain cases like making a separated list like `seq 10 | cut -f1- -d$'\n' --output-delimiter=,` although other tools like head(1) and paste(1) are more suited to this operation. This patch reinstates that functionality but restricts the "line behind" buffering behavior to only the -d$'\n' case. We also fix the following related edge case to be more consistent: before> printf "\n" | cut -s -d$'\n' -f1- | wc -l 2 before> printf "\n" | cut -d$'\n' -f1- | wc -l 1 after > printf "\n" | cut -s -d$'\n' -f1- | wc -l 1 after > printf "\n" | cut -d$'\n' -f1- | wc -l 1 * src/cut.c (cut_fields): Adjust as discussed above. * tests/misc/cut.pl: Likewise. * NEWS: Mention the change in behavior both for v8.21 and this effective revert. * cfg.mk (old_NEWS_hash): Adjust for originally omitted v8.21 entry. * src/paste.c: s/delimeter/delimiter/ comment typo fix.
2014-05-21tests: use chroot --user rather than internal setuidgid toolPádraig Brady
* init.cfg (require_root_): Adjust to use chroot, and make `require_built_ chroot` implicit when chroot used in the test. * po/POTFILES.in: Remove reference to setuidgid tool. * src/.gitignore: Likewise. * src/local.mk: Likewise. * src/setuidgid.c: Remove. * tests/cp/preserve-gid.sh: s/setuidgid/chroot --user/. * tests/cp/special-bits.sh: Likewise. * tests/id/setgid.sh: Likewise. * tests/misc/truncate-owned-by-other.sh * tests/mv/sticky-to-xpart.sh: Likewise. * tests/rm/fail-2eperm.sh: Likewise. * tests/rm/no-give-up.sh: Likewise. * tests/touch/now-owned-by-other.sh: Likewise. * tests/misc/chroot-fail.sh: Skip if chroot not built.
2014-05-21chroot: make changing root check more robustPádraig Brady
* src/chroot.c (is_root): A new helper function to determine if the passed argument is the root directory based on inode comparison. (main): Use the new helper rather than comparing strings. * tests/misc/chroot-fail.sh: Add cases for alternative root paths.
2014-05-21chroot: with --userspec clear root's supplemental groupsPádraig Brady
It's dangerous and confusing to leave root's supplemental groups in place when specifying other users with --userspec. In the edge case that that is desired one can explicitly specify --groups. Also we implicitly set the system defined supplemental groups for a user. The existing mechanism where supplemental groups needed to be explicitly specified is confusing and not general when the lookup needs to be done within the chroot. Also we extend the --groups syntax slightly to allow clearing the set of supplementary groups using --groups=''. * src/chroot.c (setgroups): On systems without supplemental groups, clearing then is a noop and so should return success. (main): Lookup the primary GID with getpwuid() when just a numeric uid is specified, and also infer the USERNAME from this call, needed when we're later looking up the supplemental groups for a user. Support clearing supplemental groups, either implicitly for unknown users, or explicitly when --groups='' is specified. * tests/misc/chroot-credentials.sh: Various new test cases * doc/coreutils.texi (chroot invocation): Adjust for the new behavior. * NEWS: Mention the change in behavior.
2014-05-16chroot: don't chdir() if not changing rootPádraig Brady
This allows chroot to be used as a light weight tool to change user identification for a command, while not changing the current working directory. It also makes `chroot / true` consistently succeed on all platforms for non root users. * src/chroot.c (main): If the same root is specified. i.e. '/' then don't change the current working directory, and avoid the overhead of the other redundant calls. * tests/misc/chroot-fail.sh: Remove failure guard previously needed on some systems. Also add an explicit case to ensure we don't change directory. * NEWS: Mention the change in behavior.
2014-05-10shred: don't infloop upon negative sizeJim Meyering
* src/shred.c (main): With the preceding change, shred -s-2 FILE would write 64KB blocks forever -- or until disk full. This change makes shred reject a negative size. * tests/misc/shred-negative.sh: New file. * tests/local.mk (all_tests): Add it.
2014-05-07tests: improve diagnostics when asserting empty filesPádraig Brady
* tests/chmod/c-option.sh: Use `compare /dev/null ... || fail=1` rather than `test -s ... && fail=1`, so that the file contents are output, thus improving diagnostics for failing tests. * tests/cp/acl.sh: Likewise. * tests/cp/cp-a-selinux.sh: Likewise. * tests/cp/cp-mv-enotsup-xattr.sh: Likewise. * tests/cp/reflink-perm.sh: Likewise. * tests/dd/misc.sh: Likewise. * tests/misc/env-null.sh: Likewise. * tests/misc/env.sh: Likewise. * tests/misc/nice.sh: Likewise. * tests/misc/nohup.sh: Likewise. * tests/misc/printenv.sh: Likewise. * tests/misc/xattr.sh: Likewise. * tests/mv/update.sh: Likewise. * tests/rm/deep-2.sh: Likewise. * tests/rm/read-only.sh: Likewise. * tests/split/r-chunk.sh: Likewise. * tests/tail-2/follow-stdin.sh: Likewise. * tests/tail-2/inotify-race.sh: Likewise. * tests/tail-2/wait.sh: Likewise. * tests/touch/no-dereference.sh: Likewise. * cfg.mk (sc_prohibit_test_empty:): New syntax-check. * tests/cp/proc-zero-len.sh: Adjust to avoid false syntax-check failure. * tests/cp/proc-zero-len.sh: Likewise. * tests/mv/part-symlink.sh: Likewise. * tests/tail-2/infloop-1.sh: Likewise.
2014-05-02numfmt: support zero padding using --format="%010f"Pádraig Brady
* src/numfmt.c (setup_padding_buffer): Simplify the code by not explicitly dealing with heap exhaustion. (parse_format_string): Likewise. Handle multiple grouping modifiers as does the standard printf. Handle the new leading zero --format modifier. (double_to_human): Use more defensive coding against overwriting stack buffers. Honor the leading zeros width. (usage): Mention the leading zero --format modifier. (main): Allow --padding in combo with a --format (width), as the number of leading zeros are useful independent of the main field width. * doc/coreutils.texi (numfmt invocation): Likewise. * tests/misc/numfmt.pl: Add new test cases. * NEWS: Mention the improvement.
2014-04-29ptx: fix whitespace trimming with multiple filesPádraig Brady
This issue was identified by running the test suite with http://code.google.com/p/address-sanitizer/ which is included in GCC 4.8 and enabled with -fsanitize=address This was checked on Fedora 20 with GCC 4.8 as follows: $ yum install libasan # http://bugzilla.redhat.com/991003 $ rm -f src/ptx.o $ make check AM_CFLAGS='-fsanitize=address' SUBDIRS=. VERBOSE=yes $ failure identified in tests/test-suite.log To see this particular failure triggered with multiple files: $ src/ptx <(echo a) <(echo a) 2>&1 | asan_symbolize.py -d ================================================================= ==32178==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000e74f at pc 0x435442 bp 0x7fffe8a1b290 sp 0x7fffe8a1b288 READ of size 1 at 0x60200000e74f thread T0 #0 0x435441 in define_all_fields coreutils/src/ptx.c:1425 #1 0x7fa206d31d64 in __libc_start_main ??:? #2 0x42f77c in _start ??:? 0x60200000e74f is located 1 bytes to the left of 3-byte region [0x60200000e750,0x60200000e753) allocated by thread T0 here: #0 0x421809 in realloc ??:? #1 0x439b4e in fread_file coreutils/lib/read-file.c:97 Shadow bytes around the buggy address: 0x0c047fff9c90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9ca0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cb0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fd fd =>0x0c047fff9ce0: fa fa 03 fa fa fa fd fd fa[fa]03 fa fa fa 00 00 0x0c047fff9cf0: fa fa 04 fa fa fa 04 fa fa fa fd fa fa fa fd fa 0x0c047fff9d00: fa fa 00 fa fa fa fd fa fa fa 00 fa fa fa 00 fa 0x0c047fff9d10: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa 0x0c047fff9d20: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa 0x0c047fff9d30: fa fa fd fa fa fa 00 fa fa fa 00 fa fa fa 00 fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Heap right redzone: fb Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack partial redzone: f4 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 ASan internal: fe ==32178==ABORTING The initial report and high level analysis were from Jim Meyering... "The underlying problem is that swallow_file_in_memory() is setting the contents of the global text_buffer for the first file, then updating it (clobbering old value) for the second file. Yet, some pointers to the initial buffer have been squirreled away and later, one of them (keyafter) is presumed to point into the new "text_buffer", which it does not. The subsequent SKIP_WHITE_BACKWARDS use backs up "cursor" and goes out of bounds." * src/ptx.c (text_buffers): Maintain references for the limits of each buffer corresponding to each file, rather than just the last processed. (struct OCCURS): Add a member to map back to the corresponding file. Note normally this could be computed from the "reference" member rather than needing the extra storage, however this is not possible when in --references mode. (find_occurs_in_text): Reference the array rather than a single entry. (define_all_fields): Likewise. Also avoid computing the file index since this is now stored directly. (main): Update text_buffers[] array rather than a single text_buffer. * tests/misc/ptx-overrun.sh: Even though this issue is already triggered with AddressSanitizer, add a new case to demonstrate the whitespace trimming issue, and to trigger without AddressSanitizer. Fixes https://bugs.gnu.org/16171
2014-04-18tests: make ls tests independent of COLORTERM envPádraig Brady
Since the recent commit v8.22-68-g08783f1, ls coloring is now dependent on the COLORTERM environment variable. * tests/envvar-check: Unset COLORTERM from test environment. * tests/ls/color-dtype-dir.sh: Ensure coloring is used. * tests/misc/ls-misc.pl: Likewise. Prompted by the continuous integration build failure at: http://hydra.nixos.org/build/10397646
2014-04-05maint: various cleanupsPádraig Brady
* tests/misc/numfmt.pl: Fix comment misspelling. * src/cut.c: Likewise. * src/tsort.c (detect_loop): Replace an fprintf() with error().
2014-04-05shred: overwrite inode storage used by some file systemsPádraig Brady
* doc/coreutils.texi (shred invocation): Mention some reasons why clearing slack space might be useful. * src/shred.c (do_wipefd): Add initial writes for each pass for small regular files in case the storage for those is in the inode, and thus a larger write up to a block size would bypass that. Move the direct I/O control to... (dopass): ... here so we can avoid enabling it for these small initial writes. It's better to retry direct I/O for each pass anyway to handle the case where direct I/O is disabled for only the last portion of a file when the size is not a multiple of the block size. Note we don't avoid the sync for the initial write as it will be small but more importantly could be on a different part of the disk and so worth doing independently to ensure the write is not discarded. * tests/misc/shred-exact.sh: Check some more direct I/O cases. * NEWS: Mention the improvements. The inode storage issue was mentioned by Paul Eggert.
2014-04-04shred: avoid a data pass on empty filesPádraig Brady
* src/shred.c (do_wipefd): Don't increase the size written for an empty file up to a full block. Also increase the size to OFF_T_MAX in the edge case where we do overflow. * NEWS: Mention the shred improvements from recent changes. * tests/misc/shred-passes.sh: Adjust as we no longer write a BLKSIZE of data for empty files.
2014-03-31head: fix bug with head -c-N when stdin is not at startPaul Eggert
* src/head.c (elide_tail_bytes_file): Fix typo in lseek invocation. * tests/misc/head-c.sh: Add test for this bug. * NEWS: Document this.
2014-03-26ptx: avoid --format long option falling through into --helpBernhard Voelker
* src/ptx.c (main): Add a 'break' after the --format handling case. Otherwise it would fall through into the usage case. * tests/misc/ptx.pl: Add test cases for --format=tex and --format=roff. * NEWS (Bug fixes): Mention the fix. Bug introduced in 1999-04-04 commit, SH-UTILS-1_16f-269-gd815c15. Spotted by coverity (MISSING_BREAK).
2014-03-13chroot: improve --userspec and --groups look-upPádraig Brady
- Support arbitrary numbers in --groups, consistent with what is already done for --userspec - Avoid look-ups entirely for --groups items with a leading '+' - Support names that are actually numbers in --groups - Ignore an empty --groups="" option for consistency with --userspec - Look up both inside and outside the chroot with inside taking precedence. The look-up outside may load required libraries to complete the look-up inside the chroot. This can happen for example with a 32 bit chroot on a 64 bit system, where the 32 bit NSS plugins within the chroot fail to load. * src/chroot.c (parse_additional_groups): A new function refactored from set_addition_groups(), to just do the parsing. The actual setgroups() call is separated out for calling from the chroot later. (main): Call parse_user_spec() and parse_additional_groups() both outside and inside the chroot for the reasons outlined above. * tests/misc/chroot-credentials.sh: Ensure arbitrary numeric IDs can be specified without causing look-up errors. * NEWS: Mention the improvements. * THANKS.in: Add Norihiro Kamae who initially reported the issue with a proposed patch. Also thanks to Dmitry V. Levin for his diagnosis and sample patch.
2014-03-05tests: avoid the :> construct which can hide errorsPádraig Brady
On most shells `:>file || framework_failure_` will not evaluate the framework_failure_ even if there was an error writing the file. shells which do evaluate the failure are ksh 93u+ and bash 4.2, while shells wich don't include bash 4.3, solaris, freebsd, dash. Furthermore this construct is problematic on Solaris 10 sh, which will try to optimize away a `:' command in a loop after the first iteration, even if it is redirected. * tests/cp/link-deref.sh: Remove the leading colon on redirections. * tests/cp/reflink-perm.sh: Likewise. * tests/id/zero.sh: Likewise. * tests/install/install-C.sh: Likewise. * tests/misc/env.sh: Likewise. * tests/misc/md5sum-bsd.sh: Likewise. * tests/misc/runcon-no-reorder.sh: Likewise. * tests/mv/partition-perm.sh: Likewise. * tests/rm/r-root.sh: Likewise. * tests/split/l-chunk.sh: Likewise. * tests/split/line-bytes.sh: Likewise. * tests/tail-2/inotify-rotate.sh: Likewise. * tests/tail-2/retry.sh: Likewise. * tests/tail-2/symlink.sh: Likewise. * tests/tail-2/wait.sh: Likewise. * tests/touch/read-only.sh: Likewise. + cfg.mk (sc_prohibit_colon_redirection): A new syntax check to avoid further instances of this creeping in.
2014-03-05tests: fix false failure in nohup.sh in non tty buildsPádraig Brady
* tests/misc/nohup.sh: When running tests without a controlling tty, an exec failure is triggered in a subshell, which causes POSIX shells to immediately exit the subshell. This was brought to notice by the newly conforming bash 4.3. Fixes http:/bugs.gnu.org/16940
2014-02-27date: fix crash or infinite loop when parsing a malformed TZ=""Pádraig Brady
* NEWS: Mention the fix. * gnulib: Update to incorporate the fix. This is the only change in this gnulib update. * tests/misc/date.pl: Add a test for this case. Fixes http://bugs.gnu.org/16872
2014-02-23shuf: with -r, don't dump core if the input is emptyPaul Eggert
Problem reported by valiant xiao in <http://bugs.gnu.org/16855>. * NEWS: Document this. * src/shuf.c (main): With -r, report an error if the input is empty. * tests/misc/shuf.sh: Test for the bug.
2014-02-09head,tail: consistently diagnose write errorsPádraig Brady
If we can't output more data, we should immediately diagnose the issue and exit rather than consuming all of input (in some cases). * src/tail.c (xwrite_stdout): Also diagnose the case where only some data is written. Also clearerr() to avoid the redundant less specific error from atexit (close_stdout); * src/head.c (xwrite_stdout): Copy this new function from tail, and use it to write all output. * tests/misc/head-write-error.sh: A new test to ensure we exit immediately on write error. * tests/local.mk: Reference the new test.
2014-02-09head: fix --lines=-0 outputting nothing if no newline at EOFАлексей Шилин
* src/head.c (elide_tail_lines_pipe): Just output all input in this case to avoid the issue and also avoid redundant '\n' processing. (elide_tail_lines_seekable): Likewise. * tests/misc/head-elide-tail.pl: Add tests for no '\n' at EOF. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/16329
2014-02-09od: add an --endian option to control byte swappingNiels Möller
* src/od.c (main): Handle the new --endian option, taking "little" and "big" as parameters. (usage): Describe the new option. (PRINT_FIELDS): Adjust to swap bytes if required. * tests/misc/od-endian.sh: A new test to verify the byte swapping operations for hex (ints) and floats for all sizes between 1 and 16 inclusive. * test/local.mk: Reference the new test. * doc/coreutils.texi (od invocation): Describe the new option. * NEWS: Mention the new feature.
2014-01-02maint: update all copyright year number rangesBernhard Voelker
Run "make update-copyright", but then also run this, perl -pi -e 's/2\d\d\d-//' tests/sample-test to make that one script use the single most recent year number.
2013-12-07tests: avoid a false memory leak test failurePádraig Brady
* tests/misc/shuf-reservoir.sh: Restrict the valgrind "exit on leak" behavior to developer environments where specific "lint" code is enabled to avoid inconsequential leaks.