summaryrefslogtreecommitdiff
path: root/tests/local.mk
AgeCommit message (Collapse)Author
2016-11-22ls: improve alignment of quoted namesPádraig Brady
This provides better alignment when some names are quoted, which also provides better indication that quotes are not part of the name. * src/ls.c (align_variable_outer_quotes): A new variable set when ls is aligning columns (not using -m, non-zero -w), and has a variable quoting style (shell, shell-escape, c-maybe). (quote_name_buf): Writes to buffer rather than FILE, taking care to avoid data copying if possible. Refactored from... (quote_name): ...here. This now manages the buffer passed to quote_name_buf() and outputs the padding, colors and name in the appropriate order, while managing the --dired offsets. (get_color_indicator): A new function to return the color sequence, refactored from... (print_color_indicator): ...here. This now simply outputs. (print_dir): Refactor common parts to quote_name(). (clear_files): Reset the flag indicating at least one file is quoted in the current directory. (needs_quoting): A new function to indicate at the scan stage whether a name needs quoting. Called from... (gobble_file): ...here, until we find the first quoted file. (print_name_with_quoting): Mostly refactored to quote_name(). * tests/ls/quote-align.sh: A new test for various output formats. * tests/local.mk: Reference the new test. * NEWS: Mention the improvement.
2016-11-22b2sum: a new checksum utility with md5sum like interfacePádraig Brady
Note we don't support the --algorithm option of the b2sum command in the external BLAKE2 project, as that was deemed too confusing for users. "BLAKE2b" was chosen as the default algorithm to use, which is single threaded but performs well on 64 bit. * src/blake2: CC0 source copied from external project. * cfg.mk[VC_LIST_ALWAYS_EXCLUDE_REGEX]: Exclude blake2/ from syntax checks, make update-copyright, etc. * src/local.mk: Reference the sources for b2sum, and set the compilation flags. * doc/coreutils.texi (b2sum invocation): Reference the md5sum invocation node, and add descriptions of -l. * tests/misc/b2sum.sh: Add new test. * tests/local.mk: Reference new test. * AUTHORS: Add new binary. * README: Likewise. * build-aux/gen-lists-of-programs.sh: Likewise. * man/.gitignore: Likewise. * scripts/git-hooks/commit-msg: Likewise. * man/b2sum.x: New man page template. * man/local.mk: Reference new template. * src/.gitignore: Ignore new binaries. * src/blake2/.gitignore: Ignore new build atrifacts. * src/md5sum.c (usage): Describe the new -l option. * NEWS: Mention the new program.
2016-09-08date: add '--debug' optionAssaf Gordon
Usage example: date --debug -d 'now + 3 days' Print parsing and debugging information to stderr when using date's -d/--date option. See: http://lists.gnu.org/archive/html/coreutils/2016-01/msg00076.html * src/date.c (main): Add '--debug' option, enable debugging in gnulib's parse-datetime.y module. * tests/misc/date.pl: Repeat tests with '--debug' enable, ensure no regression. * tests/misc/date-debug.sh: Test output of '--debug' option. * tests/local.mk: Add above test. * NEWS: Mention new option. * doc/coreutils.texi: Likewise.
2016-07-18sort: make -h work with -k and blank used as thousands separatorKamil Dudka
* src/sort.c (traverse_raw_number): Allow to skip only one occurrence of thousands_sep to avoid finding the unit in the next column in case thousands_sep matches as blank and is used as column delimiter. * tests/misc/sort-h-thousands-sep.sh: Add regression test for this bug. * tests/local.mk: Reference the test. * NEWS: Mention the bug fix. Reported at https://bugzilla.redhat.com/1355780 Fixes http://bugs.gnu.org/24015
2016-07-08install: with -Z, set default SELinux context for created directoriesKamil Dudka
* doc/coreutils.texi (install invocation): Update -Z documentation. * src/install.c (make_ancestor): Set default security context before calling mkdir() if the -Z option is given. (process_dir): Call restorecon() on the destination directory if the -Z option is given. (usage): Update -Z documentation. * tests/install/install-Z-selinux.sh: A new test for 'install -Z -D' and 'install -Z -d' based on tests/mkdir/restorecon.sh. * tests/local.mk: Reference the test. * NEWS: Mention the improvement. Reported at https://bugzilla.redhat.com/1339135 Fixes http://bugs.gnu.org/23868
2016-06-26tests: verify that fts diagnoses readdir() failuresPeter Benie
* tests/rm/rm-readdir-fail.sh: A new test to simulate readdir() failing immediately or after returning a few entries, and verifying that rm does the appropriate thing. This was initially reported at: http://bugzilla.opensuse.org/show_bug.cgi?id=984910 where it was mentioned that readdir() may fail when an NFS server has a poor readdir cookie implementation.
2016-04-24seq: detect and report I/O errors immediatelyAssaf Gordon
Ensure I/O errors are detected (and terminate seq), preventing seq from infloop (or running for long time with a large range) upon write errors or ignored SIGPIPE. Examples: seq 1 inf > /dev/full (seq_fast) seq 1.1 0.1 inf >/dev/full (print_numbers) * src/seq.c (io_error): A new function to diagnose appropriate stdio errors and exit the program with failure status. (seq_fast, print_numbers): Explicitly check for write errors and terminate the program with diagnostic. * tests/misc/seq-io-errors.sh: Test error detection with /dev/full. * tests/misc/seq-epipe.sh: Test error detection with broken pipes. * tests/local.mk: Add new tests. * NEWS: Mention the fix.
2016-04-21sleep,timeout: support overflowing floating point valuesPádraig Brady
* src/sleep.c (main): Allow ERANGE since we allow "inf" values. * src/timeout.c (parse_duration): Likewise. * tests/misc/sleep.sh: New file. Tests for sleep(1). * tests/misc/timeout-parameters.sh: Add case for newly allowed $LDBL_MAX. Also use returns_ throughout the file. Also avoid small timeout values which might give false failures under load.
2016-03-17date ls pr: fix time zone abbrs on SysV platformsPaul Eggert
The problematic code computed a struct tm in one time zone, and then printed it or converted it to a string in another. To be portable the same time zone needs to be used for both operations. On GNU platforms this is not an issue, but incorrect output can be generated on System V style platforms like AIX where time zone abbreviations are available only in the 'tzname' global variable. Problem reported by Assaf Gordon in: http://bugs.gnu.org/23035 * NEWS: Document the bug. * src/date.c (show_date): * src/ls.c (long_time_expected_width, print_long_format): * src/pr.c (init_header): * src/stat.c (human_time): Use localtime_rz instead of localtime, so that the time zone information is consistent for both localtime and time-formatting functions like fprintftime and nstrftime. For 'stat' this change is mostly just a code cleanup but it also causes stat to also print nanoseconds when printing time stamps that are out of localtime range, as this is more consistent with what other programs do. For programs other than 'stat' this fixes bugs with time zone formats that use %Z. * src/du.c, src/pr.c (localtz): New static var. (main): Initialize it. * src/du.c (show_date): New time zone argument, so that localtime and fprintftime use the same time zone information. All callers changed. * tests/misc/time-style.sh: New file. * tests/local.mk (all_tests): Add it. * tests/misc/date.pl: Test alphabetic time zone abbreviations.
2016-01-13mv: fix data loss with repeated source dir and same destinationPádraig Brady
commit v8.23-31-g90aa291 failed to consider this case, where the previous rename has failed, thus causing the following to remove the specified directory: mv dir dir dir * src/copy.c (copy_internal): Assume this rename attempt has succeeded, as a previous failure will already have been handled, and we don't want to remove the source directory in this case. * tests/cp/duplicate-sources.sh: Consolidate this test file to... * tests/mv/dup-source.sh: ...here. Add test cases for same source and dest. * tests/local.mk: Remove the consolidated test. * NEWS: Mention the bug fix. Reported at https://bugzilla.redhat.com/1297464
2016-01-01maint: update all copyright year number rangesPádraig Brady
Run "make update-copyright" and then... * gnulib: Update to latest with copyright year adjusted. * tests/init.sh: Sync with gnulib to pick up copyright year. * bootstrap: Likewise. * tests/sample-test: Adjust to use the single most recent year.
2015-12-14tests: add tests for the kill commandPádraig Brady
* tests/misc/kill.sh: Add first tests for kill. * tests/local.mk: Reference the new test.
2015-11-24copy: fix copying of extents beyond the apparent file sizeDmitry Monakhov
fallocate can allocate extents beyond EOF via FALLOC_FL_KEEP_SIZE. Where there is a gap (hole) between the extents, and EOF is within that gap, the final hole wasn't reproduced, resulting in silent data corruption in the copied file (size too small). * src/copy.c (extent_copy): Ensure we don't process extents beyond the apparent file size, since processing and allocating those is not currently supported. * tests/cp/fiemap-extents.sh: Renamed from tests/cp/fiemap-empty.sh and re-enable parts checking the extents at and beyond EOF. * tests/local.mk: Reference the renamed test. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/21790
2015-11-12csplit: check and report fwrite errors with errnoAssaf Gordon
discussed in: http://lists.gnu.org/archive/html/coreutils/2015-10/msg00091.html * src/csplit.c: (save_line_to_file): check fwrite failures, report and exit immediately instead of deferring to 'close_output'. * tests/misc/csplit-io-err.sh: test fwrite failure using LD_PRELOAD. * tests/local.mk: add new test.
2015-11-10tests: ensure programs are built before testingPádraig Brady
programs may not be built due to missing system dependencies, or any program can be excluded at configure time with --enable-no-install-program. So ensure we're not testing the system versions in these cases. * init.cfg (print_ver_): Call require_built_ first. * tests/misc/tty-eof.pl: Skip programs not built. * tests/Coreutils.pm (run_tests): Likewise. * tests/misc/ls-misc.pl: Use 'env test' rather than abs path. * tests/misc/test-diag.pl: Likewise. * tests/local.mk: Adjust include order for dependencies. * tests/misc/arch.sh: Remove redundant calls to require_built_. * tests/misc/chroot-fail.sh: Likewise. * tests/misc/groups-dash.sh: Likewise. * tests/misc/groups-version.sh: Likewise. * tests/misc/stdbuf.sh: Likewise. * tests/cp/acl.sh: Remove problematic call to print_ver_ [gs]etfacl. * tests/mv/acl.sh: Likewise. * cfg.mk (sc_env_test_dependencies): A new syntax check to enforce specifying dependencies with print_ver_ for programs specified through the env command. * du/bigtime.sh: Add new print_ver_ dependencies. * du/max-depth.sh: Likewise. * dd/ascii.sh: Likewise. * tests/ls/capability.sh: Likewise. * tests/ls/root-rel-symlink-color.sh: Likewise. * tests/misc/chroot-fail.sh: Likewise. * tests/misc/readlink-fp-loop.sh: Likewise. * tests/misc/sort-debug-keys.sh: Likewise. * tests/readlink/can-e.sh: Likewise. * tests/readlink/can-f.sh: Likewise. * tests/readlink/can-m.sh: Likewise. * tests/tail-2/inotify-race.sh: Likewise. * tests/tail-2/inotify-race2.sh: Likewise. * tests/touch/no-create-missing.sh: Likewise. * tests/touch/no-dereference.sh: Likewise. * tests/misc/printenv.sh: Tweak to avoid syntax check trigger. * tests/misc/help-version.sh: Likewise. * tests/misc/yes.sh: Likewise. * tests/misc/printf-quote.sh: Use previously unused $prog. * configure.ac (EXTRA_MANS): Add $gl_no_install_prog to the list so that check-x-vs-1 syntax check is satisfied.
2015-11-04printf: support the %q format to quote for shellPádraig Brady
* src/printf.c (usage): Mention the new format. (print_formatted): Handle the quoting by calling out to the quotearg module with "shell-escape" mode. * doc/coreutils.texi (printf invocation): Document %q. * tests/misc/printf-quote.sh: New test. * tests/local.mk: Reference new test. * NEWS: Mention the new feature.
2015-10-21ls: allow -w0 to mean no limit on line lengthPádraig Brady
* src/ls.c (print_with_separator): Renamed from print_with_commas, and parameterized to accept the separator to print. Also fix an edge case where '\n' not printed when the POS variable overflows SIZE_MAX. (print_current_files): Degenerate -x and -C to using the cheaper print_with_separator() in the -w0 case. * doc/coreutils.texi (ls invocation): Document the new feature. * tests/ls/w-option.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/21325
2015-10-18maint: avoid duplicate test runsBernhard Voelker
* tests/local.mk (all_tests): Remove the tests 'tests/id/setgid.sh' and 'tests/mkdir/smack-root.sh' because they are mentioned in the 'all_root_tests' list; these tests are skipped anyway during a non-root run because flagged with 'require_root_'.
2015-07-03tests: avoid side effects of $SHELL environment variablePádraig Brady
Since non interactive shells don't generally set $SHELL, its value is propagated through the tests and may cause issues; for example if $SHELL implicitly adjusts $PATH when run. Instead we set $SHELL to that determined by the posix-shell module, and use that consistently for all test sub scripts, including those created thorugh the `split --filter` command. * tests/local.mk: Explicitly set $SHELL to $(PREFERABLY_POSIX_SHELL) which defaults to $CONFIG_SHELL and thus usually /bin/sh. * tests/envvar-check: Remove bash environment variables with side effects, in case /bin/bash was selected for $SHELL. * tests/misc/help-version.sh: Remove redundant initialization of $SHELL. * tests/install/strip-program.sh: Use $SHELL for sub script. * tests/misc/sort-compress-hang.sh: Likewise. * tests/misc/sort-compress-proc.sh: Likewise. * tests/misc/sort-compress.sh: Likewise. * tests/misc/timeout-group.sh: Likewise. * tests/rm/fail-eperm.xpl: Remove redundant elision of bash env vars. * tests/misc/pwd-long.sh: Likewise.
2015-06-24factor: avoid interspersed lines for parallel runsPádraig Brady
* src/factor.c (n_out): A new global variable to track how much data has been written to stdout. (print_factors_single): Use n_out to determine whether to flush the current (and previous) lines. * tests/misc/factor-parallel.sh: Add a new test. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix.
2015-06-24seq: support inf last item more generally/efficientlyPádraig Brady
* src/seq.c (main): Call seq_fast for infinite last value. This implicitly avoids format conversion on the 999999 -> 1000000 transition. * src/seq.c (seq_fast): Generalize the buffer handling, and adjust to handle the "inf" last value specifics. * tests/misc/seq-precision.sh: A new test. * tests/local.mk: Reference the new test.
2015-06-09tail: display file headers correctly with inotifyPádraig Brady
* src/tail.c (tail_forever_inotify): Use the fspec pointer to distinguish previously output files, rather than a descriptor from the inotify event. That event descriptor was that of the parent directory when files were created or renamed etc. (check_fspec): Adjust for the new comparison. Also show the header when the file is truncated, since we show data in this case also. * tests/tail-2/F-headers.sh: A new test case. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix.
2015-05-17build: avoid issues with case insensitive file systemsPádraig Brady
* cfg.mk (sc_case_insensitive_file_names): A new syntax-check rule. * tests/tail-2/descriptor-vs-rename.sh: Rename from tests/tail-2/f-vs-rename.sh * tests/local.mk: Reference the renamed test. Reported by Jim Meyering.
2015-05-11tail: consistently output all data for truncated filesPádraig Brady
Generally if logs are truncated, they're truncated to 0 length, so output all existing data when our heuristic determines truncation. Note with inotify, truncate() and write() are often determined independently and so all data would be written if that was the case. * src/tail.c (check_fspec): Reset file offset to 0 upon truncation. (tail_forever): Likewise. (recheck): Add a FIXME for the related issue where tail may lose data due to tail discounting older log files too early. * tests/tail-2/truncate.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the fix.
2015-05-11tail: fix inotify startup racesPádraig Brady
The previous fixes to races in the various tail tests, identified actual races in the tail inotify implementation. With --follow=descriptor, if the tailed file was replaced before the inotify watch was added, then any subsequent changes were ignored. Similarly in --follow=name mode, all changes to a new name were effectively ignored if that name was created after the original open() but before the inotify_add_watch(). * src/tail.c (tail_forever_inotify): Fix 3 cases. 1. With -f, don't stop tailing when file removed before watch. 2. With -f, watch right file when file replaced before watch. 3. With -F, inspect correct file when replaced before watch. Existing tests identify these when tail compiled with TAIL_TEST_SLEEP. * tests/tail-2/inotify-rotate-resources.sh: This test also identifies the issue with --follow=name when TAIL_TEST_SLEEP is used. Adjust so the test is immune to such races, and also fail quicker on remote file systems. * tests/tail-2/inotify-race2.sh: A new test using GDB, based on inotify-race.sh, which tests the -F race without needed recompilation with sleeps. * tests/local.mk: Reference the new test. * NEWS: Mention the bug.
2015-05-11tests: fix races in and standardize the tail testsPádraig Brady
* tests/tail-2/F-vs-missing.sh: Use standard "fastpoll" options (-s.1 --max-unchanged-stats=1) to speedup the non-inotify case. Add the non-inotify case to the test. `wait` on the background tail process to terminate which should avoid the need for the non standard `retry_delay_ cleanup ...` on NFS. * tests/tail-2/F-vs-rename.sh: Remove 'out' at the start of the loop, to avoid a race in checking its contents. Also ensure 'a' & 'b' files are present before the tail process starts. Use the standard "fastpoll" options as above. * tests/tail-2/f-vs-rename.sh: Likewise. * tests/tail-2/append-only.sh: Use more standard variable names. * tests/tail-2/flush-initial.sh: Use "fastpoll" options for non-inotify platforms. Also `wait` on the background tail to avoid stray processes and file cleanup issues on NFS. * tests/tail-2/inotify-hash-abuse.sh: Always run non-inotify case. Use "fastpoll" options. Use a more standard retry_delay_ instead of a hardcoded sleep loop. Add a `wait` on the background tail. * tests/tail-2/inotify-hash-abuse2.sh: Likewise. * tests/tail-2/inotify-rotate-resources.sh: Wait just on the specific tail $pid needed. * tests/tail-2/inotify-rotate.sh: Use "fastpoll" options. * tests/tail-2/pid.sh: Use standard variable names. Add a `wait` on the background tails. * tests/tail-2/pipe-f2.sh: Likewise. * tests/tail-2/tail-n0f.sh: Likewise. * tests/tail-2/retry.sh: Use "fastpoll" options. * tests/tail-2/symlink.sh: Likewise. * tests/tail-2/wait.sh: Likewise. Speedup by using sub second parameters to timeout(1). Improve the part ensuring that -F never follows a renamed file. * tests/tail-2/infloop-1.sh: Remove invalid test. tail(1) was not being passed the --pid=$yes_pid option, retry_delay_ wasn't used to avoid races, and yes could write huge files before being killed. * tests/local.mk: Remove the invalid test reference. * tests/tail-2/assert-2.sh: Rewrite using retry_delay_(). Since no longer hardcoding large delays, remove the VERY_EXPENSIVE tag. * tests/tail-2/assert.sh: Likewise.
2015-03-31tail: fix -f to follow changes after a renameStephane Chazelas
* src/tail.c (tail_forever_inotify): Only monitor write()s and truncate()s to files in --follow=descriptor mode, thus avoiding the bug where we removed the watch on renamed files. Also adjust the inotify event processing code that is now significant only in --follow=name mode. * tests/tail-2/F-vs-rename.sh: Improve this existing test by running in both polling and inotify modes. * tests/tail-2/f-vs-rename.sh: A new test based on the existing one. * tests/local.mk: Reference the new test. * NEWS: Mention the bug. Fixes http://bugs.gnu.org/19760
2015-03-10yes: output data more efficientlyPádraig Brady
yes(1) may be used to generate repeating patterns of text for test inputs etc., so adjust to be more efficient. Profiling the case where yes(1) is outputting small items through stdio (which was the default case), shows the overhead of continuously processing small items in main() and in stdio: $ yes >/dev/null & perf top -p $! 31.02% yes [.] main 27.36% libc-2.20.so [.] _IO_file_xsputn@@GLIBC_2.2.5 14.51% libc-2.20.so [.] fputs_unlocked 13.50% libc-2.20.so [.] strlen 10.66% libc-2.20.so [.] __GI___mempcpy 1.98% yes [.] fputs_unlocked@plta Sending more data per stdio call improves the situation, but still, there is significant stdio overhead due to memory copies, and the repeated string length checking: $ yes "`echo {1..1000}`" >/dev/null & perf top -p $! 42.26% libc-2.20.so [.] __GI___mempcpy 17.38% libc-2.20.so [.] strlen 5.21% [kernel] [k] __srcu_read_lock 4.58% [kernel] [k] __srcu_read_unlock 4.27% libc-2.20.so [.] _IO_file_xsputn@@GLIBC_2.2.5 2.50% libc-2.20.so [.] __GI___libc_write 2.45% [kernel] [k] system_call 2.40% [kernel] [k] system_call_after_swapgs 2.27% [kernel] [k] vfs_write 2.09% libc-2.20.so [.] _IO_do_write@@GLIBC_2.2.5 2.01% [kernel] [k] fsnotify 1.95% libc-2.20.so [.] _IO_file_write@@GLIBC_2.2.5 1.44% yes [.] main We can avoid all stdio overhead by building up the buffer _once_ and outputting that, and the profile below shows the bottleneck moved to the kernel: $ src/yes >/dev/null & perf top -p $! 15.42% [kernel] [k] __srcu_read_lock 12.98% [kernel] [k] __srcu_read_unlock 9.41% libc-2.20.so [.] __GI___libc_write 9.11% [kernel] [k] vfs_write 8.35% [kernel] [k] fsnotify 8.02% [kernel] [k] system_call 5.84% [kernel] [k] system_call_after_swapgs 4.54% [kernel] [k] __fget_light 3.98% [kernel] [k] sys_write 3.65% [kernel] [k] selinux_file_permission 3.44% [kernel] [k] rw_verify_area 2.94% [kernel] [k] __fsnotify_parent 2.76% [kernel] [k] security_file_permission 2.39% yes [.] main 2.17% [kernel] [k] __fdget_pos 2.13% [kernel] [k] sysret_check 0.81% [kernel] [k] write_null 0.36% yes [.] write@plt Note this change also ensures that yes(1) will only write complete lines for lines shorter than BUFSIZ. * src/yes.c (main): Build up a BUFSIZ buffer of lines, and output that, rather than having stdio process each item. * tests/misc/yes.sh: Add a new test for various buffer sizes. * tests/local.mk: Reference the new test. Fixes http://bugs.gnu.org/20029
2015-02-20tee: treat '-' operand as file name as mandated by POSIXBernhard Voelker
Since v5.2.1-1247-g8dafbe5, tee(1) treated '-' as stdout while POSIX explicitly requires to treat this as a file name. Revert this change, as the interleaved output - due to sending another copy of input to stdout - is not considered to be useful. Discussed in http://lists.gnu.org/archive/html/coreutils/2015-02/msg00085.html * src/tee.c (tee_files): Remove the special handling for "-" operands. (usage): Remove the corresponding sentence. * doc/coreutils.texi (common options): Remove the "tee -" example. (tee invocation): Document that tee(1) now treats "-" as a file name. * tests/misc/tee.sh: Add a test case for "tee -". While at it, re-indent the above multi-argument processing case and extend that to 13 operands, as POSIX mandates that, too. * tests/misc/tee-dash.sh: Remove now-obsolete test. * tests/local.mk (all_tests): Remove the above test. * NEWS (Changes in behavior): Mention the change.
2015-02-06tail: return inotify resources where possiblePádraig Brady
Each user has a maximum number of inotify watches, so handle the cases where we exhaust these resources. * src/tail.c (tail_forever_inotify): Ensure we inotify_rm_watch() the watch for an inode, when replacing with a new watch for a name. Return all used inotify resources when reverting to polling. Revert to polling upon first indication of inotify resource exhaustion. Revert to polling on any inotify resource exhaustion. Diagnose resource exhaustion correctly in all cases. Avoid redundant reinsertion in the hash for unchanged watches (where only attributes of the file are changed). * tests/tail-2/retry.sh: Avoid false failure when reverting to polling. * tests/tail-2/inotify-rotate.sh: Likewise. * tests/tail-2/symlink.sh: Likewise. * tests/tail-2/inotify-rotate-resources.sh: New test to check that we're calling inotify_rm_watch() for replaced files. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix. * THANKS.in: Thanks for reporting and problem identification.
2015-01-28sync: support syncing specified argumentsGiuseppe Scrivano
* m4/jm-macros.m4 (coreutils_MACROS): Check for syncfs(). * man/sync.x: Add references to syncfs, fsync and fdatasync. * doc/coreutils.texi (sync invocation): Document the new feature. * src/sync.c: Include "quote.h". (AUTHORS): Include myself. (MODE_FILE, MODE_DATA, MODE_FILE_SYSTEM, MODE_SYNC): New enum values. (long_options): Define. (sync_arg): New function. (usage): Describe that arguments are now accepted. (main): Add arguments parsing and add support for fsync(2), fdatasync(2) and syncfs(2). * tests/misc/sync.sh: New (and only) test for sync. * tests/local.mk: Reference the new test. * AUTHORS: Add myself to sync's authors. * NEWS: Mention the new feature.
2015-01-19split: new -t option to select record separatorAssaf Gordon
* src/split.c (eolchar): A new variable to hold the separator character (unibyte for now). This is reference throughout rather than hardcoding '\n'. (usage): Describe the new --separator option, and mention records along with lines so there is no ambiguity that all options treat lines and records equivalently. (main): Have -t update eolchar, or default to '\n'. * tests/split/record-sep.sh: New test case. * tests/local.mk: Reference the new test. * doc/coreutils.texi (split invocation): Document the new option. Adjust --lines, --line-bytes, --number=[lr]/... to mention they pertain to records if --separator is specified. * NEWS: Mention the new feature.
2015-01-01maint: update all copyright year number rangesPádraig Brady
Run "make update-copyright" and then... * tests/sample-test: Adjust to use the single most recent year. * tests/du/bind-mount-dir-cycle-v2.sh: Fix case in copyright message, so that year is updated automatically in future.
2014-12-19diagnose too-large numbers betterPádraig Brady
Following on from commit v8.23-82-gaddae94, consistently diagnose numbers that are too large, so as to distinguish from other errors, and make the limits obvious. * gl/modules/xdectoint: A new module implementing xdecto[iu]max(), which handles the common case of parsing a bounded integer and exiting with a diagnostic on error. * gl/lib/xdectoimax.c: The signed variant. * gl/lib/xdectoint.c: The parameterized implementation. * gl/lib/xdectoint.h: The interface. * gl/lib/xdectoumax.c: The unsigned variant. * bootstrap.conf: Reference the new module. * cfg.mk (exclude_file_name_regexp--sc_require_config_h_first): Exclude the parameterized templates. * src/csplit.c: Output EOVERFLOW or ERANGE errors if appropriate. * src/fmt.c: Likewise. * src/fold.c: Likewise. * src/head.c: Likewise. * src/ls.c: Likewise. * src/nl.c: Likewise. * src/nproc.c: Likewise. * src/shred.c: Likewise. * src/shuf.c: Likewise. * src/stdbuf.c: Likewise. * src/stty.c: Likewise. * src/tail.c: Likewise. * src/truncate.c: Likewise. * src/split.c: Likewise. * src/pr.c: Likewise. * tests/pr/pr-tests.pl: Adjust to avoid matching errno diagnostic. * tests/fmt/base.pl: Likewise. * tests/split/l-chunk.sh: Likewise. * tests/misc/shred-negative.sh: Likewise. * tests/misc/tail.pl: Likewise. Also remove the redundant existing ERR_SUBST from test err-6. * tests/ls/hex-option.sh: Check HEX/OCT options. * tests/misc/shred-size.sh: Likewise. * tests/misc/stty-row-col.sh: Likewise.
2014-12-02du: handle sub-bind-mount cycles gracefullyBoris Ranto
This patch fixes the handling of sub-bind-mount cycles which are incorrectly detected as the file system errors. If you bind mount the directory 'a' to its subdirectory 'a/b/c' and then run 'du a/b' you will get the circular dependency warning even though nothing is wrong with the file system. This happens because the first directory that is traversed twice in this case is not a bind mount but a child of bind mount. The solution is to traverse all the directories in the cycle that fts detected and check whether they are not a (bind) mount. * src/du.c (mount_point_in_fts_cycle): New function that checks whether any of the directories in the cycle that fts detected is a mount point. * src/du.c (process_file): Update the function to use the new function that looks up all the directories in the fts cycle instead of only the last one. * tests/du/bind-mount-dir-cycle-v2.sh: New test case that exhibits the described behavior. * tests/local.mk: Reference the new root test. * NEWS: Mention the bug fix.
2014-11-22tests: add a case verifying mv on case insensitive file systemsPádraig Brady
* NEWS: Update the recent entry to also mention the avoidance of incorrectly unlinking a multi-hardlinked "source" file when presented with source and dest that only differ in case. * src/copy.c (same_file_ok): Mention the case issue with same_name(). * tests/mv/hardlink-case.sh: Test the issue on HFS+. * tests/local.mk: Reference the new test case. * tests/mv/vfat: Remove an old related but unused test case.
2014-11-21mv: fail when moving a file to a hardlinkBoris Ranto
We may run into a race condition if we treat hard links to the same file as distinct files. If we do 'mv a b' and 'mv b a' in parallel, both a and b can disappear from the file system. The reason is that in this case the unlink on src is called and the system calls can end up being run in the order where unlink(a) and unlink(b) are the last two system calls. Therefore exit with an error code so that we avoid the potential data loss. * src/copy.c (same_file_ok): Don't set unlink_src that was used by mv, and return false for two hardlinks to a file in move_mode. *src/copy.c (copy_internal): No longer honor the unlink_src option, used only by mv. NEWS: Mention the change in behavior. * tests/cp/same-file.sh: Augment to cover the `cp -a hlsl1 sl1` case. * tests/mv/hard-verbose.sh: Remove no longer needed test. * tests/local.mk: Remove the reference to hard-verbose.sh. * tests/mv/hard-4.sh: Adjust so we fail in this case. * tests/mv/i-4.sh: Likewise. * tests/mv/symlink-onto-hardlink-to-self.sh: Likewise.
2014-10-07wc: don't miscount /sys and similar file systemsPaul Eggert
Fix similar problems in head, od, split, tac, and tail. Reported by George Shuklin in: http://bugs.gnu.org/18621 * NEWS: Document this. * src/head.c (elseek): Move up. (elide_tail_bytes_pipe, elide_tail_lines_pipe): New arg CURRENT_POS. All uses changed. (elide_tail_bytes_file, elide_tail_lines_file): New arg ST and remove arg SIZE. All uses changed. * src/head.c (elide_tail_bytes_file): * src/od.c (skip): Avoid optimization for /sys files, where st_size is bogus and st_size == st_blksize. Don't report error at EOF when not optimizing. * src/head.c, src/od.c, src/tail.c: Include "stat-size.h". * src/split.c (input_file_size): New function. (bytes_split, lines_chunk_split, bytes_chunk_extract): New arg INITIAL_READ. All uses changed. Use it to double-check st_size. * src/tac.c (tac_seekable): New arg FILE_POS. All uses changed. (copy_to_temp): Return size of temp file. All uses changed. * src/tac.c (tac_seekable): * src/tail.c (tail_bytes): * src/wc.c (wc): Don't trust st_size; double-check by reading. * src/wc.c (wc): New arg CURRENT_POS. All uses changed. * tests/local.mk (all_tests): Add tests/misc/wc-proc.sh, tests/misc/od-j.sh, tests/tail-2/tail-c.sh. * tests/misc/head-c.sh: * tests/misc/tac-2-nonseekable.sh: * tests/split/b-chunk.sh: Add tests for problems with /proc and /sys files. * tests/misc/od-j.sh, tests/misc/wc-proc.sh, tests/tail-2/tail-c.sh: New files.
2014-09-30dd: use more robust SIGUSR1 handlingPádraig Brady
* src/dd.c (ifd_reopen): A new wrapper to ensure we don't exit upon receiving a SIGUSR1 in a blocking open() on a fifo for example. (iftruncate): Likewise for ftruncate(). (iread): Process signals also after a short read. (install_signal_handlers): Install SIGINFO/SIGUSR1 handler even if set to SIG_IGN, as this is what the parent can easily set from a shell script that can send SIGUSR1 without the possiblity of inadvertently killing the dd process. * doc/coreutils.texi (dd invocation): Improve the example to show robust usage wrt signal races and short reads. * tests/dd/stats.sh: A new test for various signal races. * tests/local.mk: Reference the new test. * NEWS: Mention the fix.
2014-09-19cp: issue correct warning and ignore duplicate source dirsPádraig Brady
* src/copy.c (copy_internal): Handle the case where we have the same destination directory as already encountered, which can only be due to the corresponding source directory being specified multiple times. * tests/cp/duplicate-sources.sh: Add a test for the new multiply specified directory case, and the existing multiply specified file case. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix.
2014-09-11cat: allow copying empty files to themselvesPaul Eggert
Problem reported by Vincent Lefevre in: http://bugs.gnu.org/18449 * src/cat.c (main): Allow copying an empty file to itself. * tests/misc/cat-self.sh: New test. * tests/local.mk (all_tests): Add it.
2014-07-13build: support building all tools in a single binaryAlex Deymo
Add the --enable-single-binary option to the configure file. When enabled, this option builds a single binary file containing the selected tools. Which tool gets executed depends on the value of argv[0] which can be set implicitly through symlinks to the single program. This setup reduces significantly the size of a complete coreutils install, since code from lib/libcoreutils.a is not duplicated in every one of the more than 100 binaries. Runtime overhead is increased due to more dynamic libraries being loaded, and extra initialization being performed for all utils. Also initially a larger binary is loaded from storage, though this is usually alleviated due to caching and lazy mmaping of unused blocks, and in fact the single binary should have better caching characteristics. Comparing the size of the individual versus single binary on x86_64: $ cd src $ size coreutils $ size -t $(../build-aux/gen-lists-of-programs.sh --list-progs | grep -Ev '(coreutils|libstdbuf)') | tail -n1 text data bss dec hex filename 1097416 5388 88432 1191236 122d44 src/coreutils 4901010 124964 163768 5189742 4f306e (TOTALS) Storage requirements are reduced similarly: $ cd src $ du -h coreutils $ du -ch $(../build-aux/gen-lists-of-programs.sh --list-progs | grep -Ev '(coreutils|libstdbuf)') | tail -n1 1.2M coreutils 5.3M total When installing, the makefile will create either symlinks or shebangs based on the --enable-single-binary setting, for each configured tool. In this way, all the tools are still callable individually, but they are all implemented by the same "coreutils" binary installed on the same directory. * .gitignore: Add new generated files. * Makefile.am: New rules to generate build-aux/gen-single-binary.sh and install symlinks. * NEWS: Mention the new feature. * README: Add "coreutils" to the list of utils. * bootstrap.conf: Regenerate src/single-binary.mk * build-aux/gen-lists-of-programs.sh: New --list-progs option. * build-aux/gen-single-binary.sh: Regenerate * configure.ac: New --enable-single-binary option and other variables. Disallow --enable-single-binary=symlinks with --program-prefix et. al. * man/coreutils.x: Manpage hook. * man/local.mk: Add manpage hook and fix dependencies. * src/coreutils.c: Multicall implementation. * src/local.mk: New rules for the single binary option. * tests/local.mk: Add $single_binary_progs to support require_built_() from init.cfg * tests/misc/env.sh: Avoid the use of symlink to echo. * tests/misc/help-version.sh: Add exception for coreutils. * tests/install/basic-1.sh: Really avoid using ginstall strip functionality if there is an issue with the independent strip command. * src/kill.c: Changes to call exit() in main. * src/readlink.c: Likewise. * src/shuf.c: Likewise. * src/timeout.c: Likewise. * src/truncate.c: Likewise.
2014-06-25df: report correct device in presence of eclipsed mountsPádraig Brady
* src/df.c (last_device_for_mount): A new function to identify the last device mounted for a mount point. (get_disk): Use the above to discard mount entries for a device, where a later mount entry uses a different device name than that of the user specified device. * tests/df/over-mount-device.sh: A new root test. * tests/local.mk: Reference the new test. * NEWS: Reword for all these related recent fixes. Discussed at: http://bugs.gnu.org/16539#69
2014-05-10shred: don't infloop upon negative sizeJim Meyering
* src/shred.c (main): With the preceding change, shred -s-2 FILE would write 64KB blocks forever -- or until disk full. This change makes shred reject a negative size. * tests/misc/shred-negative.sh: New file. * tests/local.mk (all_tests): Add it.
2014-05-06dd: fix conv=ascii, conv=ebcdic, conv=ibm to match POSIXPaul Eggert
Problem reported by Don Baggett in <http:/bugs.gnu.org/17422>. * NEWS: * doc/coreutils.texi (dd invocation): Document this. * src/dd.c (conversions): conv=ascii implies conv=unblock. conv=ebcdic and conv=ibm imply conv=block. (ascii_to_ebcdic, ebcdic_to_ascii): Correct to match POSIX 1003.1-2013. * tests/dd/ascii.sh: New file. * tests/local.mk (all_tests): Add it.
2014-05-05tests: initial SMACK testsJarkko Sakkinen
* init.cfg (require_smack_): New function. * local.mk: Referenced new tests. * tests/id/smack.sh: SMACK tests (new file). * tests/mkdir/smack-no-root.sh: SMACK tests (new file). * tests/mkdir/smack-root.sh: SMACK tests (new file).
2014-04-17ls: don't output colors with unknown TERM env variableGuilherme de Almeida Suckevicz
--colors controls whether to output colors depending on whether we're connected to a terminal or not, while this change gives control over which terminals we output colors to. * NEWS: Mention the change in behavior. * src/ls.c (known_term_type): A new function to search the static list from dircolors.h (parse_ls_colors): Honor the TERM when both LS_COLORS and COLORTERM are non empty. * tests/ls/color-term.sh: A new test. * tests/local.mk: Reference the new test. Fixes http://bugs.gnu.org/15992
2014-02-09head,tail: consistently diagnose write errorsPádraig Brady
If we can't output more data, we should immediately diagnose the issue and exit rather than consuming all of input (in some cases). * src/tail.c (xwrite_stdout): Also diagnose the case where only some data is written. Also clearerr() to avoid the redundant less specific error from atexit (close_stdout); * src/head.c (xwrite_stdout): Copy this new function from tail, and use it to write all output. * tests/misc/head-write-error.sh: A new test to ensure we exit immediately on write error. * tests/local.mk: Reference the new test.
2014-02-09od: add an --endian option to control byte swappingNiels Möller
* src/od.c (main): Handle the new --endian option, taking "little" and "big" as parameters. (usage): Describe the new option. (PRINT_FIELDS): Adjust to swap bytes if required. * tests/misc/od-endian.sh: A new test to verify the byte swapping operations for hex (ints) and floats for all sizes between 1 and 16 inclusive. * test/local.mk: Reference the new test. * doc/coreutils.texi (od invocation): Describe the new option. * NEWS: Mention the new feature.
2014-01-13copy: fix a segfault in SELinux context copying codeNicolas Iooss
* src/selinux.c (restorecon_private): On ArchLinux the `fakeroot cp -a file1 file2` command segfaulted due to getfscreatecon() returning a NULL context. So map this to the sometimes ignored ENODATA error, rather than crashing. * tests/cp/no-ctx.sh: Add a new test case. * tests/local.mk: Reference the new test. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/16335