| Age | Commit message (Collapse) | Author |
|---|
|
Prompted by the continuous integration build failure at:
http://hydra.nixos.org/build/5584121
du(1) uses the first file object of the two test files linked to the
same inode, 'd/f' and 'd/h', whatever the system returns first.
Use 'd/f' in both the expected and the actual output.
* test/du/inodes.sh: Change the expected output as described above
when returning the --all directory entries (without -l). Also replace
the name of the hardlink 'd/h' by 'd/f' in the actual output.
|
|
* test/du/inodes.sh: In the cases where compare() fails, that function
would show the unified diff automatically. Therefore, remove the
excess "cat out".
In the cases where expecting an empty file, use compare() again
rather than the simpler "test -s" because possible error reports
will then include the file's content for the same reason as above.
|
|
Prompted by the continuous integration build failure at:
http://hydra.nixos.org/build/5582213
* test/du/inodes.sh: Due to undefined order in returned directory
entries, the expected output might not match, so sort both expected
and actual output when returning --all directory entries.
Also use a simpler test for ensuring no errors are output.
|
|
This new option can be used to find directories with a huge
amount of files. The GNU find utility has the printf format
"%h" which prints the number of entries in a directory, but
this is non-cumulative and doesn't handle hard links.
* src/du.c (struct duinfo): Add new member for counting inodes.
(duinfo_init): Initialize inodes member with Zero.
(duinfo_set): Set inodes counter to 1.
(duinfo_add): Sum up the 2 given inodes counters.
(opt_inodes): Add new boolean flag to remember if the --inodes
option has been specified.
(INODES_OPTION): Add new enum value to be used ...
(long_options): ... here.
(usage): Add description of the new option.
(print_size): Pass inodes counter or size to print_only_size,
depending on the inodes mode.
(process_file): Adapt threshold handling: with --inodes, print or
elide the entries according to the struct member inodes.
(main): Add a case for accepting the new INODES_OPTION.
Print a warning diagnostic when --inodes is used together with the
option --apparent-size or -b.
Reset the output_block_size to 1 ... and thus ignoring the
options -m and -k.
* tests/du/inodes.sh: Add a new test.
* tests/local.mk (all_tests): Mention it.
* doc/coreutils.texi (du invocation): Document the new option.
* NEWS: Mention the new option.
|
|
src/copy.c (copy_internal): Use rmdir() rather than unlink()
when the source is a directory, so that empty directories
are replaced in the destination as per POSIX.
* tests/mv/part-rename.sh: Augment with various combinations.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/14763
|
|
Some newer test scripts - partially ones from me - are not executable.
It does not seem to be a problem, but for consistency and to avoid
future problems on unusual platforms or shells change the permissions
by adding the executable bit.
* cfg.mk (sc_tests_executable): Add new syntax-check rule to ensure
that all test scripts are executable.
* tests/df/df-output.sh: Change file mode from 644 to 755.
* tests/du/threshold.sh: Likewise.
* tests/factor/run.sh: Likewise.
* tests/init.sh: Likewise.
* tests/misc/csplit-suppress-matched.pl: Likewise.
* tests/misc/numfmt.pl: Likewise.
* tests/tail-2/retry.sh: Likewise.
|
|
Include the number of arguments which rm received in the "Remove all
arguments?" prompt. This is useful in the, presumably, common case
where the arguments were not provided by hand, but instead were the
result of various shell expansions. A simple, if somewhat contrived,
example (assuming rm is aliased to rm -I) could be:
rm * .o
where the prompt "Remove 120 arguments?" is more likely to make
the user catch the problem.
* src/rm.c (main): Include correctly pluralized n_files
in the output message. Also remove the now redudant "all".
* tests/rm/interactive-always.sh: Adjust to the new prompt.
* tests/rm/interactive-once.sh: Likewise.
|
|
* src/dd.c (STATUS_NONE): Simplify the enum so that
it's more general than just suppressing transfer counts.
Then test this in all locations where non fatal diagnostics
are output.
* tests/dd/misc.sh: Ensure the diagnostic about
being unable to skip past the end of input is suppressed.
* NEWS: Mention the change in behavior.
Fixes http://bugs.gnu.org/14897
|
|
main(): Process new option. Replace input_numbers_option_used()
with a local variable. Re-organize argument processing.
usage(): Describe the new option.
(write_random_numbers): A new function to generate a
permutation of the specified input range with repetition.
(write_random_lines): Likewise for stdin and --echo.
(write_permuted_numbers): New function refactored from
write_permuted_output().
(write_permuted_lines): Likewise.
* tests/misc/shuf.sh: Add tests for --repetitions option.
* doc/coreutils.texi: Mention --repetitions, add examples.
* TODO: Mention an optimization to avoid needing to
read all of the input into memory with --repetitions.
* NEWS: Mention new shuf option.
|
|
* test/misc/shuf.sh: Add tests for erroneous conditions
like multiple '-o' and '--random-source'.
|
|
* src/df.c (filter_mount_list): Initialize devlist->dev_num correctly
when unable to stat() a mount point. This will avoid possible invalid
deduplication done on the list due to use of uninitialized memory.
* tests/df/skip-duplicates.sh: Ensure this code path is exercised.
Also refactor the test to be table driven.
* NEWS: Mention the bug fix.
|
|
* tests/misc/shuf.sh: Restrict the test to the significant
case where we can't in fact read the "unreadable" file.
|
|
This is consistent with the documented interface and
avoids any ambiguity in a user thinking that stdbuf without options
might reset to a "standard" buffering setup.
* src/stdbuf.c (set_libstdbuf_options): Indicate with the return value
whether any env variables were actually set.
(main): Fail unless some env variables were set.
* tests/misc/stdbuf.sh: Ensure this constraint is enforced.
* NEWS: Mention the small change in behavior.
|
|
Prompted by the continuous integration build failure at:
http://hydra.nixos.org/build/5221053
* tests/tail-2/retry.sh: Ensure the 'out' file is truncated,
as it's used to arbitrate the run order of commands.
Relying on the truncation in the background tail command
is racy because the truncation can occur after the fork
of the background shell and thus wait4lines would not wait
for output to occur in 'out', which would mean that the
'missing' file could be populated by the time tail(1)
gets to process it.
|
|
* src/od.c (PRINT_FIELDS): Declare "i" to be of type uintmax_t, so that
the numerator in the expression for "next_pad" does not overflow.
(print_named_ascii): Likewise.
(print_ascii): Likewise.
Bug introduced via commit v6.12-42-g20c0b87.
* tests/misc/od.pl: Exercise each of the three affected code paths.
* NEWS (Bug fixes): Mention it.
Reported by Rich Burridge.
|
|
* tests/misc/head-c.sh: Don't try to elide 1 exabytes, since on
32-bit systems, that number is not representable as a size_t.
This command would fail on 32-bit systems, where SIZE_MAX < 1E:
head --bytes=-E < /dev/null
Instead of "E", use $SSIZE_MAX.
For discussion, see http://bugs.gnu.org/13530
|
|
* tests/split/line-bytes.sh: Since we've limited virtual memory to
20MB, choose a smaller size, 1GiB (which is <= SIZE_MAX) rather than
1EiB, which is larger than SIZE_MAX on 32-bit systems.
I confirmed that this test still fails when the split.c-modifying
part of v8.21-58-gfec363c is backed out.
|
|
* tests/ls/block-size.sh (size_etc): The sed expression through which
we filtered the output of "ls -l ..." assumed that the user and group
name components of each line would not contain spaces. Avoid the
problem by using -og instead of -l, thus not printing either of those
fields. Adjust the sed expression accordingly.
|
|
* tests/misc/numfmt.pl: Avoid a spurious failure when
/bin/sh is dash (as can happen on Debian systems).
|
|
* src/split.c (line_bytes_split): Rewrite to only buffer
when necessary. I.E. only increase the buffer when we've
already lines output in a split and we encounter a line
larger than the input buffer size, in which case a hold
buffer will be increased in increments of the input buffer size.
(lines_rr): Use the more abstract xalloc_die() just like
we did in line_bytes_split(), rather than explicitly
printing the "memory exhausted" message and exiting.
* tests/split/line-bytes.sh: Add a new test for this
function which previously had no test coverage.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
Fixes http://bugs.gnu.org/13537
|
|
* tests/misc/cut-huge-range.sh (subtract_one): New string.
(CUT_MAX): Don't pass a too-large integer to 'expr'.
|
|
This fixes Bug#14371, reported by Killer Bassist.
* NEWS: Document this.
* src/mkdir.c (struct mkdir_options): Remove member ancestor_mode.
New member umask_value. All uses changed.
* src/mkdir.c (make_ancestor): Fix umask assumption.
* src/mkdir.c, src/mkfifo.c, src/mknod.c (main):
Leave umask alone. This requires invoking lchmod after creating
the file, which introduces a race condition, but this can't be
avoided on hosts with "POSIX" default ACLs, and there's no easy
way with network file systems to tell what kind of host the
directory is on.
* tests/local.mk (all_tests): Add tests/mkdir/p-acl.sh.
* tests/mkdir/p-acl.sh: New file.
|
|
Use a sentinel value that's checked implicitly, rather than
a bit array, to determine if an item should be output.
Benchmark results for this change are:
$ yes abcdfeg | head -n1MB > big-file
$ for c in orig sentinel; do
src/cut-$c 2>/dev/null
echo -ne "\n== $c =="
time src/cut-$c -b1,3 big-file > /dev/null
done
== orig ==
real 0m0.049s
user 0m0.044s
sys 0m0.005s
== sentinel ==
real 0m0.035s
user 0m0.032s
sys 0m0.002s
## Again with --output-delimiter ##
$ for c in orig sentinel; do
src/cut-$c 2>/dev/null
echo -ne "\n== $c =="
time src/cut-$c -b1,3 --output-delimiter=: big-file > /dev/null
done
== orig ==
real 0m0.106s
user 0m0.103s
sys 0m0.002s
== sentinel ==
real 0m0.055s
user 0m0.052s
sys 0m0.003s
eol_range_start: Removed. 'n-' is no longer treated specially,
and instead SIZE_MAX is set for the 'hi' limit, and tested implicitly.
complement_rp: Used to complement 'rp' when '--complement' is specified.
ADD_RANGE_PAIR: Macro renamed to 'add_range_pair' function.
* tests/misc/cut-huge-range.sh: Adjust to the SENTINEL value.
Also remove the overlapping range test as this is no longer
dependent on large ranges and also is already handled with
the EOL-subsumed-3 test in cut.pl.
|
|
This issue was introduced in commit v8.21-43-g3e466ad
* src/cut.c (set_fields): Process all range pairs when merging.
* tests/misc/cut-huge-range.sh: Add a test for this edge case.
Also fix an issue where we could miss reported errors due
to truncation of the 'err' file.
|
|
print_kth() is the central function of cut used to
determine if an item is to be output or not,
so simplify it by moving some logic outside.
Benchmark results for this change are:
$ yes abcdfeg | head -n1MB > big-file
$ for c in orig split; do
src/cut-$c 2>/dev/null
echo -ne "\n== $c =="
time src/cut-$c -b1,3 big-file > /dev/null
done
== orig ==
real 0m0.111s
user 0m0.108s
sys 0m0.002s
== split ==
real 0m0.088s
user 0m0.081s
sys 0m0.007s
* src/cut.c (print_kth): Refactor a branch to outside the function.
Related to http://bugs.gnu.org/13127
|
|
The current implementation of cut, uses a bit array,
an array of `struct range_pair's, and (when --output-delimiter
is specified) a hash_table. The new implementation will use
only an array of `struct range_pair's.
The old implementation is memory inefficient because:
1. When -b with a big num is specified, it allocates a lot of
memory for `printable_field'.
2. When --output-delimiter is specified, it will allocate 31 buckets.
Even if only a few ranges are specified.
Note CPU overhead is increased to determine if an item is to be printed,
as shown by:
$ yes abcdfeg | head -n1MB > big-file
$ for c in with-bitarray without-bitarray; do
src/cut-$c 2>/dev/null
echo -ne "\n== $c =="
time src/cut-$c -b1,3 big-file > /dev/null
done
== with-bitarray ==
real 0m0.084s
user 0m0.078s
sys 0m0.006s
== without-bitarray ==
real 0m0.111s
user 0m0.108s
sys 0m0.002s
Subsequent patches will reduce this overhead.
* src/cut.c (set_fields): Set and initialize RP
instead of printable_field.
* src/cut.c (is_range_start_index): Use CURRENT_RP rather than a hash.
* tests/misc/cut.pl: Check if `eol_range_start' is set correctly.
* tests/misc/cut-huge-range.sh: Rename from cut-huge-to-eol-range.sh,
and add a test to verify large amounts of mem aren't allocated.
Fixes http://bugs.gnu.org/13127
|
|
* init.cfg (require_ulimit_v_): Renamed from require_ulimit_
as this only checks for ulimit -v support. Other uses of
ulimit -t and ulimit -n in tests shouldn't cause false failures
if not supported.
* cfg.mk (sc_prohibit_test_ulimit_without_require_): A new syntax check
to ensure that require_ulimit_v_() is used iff required.
* tests/misc/head-c.sh: Add missing call to require_ulimit_v_.
* tests/rm/many-dir-entries-vs-OOM.sh: Likewise.
* tests/split/r-chunk.sh: Remove non mandatory require_ulimit_ call.
* tests/misc/sort-merge-fdlimit.sh: Likewise.
* tests/cp/link-heap.sh: Adjust to renamed require_ulimit_v_.
* tests/dd/no-allocate.sh: Likewise.
* tests/misc/csplit-heap.sh: Likewise.
* tests/misc/cut-huge-to-eol-range.sh: Likewise.
* tests/misc/printf-surprise.sh: Likewise.
|
|
As a side effect of the previous commit which fixes 'tail -f --retry'
to wait for a file to appear, tail would not exit when the last file
appears untailable and gives up on this file.
This can happen, for example, when the argument file name appears
as directory. Tail sets the 'ignore' flag of this file to true,
but instead of exiting the program, tail would continue the loop.
* src/tail.c (any_live_files): Change the function to return true
if any of the files is still tailable or if tail should continue to
try to check again.
(tail_forever): Change the condition to break the loop in the
"no files remaining" case, because now any_live_files() will care
about it, as mentioned above.
(parse_options): When --retry is used without any follow mode,
then reset reopen_inaccessible_files to false.
* tests/tail-2/retry.sh: Add test case.
|
|
The --retry option is indeed useful for both following modes
by name and by file descriptor. The difference is that in the
latter case, it is effective only during the initial open.
As a regression of the implementation of the inotify support,
tail -f --retry would immediately exit if the given file is
inaccessible.
* src/tail.c (usage): Change the description of the --retry option:
remove the note that this option would mainly be useful when
following by name.
(main): Change diagnosing dubios uses of --retry option:
when the --retry option is used without following, then issue
a warning that this option is ignored; when it is used together
with --follow=descriptor, then issue a warning that it is only
effective for the initial open.
Disable inotify also in the case when the initial open in tail_file()
failed (which is the actual bug fix).
* init.cfg (retry_delay_): Pass excess arguments to the test function.
* tests/tail-2/retry.sh: Add new tests.
* tests/local.mk (all_tests): Mention it.
* doc/coreutils.texi (tail invocation): Enhance the documentation
of the --retry option. Clarify the difference in tail's behavior
regarding the --retry option when combined with the following modes
name versus descriptor.
* NEWS (Bug fixes): Mention the fix.
Reported by Noel Morrison in:
http://lists.gnu.org/archive/html/coreutils/2013-04/msg00003.html
|
|
On OS X it was seen that the group ID used for new files,
are set to a that of the directory rather than the current user.
It's not currently understood when this happens, but it was confirmed
that ACLs, extended attributes and setgid bits are _not_ involved.
* init.cfg (skip_if_nondefault_group_): A new function to detect
and avoid this situation. Document with links to the discussions
for hopefully future clarification.
* tests/install/install-C-root.sh: Use the new function.
* tests/install/install-C-selinux.sh: Likewise.
* tests/install/install-C.sh: Likewise.
|
|
* src/head.c (elide_tail_bytes_pipe): Don't use calloc as that
bypasses memory overcommit due to the zeroing requirement.
Also realloc rather than malloc the pointer array to avoid
dependence on overcommit entirely.
* tests/misc/head-c.sh: Add a test case.
Fixes http://bugs.gnu.org/13530
|
|
* src/dd.c: Add new static global variable ibuf.
(alloc_ibuf, alloc_obuf): New functions factored from dd_copy().
(dd_copy): Call the new functions to allocate memory for
ibuf and obuf when necessary.
(skip): Likewise.
* tests/dd/no-allocate.sh: New test.
* tests/local.mk: Reference the test.
|
|
With --suppress-matched, the lines that match the pattern will not be
printed in the output files. I.E. the first line from the second
and subsequent splits will be suppressed.
* src/csplit.c: process_regexp(),process_line_count(): Don't output the
matched lines. Since csplit includes "up to but not including" matched
lines in each split, the first line (in the next group) is the matched
line - so just skip it.
main(): Handle new option.
usage(): Mention new option.
* doc/coreutils.texi (csplit invocation): Mention new option, examples.
* tests/misc/csplit-suppress-matched.pl: New test script.
* tests/local.mk: Reference the new test.
* NEWS: Mention new feature.
|
|
* init.cfg (require_gcc_shared_): A new function to check
that we can build shared libraries in the particular manner
we use in our tests.
* tests/cp/nfs-removal-race.sh: Use require_gcc_shared_.
Then fail rather than skip, if the actual shared lib build fails.
* tests/df/no-mtab-status.sh: Likewise.
* tests/df/skip-duplicates.sh: Likewise.
* tests/ls/getxattr-speedup.sh: Likewise.
Reported in http://bugs.gnu.org/14024
|
|
* src/tail.c (main): If -n0 or -c0 were specified without -f,
then no data would ever be output, so exit without reading input.
* tests/tail-2/tail-n0f.sh: Augment the related test with this case.
|
|
* src/shuf.c (main): If -n0 specified then no data would ever be output,
so exit without reading input.
* tests/misc/shuf.sh: Augment the related test with this case.
|
|
* doc/coreutils.texi (ln invocation): Describe how symlinks are
resolved with --relative, and give an example showing the greater
control available through realpath(1).
* tests/ln/relative.sh: Add a test to demonstrate full symlink
resolution, in a case where it might not be wanted.
|
|
Don't dereference an existing symlink being replaced.
I.E. generate the symlink relative to the symlink's containing dir,
rather than to some arbitrary place it points to.
* src/ln.c (convert_abs_rel): Don't consider the final component
of the symlink name when canonicalizing, as we want to avoid
dereferencing the final component.
* tests/ln/relative.sh: Add a test case.
* NEWS: Mention the fix.
Resolves http://bugs.gnu.org/14116
|
|
Reservoir sampling optimizes selecting K random lines from large or
unknown-sized input: http://en.wikipedia.org/wiki/Reservoir_sampling
Note this also avoids reading any input when -n0 is specified.
* src/shuf.c (main): Use reservoir-sampling when the number of output
lines is known, and the input size is large or unknown.
(input_size): A new function to get the input size for regular files.
(read_input_reservoir_sampling): New function to read lines from input,
keeping only K lines in memory, replacing lines with decreasing prob.
(write_permuted_output_reservoir): New function to output reservoir.
* tests/misc/shuf-reservoir.sh: An expensive_ test using valgrind to
exercise the reservoir-sampling code.
* tests/local.mk: Reference new test.
* NEWS: Mention the improvement.
|
|
* tests/misc/uniq.pl: Previously, if LOCALE_FR was not defined, all
tests would be skipped. Modified to skip only the relevant test.
|
|
* src/uniq.c (usage): Summarize the new option,
and adjust the --all-repeated option to be more consistent.
(check_file): Merge the --group functionality into
the core loop for the default uniq operation since
it's very similar and can output lines immediately upon reading.
(main): Handle the new --group option and make it
mutually exclusive with other selection options.
* tests/misc/uniq.pl: Add tests.
* NEWS: Mention the new feature.
* doc/coreutils.texi (uniq invocation): Describe --group.
|
|
* src/system.h (emit_ancillary_info): Link to the bug report email
addresses and general help URLs online rather than specifying directly.
This give us greater scope to present better info like describing
the difference between bug-coreutils@gnu.org and coreutils@gnu.org etc.
* tests/misc/help-version.sh: Remove the check for bug-coreutils@gnu.org
* tests/local.mk: Remove the no longer needed PACKAGE_BUGREPORT.
|
|
* tests/misc/uniq.pl: add tests for --ignore-case.
|
|
* NEWS: Mention join's new option: --zero-terminated (-z).
* src/join.c: Add new option, --zero-terminated (-z), to make
join use the NUL byte as separator/delimiter rather than newline.
(get_line): Use readlinebuffer_delim in place of readlinebuffer.
(main): Handle the new option.
(usage): Describe new option the same way sort does.
* doc/coreutils.texi (join invocation): Describe the new option.
* tests/misc/join.pl: add tests for -z option.
|
|
* src/install.c (strip): Indicate failure with a return code instead
of terminating the program.
(install_file_in_file): Handle strip's return code and unlink the
created file if necessary.
* tests/install/strip-program.sh: Add a test to cover the changes.
* NEWS (Bug fixes): Mention the fix.
Reported by John Reiser in http://bugzilla.redhat.com/632444.
|
|
* tests/du/long-from-unreadable.sh: This test requires a NAME_MAX
of at least 200, so skip the test otherwise.
* tests/rm/deep-2.sh: Likewise.
Reported by C de-Avillez with ecryptfs where NAME_MAX = 143.
|
|
* tests/du/threshold.sh: use `cut` rather than
sed to avoid using the non portable \t which
fails on sed on openbsd 5 at least.
Also remove a redundant call to `tr` and avoid
explicit setting of LANG=C which is done globally.
|
|
* tests/local.mk (check-root): Restrict to SUBDIRS=. as traversing
into gnulib-tests induces a false failure.
|
|
Both factor and numfmt recently introduced debug messages
for developers, enabled by --verbose and ---devdebug respectively.
There were a few issues though:
1. They used different mechanisms to enable these messages.
2. factor used --verbose which might be needed for something else
3. They used different methods to output the messages,
and numfmt used error() which added an unwanted newline
4. numfmt marked all these messages for translation and factor
marked a couple. We really don't need these translated.
So we fix the above issues here while renaming the enabling
option for both commands to ---debug (still undocumented).
* src/factor.c (verbose): Rename to dev_debug and change from int to
bool as it's just a toggle flag.
(long_options): Rename --verbose to ---debug.
* src/system.h (devmsg): A new inline function to output a message
if enabled by a global dev_debug variable in the compilation unit.
* src/numfmt.c: Use devmsg() rather than error().
Also remove the translation tags from these messages.
Also change debug flag to bool from int.
* tests/misc/numfmt.pl: Adjust for the ---devdebug to ---debug change.
* cfg.mk (sc_marked_devdiagnostics): Add a syntax check to ensure
translations are not added to devmsg calls.
Reported by Göran Uddeborg in http://bugs.gnu.org/13665
|
|
* tests/tail-2/inotify-rotate.sh: Avoid a subshell with bash,
which in turn causes the `kill` to be ineffective to the tail
processes (as the SIGTERM is sent to the subshell which doesn't
propagate the signal on to its children). On NFS the test
cleanup will then fail as there will be .nfs files maintained
in the directory for the files still opened by the tail processes.
Reported by Bernhard Voelker.
|
