Age | Commit message (Collapse) | Author |
|
Run "make update-copyright".
* src/numfmt.c: Update copyright year number range. This file has
obviously been added to coreutils after and without the annual update.
* tests/misc/numfmt.pl: Likewise.
|
|
Some newer test scripts - partially ones from me - are not executable.
It does not seem to be a problem, but for consistency and to avoid
future problems on unusual platforms or shells change the permissions
by adding the executable bit.
* cfg.mk (sc_tests_executable): Add new syntax-check rule to ensure
that all test scripts are executable.
* tests/df/df-output.sh: Change file mode from 644 to 755.
* tests/du/threshold.sh: Likewise.
* tests/factor/run.sh: Likewise.
* tests/init.sh: Likewise.
* tests/misc/csplit-suppress-matched.pl: Likewise.
* tests/misc/numfmt.pl: Likewise.
* tests/tail-2/retry.sh: Likewise.
|
|
main(): Process new option. Replace input_numbers_option_used()
with a local variable. Re-organize argument processing.
usage(): Describe the new option.
(write_random_numbers): A new function to generate a
permutation of the specified input range with repetition.
(write_random_lines): Likewise for stdin and --echo.
(write_permuted_numbers): New function refactored from
write_permuted_output().
(write_permuted_lines): Likewise.
* tests/misc/shuf.sh: Add tests for --repetitions option.
* doc/coreutils.texi: Mention --repetitions, add examples.
* TODO: Mention an optimization to avoid needing to
read all of the input into memory with --repetitions.
* NEWS: Mention new shuf option.
|
|
* test/misc/shuf.sh: Add tests for erroneous conditions
like multiple '-o' and '--random-source'.
|
|
* tests/misc/shuf.sh: Restrict the test to the significant
case where we can't in fact read the "unreadable" file.
|
|
This is consistent with the documented interface and
avoids any ambiguity in a user thinking that stdbuf without options
might reset to a "standard" buffering setup.
* src/stdbuf.c (set_libstdbuf_options): Indicate with the return value
whether any env variables were actually set.
(main): Fail unless some env variables were set.
* tests/misc/stdbuf.sh: Ensure this constraint is enforced.
* NEWS: Mention the small change in behavior.
|
|
* src/od.c (PRINT_FIELDS): Declare "i" to be of type uintmax_t, so that
the numerator in the expression for "next_pad" does not overflow.
(print_named_ascii): Likewise.
(print_ascii): Likewise.
Bug introduced via commit v6.12-42-g20c0b87.
* tests/misc/od.pl: Exercise each of the three affected code paths.
* NEWS (Bug fixes): Mention it.
Reported by Rich Burridge.
|
|
* tests/misc/head-c.sh: Don't try to elide 1 exabytes, since on
32-bit systems, that number is not representable as a size_t.
This command would fail on 32-bit systems, where SIZE_MAX < 1E:
head --bytes=-E < /dev/null
Instead of "E", use $SSIZE_MAX.
For discussion, see http://bugs.gnu.org/13530
|
|
* tests/misc/numfmt.pl: Avoid a spurious failure when
/bin/sh is dash (as can happen on Debian systems).
|
|
* tests/misc/cut-huge-range.sh (subtract_one): New string.
(CUT_MAX): Don't pass a too-large integer to 'expr'.
|
|
Use a sentinel value that's checked implicitly, rather than
a bit array, to determine if an item should be output.
Benchmark results for this change are:
$ yes abcdfeg | head -n1MB > big-file
$ for c in orig sentinel; do
src/cut-$c 2>/dev/null
echo -ne "\n== $c =="
time src/cut-$c -b1,3 big-file > /dev/null
done
== orig ==
real 0m0.049s
user 0m0.044s
sys 0m0.005s
== sentinel ==
real 0m0.035s
user 0m0.032s
sys 0m0.002s
## Again with --output-delimiter ##
$ for c in orig sentinel; do
src/cut-$c 2>/dev/null
echo -ne "\n== $c =="
time src/cut-$c -b1,3 --output-delimiter=: big-file > /dev/null
done
== orig ==
real 0m0.106s
user 0m0.103s
sys 0m0.002s
== sentinel ==
real 0m0.055s
user 0m0.052s
sys 0m0.003s
eol_range_start: Removed. 'n-' is no longer treated specially,
and instead SIZE_MAX is set for the 'hi' limit, and tested implicitly.
complement_rp: Used to complement 'rp' when '--complement' is specified.
ADD_RANGE_PAIR: Macro renamed to 'add_range_pair' function.
* tests/misc/cut-huge-range.sh: Adjust to the SENTINEL value.
Also remove the overlapping range test as this is no longer
dependent on large ranges and also is already handled with
the EOL-subsumed-3 test in cut.pl.
|
|
This issue was introduced in commit v8.21-43-g3e466ad
* src/cut.c (set_fields): Process all range pairs when merging.
* tests/misc/cut-huge-range.sh: Add a test for this edge case.
Also fix an issue where we could miss reported errors due
to truncation of the 'err' file.
|
|
print_kth() is the central function of cut used to
determine if an item is to be output or not,
so simplify it by moving some logic outside.
Benchmark results for this change are:
$ yes abcdfeg | head -n1MB > big-file
$ for c in orig split; do
src/cut-$c 2>/dev/null
echo -ne "\n== $c =="
time src/cut-$c -b1,3 big-file > /dev/null
done
== orig ==
real 0m0.111s
user 0m0.108s
sys 0m0.002s
== split ==
real 0m0.088s
user 0m0.081s
sys 0m0.007s
* src/cut.c (print_kth): Refactor a branch to outside the function.
Related to http://bugs.gnu.org/13127
|
|
The current implementation of cut, uses a bit array,
an array of `struct range_pair's, and (when --output-delimiter
is specified) a hash_table. The new implementation will use
only an array of `struct range_pair's.
The old implementation is memory inefficient because:
1. When -b with a big num is specified, it allocates a lot of
memory for `printable_field'.
2. When --output-delimiter is specified, it will allocate 31 buckets.
Even if only a few ranges are specified.
Note CPU overhead is increased to determine if an item is to be printed,
as shown by:
$ yes abcdfeg | head -n1MB > big-file
$ for c in with-bitarray without-bitarray; do
src/cut-$c 2>/dev/null
echo -ne "\n== $c =="
time src/cut-$c -b1,3 big-file > /dev/null
done
== with-bitarray ==
real 0m0.084s
user 0m0.078s
sys 0m0.006s
== without-bitarray ==
real 0m0.111s
user 0m0.108s
sys 0m0.002s
Subsequent patches will reduce this overhead.
* src/cut.c (set_fields): Set and initialize RP
instead of printable_field.
* src/cut.c (is_range_start_index): Use CURRENT_RP rather than a hash.
* tests/misc/cut.pl: Check if `eol_range_start' is set correctly.
* tests/misc/cut-huge-range.sh: Rename from cut-huge-to-eol-range.sh,
and add a test to verify large amounts of mem aren't allocated.
Fixes http://bugs.gnu.org/13127
|
|
* init.cfg (require_ulimit_v_): Renamed from require_ulimit_
as this only checks for ulimit -v support. Other uses of
ulimit -t and ulimit -n in tests shouldn't cause false failures
if not supported.
* cfg.mk (sc_prohibit_test_ulimit_without_require_): A new syntax check
to ensure that require_ulimit_v_() is used iff required.
* tests/misc/head-c.sh: Add missing call to require_ulimit_v_.
* tests/rm/many-dir-entries-vs-OOM.sh: Likewise.
* tests/split/r-chunk.sh: Remove non mandatory require_ulimit_ call.
* tests/misc/sort-merge-fdlimit.sh: Likewise.
* tests/cp/link-heap.sh: Adjust to renamed require_ulimit_v_.
* tests/dd/no-allocate.sh: Likewise.
* tests/misc/csplit-heap.sh: Likewise.
* tests/misc/cut-huge-to-eol-range.sh: Likewise.
* tests/misc/printf-surprise.sh: Likewise.
|
|
* src/head.c (elide_tail_bytes_pipe): Don't use calloc as that
bypasses memory overcommit due to the zeroing requirement.
Also realloc rather than malloc the pointer array to avoid
dependence on overcommit entirely.
* tests/misc/head-c.sh: Add a test case.
Fixes http://bugs.gnu.org/13530
|
|
With --suppress-matched, the lines that match the pattern will not be
printed in the output files. I.E. the first line from the second
and subsequent splits will be suppressed.
* src/csplit.c: process_regexp(),process_line_count(): Don't output the
matched lines. Since csplit includes "up to but not including" matched
lines in each split, the first line (in the next group) is the matched
line - so just skip it.
main(): Handle new option.
usage(): Mention new option.
* doc/coreutils.texi (csplit invocation): Mention new option, examples.
* tests/misc/csplit-suppress-matched.pl: New test script.
* tests/local.mk: Reference the new test.
* NEWS: Mention new feature.
|
|
* src/shuf.c (main): If -n0 specified then no data would ever be output,
so exit without reading input.
* tests/misc/shuf.sh: Augment the related test with this case.
|
|
Reservoir sampling optimizes selecting K random lines from large or
unknown-sized input: http://en.wikipedia.org/wiki/Reservoir_sampling
Note this also avoids reading any input when -n0 is specified.
* src/shuf.c (main): Use reservoir-sampling when the number of output
lines is known, and the input size is large or unknown.
(input_size): A new function to get the input size for regular files.
(read_input_reservoir_sampling): New function to read lines from input,
keeping only K lines in memory, replacing lines with decreasing prob.
(write_permuted_output_reservoir): New function to output reservoir.
* tests/misc/shuf-reservoir.sh: An expensive_ test using valgrind to
exercise the reservoir-sampling code.
* tests/local.mk: Reference new test.
* NEWS: Mention the improvement.
|
|
* tests/misc/uniq.pl: Previously, if LOCALE_FR was not defined, all
tests would be skipped. Modified to skip only the relevant test.
|
|
* src/uniq.c (usage): Summarize the new option,
and adjust the --all-repeated option to be more consistent.
(check_file): Merge the --group functionality into
the core loop for the default uniq operation since
it's very similar and can output lines immediately upon reading.
(main): Handle the new --group option and make it
mutually exclusive with other selection options.
* tests/misc/uniq.pl: Add tests.
* NEWS: Mention the new feature.
* doc/coreutils.texi (uniq invocation): Describe --group.
|
|
* src/system.h (emit_ancillary_info): Link to the bug report email
addresses and general help URLs online rather than specifying directly.
This give us greater scope to present better info like describing
the difference between bug-coreutils@gnu.org and coreutils@gnu.org etc.
* tests/misc/help-version.sh: Remove the check for bug-coreutils@gnu.org
* tests/local.mk: Remove the no longer needed PACKAGE_BUGREPORT.
|
|
* tests/misc/uniq.pl: add tests for --ignore-case.
|
|
* NEWS: Mention join's new option: --zero-terminated (-z).
* src/join.c: Add new option, --zero-terminated (-z), to make
join use the NUL byte as separator/delimiter rather than newline.
(get_line): Use readlinebuffer_delim in place of readlinebuffer.
(main): Handle the new option.
(usage): Describe new option the same way sort does.
* doc/coreutils.texi (join invocation): Describe the new option.
* tests/misc/join.pl: add tests for -z option.
|
|
Both factor and numfmt recently introduced debug messages
for developers, enabled by --verbose and ---devdebug respectively.
There were a few issues though:
1. They used different mechanisms to enable these messages.
2. factor used --verbose which might be needed for something else
3. They used different methods to output the messages,
and numfmt used error() which added an unwanted newline
4. numfmt marked all these messages for translation and factor
marked a couple. We really don't need these translated.
So we fix the above issues here while renaming the enabling
option for both commands to ---debug (still undocumented).
* src/factor.c (verbose): Rename to dev_debug and change from int to
bool as it's just a toggle flag.
(long_options): Rename --verbose to ---debug.
* src/system.h (devmsg): A new inline function to output a message
if enabled by a global dev_debug variable in the compilation unit.
* src/numfmt.c: Use devmsg() rather than error().
Also remove the translation tags from these messages.
Also change debug flag to bool from int.
* tests/misc/numfmt.pl: Adjust for the ---devdebug to ---debug change.
* cfg.mk (sc_marked_devdiagnostics): Add a syntax check to ensure
translations are not added to devmsg calls.
Reported by Göran Uddeborg in http://bugs.gnu.org/13665
|
|
* tests/misc/numfmt.pl: When the system locale grouping doesn't
match our expected format for grouping 1234 in the fr_FR locale,
reset the locale to 'C' so as to skip all locale tests.
|
|
Originally requested in Red Hat bugzilla #445213.
* src/stty.c (mode_info): Add support for DTR/DSR hardware flow control,
if available.
* doc/coreutils.texi: Document it.
* tests/misc/stty.sh: Add it to the list of serial options to avoid.
* NEWS: Mention the improvement.
|
|
* AUTHORS: Add my name.
* NEWS: Mention the new program.
* README: Reference the new program.
* src/numfmt.c: New file.
* src/.gitignore: Ignore the new binary.
* build-aux/gen-lists-of-programs.sh: Update.
* scripts/git-hooks/commit-msg: Allow numfmt: commit prefix.
* po/POTFILES.in: Add new c file.
* tests/misc/numfmt.pl: A new test file giving >93% coverage.
* tests/local.mk: Reference the new test.
* man/.gitignore: Ignore the new man page.
* man/local.mk: Reference the new man page.
* man/numfmt.x: A new template.
* doc/coreutils.texi: Document the new command.
|
|
Fixes the issue introduced in unreleased commit v8.20-60-gec48bea.
* src/cut.c (set_fields): Don't access the bit array if
we've an open ended range that's outside any finite range.
* tests/misc/cut.pl: Add tests for this case.
Reported by Marcel Böhme in http://bugs.gnu.org/13627
|
|
* src/timeout.c (unblock_signal): A new function to unblock a
specified signal, or warn if not possible.
(set_timeout): Ensure SIGALRM is unblocked before we setup the timer.
* tests/misc/timeout-blocked.pl: A new test for the issue.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
Fixes: http://bugs.gnu.org/13535
|
|
* src/seq.c (main): With 3 positive integer args we were
checking the end value was == "1", rather than the step value.
* tests/misc/seq.pl: Add tests for this case.
Reported by Marcel Böhme in http://bugs.gnu.org/13525
|
|
* src/seq.c (get_default_format): Also account for the case where '.'
is auto added to the start value, which is significant when the
number sequence narrows.
* tests/misc/seq.pl: Add two new tests for the failing cases.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/13394
|
|
* src/cut.c (cut_fields): Handle the edge case where '\n' is
the delimiter, which could be used for example to suppress
the last line if it doesn't contain a '\n'.
* test/misc/cut.pl: Add tests for this edge case.
|
|
Previously line N+1 was inspected before line N was fully output,
which causes output ordering issues at the terminal or delays
from intermittent sources like tail -f.
* src/cut.c (cut_fields): Adjust so that we record the
previous output character so we can use that info to
determine wether to output a '\n' or not.
* tests/misc/cut.pl: Add tests to ensure existing
functionality isn't broken.
* NEWS: Mention the fix.
Fixes bug http://bugs.gnu.org/13498
|
|
Run "make update-copyright", but then also run this,
perl -pi -e 's/2\d\d\d-//' tests/sample-test
to make that one script use the single most recent year number.
|
|
This regression was introduced in commit v8.19-132-g3786fb6.
* src/seq.c (seq_fast): Don't use puts() to output the first number,
and instead insert it into the buffer as for other numbers.
Also output the terminator unconditionally.
* tests/misc/seq.pl: Add some basic tests for the -s option.
* NEWS: Mention the fix.
* THANKS.in: Reported by Philipp Gortan.
|
|
The -z option has been introduced in commit v8.15-60-ga3eb71a,
i.e. in coreutils-8.16. Time to add some tests for it.
* tests/misc/basename.pl: Add tests exercising the -z option.
In the foreach loop to append a newline to the end of each
expected 'OUT' string, skip the -z tests.
|
|
* tests/misc/timeout-group.sh: The kernel might possibly delay
signal propagation to timeout.cmd long enough, that it exits
normally without running the signal handler (as sleep will
be in the same process group and so get the signal too).
So avoid this by explicitly checking that the signal handler
is called, which should always happen under normal circumstances.
Reported by Stefano Lattarini on linux-2.6.30-2-686 and bash-4.2.36.
|
|
* tests/misc/cut-huge-to-eol-range.sh: New test, showing that
the change in v8.20-51-g7d03466 is a bug fix after all.
* tests/local.mk (all_tests): Add it.
* NEWS (Bug fixes): Mention it.
|
|
* tests/misc/cut.pl: This particular bug existed up to v8.10.
|
|
* tests/misc/cut.pl: Since we now output the more
complete error message irrespective of running
in a multi-byte locale or not, adjust the test accordingly.
|
|
* src/cut.c (main): Treat a NUL delimiter (-d '') consistently
with non NUL delimiters, and disallow such a delimiter option,
unless a field is also specified.
(set_fields): Provide a more accurate error message when
a given list is invalid.
* tests/misc/cut.pl: Add a test case.
|
|
When printing output delimiters, and when a to-EOL range subsumes
at least one other range, cut would mistakenly print delimiters for
the subsumed range. This bug was probably introduced via commit
v5.2.1-639-g847e066.
* src/cut.c (set_fields): Ignore any range that is subsumed by a
to-EOL range. Also, move two declarations down.
* tests/misc/cut.pl: Add tests to exercise this.
* NEWS (Bug fixes): Mention it.
Reported by Marcel Böhme in http://bugs.gnu.org/12966
|
|
* src/cut.c (set_fields): When two right-open-ended ranges are
specified, don't blindly let the latter one take precedence over
the former. Instead, use the union of the ranges.
* tests/misc/cut.pl: Add tests to exercise this.
* NEWS (Bug fixes): Mention it.
Reported by Marcel Böhme in http://bugs.gnu.org/12966
Thanks to Berhard Voelker for catching log and NEWS typos.
|
|
* tests/misc/timeout.sh: Take advantage of recent support for
sub-second timeouts to decrease runtime from about 6s to 2s.
|
|
* src/seq.c (scan_arg): Calculate the width more accurately
for numbers specified using scientific notation.
* tests/misc/seq.pl: Add tests for cases that were mishandled.
* NEWS: Mention the fix.
* THANKS.in: Reported by Marcel Böhme.
Fixes http://bugs.gnu.org/12959
|
|
The command "echo 12345 | cut -b 0-" prints an empty line while
it should fail with "fields and positions are numbered from 1".
* src/cut.c (set_fields): Add a diagnostic for the invalid open
range which starts with Zero, i.e., the range 0-.
* tests/misc/cut.pl: Add tests to ensure the range 0- fails for
fields (-f) and for positions (-b, -c).
* NEWS: Mention the fix.
Reported by Marcel Böhme in <http://bugs.gnu.org/12903>.
|
|
It's useful for commands that support running for an indeterminite
amount of time, to not return a specific timeout exit status (124),
and instead let the command handle the timeout signal and return
a status for the work done so far.
* doc/coreutils.texi (timeout invocation): Describe the new option.
* src/timeout.c (preserve_status): A new global boolean to
enable the --preserve-status behavior.
(usage): Describe the new option.
(main): Don't return EXIT_TIMEOUT of preserve_status is set.
* tests/misc/timeout.sh: Add a test for the new option.
|
|
* tests/misc/pr.pl: Refactor this test into ...
* tests/pr/pr-tests.pl: ... here.
* tests/local.mk: Remove the reference to the removed test
Improved by Jim Meyering
|
|
* tests/misc/factor.pl: Correct the precedence and
regular expression in the command to check for GMP.
|