summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2013-05-26tests: fix an ls test not to fail when user or group name contains SPJim Meyering
* tests/ls/block-size.sh (size_etc): The sed expression through which we filtered the output of "ls -l ..." assumed that the user and group name components of each line would not contain spaces. Avoid the problem by using -og instead of -l, thus not printing either of those fields. Adjust the sed expression accordingly.
2013-05-27tests: numfmt: use the printf program, not the shell builtinStefano Lattarini
* tests/misc/numfmt.pl: Avoid a spurious failure when /bin/sh is dash (as can happen on Debian systems).
2013-05-24stat: tune for large-LDAP casePaul Eggert
* src/stat.c (print_stat): Omit unnecessary calls to setpwent, setgrent. Problem reported by Fridolín Pokorný in <http://bugs.gnu.org/14462>.
2013-05-23split: with --line-bytes only allocate memory as neededPádraig Brady
* src/split.c (line_bytes_split): Rewrite to only buffer when necessary. I.E. only increase the buffer when we've already lines output in a split and we encounter a line larger than the input buffer size, in which case a hold buffer will be increased in increments of the input buffer size. (lines_rr): Use the more abstract xalloc_die() just like we did in line_bytes_split(), rather than explicitly printing the "memory exhausted" message and exiting. * tests/split/line-bytes.sh: Add a new test for this function which previously had no test coverage. * tests/local.mk: Reference the new test. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/13537
2013-05-18maint: port --enable-gcc-warnings to clangPaul Eggert
* configure.ac: If clang, add -Wno-format-extra-args and -Wno-tautological-constant-out-of-range-compare. * gl/lib/rand-isaac.c (ind): * gl/lib/randread.c (readisaac): * src/ls.c (dev_ino_push, dev_ino_pop): * src/sort.c (buffer_linelim): * src/system.h (is_nul): * src/tail.c (tail_forever_inotify): Rewrite to avoid casts that clang dislikes. It's good to avoid casts anyway. * src/expr.c (integer_overflow): Declare only if it exists. (die): Remove; unused. * src/ls.c (dev_ino_push): New function, replacing ... (DEV_INO_PUSH): ... this removed macro. All uses changed. (decode_switches): Rewrite "str"+i to &str[i].
2013-05-18build: update gnulib submodule to latestPaul Eggert
2013-05-18tests: don't assume expr was built with GMPPaul Eggert
* tests/misc/cut-huge-range.sh (subtract_one): New string. (CUT_MAX): Don't pass a too-large integer to 'expr'.
2013-05-12doc: link to the list of rejected feature requestsPádraig Brady
* README: Update the email address best suited to discussing feature requests, and also link to the list of previously discussed and rejected requests.
2013-05-12build: avoid new syntax-check failureJim Meyering
* po/POTFILES.in: Reflect renaming.
2013-05-11maint: add FIXME commentPaul Eggert
2013-05-11mkdir: don't assume umask equals POSIX default ACL maskPaul Eggert
This fixes Bug#14371, reported by Killer Bassist. * NEWS: Document this. * src/mkdir.c (struct mkdir_options): Remove member ancestor_mode. New member umask_value. All uses changed. * src/mkdir.c (make_ancestor): Fix umask assumption. * src/mkdir.c, src/mkfifo.c, src/mknod.c (main): Leave umask alone. This requires invoking lchmod after creating the file, which introduces a race condition, but this can't be avoided on hosts with "POSIX" default ACLs, and there's no easy way with network file systems to tell what kind of host the directory is on. * tests/local.mk (all_tests): Add tests/mkdir/p-acl.sh. * tests/mkdir/p-acl.sh: New file.
2013-05-11build: update gnulib submodule to latestPaul Eggert
2013-05-08id: with -Z, show SMACK security contextJarkko Sakkinen
Adds an optional dependency on libsmack. * m4/jm-macros.m4: Look for the smack library/header. * src/id.c (main): Output the smack context if available. * src/local.mk: Link with libsmack if available. * NEWS: Mention the new feature.
2013-05-08cut: improve performance, especially with --output-delimiterCojocaru Alexandru
Use a sentinel value that's checked implicitly, rather than a bit array, to determine if an item should be output. Benchmark results for this change are: $ yes abcdfeg | head -n1MB > big-file $ for c in orig sentinel; do src/cut-$c 2>/dev/null echo -ne "\n== $c ==" time src/cut-$c -b1,3 big-file > /dev/null done == orig == real 0m0.049s user 0m0.044s sys 0m0.005s == sentinel == real 0m0.035s user 0m0.032s sys 0m0.002s ## Again with --output-delimiter ## $ for c in orig sentinel; do src/cut-$c 2>/dev/null echo -ne "\n== $c ==" time src/cut-$c -b1,3 --output-delimiter=: big-file > /dev/null done == orig == real 0m0.106s user 0m0.103s sys 0m0.002s == sentinel == real 0m0.055s user 0m0.052s sys 0m0.003s eol_range_start: Removed. 'n-' is no longer treated specially, and instead SIZE_MAX is set for the 'hi' limit, and tested implicitly. complement_rp: Used to complement 'rp' when '--complement' is specified. ADD_RANGE_PAIR: Macro renamed to 'add_range_pair' function. * tests/misc/cut-huge-range.sh: Adjust to the SENTINEL value. Also remove the overlapping range test as this is no longer dependent on large ranges and also is already handled with the EOL-subsumed-3 test in cut.pl.
2013-05-07cut: fix handling of overlapping rangesCojocaru Alexandru
This issue was introduced in commit v8.21-43-g3e466ad * src/cut.c (set_fields): Process all range pairs when merging. * tests/misc/cut-huge-range.sh: Add a test for this edge case. Also fix an issue where we could miss reported errors due to truncation of the 'err' file.
2013-04-30doc: correct a URL to older textutils sourcePádraig Brady
* doc/coreutils.texi (Putting the tools together): Adjust the textutils-1.22 URL, and add a URL for newer sources.
2013-04-29cut: reduce CPU usage for the the common casePádraig Brady
Ensure appropriate functions are inlined. This was seen to be required with gcc 4.6.0 with -O2 on x86_64 at least. It was reported that gcc 4.8.0 did inline these functions though. Also reinstate the bit vector for the common case, to further improve performance. Benchmark results for both aspects of this change are: $ yes abcdfeg | head -n1MB > big-file $ for c in orig inline inline-array; do src/cut-$c 2>/dev/null echo -ne "\n== $c ==" time src/cut-$c -b1,3 big-file > /dev/null done == orig == real 0m0.088s user 0m0.081s sys 0m0.007s == inline == real 0m0.070s user 0m0.060s sys 0m0.009s == inline-array == real 0m0.049s user 0m0.044s sys 0m0.005s * src/cut.c (set_fields): Set up the printable_field bit vector for performance, but only when it's appropriate. I.E. not when either --output-delimeter or huge ranges are specified. (next_item): Ensure it's inlined and avoid unnecessary processing. (print_kth): Ensure it's inlined and add a branch for the fast path. Related to http://bugs.gnu.org/13127
2013-04-29cut: reduce CPU overhead in determining item to outputCojocaru Alexandru
print_kth() is the central function of cut used to determine if an item is to be output or not, so simplify it by moving some logic outside. Benchmark results for this change are: $ yes abcdfeg | head -n1MB > big-file $ for c in orig split; do src/cut-$c 2>/dev/null echo -ne "\n== $c ==" time src/cut-$c -b1,3 big-file > /dev/null done == orig == real 0m0.111s user 0m0.108s sys 0m0.002s == split == real 0m0.088s user 0m0.081s sys 0m0.007s * src/cut.c (print_kth): Refactor a branch to outside the function. Related to http://bugs.gnu.org/13127
2013-04-29cut: make memory allocation independent of range widthCojocaru Alexandru
The current implementation of cut, uses a bit array, an array of `struct range_pair's, and (when --output-delimiter is specified) a hash_table. The new implementation will use only an array of `struct range_pair's. The old implementation is memory inefficient because: 1. When -b with a big num is specified, it allocates a lot of memory for `printable_field'. 2. When --output-delimiter is specified, it will allocate 31 buckets. Even if only a few ranges are specified. Note CPU overhead is increased to determine if an item is to be printed, as shown by: $ yes abcdfeg | head -n1MB > big-file $ for c in with-bitarray without-bitarray; do src/cut-$c 2>/dev/null echo -ne "\n== $c ==" time src/cut-$c -b1,3 big-file > /dev/null done == with-bitarray == real 0m0.084s user 0m0.078s sys 0m0.006s == without-bitarray == real 0m0.111s user 0m0.108s sys 0m0.002s Subsequent patches will reduce this overhead. * src/cut.c (set_fields): Set and initialize RP instead of printable_field. * src/cut.c (is_range_start_index): Use CURRENT_RP rather than a hash. * tests/misc/cut.pl: Check if `eol_range_start' is set correctly. * tests/misc/cut-huge-range.sh: Rename from cut-huge-to-eol-range.sh, and add a test to verify large amounts of mem aren't allocated. Fixes http://bugs.gnu.org/13127
2013-04-28stat,tail: improve support for snfsPádraig Brady
The StorNext distributed file system was previously known as CVFS. * src/stat.c (human_fstype): Add new file system ID definition. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/14251
2013-04-25tests: fix usage of require_ulimit_Pádraig Brady
* init.cfg (require_ulimit_v_): Renamed from require_ulimit_ as this only checks for ulimit -v support. Other uses of ulimit -t and ulimit -n in tests shouldn't cause false failures if not supported. * cfg.mk (sc_prohibit_test_ulimit_without_require_): A new syntax check to ensure that require_ulimit_v_() is used iff required. * tests/misc/head-c.sh: Add missing call to require_ulimit_v_. * tests/rm/many-dir-entries-vs-OOM.sh: Likewise. * tests/split/r-chunk.sh: Remove non mandatory require_ulimit_ call. * tests/misc/sort-merge-fdlimit.sh: Likewise. * tests/cp/link-heap.sh: Adjust to renamed require_ulimit_v_. * tests/dd/no-allocate.sh: Likewise. * tests/misc/csplit-heap.sh: Likewise. * tests/misc/cut-huge-to-eol-range.sh: Likewise. * tests/misc/printf-surprise.sh: Likewise.
2013-04-22scripts: tweak URLs in autotools-installJim Meyering
* scripts/autotools-install (tarballs): Use http:// URLs rather than ftp:// ones. The former are more likely to work, these days. Update URLs to point to the latest versions.
2013-04-20tail: exit following by descriptor when no tailable file leftBernhard Voelker
As a side effect of the previous commit which fixes 'tail -f --retry' to wait for a file to appear, tail would not exit when the last file appears untailable and gives up on this file. This can happen, for example, when the argument file name appears as directory. Tail sets the 'ignore' flag of this file to true, but instead of exiting the program, tail would continue the loop. * src/tail.c (any_live_files): Change the function to return true if any of the files is still tailable or if tail should continue to try to check again. (tail_forever): Change the condition to break the loop in the "no files remaining" case, because now any_live_files() will care about it, as mentioned above. (parse_options): When --retry is used without any follow mode, then reset reopen_inaccessible_files to false. * tests/tail-2/retry.sh: Add test case.
2013-04-20tail: let -f --retry wait for inaccessible filesBernhard Voelker
The --retry option is indeed useful for both following modes by name and by file descriptor. The difference is that in the latter case, it is effective only during the initial open. As a regression of the implementation of the inotify support, tail -f --retry would immediately exit if the given file is inaccessible. * src/tail.c (usage): Change the description of the --retry option: remove the note that this option would mainly be useful when following by name. (main): Change diagnosing dubios uses of --retry option: when the --retry option is used without following, then issue a warning that this option is ignored; when it is used together with --follow=descriptor, then issue a warning that it is only effective for the initial open. Disable inotify also in the case when the initial open in tail_file() failed (which is the actual bug fix). * init.cfg (retry_delay_): Pass excess arguments to the test function. * tests/tail-2/retry.sh: Add new tests. * tests/local.mk (all_tests): Mention it. * doc/coreutils.texi (tail invocation): Enhance the documentation of the --retry option. Clarify the difference in tail's behavior regarding the --retry option when combined with the following modes name versus descriptor. * NEWS (Bug fixes): Mention the fix. Reported by Noel Morrison in: http://lists.gnu.org/archive/html/coreutils/2013-04/msg00003.html
2013-04-14tests: avoid false failures with non default groupsPádraig Brady
On OS X it was seen that the group ID used for new files, are set to a that of the directory rather than the current user. It's not currently understood when this happens, but it was confirmed that ACLs, extended attributes and setgid bits are _not_ involved. * init.cfg (skip_if_nondefault_group_): A new function to detect and avoid this situation. Document with links to the discussions for hopefully future clarification. * tests/install/install-C-root.sh: Use the new function. * tests/install/install-C-selinux.sh: Likewise. * tests/install/install-C.sh: Likewise.
2013-04-14doc: mention caveats with using install --comparePádraig Brady
* doc/coreutils.texi (install invocation): Mention that install(1) may not correctly determine the default user or permissions for installed files, and so is best used with options specifying these attributes.
2013-04-14head: with --bytes=-N only allocate memory as neededPádraig Brady
* src/head.c (elide_tail_bytes_pipe): Don't use calloc as that bypasses memory overcommit due to the zeroing requirement. Also realloc rather than malloc the pointer array to avoid dependence on overcommit entirely. * tests/misc/head-c.sh: Add a test case. Fixes http://bugs.gnu.org/13530
2013-04-14dd: avoid buffer allocations unless neededOndrej Oprala
* src/dd.c: Add new static global variable ibuf. (alloc_ibuf, alloc_obuf): New functions factored from dd_copy(). (dd_copy): Call the new functions to allocate memory for ibuf and obuf when necessary. (skip): Likewise. * tests/dd/no-allocate.sh: New test. * tests/local.mk: Reference the test.
2013-04-10csplit: add the --suppress-matched optionAssaf Gordon
With --suppress-matched, the lines that match the pattern will not be printed in the output files. I.E. the first line from the second and subsequent splits will be suppressed. * src/csplit.c: process_regexp(),process_line_count(): Don't output the matched lines. Since csplit includes "up to but not including" matched lines in each split, the first line (in the next group) is the matched line - so just skip it. main(): Handle new option. usage(): Mention new option. * doc/coreutils.texi (csplit invocation): Mention new option, examples. * tests/misc/csplit-suppress-matched.pl: New test script. * tests/local.mk: Reference the new test. * NEWS: Mention new feature.
2013-04-07build: fix man page build failure with some permissions setupsEnrico Scholz
Use the more portable 'chmod a-w', instead of the 'chmod -w' form. The latter is not always supported. Also its operation is dependent on umask controlling the permissions bits for new files, which is not the case in the presence of POSIX default ACLs for e.g. In that case, chmod may print a warning like the following, and exit with failure status: chmod: man/hostid.1-t: new permissions are r--rw-r--, not r--r--r-- * man/local.mk: s/-w/a-w/
2013-04-04tests: avoid shared lib tests on unsupported platformsPádraig Brady
* init.cfg (require_gcc_shared_): A new function to check that we can build shared libraries in the particular manner we use in our tests. * tests/cp/nfs-removal-race.sh: Use require_gcc_shared_. Then fail rather than skip, if the actual shared lib build fails. * tests/df/no-mtab-status.sh: Likewise. * tests/df/skip-duplicates.sh: Likewise. * tests/ls/getxattr-speedup.sh: Likewise. Reported in http://bugs.gnu.org/14024
2013-04-04tail: exit without reading input if would never outputPádraig Brady
* src/tail.c (main): If -n0 or -c0 were specified without -f, then no data would ever be output, so exit without reading input. * tests/tail-2/tail-n0f.sh: Augment the related test with this case.
2013-04-04shuf: exit without reading if would never outputPádraig Brady
* src/shuf.c (main): If -n0 specified then no data would ever be output, so exit without reading input. * tests/misc/shuf.sh: Augment the related test with this case.
2013-04-04doc: add details on ln --relative symlink resolutionPádraig Brady
* doc/coreutils.texi (ln invocation): Describe how symlinks are resolved with --relative, and give an example showing the greater control available through realpath(1). * tests/ln/relative.sh: Add a test to demonstrate full symlink resolution, in a case where it might not be wanted.
2013-04-04ln: --relative: fix updating of existing symlinksRémy Lefevre
Don't dereference an existing symlink being replaced. I.E. generate the symlink relative to the symlink's containing dir, rather than to some arbitrary place it points to. * src/ln.c (convert_abs_rel): Don't consider the final component of the symlink name when canonicalizing, as we want to avoid dereferencing the final component. * tests/ln/relative.sh: Add a test case. * NEWS: Mention the fix. Resolves http://bugs.gnu.org/14116
2013-03-25shuf: use reservoir-sampling for large or unknown sized inputsAssaf Gordon
Reservoir sampling optimizes selecting K random lines from large or unknown-sized input: http://en.wikipedia.org/wiki/Reservoir_sampling Note this also avoids reading any input when -n0 is specified. * src/shuf.c (main): Use reservoir-sampling when the number of output lines is known, and the input size is large or unknown. (input_size): A new function to get the input size for regular files. (read_input_reservoir_sampling): New function to read lines from input, keeping only K lines in memory, replacing lines with decreasing prob. (write_permuted_output_reservoir): New function to output reservoir. * tests/misc/shuf-reservoir.sh: An expensive_ test using valgrind to exercise the reservoir-sampling code. * tests/local.mk: Reference new test. * NEWS: Mention the improvement.
2013-03-24stat,tail: improve support for efivarfs, exofs, f2fs and ubifsPádraig Brady
* src/stat.c (human_fstype): Add new file system ID definitions. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/14020
2013-03-24maint: improve make src/fs-magic-comparePádraig Brady
* README-release: fix the `make` command, and mention how to get the latest results without requring running a system with the latest kernel. * src/local.mk (src/fs-latest-magic.h): A new target to document how/where to place the latest magic header. (src/fs-kernel-magic): Adjust to include separately downloaded header if available. (src/fs-magic): Undefine MANPAGER as it may impact the ability to pipe the output of man(1). (fs-magic-compare): Don't echo the commands run as they're distracting from the output which needs to be examined.
2013-03-24build: fix issues when building with GMPPádraig Brady
* m4/gmp.m4 (cu_GMP): Add an extra check that gmp.h is available which is required on one Mac OS X 10.5.8 system at least, where the lib was available but the header wasn't. Also enable our GMP code on systems where GMP is not in a separate lib.
2013-03-22doc: clarify the printable characters output by odPádraig Brady
* src/od.c (usage): Mention any printable character is output, Not just ASCII. * doc/coreutils.texi (od invocation): Further clarify that only single byte characters are output (due to the alignment requirement). Also mention the fact that 3 digit octal sequences are output for non printable characters without a corresponding C escape. Reported in http://bugs.gnu.org/13947
2013-03-22doc: clarify stat the meaning of --format="%t %T"Pádraig Brady
* src/stat.c (usage): Mention that the values are only defined for character and block special files. * doc/coreutils.texi (stat invocation): Likewise. Also mention st_rdev. Reported in http://bugs.gnu.org/13927
2013-03-06doc: mention `numfmt` as an alternative to `sort -h`Pádraig Brady
* doc/coreutils.texi (sort invocation): Mention that numfmt can achieve the same results with a possibly more accurate sort.
2013-03-06doc: remove a redundant numfmt heading from texinfoPádraig Brady
* doc/coreutils.texi (detailmenu): Remove the redundant numfmt heading.
2013-03-06maint: ensure proper backslash quoting in texinfo macrosPádraig Brady
* doc/coreutils.texi (ambiguousGroupNote): Ensure '\' is escaped appropriately within the macro. This was verified to generate a single '\' in both "info" and "pdf" outputs.
2013-03-04build: fix factor build failure on aarch64Torbjörn Granlund
* src/longlong.h (__aarch64__): Make add_ssaaaa and sub_ddmmss work. * NEWS: Mention the build fix. Reported at https://bugzilla.redhat.com/917735
2013-02-28tests: don't skip all uniq tests when locale is missingAssaf Gordon
* tests/misc/uniq.pl: Previously, if LOCALE_FR was not defined, all tests would be skipped. Modified to skip only the relevant test.
2013-02-28uniq: add the --group optionAssaf Gordon
* src/uniq.c (usage): Summarize the new option, and adjust the --all-repeated option to be more consistent. (check_file): Merge the --group functionality into the core loop for the default uniq operation since it's very similar and can output lines immediately upon reading. (main): Handle the new --group option and make it mutually exclusive with other selection options. * tests/misc/uniq.pl: Add tests. * NEWS: Mention the new feature. * doc/coreutils.texi (uniq invocation): Describe --group.
2013-02-28doc: move some info from all --help messages, onlinePádraig Brady
* src/system.h (emit_ancillary_info): Link to the bug report email addresses and general help URLs online rather than specifying directly. This give us greater scope to present better info like describing the difference between bug-coreutils@gnu.org and coreutils@gnu.org etc. * tests/misc/help-version.sh: Remove the check for bug-coreutils@gnu.org * tests/local.mk: Remove the no longer needed PACKAGE_BUGREPORT.
2013-02-28tests: add '--ignore-case' tests for uniqAssaf Gordon
* tests/misc/uniq.pl: add tests for --ignore-case.
2013-02-28dircolors: add a new entry to colorize 'arc' filesJavier López
* src/dircolors.hin: Add an entry for arc. Suggested in: https://bugs.launchpad.net/ubuntu/+source/coreutils/+bug/1088131