summaryrefslogtreecommitdiff
path: root/NEWS
AgeCommit message (Collapse)Author
2014-05-02numfmt: support zero padding using --format="%010f"Pádraig Brady
* src/numfmt.c (setup_padding_buffer): Simplify the code by not explicitly dealing with heap exhaustion. (parse_format_string): Likewise. Handle multiple grouping modifiers as does the standard printf. Handle the new leading zero --format modifier. (double_to_human): Use more defensive coding against overwriting stack buffers. Honor the leading zeros width. (usage): Mention the leading zero --format modifier. (main): Allow --padding in combo with a --format (width), as the number of leading zeros are useful independent of the main field width. * doc/coreutils.texi (numfmt invocation): Likewise. * tests/misc/numfmt.pl: Add new test cases. * NEWS: Mention the improvement.
2014-04-29ptx: fix whitespace trimming with multiple filesPádraig Brady
This issue was identified by running the test suite with http://code.google.com/p/address-sanitizer/ which is included in GCC 4.8 and enabled with -fsanitize=address This was checked on Fedora 20 with GCC 4.8 as follows: $ yum install libasan # http://bugzilla.redhat.com/991003 $ rm -f src/ptx.o $ make check AM_CFLAGS='-fsanitize=address' SUBDIRS=. VERBOSE=yes $ failure identified in tests/test-suite.log To see this particular failure triggered with multiple files: $ src/ptx <(echo a) <(echo a) 2>&1 | asan_symbolize.py -d ================================================================= ==32178==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000e74f at pc 0x435442 bp 0x7fffe8a1b290 sp 0x7fffe8a1b288 READ of size 1 at 0x60200000e74f thread T0 #0 0x435441 in define_all_fields coreutils/src/ptx.c:1425 #1 0x7fa206d31d64 in __libc_start_main ??:? #2 0x42f77c in _start ??:? 0x60200000e74f is located 1 bytes to the left of 3-byte region [0x60200000e750,0x60200000e753) allocated by thread T0 here: #0 0x421809 in realloc ??:? #1 0x439b4e in fread_file coreutils/lib/read-file.c:97 Shadow bytes around the buggy address: 0x0c047fff9c90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9ca0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cb0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9cd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fd fd =>0x0c047fff9ce0: fa fa 03 fa fa fa fd fd fa[fa]03 fa fa fa 00 00 0x0c047fff9cf0: fa fa 04 fa fa fa 04 fa fa fa fd fa fa fa fd fa 0x0c047fff9d00: fa fa 00 fa fa fa fd fa fa fa 00 fa fa fa 00 fa 0x0c047fff9d10: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa 0x0c047fff9d20: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa 0x0c047fff9d30: fa fa fd fa fa fa 00 fa fa fa 00 fa fa fa 00 fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Heap right redzone: fb Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack partial redzone: f4 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 ASan internal: fe ==32178==ABORTING The initial report and high level analysis were from Jim Meyering... "The underlying problem is that swallow_file_in_memory() is setting the contents of the global text_buffer for the first file, then updating it (clobbering old value) for the second file. Yet, some pointers to the initial buffer have been squirreled away and later, one of them (keyafter) is presumed to point into the new "text_buffer", which it does not. The subsequent SKIP_WHITE_BACKWARDS use backs up "cursor" and goes out of bounds." * src/ptx.c (text_buffers): Maintain references for the limits of each buffer corresponding to each file, rather than just the last processed. (struct OCCURS): Add a member to map back to the corresponding file. Note normally this could be computed from the "reference" member rather than needing the extra storage, however this is not possible when in --references mode. (find_occurs_in_text): Reference the array rather than a single entry. (define_all_fields): Likewise. Also avoid computing the file index since this is now stored directly. (main): Update text_buffers[] array rather than a single text_buffer. * tests/misc/ptx-overrun.sh: Even though this issue is already triggered with AddressSanitizer, add a new case to demonstrate the whitespace trimming issue, and to trigger without AddressSanitizer. Fixes https://bugs.gnu.org/16171
2014-04-17ls: don't output colors with unknown TERM env variableGuilherme de Almeida Suckevicz
--colors controls whether to output colors depending on whether we're connected to a terminal or not, while this change gives control over which terminals we output colors to. * NEWS: Mention the change in behavior. * src/ls.c (known_term_type): A new function to search the static list from dircolors.h (parse_ls_colors): Honor the TERM when both LS_COLORS and COLORTERM are non empty. * tests/ls/color-term.sh: A new test. * tests/local.mk: Reference the new test. Fixes http://bugs.gnu.org/15992
2014-04-09maint: document timeout --preserve-status in v8.21 NEWSPádraig Brady
* NEWS: Add the omitted new feature. * cfk.mk (old_NEWS_hash): Adjust accordingly.
2014-04-09doc: clarify in --help that -Z doesn't take an argumentPádraig Brady
* NEWS: Fix a confusing old entry. * cfg.mk (old_NEWS_hash): Adjust accordingly. * src/cp.c (usage): Separate the -Z and --context descriptions. * src/install.c: Likewise. * src/mkdir.c: Likewise. * src/mkfifo.c: Likewise. * src/mknod.c: Likewise. Fixes http://bugs.gnu.org/17220
2014-04-05shred: overwrite inode storage used by some file systemsPádraig Brady
* doc/coreutils.texi (shred invocation): Mention some reasons why clearing slack space might be useful. * src/shred.c (do_wipefd): Add initial writes for each pass for small regular files in case the storage for those is in the inode, and thus a larger write up to a block size would bypass that. Move the direct I/O control to... (dopass): ... here so we can avoid enabling it for these small initial writes. It's better to retry direct I/O for each pass anyway to handle the case where direct I/O is disabled for only the last portion of a file when the size is not a multiple of the block size. Note we don't avoid the sync for the initial write as it will be small but more importantly could be on a different part of the disk and so worth doing independently to ensure the write is not discarded. * tests/misc/shred-exact.sh: Check some more direct I/O cases. * NEWS: Mention the improvements. The inode storage issue was mentioned by Paul Eggert.
2014-04-04shred: avoid a data pass on empty filesPádraig Brady
* src/shred.c (do_wipefd): Don't increase the size written for an empty file up to a full block. Also increase the size to OFF_T_MAX in the edge case where we do overflow. * NEWS: Mention the shred improvements from recent changes. * tests/misc/shred-passes.sh: Adjust as we no longer write a BLKSIZE of data for empty files.
2014-04-02head: port to Darwin and use simpler seeksPaul Eggert
This removes an unportable assumption that if lseek succeeds, the file is capable of seeking. See: http://bugs.gnu.org/17145 * src/head.c (elseek): New function, for consistency in reporting lseek failures. (elide_tail_bytes_file, elide_tail_lines_seekable) (elide_tail_lines_file, head_lines, head): Use it. (elide_tail_bytes_file, elide_tail_lines_file): New args CURRENT_POS and SIZE. All uses changed. Don't bother invoking lseek, since we know the file's pos and size now. (elide_tail_bytes_file): Change a local from uintmax_t to off_t, since it fits. (head): Use lseek only on regular files, since its behavior on unseekable devices is implementation-defined. * NEWS: Document this.
2014-03-31head: fix bug with head -c-N when stdin is not at startPaul Eggert
* src/head.c (elide_tail_bytes_file): Fix typo in lseek invocation. * tests/misc/head-c.sh: Add test for this bug. * NEWS: Document this.
2014-03-26ptx: avoid --format long option falling through into --helpBernhard Voelker
* src/ptx.c (main): Add a 'break' after the --format handling case. Otherwise it would fall through into the usage case. * tests/misc/ptx.pl: Add test cases for --format=tex and --format=roff. * NEWS (Bug fixes): Mention the fix. Bug introduced in 1999-04-04 commit, SH-UTILS-1_16f-269-gd815c15. Spotted by coverity (MISSING_BREAK).
2014-03-24split: avoid unnecessary input bufferingPádraig Brady
Input buffering is best avoided because it introduces delayed processing of output for intermittent input, especially when the output size is less than that of the input buffer. This is significant when output is being further processed which could happen if split is writing to precreated fifos, or through --filter. If input is arriving quickly from a pipe then this will already be buffered before we read it, so fast arriving input shouldn't be a performance issue. * src/split.c (lines_split, lines_bytes_split, bytes_split, lines_chunk_split, bytes_chunk_extract): s/full_read/safe_read/. * THANKS.in: Mention the reporter. * NEWS: Mention the improvement.
2014-03-19chmod: fix erroneous warnings with -R --changesDylan Simon
For files with "special" bits set, we would stat the relative file name in the wrong directory, giving an erroneous ENOENT diagnostic. This issue was introduced with commit v5.92-653-gc1994c1 which changed fts to not change directory on traversal. * src/chmod.c (mode_changed): Use fts->fts_cwd_fd with fstatat rather than stat. All callers changed. * tests/chmod/c-option.sh: Add a test case. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/17035
2014-03-13ln: with -sr, don't segfault for a TARGET of ''Jim Meyering
Prior to this change, "ln -sr '' F" would segfault, attempting to read path2[1] in relpath.c's path_common_prefix function. This problem arises whenever canonicalize_filename_mode returns NULL. * src/ln.c (convert_abs_rel): Call relpath only when both canonicalize_filename_mode calls return non-NULL. * tests/ln/relative.sh: Add a test to trigger this failure. * THANKS.in: List reporter's name/address. * NEWS (Bug fixes): Mention it. Reported by Erik Bernstein in 739752@bugs.debian.org. Fixes http://bugs.gnu.org/17010.
2014-03-13chroot: improve --userspec and --groups look-upPádraig Brady
- Support arbitrary numbers in --groups, consistent with what is already done for --userspec - Avoid look-ups entirely for --groups items with a leading '+' - Support names that are actually numbers in --groups - Ignore an empty --groups="" option for consistency with --userspec - Look up both inside and outside the chroot with inside taking precedence. The look-up outside may load required libraries to complete the look-up inside the chroot. This can happen for example with a 32 bit chroot on a 64 bit system, where the 32 bit NSS plugins within the chroot fail to load. * src/chroot.c (parse_additional_groups): A new function refactored from set_addition_groups(), to just do the parsing. The actual setgroups() call is separated out for calling from the chroot later. (main): Call parse_user_spec() and parse_additional_groups() both outside and inside the chroot for the reasons outlined above. * tests/misc/chroot-credentials.sh: Ensure arbitrary numeric IDs can be specified without causing look-up errors. * NEWS: Mention the improvements. * THANKS.in: Add Norihiro Kamae who initially reported the issue with a proposed patch. Also thanks to Dmitry V. Levin for his diagnosis and sample patch.
2014-02-27date: fix crash or infinite loop when parsing a malformed TZ=""Pádraig Brady
* NEWS: Mention the fix. * gnulib: Update to incorporate the fix. This is the only change in this gnulib update. * tests/misc/date.pl: Add a test for this case. Fixes http://bugs.gnu.org/16872
2014-02-26shuf: convert error diagnostic to lowercaseBernhard Voelker
* src/shuf.c (main): s/No/no/, introduced by commit v8.22-25-g9f60f37. * NEWS: Also adjust the NEWS for that recent commit to make it clear this was new bug rather than a regression. Prompted by the syntax-check rule sc_error_message_uppercase
2014-02-23shuf: with -r, don't dump core if the input is emptyPaul Eggert
Problem reported by valiant xiao in <http://bugs.gnu.org/16855>. * NEWS: Document this. * src/shuf.c (main): With -r, report an error if the input is empty. * tests/misc/shuf.sh: Test for the bug.
2014-02-09head: fix --lines=-0 outputting nothing if no newline at EOFАлексей Шилин
* src/head.c (elide_tail_lines_pipe): Just output all input in this case to avoid the issue and also avoid redundant '\n' processing. (elide_tail_lines_seekable): Likewise. * tests/misc/head-elide-tail.pl: Add tests for no '\n' at EOF. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/16329
2014-02-09od: add an --endian option to control byte swappingNiels Möller
* src/od.c (main): Handle the new --endian option, taking "little" and "big" as parameters. (usage): Describe the new option. (PRINT_FIELDS): Adjust to swap bytes if required. * tests/misc/od-endian.sh: A new test to verify the byte swapping operations for hex (ints) and floats for all sizes between 1 and 16 inclusive. * test/local.mk: Reference the new test. * doc/coreutils.texi (od invocation): Describe the new option. * NEWS: Mention the new feature.
2014-01-29stat,tail: improve support for HFS+ and HFSXPádraig Brady
* src/stat.c (human_fstype): Add new file system ID definitions. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/16336
2014-01-17ln: fix replacing symbolic links whose targets can't existPádraig Brady
* src/ln.c (errno_nonexisting): A new function to determine if the errno implies that a file doesn't or can't (currently) exist. (target_directory_operand): Use the new function to expand the set of errors we handle. * tests/ln/sf-1.sh: Add test cases for the newly handled errors. * THANKS.in: Mention the reporter. * NEWS: Mention the bug fix.
2014-01-13copy: fix a segfault in SELinux context copying codeNicolas Iooss
* src/selinux.c (restorecon_private): On ArchLinux the `fakeroot cp -a file1 file2` command segfaulted due to getfscreatecon() returning a NULL context. So map this to the sometimes ignored ENODATA error, rather than crashing. * tests/cp/no-ctx.sh: Add a new test case. * tests/local.mk: Reference the new test. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/16335
2014-01-13copy: fix SELinux context preservation for existing directoriesPádraig Brady
* src/copy.c (copy_internal): Use the global process context to set the context of existing directories before they're populated. This is more consistent with the new directory case, and fixes a bug for existing directories where we erroneously set the context to the last copied descendent, rather than to that of the source directory itself. * tests/cp/cp-a-selinux.sh: Add a test for this case. * NEWS: Mention the fix. * THANKS.in: Add reporter Michal Trunecka.
2014-01-02maint: update all copyright year number rangesBernhard Voelker
Run "make update-copyright", but then also run this, perl -pi -e 's/2\d\d\d-//' tests/sample-test to make that one script use the single most recent year number.
2013-12-13maint: post-release administriviaPádraig Brady
* NEWS: Add header line for next release. * .prev-version: Record previous version. * cfg.mk (old_NEWS_hash): Auto-update.
2013-12-13version 8.22Pádraig Brady
* NEWS: Record release date.
2013-12-13stat,tail: improve support for hostfs and smackfsPádraig Brady
hostfs is provided by the Linux UML subsystem. smackfs is provided by the Linux Smack security module. * src/stat.c (human_fstype): Add new file system ID definitions. * NEWS: Mention the improvement, and adjust for the fact that SNFS is a remote file system.
2013-12-13build: default to avoiding openssl system librariesPádraig Brady
* configure.ac: Don't change the gnulib default of 'no' for whether to link with openssl system libraries if available. Distributions can explicitly enable this as their policy allows. * NEWS: Adjust accordingly.
2013-12-06shuf: --repeat, not --repetitions; default --head-count is infinityPaul Eggert
Original problem reported by Philipp Thomas in <http://bugs.gnu.org/16061>. * NEWS: shuf --repeat, not shuf --repetitions. * doc/coreutils.texi (shuf invocation): * src/shuf.c (usage, long_opts, main): * tests/misc/shuf.sh: Likewise. Also, the default head-count is infinity.
2013-12-06doc: clarify that df now generally processes special files correctlyBernhard Voelker
Since v8.21-172-g33660b4, df not only treats symbolic link arguments differently, as stated there, but now generally processes special file arguments in a non-canonicalized form correctly: $ cd /dev && df-old sdb Filesystem 1K-blocks Used Available Use% Mounted on devtmpfs 1014572 48 1014524 1% /dev $ cd /dev && df-new sdb Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb 10190136 6039532 3609932 63% /home Document df's new behavior. * doc/coreutils.texi (df invocation): In the paragraph describing df's behavior regarding special file arguments, relax the condition for such special files from "... is an absolute name of ..." to "... resolves to ...". * NEWS (Bug fixes): Mention the new behavior also here.
2013-12-04md5sum, sha*sum: use libcrypto where availablePádraig Brady
libcrypto is generally available and has well optimized crypto hash routines particular to various systems. For example, testing sha1sum with openssl-1.0.0j on an i3-2310M, gives a performance boost of about 40%: $ time sha1sum.old --tag ~/test.iso SHA1 (/home/padraig/test.iso) = 3c27f7ed01965fd2b89e22128fd62dc51a3bef30 real 0m4.692s user 0m4.499s sys 0m0.162s $ time sha1sum.new --tag ~/test.iso SHA1 (/home/padraig/test.iso) = 3c27f7ed01965fd2b89e22128fd62dc51a3bef30 real 0m2.685s user 0m2.512s sys 0m0.170s * configure.ac: By default, enable use of libcrypto if available. * src/local.mk: Link with libcrypto. * NEWS: Mention the md5sum and sha*sum improvements.
2013-12-04df: handle bind mounts when processing device nodesPádraig Brady
* src/df.c (get_disk): Use the same heuristic used in get_point() to select the shortest file system mount point, in an attempt to show the base mounted file system. * NEWS: Mention the bug fix.
2013-12-04df: dereference symlinks to disk device nodesPádraig Brady
This is so the matching for the device is done on the canonical name of the disk node, rather than on the path of the symlink. In any case the user will generally want to use the symlink target. * src/df.c (get_disk): Canonicalize the passed file, before matching against the list of mounted file system devices. Note we pass the original symlink name to the "file" output field, as the symlink target is usually available through the "source" field. * tests/df/df-symlink.sh: Test the dereferencing operation. * tests/local.mk: Mention the new test. * NEWS: Mention the fix. Reported by Ondrej Oprala
2013-11-29df: remove long-obsolescent --megabytes optionBernhard Voelker
This option has been undocumented for 12 years [1], and warned about for a year [2]. [1] commit FILEUTILS-4_1_4-23-gd177203 [2] commit v8.17-43-g453ce92 * src/df.c (MEGABYTES_OPTION): Remove. (long_options): Remove "megabytes" element. (main): In the option parsing loop, remove the MEGABYTES_OPTION case. * NEWS: Mention the change.
2013-11-27selinux: adjust utils to run restorecon with -ZPádraig Brady
cp, mv, install, mkdir, mkfifo, mknod are adjusted so that: -Z no longer accepts an argument. -Z or --context without an argument do not warn without SELinux. --context with an argument will warn without SELinux. * src/local.mk: Reference the new selinux module where required. * src/system.h: Make the argument to --context optional. * src/mkdir.c: Likewise. Also handle the SMACK case for --context. Note we currently silently ignore -Z with SMACK. * src/mkfifo.c: Likewise. * src/mknod.c: Likewise. * src/install.c: Likewise. Note install(1) by default already set the context for target files to their system default, albeit with an older method. Use the -Z option to select between the old and new context restoration behavior, and document the differences and details for how context restoration is done in new and old methods, with a view disabling the old method entirely in future. * src/cp.c: Make the argument to --context optional. Note -Z implies --no-preserve=context. I.E. -Z overrides that aspect of -a no matter what order specified. (struct cp_options): Document the context handling options. (main): Check/adjust option combinations after all options are processed, to both simplify processing and to make handling independent of order of options on the command line. Also improve the diagnostics from a failed call to setfscreatecon(). (set_process_security_ctx): A new function, refactored to set the default context from the source file, or with the type adjusted as per the system default for the destination path. (set_file_security_ctx): A new function refactored to set the security context of an existing file, either based on the process context or the default system context for a path. (copy_internal): Use the refactored functions to simplify error handling and consistently fail or warn as needed. (copy_reg): Likewise. (copy_internal): With --preserve=context, also copy context from non regular files. Note for directories this may impact the copying of subsequent files to that directory? (copy_attr): If we're handling SELinux explicitly, then exclude to avoid the redudant copy with --preserve=context, and the problematic copy with -Z. Note SELinux attribute exclusion also now honors cp -a --no-preserve=context. Note there was a very small window over 10 years ago, where attr_copy_file was available, while attr_copy_check_permissions was not, so we don't bother adding an explicit m4 check for the latter function. * src/mv.c: Support --context, but don't allow specifying an argument. * src/chcon.c: Adjust a comment to be specific to SELinux. * src/runcon.c: Likewise. * src/copy.c: Honor the context settings to "restorecon" as appropriate. * src/copy.h: Add a new setting to select "restorecon" functionality. * tests/mkdir/selinux.sh: s/-Z/--context=/ * tests/cp/cp-a-selinux.sh: Augment this test with cases testing basic -Z functionality, and also test the various invalid option combinations and option precedence. * tests/mkdir/restorecon.sh: Add a new test for the more involved mkdir -Z handling, since the directory changing and non existent directories need to be specially handled. Also check the similar but simpler handling of -Z by mk{nod,fifo}. * tests/local.mk: Reference the new test. * doc/coreutils.texi (cp invocation): Update as per interface changes. (mv invocation): Likewise. (install invocation): Likewise. (mkfifo invocation): Likewise. (mknod invocation): Likewise. (mkdir invocation): Likewise. * NEWS: Mention the new feature and change in behavior.
2013-11-27shred: provide --remove methods to avoid excessive syncingPádraig Brady
A sync operation is very often expensive. For illustration I timed the following python script which indicated that each ext4 dir sync was taking about 2ms and 12ms, on an SSD and traditional disk respectively. import os d=os.open(".", os.O_DIRECTORY|os.O_RDONLY) for i in range(1000): os.fdatasync(d) So syncing for each character for each file can result in significant delays. Often this overhead is redundant, as only the data is sensitive and not the file name. Even if the names are sensitive, your file system may employ synchronous metadata updates, which also makes explicit syncing redundant. * tests/misc/shred-remove.sh: Ensure all the new parameters actually unlink the file. * doc/coreutils.texi (shred invocation): Describe the new parameters to the --remove option. * src/shred.c (Usage): Likewise. (main): Parse the new options. (wipename): Inspect the new enum to see which of the now optional tasks to perform. * NEWS: Mention the new feature. * THANKS.in: Add reporter Joseph D. Wagner
2013-11-27tail: improve inotify handling of symlinksPádraig Brady
Previous behavior failed to read contents of a (re)appearing file, when symlinked by tail's watched file. Also we now diagnose other edge cases when running in inotify mode, where an initially missing or regular file changes to a symlink. * src/tail.c (main): If any arg is a symlink, use polling mode. (recheck): Diagnose the edge case where a symlink appears during inotify processing. * tests/tail-2/symlink.sh: Test the fix. Mention the edge cases. * tests/local.mk: Reference the new test. * NEWS: Mention the fix. Reported by: Ondrej Oprala
2013-11-27df: add --output=file to directly output specified argumentsPádraig Brady
* src/df.c (usage): Document the new 'file' --output field. (get_dev): Add a new parameter to pass the specified argument from the command line through. Use '-' if a command line parameter is not being used. * doc/coreutils.texi (df invocation): Describe the new 'file' field. * tests/df/df-output.sh: Adjust all fields test, and add a specific test for --output=file. * NEWS: Mention the new feature.
2013-11-17stty: add support for mark/space parityColin Leitner
This adds support for using a constant or "stick" parity bit. * src/stty.c (usage): Mention the new flag. * tests/misc/stty.sh: Adjust for the new flag. * NEWS: Mention the improvement. * docs/coreutils.texi (stty invocation): Mention the new flag.
2013-11-11base64: improve encoding I/O efficiencyPádraig Brady
Since the I/O overhead is significant to the relatively simple processing done by this utility, use fputs() rather than fputc() to output '\n'. Time to process a 100MiB file was measured to decrease from 0.417s to 0.383s, i.e. an 8% improvement. Related to these changes, is a processing improvement in gnulib, which increases throughput by 60% when processing full buffers, which improves processing of a 100MiB file with standard wrapped output, down to 0.256s. http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=commit;h=43fd1e7b Also increase the encoding buffer size from 3 to 30KiB. This was seen to give a further 8% improvement, taking processing time down to 0.235s in the wrapped output case. The decoding size buffer is not adjusted, due to the noted caveat with --ignore-garbage. * src/base64.c (BLOCKSIZE): Split into ENC_ and DEC_ variants, with the former increased from 3KiB to 30KiB. (wrap_write): Use the simpler fputc() rather than fputs() to output the '\n' character. Also check against EOF rather than < 0 for errors. (do_encode): Likewise. * NEWS: Mention the large increase in performance, which with the I/O improvements in coreutils and the processing improvement in gnulib, amount to about a 60% throughput increase.
2013-11-08shred: increase I/O block size for periodic pattern casePádraig Brady
* src/shred.c (dopass): In the periodic pattern case increase the I/O block size from 12KiB to 60KiB (also a multiple of 3 and 4096). * NEWS: Adjust accordingly.
2013-11-08shred: fix direct I/O failures for last write to filePádraig Brady
Since direct I/O is now enabled with commit v8.21-139-gebaf961 we must handle the case where we write an odd size at the end of a file (with --exact), or we specify an odd --size that is larger than 64KiB, or in the very unlikely case of a device with an odd size. This issue was present since direct I/O support was first added in v5.3.0, but latent since v6.0. Theoretically this could have also been an issue after that on systems which didn't have alignment constraints, but did have size constraints for direct I/O. * src/shred.c (dopass): On the first pass for a file, always retry a write that fails with EINVAL, so we handle direct I/O failure at either the start or end of the file. Adjust the comment as the original case is out of date and implicitly handled by this more general fix. * tests/misc/shred-exact.sh: Add a test case. * NEWS: Add a "bug fix" entry for shred since there are two related issues now fixed.
2013-11-07cp: fix --link regarding the dereferencing of symbolic linksGian Piero Carrubba
* src/copy.c (create_hard_link): Add a bool 'dereference' parameter, and pass AT_SYMLINK_FOLLOW as 'flags' to linkat() when dereference is true. (should_dereference): Add new 'bool' function to determine if a file should be dereferenced or not. (copy_internal): Use the above new should_dereference() and remember its return value in a new local bool 'dereference' variable. Use that in all three calls to create_hard_link(). * src/cp.c (main): after parsing the options, if x.dereference is still DEFEF_UNDEFINED and the x.recursive is true, then only set x.dereference to DEREF_NEVER iff --link was not specified. * doc/coreutils.texi (cp invocation): Mention that cp(1) does not follow symbolic links in the source when --link is specified. Likewise in the description of the -R option when used together with that option. * tests/cp/same-file.sh: Adapt the expected results for the -fl, the -bl and the -bfl tests. * tests/cp/link-deref.sh: Add a new test. * tests/local.mk (all_tests): Reference the above new test. * NEWS (Changes in behavior): Mention the change. This fixes http://bugs.gnu.org/15173 Co-authored-by: Bernhard Voelker <mail@bernhard-voelker.de>
2013-11-06shred: write larger chunks when possiblePádraig Brady
* src/shred.c (dopass): When not needing to write periodic patterns, use a 64KiB block size to reduce the number of write system calls.
2013-11-05shred: enable direct I/O when possiblePádraig Brady
Commit v5.92-1057-g43d487b introduced a regression in coreutils 6.0 where it removed the page alignment of the buffer to write, thus disabling direct I/O. We want to use direct I/O when possible to avoid impacting the page cache at least, as we know we don't want to cache the data we're writing. * src/shred.c (dopass): Allocate the buffer on the heap, while using a more general calculation to allow to have the output size independent from the fillpattern() size constraint of a multiple of 3. Also we dispense with the union as it's no longer needed given we're aligning on a page boundary and thus don't need to explicitly handle uint32_t alignment.
2013-09-26rm: with -I, prompt before deleting a write protected fileSergio Durigan Junior
This regression was introduced in commit v6.7-71-g0928c24 * src/rm.c (main): Make the -I option behave like --interactive=once. * tests/rm/interactive-once.sh: Add cases for single and multiple files. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/9308
2013-09-22id: support specifying the user by user IDPádraig Brady
* src/id.c (usage): Remove 'name' from the synopsis, implying that one can also specify by user ID. (main): Like chown(1), call parse_user_spec() to implement user name or ID lookup with appropriate precedence. * doc/coreutils.texi (id invocation): Mention that a user ID is supported and how '+' affects lookup order. * tests/misc/id-groups.sh: Remove test now subsumed into... * tests/misc/id-uid.sh: New test covering new interface. * tests/local.mk: Rename the test. * NEWS: Mention the new feature. Addresses http://bugs.gnu.org/15421
2013-09-21id: add -z, --zero optionBernhard Voelker
* src/group-list.h (print_group_list): Add a parameter for the delimiter of type char. * src/group-list.c (print_group_list): Likewise, and use it instead of a white space character to delimit the group entries. * src/groups.c (main): Pass white space character to print_group_list(). * src/id.c (longopts): Add array element for the new long option. (usage): Document the new option. While at it, fix the alignment of the descriptions to match that of HELP_OPTION_DESCRIPTION. (main): Define the bool flag opt_zero indicating the use of the new option. In the getopt_long loop, handle it. Output an error diagnostic in the case the --zero option has been specified together with the default format. In the case of -gG, pass either a NUL or a white space character to print_group_list() - depending on the above new flag. Likewise change the printing of the final newline character: output a NUL instead if the --zero option has been specified. * doc/coreutils.texi (id invocation): Document the new option. While at it, move the @exitstatus macro down after the macro @primaryAndSupplementaryGroups in order to be consistent with other texinfo documents. (groups invocation): Move @exitstatus down after the macro @primaryAndSupplementaryGroups here, too. * tests/misc/id-zero.sh: Add new test exercising the new option. * tests/local.mk (all_tests): Reference it. * NEWS (New features): Mention the new option. Fixes http://bugs.gnu.org/9987
2013-09-05ls: fix possible incorrect exit status when recursing directoriesFUJIWARA Katsunori
If there is an error reading a directory that was referenced through recursion, rather than directly on the command line, then exit with the "less serious" exit code, rather than the "serious" exit code reserved for command line arguments. This issue was introduced in commit v5.2.1-1908-gb58dea5 * src/ls.c (print_dir): Ensure that the command_line_arg param is false for directories being recursed into. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/15249
2013-07-27du: add --inodes optionBernhard Voelker
This new option can be used to find directories with a huge amount of files. The GNU find utility has the printf format "%h" which prints the number of entries in a directory, but this is non-cumulative and doesn't handle hard links. * src/du.c (struct duinfo): Add new member for counting inodes. (duinfo_init): Initialize inodes member with Zero. (duinfo_set): Set inodes counter to 1. (duinfo_add): Sum up the 2 given inodes counters. (opt_inodes): Add new boolean flag to remember if the --inodes option has been specified. (INODES_OPTION): Add new enum value to be used ... (long_options): ... here. (usage): Add description of the new option. (print_size): Pass inodes counter or size to print_only_size, depending on the inodes mode. (process_file): Adapt threshold handling: with --inodes, print or elide the entries according to the struct member inodes. (main): Add a case for accepting the new INODES_OPTION. Print a warning diagnostic when --inodes is used together with the option --apparent-size or -b. Reset the output_block_size to 1 ... and thus ignoring the options -m and -k. * tests/du/inodes.sh: Add a new test. * tests/local.mk (all_tests): Mention it. * doc/coreutils.texi (du invocation): Document the new option. * NEWS: Mention the new option.