| Age | Commit message (Collapse) | Author |
|---|
|
* src/tail.c (tail_forever_inotify): Use the fspec pointer to
distinguish previously output files, rather than a descriptor
from the inotify event. That event descriptor was that of
the parent directory when files were created or renamed etc.
(check_fspec): Adjust for the new comparison. Also show the
header when the file is truncated, since we show data
in this case also.
* tests/tail-2/F-headers.sh: A new test case.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
|
|
* cfg.mk (sc_case_insensitive_file_names): A new syntax-check rule.
* tests/tail-2/descriptor-vs-rename.sh: Rename from
tests/tail-2/f-vs-rename.sh
* tests/local.mk: Reference the renamed test.
Reported by Jim Meyering.
|
|
Generally if logs are truncated, they're truncated to 0 length,
so output all existing data when our heuristic determines truncation.
Note with inotify, truncate() and write() are often determined
independently and so all data would be written if that was the case.
* src/tail.c (check_fspec): Reset file offset to 0 upon truncation.
(tail_forever): Likewise.
(recheck): Add a FIXME for the related issue where tail may lose
data due to tail discounting older log files too early.
* tests/tail-2/truncate.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
|
|
The previous fixes to races in the various tail tests,
identified actual races in the tail inotify implementation.
With --follow=descriptor, if the tailed file was replaced before
the inotify watch was added, then any subsequent changes were ignored.
Similarly in --follow=name mode, all changes to a new name were
effectively ignored if that name was created after the original open()
but before the inotify_add_watch().
* src/tail.c (tail_forever_inotify): Fix 3 cases.
1. With -f, don't stop tailing when file removed before watch.
2. With -f, watch right file when file replaced before watch.
3. With -F, inspect correct file when replaced before watch.
Existing tests identify these when tail compiled with TAIL_TEST_SLEEP.
* tests/tail-2/inotify-rotate-resources.sh:
This test also identifies the issue with --follow=name
when TAIL_TEST_SLEEP is used. Adjust so the test is immune
to such races, and also fail quicker on remote file systems.
* tests/tail-2/inotify-race2.sh: A new test using GDB,
based on inotify-race.sh, which tests the -F race
without needed recompilation with sleeps.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug.
|
|
* tests/tail-2/F-vs-missing.sh: Use standard "fastpoll" options
(-s.1 --max-unchanged-stats=1) to speedup the non-inotify case.
Add the non-inotify case to the test. `wait` on the background
tail process to terminate which should avoid the need for the
non standard `retry_delay_ cleanup ...` on NFS.
* tests/tail-2/F-vs-rename.sh: Remove 'out' at the start of the loop,
to avoid a race in checking its contents. Also ensure 'a' & 'b'
files are present before the tail process starts. Use the standard
"fastpoll" options as above.
* tests/tail-2/f-vs-rename.sh: Likewise.
* tests/tail-2/append-only.sh: Use more standard variable names.
* tests/tail-2/flush-initial.sh: Use "fastpoll" options for
non-inotify platforms. Also `wait` on the background tail to avoid
stray processes and file cleanup issues on NFS.
* tests/tail-2/inotify-hash-abuse.sh: Always run non-inotify case.
Use "fastpoll" options. Use a more standard retry_delay_ instead
of a hardcoded sleep loop. Add a `wait` on the background tail.
* tests/tail-2/inotify-hash-abuse2.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: Wait just on the
specific tail $pid needed.
* tests/tail-2/inotify-rotate.sh: Use "fastpoll" options.
* tests/tail-2/pid.sh: Use standard variable names.
Add a `wait` on the background tails.
* tests/tail-2/pipe-f2.sh: Likewise.
* tests/tail-2/tail-n0f.sh: Likewise.
* tests/tail-2/retry.sh: Use "fastpoll" options.
* tests/tail-2/symlink.sh: Likewise.
* tests/tail-2/wait.sh: Likewise. Speedup by using sub second
parameters to timeout(1). Improve the part ensuring that
-F never follows a renamed file.
* tests/tail-2/infloop-1.sh: Remove invalid test. tail(1) was not
being passed the --pid=$yes_pid option, retry_delay_ wasn't used
to avoid races, and yes could write huge files before being killed.
* tests/local.mk: Remove the invalid test reference.
* tests/tail-2/assert-2.sh: Rewrite using retry_delay_(). Since
no longer hardcoding large delays, remove the VERY_EXPENSIVE tag.
* tests/tail-2/assert.sh: Likewise.
|
|
* src/tail.c (tail_forever_inotify): Only monitor write()s and
truncate()s to files in --follow=descriptor mode, thus avoiding
the bug where we removed the watch on renamed files.
Also adjust the inotify event processing code that is
now significant only in --follow=name mode.
* tests/tail-2/F-vs-rename.sh: Improve this existing test by running
in both polling and inotify modes.
* tests/tail-2/f-vs-rename.sh: A new test based on the existing one.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug.
Fixes http://bugs.gnu.org/19760
|
|
yes(1) may be used to generate repeating patterns of text
for test inputs etc., so adjust to be more efficient.
Profiling the case where yes(1) is outputting small items
through stdio (which was the default case), shows the overhead
of continuously processing small items in main() and in stdio:
$ yes >/dev/null & perf top -p $!
31.02% yes [.] main
27.36% libc-2.20.so [.] _IO_file_xsputn@@GLIBC_2.2.5
14.51% libc-2.20.so [.] fputs_unlocked
13.50% libc-2.20.so [.] strlen
10.66% libc-2.20.so [.] __GI___mempcpy
1.98% yes [.] fputs_unlocked@plta
Sending more data per stdio call improves the situation,
but still, there is significant stdio overhead due to memory copies,
and the repeated string length checking:
$ yes "`echo {1..1000}`" >/dev/null & perf top -p $!
42.26% libc-2.20.so [.] __GI___mempcpy
17.38% libc-2.20.so [.] strlen
5.21% [kernel] [k] __srcu_read_lock
4.58% [kernel] [k] __srcu_read_unlock
4.27% libc-2.20.so [.] _IO_file_xsputn@@GLIBC_2.2.5
2.50% libc-2.20.so [.] __GI___libc_write
2.45% [kernel] [k] system_call
2.40% [kernel] [k] system_call_after_swapgs
2.27% [kernel] [k] vfs_write
2.09% libc-2.20.so [.] _IO_do_write@@GLIBC_2.2.5
2.01% [kernel] [k] fsnotify
1.95% libc-2.20.so [.] _IO_file_write@@GLIBC_2.2.5
1.44% yes [.] main
We can avoid all stdio overhead by building up the buffer
_once_ and outputting that, and the profile below shows
the bottleneck moved to the kernel:
$ src/yes >/dev/null & perf top -p $!
15.42% [kernel] [k] __srcu_read_lock
12.98% [kernel] [k] __srcu_read_unlock
9.41% libc-2.20.so [.] __GI___libc_write
9.11% [kernel] [k] vfs_write
8.35% [kernel] [k] fsnotify
8.02% [kernel] [k] system_call
5.84% [kernel] [k] system_call_after_swapgs
4.54% [kernel] [k] __fget_light
3.98% [kernel] [k] sys_write
3.65% [kernel] [k] selinux_file_permission
3.44% [kernel] [k] rw_verify_area
2.94% [kernel] [k] __fsnotify_parent
2.76% [kernel] [k] security_file_permission
2.39% yes [.] main
2.17% [kernel] [k] __fdget_pos
2.13% [kernel] [k] sysret_check
0.81% [kernel] [k] write_null
0.36% yes [.] write@plt
Note this change also ensures that yes(1) will only write
complete lines for lines shorter than BUFSIZ.
* src/yes.c (main): Build up a BUFSIZ buffer of lines,
and output that, rather than having stdio process each item.
* tests/misc/yes.sh: Add a new test for various buffer sizes.
* tests/local.mk: Reference the new test.
Fixes http://bugs.gnu.org/20029
|
|
Since v5.2.1-1247-g8dafbe5, tee(1) treated '-' as stdout while POSIX
explicitly requires to treat this as a file name. Revert this change,
as the interleaved output - due to sending another copy of input to
stdout - is not considered to be useful. Discussed in
http://lists.gnu.org/archive/html/coreutils/2015-02/msg00085.html
* src/tee.c (tee_files): Remove the special handling for "-" operands.
(usage): Remove the corresponding sentence.
* doc/coreutils.texi (common options): Remove the "tee -" example.
(tee invocation): Document that tee(1) now treats "-" as a file name.
* tests/misc/tee.sh: Add a test case for "tee -".
While at it, re-indent the above multi-argument processing case and
extend that to 13 operands, as POSIX mandates that, too.
* tests/misc/tee-dash.sh: Remove now-obsolete test.
* tests/local.mk (all_tests): Remove the above test.
* NEWS (Changes in behavior): Mention the change.
|
|
Each user has a maximum number of inotify watches,
so handle the cases where we exhaust these resources.
* src/tail.c (tail_forever_inotify): Ensure we inotify_rm_watch()
the watch for an inode, when replacing with a new watch for a name.
Return all used inotify resources when reverting to polling.
Revert to polling upon first indication of inotify resource exhaustion.
Revert to polling on any inotify resource exhaustion.
Diagnose resource exhaustion correctly in all cases.
Avoid redundant reinsertion in the hash for unchanged watches
(where only attributes of the file are changed).
* tests/tail-2/retry.sh: Avoid false failure when reverting to polling.
* tests/tail-2/inotify-rotate.sh: Likewise.
* tests/tail-2/symlink.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: New test to check
that we're calling inotify_rm_watch() for replaced files.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
* THANKS.in: Thanks for reporting and problem identification.
|
|
* m4/jm-macros.m4 (coreutils_MACROS): Check for syncfs().
* man/sync.x: Add references to syncfs, fsync and fdatasync.
* doc/coreutils.texi (sync invocation): Document the new feature.
* src/sync.c: Include "quote.h".
(AUTHORS): Include myself.
(MODE_FILE, MODE_DATA, MODE_FILE_SYSTEM, MODE_SYNC): New enum values.
(long_options): Define.
(sync_arg): New function.
(usage): Describe that arguments are now accepted.
(main): Add arguments parsing and add support for fsync(2),
fdatasync(2) and syncfs(2).
* tests/misc/sync.sh: New (and only) test for sync.
* tests/local.mk: Reference the new test.
* AUTHORS: Add myself to sync's authors.
* NEWS: Mention the new feature.
|
|
* src/split.c (eolchar): A new variable to hold
the separator character (unibyte for now).
This is reference throughout rather than hardcoding '\n'.
(usage): Describe the new --separator option, and
mention records along with lines so there is no ambiguity
that all options treat lines and records equivalently.
(main): Have -t update eolchar, or default to '\n'.
* tests/split/record-sep.sh: New test case.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (split invocation): Document the new option.
Adjust --lines, --line-bytes, --number=[lr]/... to mention
they pertain to records if --separator is specified.
* NEWS: Mention the new feature.
|
|
Run "make update-copyright" and then...
* tests/sample-test: Adjust to use the single most recent year.
* tests/du/bind-mount-dir-cycle-v2.sh: Fix case in copyright message,
so that year is updated automatically in future.
|
|
Following on from commit v8.23-82-gaddae94, consistently diagnose
numbers that are too large, so as to distinguish from other errors,
and make the limits obvious.
* gl/modules/xdectoint: A new module implementing xdecto[iu]max(),
which handles the common case of parsing a bounded integer and
exiting with a diagnostic on error.
* gl/lib/xdectoimax.c: The signed variant.
* gl/lib/xdectoint.c: The parameterized implementation.
* gl/lib/xdectoint.h: The interface.
* gl/lib/xdectoumax.c: The unsigned variant.
* bootstrap.conf: Reference the new module.
* cfg.mk (exclude_file_name_regexp--sc_require_config_h_first):
Exclude the parameterized templates.
* src/csplit.c: Output EOVERFLOW or ERANGE errors if appropriate.
* src/fmt.c: Likewise.
* src/fold.c: Likewise.
* src/head.c: Likewise.
* src/ls.c: Likewise.
* src/nl.c: Likewise.
* src/nproc.c: Likewise.
* src/shred.c: Likewise.
* src/shuf.c: Likewise.
* src/stdbuf.c: Likewise.
* src/stty.c: Likewise.
* src/tail.c: Likewise.
* src/truncate.c: Likewise.
* src/split.c: Likewise.
* src/pr.c: Likewise.
* tests/pr/pr-tests.pl: Adjust to avoid matching errno diagnostic.
* tests/fmt/base.pl: Likewise.
* tests/split/l-chunk.sh: Likewise.
* tests/misc/shred-negative.sh: Likewise.
* tests/misc/tail.pl: Likewise. Also remove the redundant
existing ERR_SUBST from test err-6.
* tests/ls/hex-option.sh: Check HEX/OCT options.
* tests/misc/shred-size.sh: Likewise.
* tests/misc/stty-row-col.sh: Likewise.
|
|
This patch fixes the handling of sub-bind-mount cycles which are
incorrectly detected as the file system errors. If you bind mount the
directory 'a' to its subdirectory 'a/b/c' and then run 'du a/b' you
will get the circular dependency warning even though nothing is wrong
with the file system. This happens because the first directory that is
traversed twice in this case is not a bind mount but a child of bind
mount. The solution is to traverse all the directories in the cycle
that fts detected and check whether they are not a (bind) mount.
* src/du.c (mount_point_in_fts_cycle): New function that checks whether
any of the directories in the cycle that fts detected is a mount point.
* src/du.c (process_file): Update the function to use the new function
that looks up all the directories in the fts cycle instead of only the
last one.
* tests/du/bind-mount-dir-cycle-v2.sh: New test case that exhibits the
described behavior.
* tests/local.mk: Reference the new root test.
* NEWS: Mention the bug fix.
|
|
* NEWS: Update the recent entry to also mention the avoidance
of incorrectly unlinking a multi-hardlinked "source" file when
presented with source and dest that only differ in case.
* src/copy.c (same_file_ok): Mention the case issue with same_name().
* tests/mv/hardlink-case.sh: Test the issue on HFS+.
* tests/local.mk: Reference the new test case.
* tests/mv/vfat: Remove an old related but unused test case.
|
|
We may run into a race condition if we treat hard links to the same file
as distinct files. If we do 'mv a b' and 'mv b a' in parallel, both a
and b can disappear from the file system. The reason is that in this
case the unlink on src is called and the system calls can end up being
run in the order where unlink(a) and unlink(b) are the last two system
calls. Therefore exit with an error code so that we avoid the potential
data loss.
* src/copy.c (same_file_ok): Don't set unlink_src that was used by mv,
and return false for two hardlinks to a file in move_mode.
*src/copy.c (copy_internal): No longer honor the unlink_src option,
used only by mv.
NEWS: Mention the change in behavior.
* tests/cp/same-file.sh: Augment to cover the `cp -a hlsl1 sl1` case.
* tests/mv/hard-verbose.sh: Remove no longer needed test.
* tests/local.mk: Remove the reference to hard-verbose.sh.
* tests/mv/hard-4.sh: Adjust so we fail in this case.
* tests/mv/i-4.sh: Likewise.
* tests/mv/symlink-onto-hardlink-to-self.sh: Likewise.
|
|
Fix similar problems in head, od, split, tac, and tail.
Reported by George Shuklin in: http://bugs.gnu.org/18621
* NEWS: Document this.
* src/head.c (elseek): Move up.
(elide_tail_bytes_pipe, elide_tail_lines_pipe): New arg
CURRENT_POS. All uses changed.
(elide_tail_bytes_file, elide_tail_lines_file):
New arg ST and remove arg SIZE. All uses changed.
* src/head.c (elide_tail_bytes_file):
* src/od.c (skip): Avoid optimization for /sys files, where
st_size is bogus and st_size == st_blksize.
Don't report error at EOF when not optimizing.
* src/head.c, src/od.c, src/tail.c: Include "stat-size.h".
* src/split.c (input_file_size): New function.
(bytes_split, lines_chunk_split, bytes_chunk_extract): New arg
INITIAL_READ. All uses changed. Use it to double-check st_size.
* src/tac.c (tac_seekable): New arg FILE_POS. All uses changed.
(copy_to_temp): Return size of temp file. All uses changed.
* src/tac.c (tac_seekable):
* src/tail.c (tail_bytes):
* src/wc.c (wc):
Don't trust st_size; double-check by reading.
* src/wc.c (wc): New arg CURRENT_POS. All uses changed.
* tests/local.mk (all_tests): Add tests/misc/wc-proc.sh,
tests/misc/od-j.sh, tests/tail-2/tail-c.sh.
* tests/misc/head-c.sh:
* tests/misc/tac-2-nonseekable.sh:
* tests/split/b-chunk.sh:
Add tests for problems with /proc and /sys files.
* tests/misc/od-j.sh, tests/misc/wc-proc.sh, tests/tail-2/tail-c.sh:
New files.
|
|
* src/dd.c (ifd_reopen): A new wrapper to ensure we
don't exit upon receiving a SIGUSR1 in a blocking open()
on a fifo for example.
(iftruncate): Likewise for ftruncate().
(iread): Process signals also after a short read.
(install_signal_handlers): Install SIGINFO/SIGUSR1 handler
even if set to SIG_IGN, as this is what the parent can easily
set from a shell script that can send SIGUSR1 without the
possiblity of inadvertently killing the dd process.
* doc/coreutils.texi (dd invocation): Improve the example to
show robust usage wrt signal races and short reads.
* tests/dd/stats.sh: A new test for various signal races.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
|
|
* src/copy.c (copy_internal): Handle the case where we have the
same destination directory as already encountered, which can only
be due to the corresponding source directory being specified multiple
times.
* tests/cp/duplicate-sources.sh: Add a test for the new multiply
specified directory case, and the existing multiply specified file case.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
|
|
Problem reported by Vincent Lefevre in: http://bugs.gnu.org/18449
* src/cat.c (main): Allow copying an empty file to itself.
* tests/misc/cat-self.sh: New test.
* tests/local.mk (all_tests): Add it.
|
|
Add the --enable-single-binary option to the configure file.
When enabled, this option builds a single binary file containing
the selected tools. Which tool gets executed depends on the value
of argv[0] which can be set implicitly through symlinks to the
single program.
This setup reduces significantly the size of a complete coreutils
install, since code from lib/libcoreutils.a is not duplicated in
every one of the more than 100 binaries. Runtime overhead is
increased due to more dynamic libraries being loaded, and extra
initialization being performed for all utils. Also initially
a larger binary is loaded from storage, though this is usually
alleviated due to caching and lazy mmaping of unused blocks,
and in fact the single binary should have better caching
characteristics.
Comparing the size of the individual versus single binary on x86_64:
$ cd src
$ size coreutils
$ size -t $(../build-aux/gen-lists-of-programs.sh --list-progs |
grep -Ev '(coreutils|libstdbuf)') | tail -n1
text data bss dec hex filename
1097416 5388 88432 1191236 122d44 src/coreutils
4901010 124964 163768 5189742 4f306e (TOTALS)
Storage requirements are reduced similarly:
$ cd src
$ du -h coreutils
$ du -ch $(../build-aux/gen-lists-of-programs.sh --list-progs |
grep -Ev '(coreutils|libstdbuf)') | tail -n1
1.2M coreutils
5.3M total
When installing, the makefile will create either symlinks or
shebangs based on the --enable-single-binary setting, for
each configured tool. In this way, all the tools are still
callable individually, but they are all implemented by the same
"coreutils" binary installed on the same directory.
* .gitignore: Add new generated files.
* Makefile.am: New rules to generate build-aux/gen-single-binary.sh
and install symlinks.
* NEWS: Mention the new feature.
* README: Add "coreutils" to the list of utils.
* bootstrap.conf: Regenerate src/single-binary.mk
* build-aux/gen-lists-of-programs.sh: New --list-progs option.
* build-aux/gen-single-binary.sh: Regenerate
* configure.ac: New --enable-single-binary option and other variables.
Disallow --enable-single-binary=symlinks with --program-prefix et. al.
* man/coreutils.x: Manpage hook.
* man/local.mk: Add manpage hook and fix dependencies.
* src/coreutils.c: Multicall implementation.
* src/local.mk: New rules for the single binary option.
* tests/local.mk: Add $single_binary_progs to support
require_built_() from init.cfg
* tests/misc/env.sh: Avoid the use of symlink to echo.
* tests/misc/help-version.sh: Add exception for coreutils.
* tests/install/basic-1.sh: Really avoid using ginstall strip
functionality if there is an issue with the independent strip command.
* src/kill.c: Changes to call exit() in main.
* src/readlink.c: Likewise.
* src/shuf.c: Likewise.
* src/timeout.c: Likewise.
* src/truncate.c: Likewise.
|
|
* src/df.c (last_device_for_mount): A new function to identify
the last device mounted for a mount point.
(get_disk): Use the above to discard mount entries for a device,
where a later mount entry uses a different device name than
that of the user specified device.
* tests/df/over-mount-device.sh: A new root test.
* tests/local.mk: Reference the new test.
* NEWS: Reword for all these related recent fixes.
Discussed at: http://bugs.gnu.org/16539#69
|
|
* src/shred.c (main): With the preceding change, shred -s-2 FILE
would write 64KB blocks forever -- or until disk full. This change
makes shred reject a negative size.
* tests/misc/shred-negative.sh: New file.
* tests/local.mk (all_tests): Add it.
|
|
Problem reported by Don Baggett in <http:/bugs.gnu.org/17422>.
* NEWS:
* doc/coreutils.texi (dd invocation): Document this.
* src/dd.c (conversions): conv=ascii implies conv=unblock.
conv=ebcdic and conv=ibm imply conv=block.
(ascii_to_ebcdic, ebcdic_to_ascii): Correct to match
POSIX 1003.1-2013.
* tests/dd/ascii.sh: New file.
* tests/local.mk (all_tests): Add it.
|
|
* init.cfg (require_smack_): New function.
* local.mk: Referenced new tests.
* tests/id/smack.sh: SMACK tests (new file).
* tests/mkdir/smack-no-root.sh: SMACK tests (new file).
* tests/mkdir/smack-root.sh: SMACK tests (new file).
|
|
--colors controls whether to output colors depending on
whether we're connected to a terminal or not, while this
change gives control over which terminals we output colors to.
* NEWS: Mention the change in behavior.
* src/ls.c (known_term_type): A new function to search the static
list from dircolors.h
(parse_ls_colors): Honor the TERM when both LS_COLORS and COLORTERM
are non empty.
* tests/ls/color-term.sh: A new test.
* tests/local.mk: Reference the new test.
Fixes http://bugs.gnu.org/15992
|
|
If we can't output more data, we should immediately
diagnose the issue and exit rather than consuming all
of input (in some cases).
* src/tail.c (xwrite_stdout): Also diagnose the case where
only some data is written. Also clearerr() to avoid the
redundant less specific error from atexit (close_stdout);
* src/head.c (xwrite_stdout): Copy this new function from tail,
and use it to write all output.
* tests/misc/head-write-error.sh: A new test to ensure we
exit immediately on write error.
* tests/local.mk: Reference the new test.
|
|
* src/od.c (main): Handle the new --endian option,
taking "little" and "big" as parameters.
(usage): Describe the new option.
(PRINT_FIELDS): Adjust to swap bytes if required.
* tests/misc/od-endian.sh: A new test to verify
the byte swapping operations for hex (ints) and floats
for all sizes between 1 and 16 inclusive.
* test/local.mk: Reference the new test.
* doc/coreutils.texi (od invocation): Describe the new option.
* NEWS: Mention the new feature.
|
|
* src/selinux.c (restorecon_private): On ArchLinux the
`fakeroot cp -a file1 file2` command segfaulted due
to getfscreatecon() returning a NULL context.
So map this to the sometimes ignored ENODATA error,
rather than crashing.
* tests/cp/no-ctx.sh: Add a new test case.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/16335
|
|
Run "make update-copyright", but then also run this,
perl -pi -e 's/2\d\d\d-//' tests/sample-test
to make that one script use the single most recent year number.
|
|
This is so the matching for the device is done on the canonical name
of the disk node, rather than on the path of the symlink.
In any case the user will generally want to use the symlink target.
* src/df.c (get_disk): Canonicalize the passed file,
before matching against the list of mounted file system devices.
Note we pass the original symlink name to the "file" output field,
as the symlink target is usually available through the "source" field.
* tests/df/df-symlink.sh: Test the dereferencing operation.
* tests/local.mk: Mention the new test.
* NEWS: Mention the fix.
Reported by Ondrej Oprala
|
|
* tests/rm/r-root.sh: Add a non-root test.
* tests/local.mk (all_tests): Mention the test.
|
|
cp, mv, install, mkdir, mkfifo, mknod are adjusted so that:
-Z no longer accepts an argument.
-Z or --context without an argument do not warn without SELinux.
--context with an argument will warn without SELinux.
* src/local.mk: Reference the new selinux module where required.
* src/system.h: Make the argument to --context optional.
* src/mkdir.c: Likewise. Also handle the SMACK case for --context.
Note we currently silently ignore -Z with SMACK.
* src/mkfifo.c: Likewise.
* src/mknod.c: Likewise.
* src/install.c: Likewise. Note install(1) by default already
set the context for target files to their system default,
albeit with an older method. Use the -Z option to select between
the old and new context restoration behavior, and document
the differences and details for how context restoration
is done in new and old methods, with a view disabling the
old method entirely in future.
* src/cp.c: Make the argument to --context optional.
Note -Z implies --no-preserve=context. I.E. -Z overrides
that aspect of -a no matter what order specified.
(struct cp_options): Document the context handling options.
(main): Check/adjust option combinations after all
options are processed, to both simplify processing
and to make handling independent of order of options
on the command line. Also improve the diagnostics
from a failed call to setfscreatecon().
(set_process_security_ctx): A new function,
refactored to set the default context from the source file,
or with the type adjusted as per the system default for
the destination path.
(set_file_security_ctx): A new function refactored to
set the security context of an existing file, either based on
the process context or the default system context for a path.
(copy_internal): Use the refactored functions to simplify
error handling and consistently fail or warn as needed.
(copy_reg): Likewise.
(copy_internal): With --preserve=context, also copy
context from non regular files. Note for directories this may
impact the copying of subsequent files to that directory?
(copy_attr): If we're handling SELinux explicitly,
then exclude to avoid the redudant copy with --preserve=context,
and the problematic copy with -Z. Note SELinux attribute exclusion
also now honors cp -a --no-preserve=context. Note there was a
very small window over 10 years ago, where attr_copy_file was
available, while attr_copy_check_permissions was not, so we
don't bother adding an explicit m4 check for the latter function.
* src/mv.c: Support --context, but don't allow specifying an argument.
* src/chcon.c: Adjust a comment to be specific to SELinux.
* src/runcon.c: Likewise.
* src/copy.c: Honor the context settings to "restorecon" as appropriate.
* src/copy.h: Add a new setting to select "restorecon" functionality.
* tests/mkdir/selinux.sh: s/-Z/--context=/
* tests/cp/cp-a-selinux.sh: Augment this test with cases
testing basic -Z functionality, and also test the various
invalid option combinations and option precedence.
* tests/mkdir/restorecon.sh: Add a new test for the
more involved mkdir -Z handling, since the directory changing
and non existent directories need to be specially handled.
Also check the similar but simpler handling of -Z by mk{nod,fifo}.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (cp invocation): Update as per interface changes.
(mv invocation): Likewise.
(install invocation): Likewise.
(mkfifo invocation): Likewise.
(mknod invocation): Likewise.
(mkdir invocation): Likewise.
* NEWS: Mention the new feature and change in behavior.
|
|
Previous behavior failed to read contents of a (re)appearing file,
when symlinked by tail's watched file. Also we now diagnose other
edge cases when running in inotify mode, where an initially
missing or regular file changes to a symlink.
* src/tail.c (main): If any arg is a symlink, use polling mode.
(recheck): Diagnose the edge case where a symlink appears during
inotify processing.
* tests/tail-2/symlink.sh: Test the fix. Mention the edge cases.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
Reported by: Ondrej Oprala
|
|
* tests/local.mk (factor-tests:) Add -f to the mv command that
replaces any existing generated tests. This is required to avoid
prompts when root initially generates the tests, and they subsequently
need to be regenerated by a non root user.
|
|
* src/copy.c (create_hard_link): Add a bool 'dereference' parameter,
and pass AT_SYMLINK_FOLLOW as 'flags' to linkat() when dereference
is true.
(should_dereference): Add new 'bool' function to determine if a
file should be dereferenced or not.
(copy_internal): Use the above new should_dereference() and remember
its return value in a new local bool 'dereference' variable. Use that
in all three calls to create_hard_link().
* src/cp.c (main): after parsing the options, if x.dereference is
still DEFEF_UNDEFINED and the x.recursive is true, then only set
x.dereference to DEREF_NEVER iff --link was not specified.
* doc/coreutils.texi (cp invocation): Mention that cp(1) does not
follow symbolic links in the source when --link is specified.
Likewise in the description of the -R option when used together with
that option.
* tests/cp/same-file.sh: Adapt the expected results for the -fl,
the -bl and the -bfl tests.
* tests/cp/link-deref.sh: Add a new test.
* tests/local.mk (all_tests): Reference the above new test.
* NEWS (Changes in behavior): Mention the change.
This fixes http://bugs.gnu.org/15173
Co-authored-by: Bernhard Voelker <mail@bernhard-voelker.de>
|
|
* tests/misc/id-context.sh -> tests/id/context.sh
* tests/misc/id-setgid.sh -> tests/id/setgid.sh
* tests/misc/id-uid.sh -> tests/id/uid.sh
* tests/misc/id-zero.sh -> tests/id/zero.sh
* tests/local.mk: Reference the renamed tests
|
|
* src/id.c (usage): Remove 'name' from the synopsis,
implying that one can also specify by user ID.
(main): Like chown(1), call parse_user_spec() to implement
user name or ID lookup with appropriate precedence.
* doc/coreutils.texi (id invocation): Mention that
a user ID is supported and how '+' affects lookup order.
* tests/misc/id-groups.sh: Remove test now subsumed into...
* tests/misc/id-uid.sh: New test covering new interface.
* tests/local.mk: Rename the test.
* NEWS: Mention the new feature.
Addresses http://bugs.gnu.org/15421
|
|
* src/group-list.h (print_group_list): Add a parameter for the
delimiter of type char.
* src/group-list.c (print_group_list): Likewise, and use it instead
of a white space character to delimit the group entries.
* src/groups.c (main): Pass white space character to print_group_list().
* src/id.c (longopts): Add array element for the new long option.
(usage): Document the new option. While at it, fix the alignment
of the descriptions to match that of HELP_OPTION_DESCRIPTION.
(main): Define the bool flag opt_zero indicating the use of the
new option. In the getopt_long loop, handle it.
Output an error diagnostic in the case the --zero option has been
specified together with the default format.
In the case of -gG, pass either a NUL or a white space character to
print_group_list() - depending on the above new flag.
Likewise change the printing of the final newline character: output
a NUL instead if the --zero option has been specified.
* doc/coreutils.texi (id invocation): Document the new option.
While at it, move the @exitstatus macro down after the macro
@primaryAndSupplementaryGroups in order to be consistent with
other texinfo documents.
(groups invocation): Move @exitstatus down after the macro
@primaryAndSupplementaryGroups here, too.
* tests/misc/id-zero.sh: Add new test exercising the new option.
* tests/local.mk (all_tests): Reference it.
* NEWS (New features): Mention the new option.
Fixes http://bugs.gnu.org/9987
|
|
This new option can be used to find directories with a huge
amount of files. The GNU find utility has the printf format
"%h" which prints the number of entries in a directory, but
this is non-cumulative and doesn't handle hard links.
* src/du.c (struct duinfo): Add new member for counting inodes.
(duinfo_init): Initialize inodes member with Zero.
(duinfo_set): Set inodes counter to 1.
(duinfo_add): Sum up the 2 given inodes counters.
(opt_inodes): Add new boolean flag to remember if the --inodes
option has been specified.
(INODES_OPTION): Add new enum value to be used ...
(long_options): ... here.
(usage): Add description of the new option.
(print_size): Pass inodes counter or size to print_only_size,
depending on the inodes mode.
(process_file): Adapt threshold handling: with --inodes, print or
elide the entries according to the struct member inodes.
(main): Add a case for accepting the new INODES_OPTION.
Print a warning diagnostic when --inodes is used together with the
option --apparent-size or -b.
Reset the output_block_size to 1 ... and thus ignoring the
options -m and -k.
* tests/du/inodes.sh: Add a new test.
* tests/local.mk (all_tests): Mention it.
* doc/coreutils.texi (du invocation): Document the new option.
* NEWS: Mention the new option.
|
|
* src/split.c (line_bytes_split): Rewrite to only buffer
when necessary. I.E. only increase the buffer when we've
already lines output in a split and we encounter a line
larger than the input buffer size, in which case a hold
buffer will be increased in increments of the input buffer size.
(lines_rr): Use the more abstract xalloc_die() just like
we did in line_bytes_split(), rather than explicitly
printing the "memory exhausted" message and exiting.
* tests/split/line-bytes.sh: Add a new test for this
function which previously had no test coverage.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
Fixes http://bugs.gnu.org/13537
|
|
This fixes Bug#14371, reported by Killer Bassist.
* NEWS: Document this.
* src/mkdir.c (struct mkdir_options): Remove member ancestor_mode.
New member umask_value. All uses changed.
* src/mkdir.c (make_ancestor): Fix umask assumption.
* src/mkdir.c, src/mkfifo.c, src/mknod.c (main):
Leave umask alone. This requires invoking lchmod after creating
the file, which introduces a race condition, but this can't be
avoided on hosts with "POSIX" default ACLs, and there's no easy
way with network file systems to tell what kind of host the
directory is on.
* tests/local.mk (all_tests): Add tests/mkdir/p-acl.sh.
* tests/mkdir/p-acl.sh: New file.
|
|
The current implementation of cut, uses a bit array,
an array of `struct range_pair's, and (when --output-delimiter
is specified) a hash_table. The new implementation will use
only an array of `struct range_pair's.
The old implementation is memory inefficient because:
1. When -b with a big num is specified, it allocates a lot of
memory for `printable_field'.
2. When --output-delimiter is specified, it will allocate 31 buckets.
Even if only a few ranges are specified.
Note CPU overhead is increased to determine if an item is to be printed,
as shown by:
$ yes abcdfeg | head -n1MB > big-file
$ for c in with-bitarray without-bitarray; do
src/cut-$c 2>/dev/null
echo -ne "\n== $c =="
time src/cut-$c -b1,3 big-file > /dev/null
done
== with-bitarray ==
real 0m0.084s
user 0m0.078s
sys 0m0.006s
== without-bitarray ==
real 0m0.111s
user 0m0.108s
sys 0m0.002s
Subsequent patches will reduce this overhead.
* src/cut.c (set_fields): Set and initialize RP
instead of printable_field.
* src/cut.c (is_range_start_index): Use CURRENT_RP rather than a hash.
* tests/misc/cut.pl: Check if `eol_range_start' is set correctly.
* tests/misc/cut-huge-range.sh: Rename from cut-huge-to-eol-range.sh,
and add a test to verify large amounts of mem aren't allocated.
Fixes http://bugs.gnu.org/13127
|
|
The --retry option is indeed useful for both following modes
by name and by file descriptor. The difference is that in the
latter case, it is effective only during the initial open.
As a regression of the implementation of the inotify support,
tail -f --retry would immediately exit if the given file is
inaccessible.
* src/tail.c (usage): Change the description of the --retry option:
remove the note that this option would mainly be useful when
following by name.
(main): Change diagnosing dubios uses of --retry option:
when the --retry option is used without following, then issue
a warning that this option is ignored; when it is used together
with --follow=descriptor, then issue a warning that it is only
effective for the initial open.
Disable inotify also in the case when the initial open in tail_file()
failed (which is the actual bug fix).
* init.cfg (retry_delay_): Pass excess arguments to the test function.
* tests/tail-2/retry.sh: Add new tests.
* tests/local.mk (all_tests): Mention it.
* doc/coreutils.texi (tail invocation): Enhance the documentation
of the --retry option. Clarify the difference in tail's behavior
regarding the --retry option when combined with the following modes
name versus descriptor.
* NEWS (Bug fixes): Mention the fix.
Reported by Noel Morrison in:
http://lists.gnu.org/archive/html/coreutils/2013-04/msg00003.html
|
|
* src/dd.c: Add new static global variable ibuf.
(alloc_ibuf, alloc_obuf): New functions factored from dd_copy().
(dd_copy): Call the new functions to allocate memory for
ibuf and obuf when necessary.
(skip): Likewise.
* tests/dd/no-allocate.sh: New test.
* tests/local.mk: Reference the test.
|
|
With --suppress-matched, the lines that match the pattern will not be
printed in the output files. I.E. the first line from the second
and subsequent splits will be suppressed.
* src/csplit.c: process_regexp(),process_line_count(): Don't output the
matched lines. Since csplit includes "up to but not including" matched
lines in each split, the first line (in the next group) is the matched
line - so just skip it.
main(): Handle new option.
usage(): Mention new option.
* doc/coreutils.texi (csplit invocation): Mention new option, examples.
* tests/misc/csplit-suppress-matched.pl: New test script.
* tests/local.mk: Reference the new test.
* NEWS: Mention new feature.
|
|
Reservoir sampling optimizes selecting K random lines from large or
unknown-sized input: http://en.wikipedia.org/wiki/Reservoir_sampling
Note this also avoids reading any input when -n0 is specified.
* src/shuf.c (main): Use reservoir-sampling when the number of output
lines is known, and the input size is large or unknown.
(input_size): A new function to get the input size for regular files.
(read_input_reservoir_sampling): New function to read lines from input,
keeping only K lines in memory, replacing lines with decreasing prob.
(write_permuted_output_reservoir): New function to output reservoir.
* tests/misc/shuf-reservoir.sh: An expensive_ test using valgrind to
exercise the reservoir-sampling code.
* tests/local.mk: Reference new test.
* NEWS: Mention the improvement.
|
|
* src/system.h (emit_ancillary_info): Link to the bug report email
addresses and general help URLs online rather than specifying directly.
This give us greater scope to present better info like describing
the difference between bug-coreutils@gnu.org and coreutils@gnu.org etc.
* tests/misc/help-version.sh: Remove the check for bug-coreutils@gnu.org
* tests/local.mk: Remove the no longer needed PACKAGE_BUGREPORT.
|
|
* tests/local.mk (check-root): Restrict to SUBDIRS=. as traversing
into gnulib-tests induces a false failure.
|
|
* AUTHORS: Add my name.
* NEWS: Mention the new program.
* README: Reference the new program.
* src/numfmt.c: New file.
* src/.gitignore: Ignore the new binary.
* build-aux/gen-lists-of-programs.sh: Update.
* scripts/git-hooks/commit-msg: Allow numfmt: commit prefix.
* po/POTFILES.in: Add new c file.
* tests/misc/numfmt.pl: A new test file giving >93% coverage.
* tests/local.mk: Reference the new test.
* man/.gitignore: Ignore the new man page.
* man/local.mk: Reference the new man page.
* man/numfmt.x: A new template.
* doc/coreutils.texi: Document the new command.
|
