| Age | Commit message (Collapse) | Author |
|---|
|
* src/chroot.c: Explicitly cast int to pointer type.
|
|
This didn't seem to cause any invalid operation on GNU/Linux at least,
but depending on the implementation, mutex deadlocks could occur.
For example this might be the cause of lockups seen on Solaris:
http://lists.gnu.org/archive/html/coreutils/2013-03/msg00048.html
This was identified with valgrind 3.9.0 with this setup:
seq 200000 > file.sort
valgrind --tool=drd src/sort file.sort -o file.sort
With that, valgrind would _intermittently_ report the following:
Destroying locked mutex: mutex 0x5419548, recursion count 1, owner 2.
at 0x4C2E3F0: pthread_mutex_destroy(in vgpreload_drd-amd64-linux.so)
by 0x409FA2: sortlines (sort.c:3649)
by 0x409E26: sortlines (sort.c:3621)
by 0x40AA9E: sort (sort.c:3955)
by 0x40C5D9: main (sort.c:4739)
mutex 0x5419548 was first observed at:
at 0x4C2DE82: pthread_mutex_init(in vgpreload_drd-amd64-linux.so)
by 0x409266: init_node (sort.c:3276)
by 0x4092F4: init_node (sort.c:3286)
by 0x4090DD: merge_tree_init (sort.c:3234)
by 0x40AA5A: sort (sort.c:3951)
by 0x40C5D9: main (sort.c:4739)
Thread 2:
The object at address 0x5419548 is not a mutex.
at 0x4C2F4A4: pthread_mutex_unlock(in vgpreload_drd-amd64-linux.so)
by 0x4093CA: unlock_node (sort.c:3323)
by 0x409C85: merge_loop (sort.c:3531)
by 0x409F8F: sortlines (sort.c:3644)
by 0x409CE3: sortlines_thread (sort.c:3574)
by 0x4E44F32: start_thread (in /usr/lib64/libpthread-2.18.so)
by 0x514EEAC: clone (in /usr/lib64/libc-2.18.so)
* src/sort.c (sortlines): Move pthread_mutex_destroy() out to
merge_tree_destroy(), so that we don't overlap mutex destruction
with threads still operating on the nodes.
(sort): Call the destructors only with "lint" defined, as the
memory used will be deallocated implicitly at process end.
* NEWS: Mention the bug fix.
|
|
Neither issue impacts on the correct operation of sort.
The issues were detected by both valgrind 3.8.1 and 3.9.0 using:
seq 200000 > file.sort
valgrind --tool=drd src/sort file.sort -o file.sort
For tool usage and error details see:
http://valgrind.org/docs/manual/drd-manual.html
* src/sort.c (queue_insert): Unlock mutex _after_ signalling the
associated condition variable. Valgrind flags this with:
"Probably a race condition: condition variable 0xffeffffb0 has been
signaled but the associated mutex 0xffeffff88 is not locked by the
signalling thread."
The explanation at the above URL is:
"Sending a signal to a condition variable while no lock is held on
the mutex associated with the condition variable. This is a common
programming error which can cause subtle race conditions and
unpredictable behavior."
This should at least give more defined scheduling behavior.
(merge_tree_destroy): Make symmetrical with merge_tree_init() thus
destroying the correct mutex. Valgrind flags this with:
"The object at address 0x5476cf8 is not a mutex."
|
|
* src/df.c (get_disk): Avoid an inconsequential mem leak
spotted by coverity. Also s/duplicities/duplicates/.
|
|
Add the --enable-single-binary option to the configure file.
When enabled, this option builds a single binary file containing
the selected tools. Which tool gets executed depends on the value
of argv[0] which can be set implicitly through symlinks to the
single program.
This setup reduces significantly the size of a complete coreutils
install, since code from lib/libcoreutils.a is not duplicated in
every one of the more than 100 binaries. Runtime overhead is
increased due to more dynamic libraries being loaded, and extra
initialization being performed for all utils. Also initially
a larger binary is loaded from storage, though this is usually
alleviated due to caching and lazy mmaping of unused blocks,
and in fact the single binary should have better caching
characteristics.
Comparing the size of the individual versus single binary on x86_64:
$ cd src
$ size coreutils
$ size -t $(../build-aux/gen-lists-of-programs.sh --list-progs |
grep -Ev '(coreutils|libstdbuf)') | tail -n1
text data bss dec hex filename
1097416 5388 88432 1191236 122d44 src/coreutils
4901010 124964 163768 5189742 4f306e (TOTALS)
Storage requirements are reduced similarly:
$ cd src
$ du -h coreutils
$ du -ch $(../build-aux/gen-lists-of-programs.sh --list-progs |
grep -Ev '(coreutils|libstdbuf)') | tail -n1
1.2M coreutils
5.3M total
When installing, the makefile will create either symlinks or
shebangs based on the --enable-single-binary setting, for
each configured tool. In this way, all the tools are still
callable individually, but they are all implemented by the same
"coreutils" binary installed on the same directory.
* .gitignore: Add new generated files.
* Makefile.am: New rules to generate build-aux/gen-single-binary.sh
and install symlinks.
* NEWS: Mention the new feature.
* README: Add "coreutils" to the list of utils.
* bootstrap.conf: Regenerate src/single-binary.mk
* build-aux/gen-lists-of-programs.sh: New --list-progs option.
* build-aux/gen-single-binary.sh: Regenerate
* configure.ac: New --enable-single-binary option and other variables.
Disallow --enable-single-binary=symlinks with --program-prefix et. al.
* man/coreutils.x: Manpage hook.
* man/local.mk: Add manpage hook and fix dependencies.
* src/coreutils.c: Multicall implementation.
* src/local.mk: New rules for the single binary option.
* tests/local.mk: Add $single_binary_progs to support
require_built_() from init.cfg
* tests/misc/env.sh: Avoid the use of symlink to echo.
* tests/misc/help-version.sh: Add exception for coreutils.
* tests/install/basic-1.sh: Really avoid using ginstall strip
functionality if there is an issue with the independent strip command.
* src/kill.c: Changes to call exit() in main.
* src/readlink.c: Likewise.
* src/shuf.c: Likewise.
* src/timeout.c: Likewise.
* src/truncate.c: Likewise.
|
|
Avoid complicated and error-prone parsing of df's output via
sed(1), cut(1), etc., and instead use df's more modern --output
option.
* src/ioblksize.h (in a comment): Simplify the extraction of the
device name of the mounted file system from df's output.
* tests/dd/skip-seek-past-dev.sh: Likewise.
* tests/du/2g.sh: Likewise for the 'avail' column here.
Also avoid the deprecated use of "tail -NUM".
* tests/misc/stat-mount.sh: While at it, remove the determination
of the mount point of "." via df(1) plus sed(1) as it is unused
since commit v8.5-159-gf57cb37 anyway. Instead, improve this test
by verifying that the output of "stat -c%m ." at least starts with
a slash '/'.
|
|
Revert commit v8.22-131-g3e89d5b as even though POSIX
states that the default mode should be -L,
common practice for stand-alone pwd implementations
is to default to -P.
* src/pwd.c (usage): Retain mention of the default mode of operation.
Suggested by Bob Proulx
|
|
On systems without libselinux, context_t is defined as an int type,
but defined as a pointer type by SELinux.
* src/chcon.c: Init with 0 rather than NULL to avoid warning.
|
|
The security_context_t type was always an artificial separation
from a standard char* string, and various libselinux using code
assumed both were synonymous. In addition, prior to libselinux 2.3
function declarations were incorrect wrt constness of this type.
Here we replace security_context_t with char*, and also
provide a wrapper function to cater for the const issue on
older libselinux.
* src/system.h (se_const): A new function to avoid and identify
the const issue on older libselinux.
* src/copy.c: s/security_context_t/char */.
* src/cp.c: Likewise.
* src/id.c: Likewise.
* src/install.c: Likewise.
* src/ls.c: Likewise.
* src/mkdir.c: Likewise.
* src/mkfifo.c: Likewise.
* src/mknod.c: Likewise.
* src/runcon.c: Likewise.
* src/selinux.c: Likewise.
* tests/cp/no-ctx.sh: Likewise.
* src/chcon.c: Likesize.
|
|
Since context is verified by security_check_context() it can be used in
change_file_context() without converting to context_t every time.
* src/chcon.c (change_file_context): Use specified_context directly.
|
|
context_new() and _free() are used for checking validity of a
specified context. libselinux provides security_check_context
for this purpose so use it.
Note that context_new() can fail for a valid context - e.g. ENOMEM.
* src/chcon.c (main): Use security_check_context().
|
|
* src/pwd.c (main): Adjust default mode to be "logical"
and independent of the POSIXLY_CORRECT env var.
(usage): Mention the default mode of operation.
* doc/coreutils.texi (pwd invocation): Adjust accordingly.
* tests/misc/pwd-option.sh: Likewise.
* NEWS: Mention the change in behavior.
|
|
* src/id.c (print_full_info): When no user is specified,
output the effective group for the _process_, rather than
the default group from the system database, which may be different.
* tests/id/setgid.sh: Add a case for `id` as well as `id -G`.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/7320
Reported at http://bugzilla.redhat.com/1016163
|
|
* src/df.c (last_device_for_mount): A new function to identify
the last device mounted for a mount point.
(get_disk): Use the above to discard mount entries for a device,
where a later mount entry uses a different device name than
that of the user specified device.
* tests/df/over-mount-device.sh: A new root test.
* tests/local.mk: Reference the new test.
* NEWS: Reword for all these related recent fixes.
Discussed at: http://bugs.gnu.org/16539#69
|
|
* src/df.c (get_disk): Include whether we can access the mount dir,
in the mount entry selection criteria. This handles the case where
a device is (bind) mounted multiple times with the shortest mount path
not being accessible, while some of the other mount points are.
Discussed at: http://bugs.gnu.org/16539#63
|
|
A system provided mount entry may be unavailable due to TOCTOU race,
or if another device has been over-mounted at that position, or due to
access permissions. In all these cases output "-" placeholder values
rather than either producing an error, or in the over-mount case
outputting values for the wrong device.
* src/df.c (device_list): A new global list now updated by
filter_mount_list().
(filter_mount_list): Adjust to take a parameter as to whether
update the global mount list, or only the mount <-> device ID mapping.
(get_dev): Use the device ID mapping to ensure we're not outputting
stats for the wrong device. Also output placeholder values when we
can't access a system specified mount point.
(get_all_entries): Set the DEVICE_ONLY param for filter_mount_list().
(devname_for_dev): A new function to search the mount <-> dev mapping.
* test/df/skip-duplicates.sh: Adjust accordingly.
* NEWS: Mention the bug fixes.
Discussed at: http://bugs.gnu.org/16539
|
|
* src/du.c (process_file): Treat cycles due to bind mounts
like cycles due to following symlinks.
* tests/du/bind-mount-dir-cycle.sh: Adjust accordingly.
* NEWS: Mention the change in behavior.
Reported at http://bugzilla.redhat.com/836557
|
|
* src/ls.c (usage): State that --human changes --size format.
Fixes http://bugs.gnu.org/17838
|
|
* src/df.c (filter_mount_list): Recent commit v8.22-108-g25a2c94
failed to copy file system type along with the updated device name.
Therefore simply replace the existing mount entry with the
current one with all the latest device details. Note the name,
even if not shorter in this entry, will be replaced with a shorter
name in a subsequent mount entry.
* tests/df/skip-duplicates.sh: Add a test case.
|
|
* src/seq.c (main): Avoid seq_fast() with a start or end of -0.
* tests/misc/seq.pl: Add test cases.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/17800
|
|
Veritas File System can run in single instance or clustered mode,
so mark as remote to avoid using inotify for the latter case.
* src/stat.c (human_fstype): Tag VXFS as remote, to use polling
for the clustered variant (VXCFS).
* NEWS: Mention the bug fix.
Reported by Ondřej Vašík in http://bugzilla.redhat.com/1104244
Fixes http://bugs.gnu.org/17770
|
|
* src/numfmt.c (double_to_human): The printf format is built up in
a stack buffer which is big enough to hold any of the possible formats.
However the size parameter passed to snprintf was too big by 1
when GROUP was true. So decrease the buffer available to snprintf
to avoid this theoretical in practise but valid coverity warning.
|
|
* src/install.c (install_file_in_file_parents): Factor out the
creation of any parent directories into ...
(mkancesdirs_safe_wd): ... this new function.
(install_file_in_dir): Add the parameter 'mkdir_and_install', and
call the above new function if it evaluates to true.
(main): During parsing of the -t option, move the check whether
the target_directory exists down after the option parsing loop,
and do not complain about stat(optarg,...) failing if -D was given.
Pass 'mkdir_and_install' to install_file_in_dir().
* doc/coreutils.texi (install invocation): Remove the (false)
restriction that -D would be ignored together with -t. Instead,
clarify install's new bahavior.
Fix the node's reference in the top-level @direntry for consistency.
* src/install/basic-1.sh: Add tests for the now-allowed combination
of the -D and -t options.
* NEWS: Mention the improvement.
|
|
* src/true.c (main): Add a comment about the possibility
of true returning EXIT_FAILURE due to write failure.
* tests/misc/false-status.sh: Fix so we're testing
the tool and not the shell builtin. Add a case for true(1).
* tests/misc/help-version.sh: Skip /dev/full test
for true as well as false since the exit status is tested separately.
Also remove the iterations for different LC_MESSAGES, as this was only
applied for false(1). Translations are not honored in the test dir
and so would need separate handling in any case.
|
|
* src/df.c (get_dev): asssert() on Solaris 10 is not marked as
__noreturn__ and thus the compiler may think V is uninitialized
later on in the function.
* THANKS.in: Remove the now committer.
|
|
commits v8.20-98-g51ce0bf and v8.20-99-gd302aed changed cut(1)
to process each line independently and thus promptly output
each line without buffering. As part of those changes we removed
the special handling of --delimiter=$'\n' --fields=... which
could be used to select arbitrary (ranges of) lines, so as to
simplify and optimize the implementation while also matching the
behavior of different cut(1) implementations.
However that GNU behavior was in place for a long time, and
could be useful in certain cases like making a separated list like
`seq 10 | cut -f1- -d$'\n' --output-delimiter=,` although other tools
like head(1) and paste(1) are more suited to this operation.
This patch reinstates that functionality but restricts the
"line behind" buffering behavior to only the -d$'\n' case.
We also fix the following related edge case to be more consistent:
before> printf "\n" | cut -s -d$'\n' -f1- | wc -l
2
before> printf "\n" | cut -d$'\n' -f1- | wc -l
1
after > printf "\n" | cut -s -d$'\n' -f1- | wc -l
1
after > printf "\n" | cut -d$'\n' -f1- | wc -l
1
* src/cut.c (cut_fields): Adjust as discussed above.
* tests/misc/cut.pl: Likewise.
* NEWS: Mention the change in behavior both for v8.21
and this effective revert.
* cfg.mk (old_NEWS_hash): Adjust for originally omitted v8.21 entry.
* src/paste.c: s/delimeter/delimiter/ comment typo fix.
|
|
* src/stat.c (find_bind_mount): NAME is invariant in the loop,
so only stat(NAME) outside the loop.
|
|
The device name reported for a particular mount entry
may no longer be valid if the mount point was subsequently
mounted on a different device. Therefore honor the order
of the mount list returned by the system and use the last
reported device name.
* src/df.c (filter_mount_list): When discarding the current
mount entry, ensure that a new device name is not also discarded.
* tests/df/skip-duplicates.sh: Add a test case. Also fix
a false failure in the edge case of a system with only a
single file system.
* NEWS: Mention the fix.
|
|
* src/stdbuf.c (set_LD_PRELOAD): Adjust to use Mac OS X
specific environment variables on __APPLE__ platforms.
Fixes http://bugs.gnu.org/17590
|
|
* src/uname.c (main) [__APPLE__]: Rename inner "s" to "cs",
to avoid gcc shadowing warning.
|
|
* src/join.c (usage): Reword to avoid implication that
the NUL byte is only generated as the output delimeter.
* src/sort.c (usage): Likewise.
* src/shuf.c (usage): Likewise. Also since we're changing the
translation string take the opportunity to separate out
the description to a separate string to reduce translation overhead.
* src/uniq.c (usage): Likewise.
* src/stty.c (usage): s/null/NUL/ for consistency.
* src/basename.c (usage): Reword for accuracy/consistency.
* src/dirname.c (usage): Likewise.
* src/du.c (usage): Likewise.
* src/env.c (usage): Likewise.
* src/printenv.c (usage): Likewise.
* src/readlink.c (usage): Likewise.
* src/realpath.c (usage): Likewise.
* doc/coreutils.texi: Consolidate/share the descriptions of
--null, --zero and --zero-terminated.
|
|
* src/local.mk (src_libstdbuf_so_LDADD): Add $(LIBINTL),
to avoid link failure on OS X.
|
|
* src/ioblksize.h (IO_BUFSIZE): Double the size, to 128KB.
Add four more columns to the blksize-vs-bandwidth table
|
|
* src/stat.c (human_fstype): Add new file system ID definitions.
* NEWS: Mention the improvement.
|
|
* src/stat.c (human_fstype): Adjust a couple of existing constants
to be a consistent width and capitalization so that the
src/fs-magic-compare target works without reporting false positives.
* cfg.mk (sc_fs-magic-compare): A new syntax check to enforce this.
Improved by: Jim Meyering
|
|
__SUNPRO_C >= 0x590 /*12.0*/ supports __attribute((constructor))
required by stdbuf, so use a more direct check for this.
Note ensure that --libexecdir is set to the appropriate
install location for libstdbuf.so so that stdbuf works
when installed on the system like it does when running
tests in the build directory.
* configure.ac (stdbuf_supported): Use a test prog to determine support.
* src/libstdbuf.c (stdbuf): Define appropriately for non GCC compilers,
and provide early feedback (compilation warning) if trying to compile
libstdbuf without the necessary support.
* src/stdbuf.c (set_LD_PRELOAD): Add a note on having stdbuf
look for libstdbuf.so in the default lib search path.
* cfg.mk (sc_prohibit-gl-attributes): Adjust so we can exclude
libstdbuf.so from prohibiting '__attribute', since we want
this form to avoid silently eliding this required attribute on non GCC.
Reported and tested by Rich Burridge.
|
|
* init.cfg (require_root_): Adjust to use chroot, and make
`require_built_ chroot` implicit when chroot used in the test.
* po/POTFILES.in: Remove reference to setuidgid tool.
* src/.gitignore: Likewise.
* src/local.mk: Likewise.
* src/setuidgid.c: Remove.
* tests/cp/preserve-gid.sh: s/setuidgid/chroot --user/.
* tests/cp/special-bits.sh: Likewise.
* tests/id/setgid.sh: Likewise.
* tests/misc/truncate-owned-by-other.sh
* tests/mv/sticky-to-xpart.sh: Likewise.
* tests/rm/fail-2eperm.sh: Likewise.
* tests/rm/no-give-up.sh: Likewise.
* tests/touch/now-owned-by-other.sh: Likewise.
* tests/misc/chroot-fail.sh: Skip if chroot not built.
|
|
* src/chroot.c (is_root): A new helper function to
determine if the passed argument is the root directory
based on inode comparison.
(main): Use the new helper rather than comparing strings.
* tests/misc/chroot-fail.sh: Add cases for alternative root paths.
|
|
* src/chroot.c (main): Consistently exit with failure status immediately
upon hitting a terminal issue, rather than diagnosing multiple issues
lest users think previous failing actions are optional.
|
|
It's dangerous and confusing to leave root's supplemental
groups in place when specifying other users with --userspec.
In the edge case that that is desired one can explicitly
specify --groups.
Also we implicitly set the system defined supplemental groups
for a user. The existing mechanism where supplemental groups
needed to be explicitly specified is confusing and not general
when the lookup needs to be done within the chroot.
Also we extend the --groups syntax slightly to allow clearing
the set of supplementary groups using --groups=''.
* src/chroot.c (setgroups): On systems without supplemental groups,
clearing then is a noop and so should return success.
(main): Lookup the primary GID with getpwuid() when just a numeric
uid is specified, and also infer the USERNAME from this call,
needed when we're later looking up the supplemental groups for a user.
Support clearing supplemental groups, either implicitly for
unknown users, or explicitly when --groups='' is specified.
* tests/misc/chroot-credentials.sh: Various new test cases
* doc/coreutils.texi (chroot invocation): Adjust for the new behavior.
* NEWS: Mention the change in behavior.
|
|
This allows chroot to be used as a light weight tool
to change user identification for a command,
while not changing the current working directory.
It also makes `chroot / true` consistently succeed on
all platforms for non root users.
* src/chroot.c (main): If the same root is specified. i.e. '/'
then don't change the current working directory, and avoid the
overhead of the other redundant calls.
* tests/misc/chroot-fail.sh: Remove failure guard previously
needed on some systems. Also add an explicit case to ensure
we don't change directory.
* NEWS: Mention the change in behavior.
|
|
* src/df.c (decode_output_arg): Use only enum constants to avoid
clang "warning: comparison of constant -1 with expression of
type 'display_field_t' is always false"
|
|
Linux with network namespaces contains entries in /proc/mounts like:
proc net:[4026532464] proc rw,nosuid,nodev,noexec,relatime 0 0
resulting in a failure to stat 'net:[...]', inducing a warning
and an exit with failure status.
* src/df.c (get_dev): Ignore all relative mount points.
* tests/df/skip-duplicates.sh: Add an entry to test relative dirs.
|
|
The symlink handling in commit v8.21-172-g33660b4 was incomplete
in the case where there were symlinks in the mount list itself.
For example, in the case where /dev/mapper/fedora-home was in the
mount list and that in turn was a symlink to /dev/dm-2, we have:
before> df --out=source /dev/mapper/fedora-home
devtmpfs
after > df --out=source /dev/mapper/fedora-home
/dev/mapper/fedora-home
* src/df.c (get_disk): Compare canonicalized device names from
the mount list. Note we still display the non canonicalized name,
even if longer, as we assume that is the most representative.
* tests/df/df-symlink.sh: This could theoretically fail on some systems
depending on the content of the mount list, but adjust to fail on any
system where symlinks are present in the mount list for the current dir.
|
|
* src/df.c (filter_mountlist): Remove the constraint that
a '/' needs to be in the device name for a mount entry to
be considered for deduplication. Virtual file systems also
have storage associated with them (like tmpfs for example),
and thus need to be deduplicated since they will be shown
in the default df output and subject to --total processing also.
* test/df/skip-duplicates.sh: Add a test to ensure we deduplicate
all entries, even for virtual file systems. Also avoid possible
length operations on many remote file systems in the initial
check of df operation. Also avoid the assumption that "/root"
is on the same file system as "/".
* NEWS: Mention the change in behavior.
|
|
* src/shred.c (main): With the preceding change, shred -s-2 FILE
would write 64KB blocks forever -- or until disk full. This change
makes shred reject a negative size.
* tests/misc/shred-negative.sh: New file.
* tests/local.mk (all_tests): Add it.
|
|
* src/shred.c (main): Limit -n (number of passes) value to
ULONG_MAX, not to UINT32_MAX, since the vars are unsigned long.
Limit the -s (file size) value to OFF_T_MAX.
|
|
Problem reported by Don Baggett in <http:/bugs.gnu.org/17422>.
* NEWS:
* doc/coreutils.texi (dd invocation): Document this.
* src/dd.c (conversions): conv=ascii implies conv=unblock.
conv=ebcdic and conv=ibm imply conv=block.
(ascii_to_ebcdic, ebcdic_to_ascii): Correct to match
POSIX 1003.1-2013.
* tests/dd/ascii.sh: New file.
* tests/local.mk (all_tests): Add it.
|
|
* src/copy.c (copy_internal): Include the copy_attr() call for symlinks.
This should not dereference symlinks, since llistxattr() is used
in attr_copy_file() in libattr, and so should copy all but the filtered
extended attributes. Note we don't just move the copy_attr() call
before the set_owner() call, as that would break capabilities
for non symlinks.
* tests/cp/cp-mv-enotsup-xattr.sh: Add a test case.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/16131
|
|
The devmsg() calls that took quote_n() arguments,
didn't normally output anything, but still incurred
the overhead of those quote_n() calls.
* src/numfmt.c (devmsg): Move the inline function
with _internal_ enablement check to...
* src/system.h: ...here as a variadic macro, with
the enablement check at the outer level.
* src/factor.c: As per numfmt.c but there is no
performance change in this case.
* NEWS: Mention the significant performance improvement.
|
