summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2017-02-15maint: use xsetmode, not xfreopenPaul Eggert
This fixes a bug noted by Eric Blake. Code was using xfreopen to change files to binary mode, but this fails for stdout when in append mode. Such code should use xsetmode instead. This affects only the port on platforms like MS-Windows which distiguish text from binary I/O. * bootstrap.conf (gnulib_modules): Remove xfreopen and add xsetmode. Sort. * src/base64.c (main): * src/cat.c (main): * src/cksum.c (cksum): * src/head.c (head_file, main): * src/md5sum.c (digest_file): * src/od.c (open_next_file): * src/split.c (main): * src/sum.c (bsd_sum_file, sysv_sum_file): * src/tac.c (tac_file, main): * src/tail.c (tail_file): * src/tee.c (tee_files): * src/tr.c (main): * src/wc.c (wc_file): Use xsetmode, not xfreopen.
2017-02-12maint: tweaks so syntax tests pass for previous commitPádraig Brady
* src/force-link.h: Don't include headers already included by system.h * src/force-link.c: Likewise. Also include system.h and explicitly mark extern functions as such.
2017-02-11ln: replace destination links more atomicallyPaul Eggert
If the file B already exists, commands like 'ln -f A B' and 'cp -fl A B' no longer remove B before creating the new link. Instead, they arrange for the new link to replace B atomically. This should fix a race condition reported by Mike Crowe (Bug#25680). * NEWS, doc/coreutils.texi (cp invocation, ln invocation): Document this. * bootstrap.conf (gnulib_modules): Add symlinkat. * src/copy.c, src/ln.c: Include force-link.h. * src/copy.c (same_file_ok): It's also OK to remove a destination symlink when creating symbolic links, or when the source and destination are on the same file system and when creating hard links. * src/copy.c (create_hard_link, copy_internal): * src/ln.c (do_link): Rewrite using force_linkat and force_symlinkat, to close a window where the destination temporarily does not exist. * src/cp.c (main): Do not set x.unlink_dest_before_opening merely because we are in link-creation mode. * src/force-link.c, src/force-link.h: New files. * src/local.mk (copy_sources, src_ln_SOURCES): Add them. * tests/cp/same-file.sh: Adjust test case to match fixed behavior.
2017-02-09timeout: fix race possibly terminating wrong processTobias Stoeckmann
The race is unlikely, as timeout(1) needs to receive a signal in the few operations between waitpid() returning and exit(). Also the system needs to have reallocated the just released pid in this time window. Previously we never disabled the signal handler that sent the termination signal to the "child" pid. However once waitpid() has reaped the child, the system is free to allocate that pid, so we must ensure we don't process any further signals. * build-aux/gen-lists-of-programs.sh: Build timeout(1) optionally... * configure.ac: ...predicated on sigsuspend() being available. * src/timeout.c (block_cleanup): A new function to ensure the cleanup() handler is disabled after waitpid has returned. (main): Use sigsuspend() to wait with cleanup() enabled but disabled once it returns, and thus disabled for the waitpid() call. (monitored_pid): Change to the more accurate pid_t. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/25624
2017-02-08tail: fix output of redundant headers when resumingJanne Snabb
* src/tail.c (check_fspec): Only enable printing of the file header if we've actually read some data and this is a new file. Also move printing of the file header to... (dump_remainder): ...here, to allow printing only when data read. * tests/tail-2/overlay-headers.sh: A new test for suspension and resumption of tail. * tests/local.mk: Reference the new test. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/23539
2017-02-08tail: fix erroneous status about 'giving up' on filePádraig Brady
* src/tail.c (recheck): Set f->ignore before we use it to show the appropriate error. * tests/tail-2/retry.sh: Ensure the "giving up" message is not presented.
2017-01-25build: fix issue with HAVE_FALLOCATE on centos5Pádraig Brady
* src/copy.c (punch_hole): Work around an empty definition of HAVE_FALLOCATE which leads to a build error of: "error: #if with no expression" That was triggered by the inclusion of <linux/fs.h> in commit v8.25-68-g89e1fef with kernel-headers-2.6.18. Reported by Nelson H. F. Beebe
2017-01-25build: Properly expand cu_install_program when cross-compilingManolis Ragkousis
* src/local.mk (cu_install_program): Replace @INSTALL_PROGRAM@ with @INSTALL@ when cross-compiling; missed in commit 477a1e8e. Message-Id: <20170125163329.5690-1-manolis837@gmail.com> Copyright-paperwork-exempt: Yes
2017-01-21dircolors: highlight windows archive formatMike Swanson
* src/dircolors.hin: Match *.{wim,swm,dwn,esd}
2017-01-21maint: appease syntax-check failures due to recent updatesAssaf Gordon
* bootstrap: s/time stamp/timestamp/. * old/fileutils/NEWS: Likewise. * src/tail.c: Avoided a long line.
2017-01-20date: fix TZ= regressionPaul Eggert
Problem reported by Paul Wise for Debian, in: https://bugs.debian.org/851934 This is fallout from the fix for GNU Bug#23035. * src/date.c (batch_convert): New args TZ and TZSTRING. All uses changed. (batch_convert, main): Adjust to parse_datetime2 API change. (main): Allocate time zone object. * tests/misc/date-debug.sh: Fix incorrect test case, caught by the fix. * tests/misc/date.pl: Test the fix.
2017-01-14date: new option spelling --rfc-emailPaul Eggert
* NEWS: * doc/coreutils.texi (Time conversion specifiers) (Options for date, Examples of date): Document this. * src/date.c (rfc_email_format): Rename from rfc_2822_format. All uses changed. (usage, long_options): Support --rfc-email.
2017-01-14maint: modernize URLsPaul Eggert
A lot of this is converting http: to https:. Also, gmane went away, so remove URLs that no longer work and are not easy to figure out what they were. Some of this stuff is so old that it no longer matters anyway.
2017-01-09maint: standardize on "timestamp" as per POSIXPaul Eggert
2017-01-09stty: ensure no side effects from invalid optionsPádraig Brady
* src/stty.c (apply_settings): A new function refactored from main() that is used to both check and apply options. (main): Call apply_settings before we open the device, so all validation is done before interacting with a device. * NEWS: Mention the improvement. * tests/misc/stty.sh: Add a test case.
2017-01-01maint: update all copyright year number rangesPádraig Brady
Run "make update-copyright" and then... * gnulib: Update to latest with copyright year adjusted. * tests/init.sh: Sync with gnulib to pick up copyright year. * bootstrap: Likewise. * tests/sample-test: Adjust to use the single most recent year.
2016-12-26wc: with only --bytes, determine size more efficientlyPádraig Brady
* src/wc.c (wc): Avoid reading the end of the file when the size is not a multiple of PAGE_SIZE, as the special case handling for files in /proc and /sys is only required when st_size is 0 or a multiple of PAGE_SIZE. * tests/misc/wc-proc.sh: Add a test case.
2016-12-19wc: fix wrong byte counts when using --files-from0William R. Fraser
* src/wc.c (main): Reset fstatus[0].failed between files when reusing the fstatus[0] entry in --files-from0 mode. This ensures a stat() is done for each file, avoiding incorrect counts and redundant reading. * NEWS: Mention the bug fix. * tests/misc/wc-files0.sh: Add a test case. Fixes http://bugs.gnu.org/23073
2016-12-08factor: retry properly if Pollard rho gives a trivial factorizationTorbjörn Granlund
* src/factor.c (factor_using_pollard_rho): Handle trivial factor g = n. (factor_using_pollard_rho2): Handle trivial factor g1 = n1, g0 = n0. * tests/misc/factor.pl: Add a test case. Fixes http://bugs.gnu.org/25135
2016-12-08factor: fix infinite loop in gcd2_oddNiels Möller
* src/factor.c (gcd2_odd): Fix the case a1 == 0, a0 == 0. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/25135
2016-12-02doc: fix --help for: od -t f[SIZE]Pádraig Brady
* src/od.c (usage): SIZE is that of float, not integer.
2016-11-28head: fix processing of non-seekable input as seekablePádraig Brady
* src/head.c (elide_tail_bytes_file): Ensure we don't use st_size unless we've previously used seek() to determine the CURRENT_POS in the seekable file. This was seen to cause issue on FreeBSD 11 when the pipe buffer was filled with `yes | head --lines=-0`, in which case st_size was 64KiB while ST_BLKSIZE() was 4KiB. Reported by Assaf Gordon.
2016-11-28install,mkdir: fix handling of -DZ and -pZ, respectivelyKamil Dudka
... in the case where two or more directories nested in each other are created and each of them defaults to a different SELinux context. * src/install.c (make_ancestor): When calling defaultcon(), give it the same path that is given to mkdir(). The other path is not always valid wrt. current working directory. * src/mkdir.c (make_ancestor): Likewise. * NEWS: Mention the bug fix. Reported at https://bugzilla.redhat.com/1398913
2016-11-28tac: fix mem corruption when failing to read non seekable inputsPádraig Brady
This was detected with ASAN, but can also be seen without ASAN with: $ tac - - <&- tac: standard input: read error: Bad file descriptor *** Error in `tac': malloc(): memory corruption: 0x... * src/tac.c (copy_to_temp): Don't close our output stream on (possibly transient) output error, or on input error. (temp_stream): clearerr() on the stream about to be reused, to ensure future stream use is not impacted by transient errors. * tests/misc/tac-2-nonseekable.sh: Add a test case. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/25041
2016-11-27tail: fix uninitialized memory read when failing to read filePádraig Brady
Reproduced under UBSAN with `tail -f <&-` giving: tail.c:2220:18: runtime error: load of value 190, which is not a valid value for type ‘_Bool' * src/tail.c (tail_file): Ensure f->ignore is initialized in all cases where we can't tail the specified file. * tests/tail-2/follow-stdin.sh: Add a test case which checks stderr has no UBSAN warnings. Fixes http://bugs.gnu.org/25041
2016-11-27stat,tail: sync with latest Linux file systemsPádraig Brady
Update with the results from: kgit='https://git.kernel.org/cgit/linux/kernel/git' wget -q $kgit/torvalds/linux.git/plain/include/uapi/linux/magic.h \ -O src/fs-latest-magic.h make src/fs-magic-compare * src/stat.c (human_fstype): Add entries for: BALLOON_KVM, CGROUP2, DAXFS, ZSMALLOC.
2016-11-27build: fix potential factor build failure on arm and riscPádraig Brady
* src/longlong.h: Sync from gmp repo incorporating: Protect umul_ppmm with do ... while (0) Replace obsolete ARC asm 'J' constraints with 'Cal' Provide umul_ppmm for riscv64
2016-11-27shred,sort: ensure faster unaligned access to rand modulePádraig Brady
glibc has changed the public define from _STRING_ARCH_unaligned to _STRING_INLINE_unaligned as per https://sourceware.org/bugzilla/show_bug.cgi?id=19462 * gl/lib/rand-isaac.c: Cater for both defines. * gl/lib/randread.c: Likewise. * src/system.h: Update commented out code.
2016-11-25numfmt: pacify Sun C 5.14Paul Eggert
* src/numfmt.c (main): Don't implicitly coerce pointer in bool initializer. Although it's portable C99 code, it's confusing.
2016-11-25pr: fix read from invalid memory with tabs in separatorPádraig Brady
This was detected with: echo a > a; pr "-S$(printf "\t\t\t")" a -m a > /dev/null Resulting in ASAN triggering: ==================================================== ERROR: AddressSanitizer: global-buffer-overflow READ of size 1 at 0x00000041b622 thread T0 #0 0x40506a in print_sep_string ../src/pr.c:2241 #1 0x407ec4 in read_line ../src/pr.c:2493 #2 0x40985c in print_page ../src/pr.c:1802 #3 0x40985c in print_files ../src/pr.c:1618 #4 0x4036e0 in main ../src/pr.c:1136 * src/pr.c (init_parameters): Ensure we only override the specified separator when it's a single tab, thus matching the calculated separator length. * tests/pr/pr-tests.pl: Add a test case. * NEWS: Mention the fix.
2016-11-24ptx: fix an invalid heap reference with short --widthPádraig Brady
* src/ptx.c (fix_output_parameters): Ensure line_width doesn't go negative, which can happen when the --width is less than the --gap-size. * tests/misc/ptx-overrun.sh: Add a test case that triggers with ASAN. (Note the longer filename is needed to trigger). Fixes http://bugs.gnu.org/25011
2016-11-24doc: clarify that readlink's --quiet option is on by defaultBernhard Voelker
* src/readlink.c (usage): Mark the --quiet/--silent option as active by default. * doc/coreutils.texi (readlink invocation): Likewise.
2016-11-24split: fix memory corruption during chunk extractionPádraig Brady
ASAN reported this error for: split -n2/3 /dev/null ERROR: AddressSanitizer: negative-size-param: (size=-1) #0 0x7f0d4c36951d in __asan_memmove (/lib64/libasan.so.2+0x8d51d) #1 0x404e06 in memmove /usr/include/bits/string3.h:59 #2 0x404e06 in bytes_chunk_extract src/split.c:988 #3 0x404e06 in main src/split.c:1626 Specifically there would be invalid memory access and subsequent processing if the chunk to be extracted was beyond the initial amount read from file (which is currently capped at 128KiB). This issue is not in a released version, only being introduced in commit v8.25-4-g62e7af0 * src/split.c (bytes_chunk_extract): The initial_read != SIZE_MAX should have been combined with && rather than ||, but also this condition is always true in this function so remove entirely. * tests/split/b-chunk.sh: Add a test case. Fixes http://bugs.gnu.org/25003
2016-11-24tail: fix checking of remoteness when not using inotifyPádraig Brady
In recent commit v8.25-93-g7fc7206 we used the f->remote flag which wasn't set in all cases. This was detected with ASAN giving this error when reading f->remote; runtime error: load of value 190, which is not a valid value for type '_Bool' * src/tail.c (fremote): Query the system even without inotify. (recheck): Always set f->fremote for valid files.
2016-11-23build: avoid racy failure of "make -jN install"Jim Meyering
Installing with -j2 or greater could result in a failure like /bin/sh: line 29: /P/bin/install: Permission denied when /P/bin/install specifies your PATH-selected install program. This would arise because we're using "install" to install all man/*.1 files, and that command would run concurrently with the one that installs "/P/bin/install" itself. We would run this command: "src/ginstall src/ginstall /P/bin/install", and it would result in intervals during which the destination file does not exist, is empty or incomplete and not executable. We addressed this problem long ago for installation of actual binaries by telling the installation rules to use our just-built bin/ginstall (only when not cross-compiling) rather than the PATH-resolved "install" program. This change is to do the same for those .1 files. * src/local.mk (INSTALL): Override automake's default of something like "INSTALL = /P/bin/install -c". (INSTALL_PROGRAM): Now that we set INSTALL, there is no longer any need to set this derived variable. Its default definition, "INSTALL_PROGRAM = ${INSTALL}" does what we require. Improved by Eric Blake.
2016-11-23pr: fix integer overflow in buffer size calcsPaul Eggert
Problem reported by Marcel Böhme (Bug#24996). * configure.ac (WERROR_CFLAGS): Avoid -Wtype-limits. * src/pr.c (col_sep_string): Now a const pointer. All uses changed. (integer_overflow): New function. (separator_string, main, init_parameters, init_store_cols): Check for integer overflow. (align_column, read_line, print_stored): Avoid integer overflow.
2016-11-22comm: add --total optionBernhard Voelker
* src/comm.c (total_option): Add bool variable for the new option. (TOTAL_OPTION): Add enum value. (long_options): Add array element for the new option. (usage): Document the new option here. (compare_files): Count the lines in total[3], and output the summary at the end. (main): Accept the new option. * doc/coreutils.texi (comm invocation): Document it. * tests/misc/comm.pl: Test it. While at it, improve the test data to have 1 unique line in the first file, 2 unique lines in the second file, and 3 common lines. * NEWS (New Features): Mention the new option. Fixes http://bugs.gnu.org/24929
2016-11-22ls: improve alignment of quoted namesPádraig Brady
This provides better alignment when some names are quoted, which also provides better indication that quotes are not part of the name. * src/ls.c (align_variable_outer_quotes): A new variable set when ls is aligning columns (not using -m, non-zero -w), and has a variable quoting style (shell, shell-escape, c-maybe). (quote_name_buf): Writes to buffer rather than FILE, taking care to avoid data copying if possible. Refactored from... (quote_name): ...here. This now manages the buffer passed to quote_name_buf() and outputs the padding, colors and name in the appropriate order, while managing the --dired offsets. (get_color_indicator): A new function to return the color sequence, refactored from... (print_color_indicator): ...here. This now simply outputs. (print_dir): Refactor common parts to quote_name(). (clear_files): Reset the flag indicating at least one file is quoted in the current directory. (needs_quoting): A new function to indicate at the scan stage whether a name needs quoting. Called from... (gobble_file): ...here, until we find the first quoted file. (print_name_with_quoting): Mostly refactored to quote_name(). * tests/ls/quote-align.sh: A new test for various output formats. * tests/local.mk: Reference the new test. * NEWS: Mention the improvement.
2016-11-22b2sum: a new checksum utility with md5sum like interfacePádraig Brady
Note we don't support the --algorithm option of the b2sum command in the external BLAKE2 project, as that was deemed too confusing for users. "BLAKE2b" was chosen as the default algorithm to use, which is single threaded but performs well on 64 bit. * src/blake2: CC0 source copied from external project. * cfg.mk[VC_LIST_ALWAYS_EXCLUDE_REGEX]: Exclude blake2/ from syntax checks, make update-copyright, etc. * src/local.mk: Reference the sources for b2sum, and set the compilation flags. * doc/coreutils.texi (b2sum invocation): Reference the md5sum invocation node, and add descriptions of -l. * tests/misc/b2sum.sh: Add new test. * tests/local.mk: Reference new test. * AUTHORS: Add new binary. * README: Likewise. * build-aux/gen-lists-of-programs.sh: Likewise. * man/.gitignore: Likewise. * scripts/git-hooks/commit-msg: Likewise. * man/b2sum.x: New man page template. * man/local.mk: Reference new template. * src/.gitignore: Ignore new binaries. * src/blake2/.gitignore: Ignore new build atrifacts. * src/md5sum.c (usage): Describe the new -l option. * NEWS: Mention the new program.
2016-11-19cp: improve status message when omitting directoriesPádraig Brady
* src/copy.h (cp_options): Add a new flag for install(1). * src/copy.c (copy_internal): For cp, also output remediation advice which also indicates why directories aren't copied by default. The message is unchanged for install(1). * src/cp.c (cp_option_init): Init install_mode to false. * src/mv.c (cp_option_init): Likewise. * src/install.c (cp_option_init): Init install_mode to true. * tests/install/basic-1.sh: Add a test case. * tests/cp/link-deref.sh: Adjust test case. Fixes http://bugs.gnu.org/24958
2016-11-15dircolors: highlight Motion JPEG multimedia filesAntonio Ospite
* src/dircolors.hin: Add .mjpg and .mjpeg multimedia files. This resolves http://bugs.gnu.org/24436.
2016-11-14all: reduce usage of proper_name_utf8()Pádraig Brady
This reduces a standard coreutils install size by about 160K. * src/cat.c: Change to proper_name() which removes about 18K text. * src/cp.c: Likewise. * src/df.c: Likewise. * src/du.c: Likewise. * src/getlimits.c: Likewise. * src/realpath.c: Likewise. * src/split.c: Likewise. * src/stdbuf.c: Likewise. * src/timeout.c: Likewise. * src/truncate.c: Likewise. * src/local.mk: Remove -llibiconv from the above programs. * cfg.mk (sc_check-AUTHORS): Adjust to use factor(1). * AUTHORS: Adjust to use ASCII to satisfy sc_check-AUTHORS.
2016-11-10tail: only retry file open if --retry specifedPádraig Brady
* src/tail.c (tail_file): On failure to open a file, set ignore=true when --retry is not specified. * tests/tail-2/assert-2.sh: Adjust to the new behavior. * tests/tail-2/retry.sh: Add a test case. Also change from `tail ... && fail=1` to the more robust `returns_ 1 ...` construct which detects segfaults etc. * NEWS: Document the fix.
2016-11-09tail: ensure -f --retry 'missing' handles truncationPádraig Brady
* src/tail.c (tail_forever): The BLOCKING optimization is only enabled for non regular files (which can't be truncated), so ensure we don't enable that unless we've a valid file descriptor. * tests/tail-2/retry.sh: Add a test case. * NEWS: Mention the bug fix.
2016-11-09tail: avoid outputting repeated data with remote filesPádraig Brady
* src/tail.c (tail_forever): Only read up to st_size on network file systems to avoid the issue with a stale attribute cache returning a smaller st_size than we have already read(). The was seen with glusterfs at least and caused the complete file to be repeatedly output due to assuming the file was truncated in this case. * NEWS: Mention the fix.
2016-11-08tail: terminate when following pipes and untailable non pipesPádraig Brady
* src/tail.c (ignore_pipe_or_fifo): Mark the descriptor as -1 for pipes so that any_live_files() detects correctly that the entry is no longer live. * tests/tail-2/pipe-f.sh: Add a test case. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/24903 which was detected using Symbolic Execution techniques developed in the course of the SYMBIOSYS research project at COMSYS, RWTH Aachen University.
2016-11-07maint: refactor printing of backup suffix --helpPádraig Brady
* src/system.h (emit_backup_suffix_note): A new function to output the backup suffix info. The strings are unchanged, so translations are not impacted. * src/cp.c (usage): Use the new function. * src/ln.c (usage): Likewise. * src/mv.c (usage): Likewise. * src/install.c (usage): Likewise.
2016-11-07maint: simplify handling of backup --suffix in various toolsRishabh Dave
* src/cp.c (main): Avoid the getenv("SIMPLE_BACKUP_SUFFIX") call, which is now done if needed in the gnulib backupfile module. Also avoid the redundant strdup, as we don't modify this suffix. * src/install.c (main): Likewise. * src/ln.c (main): Likewise. * src/mv.c (main): Likewise. Fixes http://bugs.gnu.org/23153
2016-11-07date: add %q to output the quarter of the yearPádraig Brady
* doc/coreutils.texi (date invocation): Document %q. * src/date.c (usage): Likewise. * tests/misc/date.pl: Add a test case. * cfg.mk (sc_strftime_check): Adjust to allow %q. * NEWS: Mention the new feature.
2016-11-05stat: make --format=%N honor the QUOTING_STYLE env varPádraig Brady
* doc/coreutils.texi (stat invocation): Describe the QUOTING_STYLE values now supported. * src/stat.c (getenv_quoting_style): A new function called from main, that sets the default quoting style for quotearg. (main): Call getenv_quoting_style() when %N specified. * tests/misc/stat-fmt.sh: Add a test case. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/23422