summaryrefslogtreecommitdiff
path: root/src/shuf.c
AgeCommit message (Collapse)Author
2013-12-06shuf: --repeat, not --repetitions; default --head-count is infinityPaul Eggert
Original problem reported by Philipp Thomas in <http://bugs.gnu.org/16061>. * NEWS: shuf --repeat, not shuf --repetitions. * doc/coreutils.texi (shuf invocation): * src/shuf.c (usage, long_opts, main): * tests/misc/shuf.sh: Likewise. Also, the default head-count is infinity.
2013-07-11build: fix a build warning on 32 bit systems in shuf.cPádraig Brady
Prompted by the continuous integration build failure at: http://hydra.nixos.org/build/5508873 * src/shuf.c (write_random_numbers): Convert to an int type that matches the prinft format spec.
2013-07-11shuf: add --repetition to support repetition in outputAssaf Gordon
main(): Process new option. Replace input_numbers_option_used() with a local variable. Re-organize argument processing. usage(): Describe the new option. (write_random_numbers): A new function to generate a permutation of the specified input range with repetition. (write_random_lines): Likewise for stdin and --echo. (write_permuted_numbers): New function refactored from write_permuted_output(). (write_permuted_lines): Likewise. * tests/misc/shuf.sh: Add tests for --repetitions option. * doc/coreutils.texi: Mention --repetitions, add examples. * TODO: Mention an optimization to avoid needing to read all of the input into memory with --repetitions. * NEWS: Mention new shuf option.
2013-04-04shuf: exit without reading if would never outputPádraig Brady
* src/shuf.c (main): If -n0 specified then no data would ever be output, so exit without reading input. * tests/misc/shuf.sh: Augment the related test with this case.
2013-03-25shuf: use reservoir-sampling for large or unknown sized inputsAssaf Gordon
Reservoir sampling optimizes selecting K random lines from large or unknown-sized input: http://en.wikipedia.org/wiki/Reservoir_sampling Note this also avoids reading any input when -n0 is specified. * src/shuf.c (main): Use reservoir-sampling when the number of output lines is known, and the input size is large or unknown. (input_size): A new function to get the input size for regular files. (read_input_reservoir_sampling): New function to read lines from input, keeping only K lines in memory, replacing lines with decreasing prob. (write_permuted_output_reservoir): New function to output reservoir. * tests/misc/shuf-reservoir.sh: An expensive_ test using valgrind to exercise the reservoir-sampling code. * tests/local.mk: Reference new test. * NEWS: Mention the improvement.
2013-01-23maint: define usage note about mandatory args centrallyBernhard Voelker
Each program with at least one long option which is marked as 'required_argument' and which has also a short option for that option, should print a note about mandatory arguments. Define that well-known note centrally and use it rather than literal printf/fputs, and add it where it was missing. * src/system.h (emit_mandatory_arg_note): Add new function. * src/cp.c (usage): Use it rather than literal printf/fputs. * src/csplit.c, src/cut.c, src/date.c, src/df.c, src/du.c: * src/expand.c, src/fmt.c, src/fold.c, src/head.c, src/install.c: * src/kill.c, src/ln.c, src/ls.c, src/mkdir.c, src/mkfifo.c: * src/mknod.c, src/mv.c, src/nl.c, src/od.c, src/paste.c: * src/pr.c, src/ptx.c, src/shred.c, src/shuf.c, src/sort.c: * src/split.c, src/stdbuf.c, src/tac.c, src/tail.c, src/timeout.c: * src/touch.c, src/truncate.c, src/unexpand.c, src/uniq.c: Likewise. * src/base64.c (usage): Add call of the above new function because at least one long option has a required argument. * src/basename.c, src/chcon.c, src/date.c, src/env.c: * src/nice.c, src/runcon.c, src/seq.c, src/stat.c, src/stty.c: Likewise.
2013-01-01maint: update all copyright year number rangesJim Meyering
Run "make update-copyright", but then also run this, perl -pi -e 's/2\d\d\d-//' tests/sample-test to make that one script use the single most recent year number.
2012-01-09maint: src/*.c: change remaining quotes (without embedded spaces)Jim Meyering
Run this (twice): git grep -E -l '`[^ ]+'\' src/*.c \ |xargs perl -pi -e 's/`([^ ]+'\'')/'\''$1/'
2012-01-07maint: use new emit_try_help in place of equivalent fprintfJim Meyering
Run this command: perl -0777 -pi -e \ 's/fprintf \(stderr, _\("Try `%s --help.*\n.*;/emit_try_help ();/m'\ src/*.c
2012-01-01maint: update all copyright year number rangesJim Meyering
Run "make update-copyright".
2011-01-14maint: refactor to use read-file from gnulibPádraig Brady
* bootstrap.conf: Add the read-file module * src/ptx.c: Replace the original code which would needlessly read SIZE_MAX bytes of files larger than this. * src/shuf.c: Replace the original code.
2011-01-01maint: update all copyright year number rangesJim Meyering
Run "make update-copyright".
2010-07-22provide POSIX_FADV_SEQUENTIAL hint to appropriate utilsPádraig Brady
Following on from commit dae35bac, 01-03-2010, "sort: inform the system about our input access pattern" apply the same hint to all appropriate utils. This currently gives around a 5% speedup for reading large files from fast flash devices on GNU/Linux. * src/base64.c: Call fadvise (..., FADVISE_SEQUENTIAL); * src/cat.c: Likewise. * src/cksum.c: Likewise. * src/comm.c: Likewise. * src/cut.c: Likewise. * src/expand.c: Likewise. * src/fmt.c: Likewise. * src/fold.c: Likewise. * src/join.c: Likewise. * src/md5sum.c: Likewise. * src/nl.c: Likewise. * src/paste.c: Likewise. * src/pr.c: Likewise. * src/ptx.c: Likewise. * src/shuf.c: Likewise. * src/sum.c: Likewise. * src/tee.c: Likewise. * src/tr.c: Likewise. * src/tsort.c: Likewise. * src/unexpand.c: Likewise. * src/uniq.c: Likewise. * src/wc.c: Likewise, unless we don't actually read().
2010-05-25maint: don't emit an extra newline in each of two diagnosticsJim Meyering
* src/shuf.c (main): Remove a stray newline in a diagnostic. * src/od.c (main): Likewise. Detected via these: git grep -A1 'error *(.*,$' | grep -C1 '\\n"[,)]' git grep 'error *(.*;$' | grep '\\n"[,)]'
2010-01-01maint: update all FSF copyright year lists to include 2010Jim Meyering
Use this command: git ls-files | grep -v COPYING \ | xargs env UPDATE_COPYRIGHT_USE_INTERVALS=1 \ build-aux/update-copyright
2009-11-07build: consistently use freopen-saferEric Blake
cat, head, ptx, shuf, tac, tail, tee, tr, and uniq used freopen on stdout, and were potentially vulnerable. dircolors, du, and tsort only used it on stdin, which is unaffected by freopen_safer, but this covers all uses for consistency. * cfg.mk (sc_require_stdio_safer): New rule. * gl/modules/xfreopen (Depends-on): Add freopen-safer. * gl/lib/xfreopen.c (includes): Use stdio--.h. * src/ptx.c (includes): Likewise. * src/shuf.c (includes): Likewise. * src/uniq.c (includes): Likewise. * src/dircolors.c (includes): Likewise. * src/du.c (includes): Likewise. * src/tsort.c (includes): Likewise.
2009-09-21doc: mention the texinfo documentation in --helpPádraig Brady
* src/system.h: Rename emit_bug_reporting_address() to emit_ancillary_info() and update it to not print the translation project address in en_* locales, and _do_ print it in the 'C' (and other) locales so that it's included in the default man page. Also mention how to invoke the texinfo documentation for each command. Also move the "hard-locale.h" include to the 8 files that now use it. * man/help2man: Strip the newly added texinfo reference from the --help output as a more verbose version is already added by help2man. Suggestion from C de-Avillez
2009-08-25global: convert indentation-TABs to spacesJim Meyering
Transformed via this shell code: t=$'\t' git ls-files \ | grep -vE '(^|/)((GNU)?[Mm]akefile|ChangeLog)|\.(am|mk)$' \ | grep -vE 'tests/pr/|help2man' \ | xargs grep -lE "^ *$t" \ | xargs perl -MText::Tabs -ni -le \ '$m=/^( *\t[ \t]*)(.*)/; print $m ? expand($1) . $2 : $_'
2009-06-23maint: update all Copyright year lists to include 2009Jim Meyering
2009-04-07shred,sort,shuf: don't use /dev/urandom by defaultPádraig Brady
Suggestion from Steven Schveighoffer at: http://savannah.gnu.org/patch/?6797 to greatly speed up the random passes done by shred. * gl/lib/randread.c: Default to using the internal pseudorandom generator, rather than reading /dev/urandom * src/shred.c (usage): remove mention of /dev/urandom * src/shuf.c (usage); ditto * src/sort.c (usage): ditto * doc/coreutils.text: Document the new behaviour for aquiring random data.
2008-07-27shuf: correctly document --head-countMikael Magnusson
* src/shuf.c (usage): Correct --help output. * doc/coreutils.texi: Correct option listing. * NEWS: Mention correction.
2008-07-27shuf: honor --zero-terminated option even with --input-range=LO-HIJim Meyering
* src/shuf.c (write_permuted_output): Add EOLBYTE parameter and use it rather than hard-coding "\n". (main): Adjust sole caller. * tests/misc/shuf: Add a test to exercise this bug fix. * NEWS: Mention it.
2008-06-16remove redundant const directivesJim Meyering
In 1463824d8e7f72c31f1d803d7cfe2b608ccafc5c, I added some missing "const" directives, as well as some new, redundant ones. This removes the redundant ones. Pointed out by Eric Blake. * base64.c, cat.c, chcon.c, chgrp.c, chmod.c, chown.c, comm.c: * cp.c, csplit.c, cut.c, date.c, dd.c, df.c, dircolors.c, du.c: * env.c, expand.c, fmt.c, fold.c, groups.c, head.c, id.c: * install.c, join.c, kill.c, ln.c, ls.c, md5sum.c, mkdir.c: * mkfifo.c, mknod.c, mktemp.c, mv.c, nice.c, nl.c, od.c: * paste.c, pathchk.c, pinky.c, pr.c, ptx.c, readlink.c, rm.c: * rmdir.c, runcon.c, seq.c, shred.c, shuf.c, sort.c, split.c: * stat.c, stty.c, su.c, sum.c, tac.c, tail.c, tee.c, timeout.c: * touch.c, tr.c, truncate.c, tty.c, uname.c, unexpand.c, uniq.c: * wc.c, who.c: Remove redundant const directives. * maint.mk (sc_const_long_option): Don't require redundant "const".
2008-06-14add "const" attribute, where possibleJim Meyering
* maint.mk (sc_const_long_option): New rule. Enforce global change. * src/base64.c (long_options): Use "const" where possible. * src/cat.c (main): Likewise. * src/chcon.c (long_options): Likewise. * src/chgrp.c (long_options): Likewise. * src/chmod.c (long_options): Likewise. * src/chown.c (long_options): Likewise. * src/comm.c (long_options, OUTPUT_DELIMITER_OPTION): Likewise. * src/cp.c (long_opts): Likewise. * src/csplit.c (longopts): Likewise. * src/cut.c (longopts): Likewise. * src/date.c (long_options): Likewise. * src/dd.c (conversions, flags, statuses): Likewise. * src/df.c (long_options): Likewise. * src/dircolors.c (long_options): Likewise. * src/du.c (long_options): Likewise. * src/env.c (longopts): Likewise. * src/expand.c (longopts): Likewise. * src/fmt.c (long_options): Likewise. * src/fold.c (longopts): Likewise. * src/groups.c (longopts): Likewise. * src/head.c (long_options): Likewise. * src/id.c (longopts): Likewise. * src/install.c (long_options): Likewise. * src/join.c (longopts): Likewise. * src/kill.c (long_options): Likewise. * src/ln.c (long_options): Likewise. * src/ls.c (long_time_format, long_options, sort_functions): Likewise. * src/md5sum.c (long_options): Likewise. * src/mkdir.c (longopts): Likewise. * src/mkfifo.c (longopts): Likewise. * src/mknod.c (longopts): Likewise. * src/mktemp.c (longopts): Likewise. * src/mv.c (long_options): Likewise. * src/nice.c (longopts): Likewise. * src/nl.c (longopts): Likewise. * src/od.c (charname, long_options): Likewise. * src/paste.c (longopts): Likewise. * src/pathchk.c (longopts): Likewise. * src/pinky.c (longopts): Likewise. * src/pr.c (long_options): Likewise. * src/ptx.c (long_options): Likewise. * src/readlink.c (longopts): Likewise. * src/rm.c (long_opts): Likewise. * src/rmdir.c (longopts): Likewise. * src/runcon.c (long_options): Likewise. * src/seq.c (long_options): Likewise. * src/shred.c (long_opts): Likewise. * src/shuf.c (long_opts): Likewise. * src/sort.c (monthtab, long_options): Likewise. * src/split.c (longopts): Likewise. * src/stat.c (long_options): Likewise. * src/stty.c (mode_info, control_info, longopts, set_mode) Likewise. (set_control_char, speeds): Likewise. * src/su.c (longopts): Likewise. * src/sum.c (longopts): Likewise. * src/tac.c (longopts): Likewise. * src/tail.c (long_options): Likewise. * src/tee.c (long_options): Likewise. * src/timeout.c (long_options): Likewise. * src/touch.c (longopts): Likewise. * src/tr.c (long_options): Likewise. * src/truncate.c (longopts): Likewise. * src/tty.c (longopts): Likewise. * src/uname.c (uname_long_options, arch_long_options): Likewise. * src/unexpand.c (longopts): Likewise. * src/uniq.c (longopts): Likewise. * src/wc.c (longopts): Likewise. * src/who.c (longopts): Likewise.
2008-06-03use gnulib's progname moduleJim Meyering
* bootstrap.conf (gnulib_modules): Add progname. * src/*.c (program_name): Remove declaration. * (main): Call set_program_name rather than setting program_name. * src/nice.c (main): Cast program_name to "(char *)". * src/prog-fprintf.c: Include "system.h" * src/system.h: Include "progname.h". * maint.mk (sc_program_name): Adjust rule. Suggestion from Eric Blake.
2008-06-02declare program_name consistentlyJim Meyering
* src/base64.c: Likewise. * src/basename.c: Likewise. * src/cat.c: Likewise. * src/chcon.c: Likewise. * src/chgrp.c: Likewise. * src/chmod.c: Likewise. * src/chown.c: Likewise. * src/chroot.c: Likewise. * src/cksum.c: Likewise. * src/comm.c: Likewise. * src/cp.c: Likewise. * src/csplit.c: Likewise. * src/cut.c: Likewise. * src/date.c: Likewise. * src/dd.c: Likewise. * src/df.c: Likewise. * src/dircolors.c: Likewise. * src/dirname.c: Likewise. * src/du.c: Likewise. * src/echo.c: Likewise. * src/env.c: Likewise. * src/expand.c: Likewise. * src/expr.c: Likewise. * src/factor.c: Likewise. * src/fmt.c: Likewise. * src/fold.c: Likewise. * src/groups.c: Likewise. * src/head.c: Likewise. * src/hostid.c: Likewise. * src/hostname.c: Likewise. * src/id.c: Likewise. * src/install.c: Likewise. * src/join.c: Likewise. * src/kill.c: Likewise. * src/link.c: Likewise. * src/ln.c: Likewise. * src/logname.c: Likewise. * src/ls.c: Likewise. * src/md5sum.c: Likewise. * src/mkdir.c: Likewise. * src/mkfifo.c: Likewise. * src/mknod.c: Likewise. * src/mktemp.c: Likewise. * src/mv.c: Likewise. * src/nice.c: Likewise. * src/nl.c: Likewise. * src/nohup.c: Likewise. * src/od.c: Likewise. * src/paste.c: Likewise. * src/pathchk.c: Likewise. * src/pinky.c: Likewise. * src/pr.c: Likewise. * src/printenv.c: Likewise. * src/printf.c: Likewise. * src/ptx.c: Likewise. * src/pwd.c: Likewise. * src/readlink.c: Likewise. * src/rm.c: Likewise. * src/rmdir.c: Likewise. * src/runcon.c: Likewise. * src/seq.c: Likewise. * src/setuidgid.c: Likewise. * src/shuf.c: Likewise. * src/sleep.c: Likewise. * src/sort.c: Likewise. * src/split.c: Likewise. * src/stat.c: Likewise. * src/stty.c: Likewise. * src/su.c: Likewise. * src/sum.c: Likewise. * src/sync.c: Likewise. * src/tac.c: Likewise. * src/tail.c: Likewise. * src/tee.c: Likewise. * src/test.c: Likewise. * src/timeout.c: Likewise. * src/touch.c: Likewise. * src/tr.c: Likewise. * src/true.c: Likewise. * src/tsort.c: Likewise. * src/tty.c: Likewise. * src/uname.c: Likewise. * src/unexpand.c: Likewise. * src/uniq.c: Likewise. * src/unlink.c: Likewise. * src/uptime.c: Likewise. * src/users.c: Likewise. * src/wc.c: Likewise. * src/who.c: Likewise. * src/whoami.c: Likewise. * src/yes.c: Likewise.
2008-05-26adjust copyright datesJim Meyering
2008-05-26convert single-author programs to use proper_nameJim Meyering
g grep -E -l 'define AUTHORS "[^,]+"$'|xargs perl -pi -e \ 's/(define AUTHORS) ("[^,]+")$/$1 proper_name ($2)/'
2007-07-23Update all copyright notices to use the newer form.Jim Meyering
2007-07-10Change "version 2" to "version 3" in all copyright notices.Jim Meyering
2007-03-28Help translators include translation team's web or email address.Jim Meyering
* src/system.h (emit_bug_reporting_address): New function. * src/base64.c: Use it rather than a literal printf. * src/basename.c, src/cat.c, src/chgrp.c, src/chmod.c: * src/chown.c, src/chroot.c, src/cksum.c, src/comm.c, src/cp.c: * src/csplit.c, src/cut.c, src/date.c, src/dd.c, src/df.c: * src/dircolors.c, src/dirname.c, src/du.c, src/echo.c, src/env.c: * src/expand.c, src/expr.c, src/factor.c, src/fmt.c, src/fold.c: * src/head.c, src/hostid.c, src/hostname.c, src/id.c, src/install.c: * src/join.c, src/kill.c, src/link.c, src/ln.c, src/logname.c: * src/ls.c, src/md5sum.c, src/mkdir.c, src/mkfifo.c, src/mknod.c: * src/mv.c, src/nice.c, src/nl.c, src/nohup.c, src/od.c: * src/paste.c, src/pathchk.c, src/pinky.c, src/pr.c, src/printenv.c: * src/printf.c, src/ptx.c, src/pwd.c, src/readlink.c, src/rm.c: * src/rmdir.c, src/seq.c, src/setuidgid.c, src/shred.c, src/shuf.c: * src/sleep.c, src/sort.c, src/split.c, src/stat.c, src/stty.c: * src/su.c, src/sum.c, src/sync.c, src/system.h, src/tac.c: * src/tail.c, src/tee.c, src/test.c, src/touch.c, src/tr.c: * src/true.c, src/tsort.c, src/tty.c, src/uname.c, src/unexpand.c: * src/uniq.c, src/unlink.c, src/uptime.c, src/users.c, src/wc.c: * src/who.c, src/whoami.c, src/yes.c: Likewise.
2007-02-23With -Dlint, make shuf free all heap-allocated storage.Jim Meyering
* src/shuf.c (main): Move declaration of input_lines to function scope, and initialize to NULL, so we can free it. [lint]: Free all malloc'd memory. * tests/misc/shuf: Also test shuf's -e and -i options.
2006-09-29* src/shuf.c (read_input): Fix an off-by-one error thatJim Meyering
would cause an infloop for piped input of 8KB or more. * NEWS: Mention the fix. * tests/misc/shuf: Test for the above fix.
2006-09-18* src/shuf.c (main): Quote the entire range when reporting anPaul Eggert
invalid one, rather than just the part that contained the error.
2006-08-09(next_line): New function.Paul Eggert
(read_input): Use it, to avoid relying on GCC-specific behavior with void * arithmetic.
2006-08-08New file, introduced for shuf, sort -R, and/or shred.Paul Eggert
generated by cgit v1.2.3-70-g09d2 (git 2.47.1) at 2024-12-29 06:55:05 +0000