summaryrefslogtreecommitdiff
path: root/tests
diff options
context:
space:
mode:
authorPaul Eggert <eggert@CS.UCLA.EDU>2009-03-13 15:48:30 -0700
committerJim Meyering <meyering@redhat.com>2009-03-18 21:44:37 +0100
commit8f7fae59727e0a050a4833705a6f7849ba2c4531 (patch)
tree7af8e87ff4424402ac12314a1d8f429a0dd23cba /tests
parente6d2d9479495dff8520e577c221d5195eb9bb48b (diff)
downloadcoreutils-8f7fae59727e0a050a4833705a6f7849ba2c4531.tar.xz
sort: handle fd exhaustion better when merging
This is an alternative to my 9 March patch labeled "Silently lower nmerge; don't (sometimes incorrectly) range-check" <http://lists.gnu.org/archive/html/bug-coreutils/2009-03/msg00070.html>. It differs by not using 'dup' to probe for extra file descriptors; instead, it simply calls 'open' (and 'pipe') to open files and pipes, until one of these calls fails due to file descriptor exhaustion; it then backs off by 1, does a merge with the files that it has opened, and then retries with the (now-smaller) number of files. This patch requires quite a few more changes to the source code than the earlier patch, but it is in some sense "better" because it doesn't need to call "dup" ahead of time in order to decide whether "open" or "pipe" will fail. Also, it's more robust in the case where "open" or "pipe" fails with errno==EMFILE because some system-wide limit is exhausted. * src/sort.c (create_temp_file): New arg SURVIVE_FD_EXHAUSTION. (stream_open): New function, containing guts of xfopen. (xfopen): Use it. (pipe_fork): Set errno on failure. (maybe_create_temp): New function, containing guts of create_temp. (create_temp): Use it. (open_temp): Distinguish failures due to file descriptor exhaustion from other failures, and on fd exhaustion return a notice to caller rather than dying. Don't test execlp's return value; when it returns, it *always* returns -1. (open_input_files): New function. (mergefps): New arg FPS. It's now the caller's responsibility to open the input and output files. All callers changed. (mergefiles): New function. (avoid_trashing_input, merge): Handle the case where a single merge can't merge as much as we wanted due to file descriptor exhaustion, by merging as much as we can and then retrying. * tests/Makefile.am (TESTS): Add misc/sort-continue. * tests/misc/sort-continue: New file. * THANKS: Add Glen Lenker and Matt Pham who coauthored this patch.
Diffstat (limited to 'tests')
-rw-r--r--tests/Makefile.am1
-rwxr-xr-xtests/misc/sort-continue47
2 files changed, 48 insertions, 0 deletions
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 10be0c60f..2fb01c461 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -201,6 +201,7 @@ TESTS = \
misc/shuf \
misc/sort \
misc/sort-compress \
+ misc/sort-continue \
misc/sort-files0-from \
misc/sort-merge \
misc/sort-rand \
diff --git a/tests/misc/sort-continue b/tests/misc/sort-continue
new file mode 100755
index 000000000..b1031fac3
--- /dev/null
+++ b/tests/misc/sort-continue
@@ -0,0 +1,47 @@
+#!/bin/bash
+# Tests for file descriptor exhaustion.
+
+# Copyright (C) 2009 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+if test "$VERBOSE" = yes; then
+ set -x
+ sort --version
+fi
+
+. $srcdir/test-lib.sh
+
+for i in $(seq 31); do
+ echo $i | tee -a in > __test.$i || framework_failure
+done
+
+fail=0
+
+(
+ ulimit -n 6
+ exec 0</dev/null 3<&- 4<&- 5<&-
+ sort -n -m __test.* > out
+) &&
+compare in out || { fail=1; echo 'file descriptor exhaustion not handled' 1>&2; }
+
+echo 32 | tee -a in > in1
+(
+ ulimit -n 6
+ exec 3<&- 4<&- 5<&-
+ cat in1 | sort -n -m __test.* - > out
+) &&
+compare in out || { fail=1; echo 'stdin not handled properly' 1>&2; }
+
+Exit $fail