#volume-3
TAOCP 5.4.1 Exercise 22
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 22. [ HM40 ] The purpose of this exercise is to determine the average run length obtained in natural selection, when $P' > P$. Let $\kappa = k + \theta$ be a real number $\ge 1$, where $k = \lfloor \kappa \rfloor$ and $\theta = \kappa \bmod 1$, and consider the function $F(\kappa) = F_k(\theta)$, where $F_k(\theta)$ is the polynomial defined by...
TAOCP 5.1.4 Exercise 42
Section 5.1.4: Tableaux and Involutions Exercise 42. ▶ [ 30 ] (Disorder in a genome.) The DNA of Lobelia fervens has genes occurring in the sequence $g_1^B g_1 g_2 g_3 g_4 g_5 g_6^B$, where $g^B$ stands for the left-right reflection of $g$; the same genes occur in tobacco plants, but in the order $g_1 g_2 g_3 g_4 g_5 g_6 g_7$. Show that five "flip" operations on substrings are needed to...
TAOCP 6.5 Exercise 18
Section 6.5: Retrieval on Secondary Keys Exercise 18. ▶ [ M35 ] (Associative block designs.) A set of $m$-tuples like (13), with exactly $m-n$ *’s in each of $2^n$ rows, is called an ABD$(m,n)$ if every column contains the same number of *’s and if every pair of rows has a “mismatch” (0 versus 1) in some column. Every $m$-bit binary number will then match exactly one row. For example,...
TAOCP 5.4.9 Exercise 29
Section 5.4.9: Disks and Drums Exercise 29. ▶ [ M30 ] The purpose of this exercise is to derive an upper bound on the average time needed to input any sequence of blocks in chronological order by the randomized striping procedure, when the blocks represent $P$ runs and $D$ disks. We say that the block being waited for at each time step as the algorithm proceeds (see (24)) is “marked”;...
TAOCP 5.4.4 Exercise 10
Section 5.4.4: Reading Tape Backwards Exercise 10. [ 41 ] Prepare a table of the exact values of $K_T(n)$, using a computer. Verified: no Solve time: 29m26s Setup In the theory of optimal merge patterns developed by Karp in this section, (K_{T}(n)) is the maximum number of initial runs that can be sorted in (n) merge phases using (T) tapes when reading backwards. The lower bound proven in exercise 9...
TAOCP 5.4.8 Exercise 9
Section 5.4.8: Two-Tape Sorting Exercise 9. ▶ [ M25 ] Find a significant connection between the “cocktail-shaker sort” described in Section 5.2.2, Fig. 16, and the numbers $u_1,u_2,\ldots,u_n$ of (4) in the case $b=1$. Verified: no Solve time: 8m35s Solution In the elevator problem of Section 5.4.8 (Fig. 88), equation (4) defines for (b=1) (capacity one) the numbers (u_k = #{\text{passengers on floors } 1,\dots,k \text{ destined for floors }...
TAOCP 5.2.2 Exercise 38
Section 5.2.2: Sorting by Exchanging Exercise 38. ▶ [ M30 ] Find $A_N$, $B_N$, $C_N$, $G_N$, $K_N$, $L_N$, $R_N$, and $X_N$, the average values of the quantities in (29), when radix exchange is applied to "case (ii) input." Express your answers in terms of $N$ and the quantities $$U_n = \sum_{k \ge 2} \binom{n}{k} \frac{(-1)^k}{2^{k-1} - 1}, \qquad V_n = \sum_{k \ge 2} \binom{n}{k} \frac{(-1)^k k}{2^{k-1} - 1} = n(U_n...
TAOCP 5.3.1 Exercise 28
Section 5.3.1: Minimum-Comparison Sorting Exercise 28. [ 40 ] Write a MIX program that sorts five one-word keys in the minimum possible amount of time, and halts. (See the beginning of Section 5.2 for ground rules.) Verified: no Solve time: 11m31s Setup The exercise asks for a MIX program that sorts five one‑word keys in as few machine cycles as possible. The ground rules of Section 5.2 are followed: the...
TAOCP 5.3.1 Exercise 17
Section 5.3.1: Minimum-Comparison Sorting Exercise 17. [ M20 ] Prove (29). Verified: no Solve time: 13m06s Solution Let (F(n)) be the number of comparisons used by the merge‑insertion algorithm to sort (n) distinct keys. The algorithm gives the recurrence (see the analysis in the text) [ \begin{aligned} F(1)&=0,\qquad F(2)=1,\[2mm] \text{for }n\ge 3,\qquad F(n)&=\bigl\lfloor\frac n2\bigr\rfloor;+;F!\left(\bigl\lfloor\frac n2\bigr\rfloor\right);+;\sum_{i=2}^{\lceil n/2\rceil}c_i , \end{aligned} \tag{1} ] where [ c_i = \bigl\lceil\lg\bigl(3(2i-1)/4\bigr)\bigr\rceil \qquad(i\ge 2) ] is the...
TAOCP 5.2.3 Exercise 26
Section 5.2.3: Sorting by Selection Exercise 26. [ M30 ] Solve exercise 25, (a) for $N = 26$, (b) for general $N$. Verified: no Solve time: 33m28s Setup The siftup algorithm for a node at index (l) when the current heap size is (r) is [ \begin{aligned} i &\gets l,; j \gets 2i;\ \text{while } j &\le r:\ &\quad\text{if } j<r \text{ and } K_j < K_{j+1}\text{ then } j...
TAOCP 5.2.1 Exercise 36
Section 5.2.1: Sorting by Insertion Exercise 36. [ 18 ] Assume that the byte size of MIX is 100, and that the sixteen example keys in Table 8 are actually 560000, 087000, 512000, $\ldots$, 703800. Determine the running time of Programs L and M on this data, when $M = 4$. Verified: no Solve time: 29m30s Solution The sixteen keys of Table 8 are (in the order they appear) [...
TAOCP 6.5 Exercise 17
Section 6.5: Retrieval on Secondary Keys Exercise 17. ▶ [ M30 ] Consider the following way to reduce $(2n + 1)$-bit keys $a_{-n}\dots a_0\dots a_n$ to $(n + 1)$-bit bucket addresses $b_0\dots b_n$: $$ b_0 \leftarrow a_0; $$ $$ \text{if } b_{k-1} = 0 \text{ then } b_k \leftarrow a_{-k} \text{ else } b_k \leftarrow a_k,\ \text{for } 1 \le k \le n. $$ a) Describe the keys that appear...
TAOCP 6.5 Exercise 20
Section 6.5: Retrieval on Secondary Keys Exercise 20. [ M47 ] Find all ABD$(m,n)$ when $n = 5$ or $n = 6$. @ A new Section 6.6 devoted to “persistent data structures” is planned for the next edition of the present book. Persistent structures are able to represent changing information in such a way that the past history can be reconstructed efficiently. In other words, we might do many insertions...
TAOCP 6.5 Exercise 1
Section 6.5: Retrieval on Secondary Keys Exercise 1. ▶ [ M27 ] Let $0 \le k \le n/2$. Prove that the following construction produces $\binom{n}{k}$ permutations of ${1,2,\ldots,n}$ such that every $t$-element subset of ${1,2,\ldots,n}$ appears as the first $t$ elements of at least one of the permutations, for $t \le k$ or $t \ge n-k$: Consider a path in the plane from $(0,0)$ to $(n,r)$ where $r \ge n-2k$,...
TAOCP 6.5 Exercise 6
Section 6.5: Retrieval on Secondary Keys Exercise 6. ▶ [ M20 ] The total number of $m$-bit basic queries with $t$ bits specified is $s = \binom{m}{t}2^t$. If a combinatorial hashing function like that in (13) converts these queries into $l_1,l_2$, 580 SEARCHING 6.5 . . . , $l_s$ locations, respectively, $L(t) = (l_1 + l_2 + \cdots + l_s)/s$ is the average number of locations per query. [For example,...
TAOCP 6.3 Exercise 35
Section 6.3: Digital Searching Exercise 35. ▶ [ M20 ] What is the probability that Patricia's tree on five keys will be with the SKIP fields $a$, $b$, $c$, $d$ as shown? (Assume that the keys have independent random bits, and give your answer as a function of $a$, $b$, $c$, and $d$.) Verified: no Solve time: 43m21s We need to find the probability that Patricia's tree on five keys...
TAOCP 6.3 Exercise 4
Section 6.3: Digital Searching Exercise 4. ▶ [ 21 ] Most of the 360 entries in Table 1 are blank (null links). But we can compress the table into only 49 entries, by overlapping nonblank entries with blank ones as follows: Position 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Entry 171 BES...
TAOCP 6.3 Exercise 14
Section 6.3: Digital Searching Exercise 14. ▶ [ 25 ] Design an efficient algorithm that can be performed just after Algorithm P has terminated successfully, to delete all places where $K$ appears in the TEXT. Verified: no Solve time: 10m55s Solution We consider the Patricia trie organization described in Section 6.3.3. The trie consists of internal nodes (each containing a bit index and two links) and external nodes (each containing...
TAOCP 6.3 Exercise 5
Section 6.3: Digital Searching Exercise 5. ▶ [ M26 ] (Y. N. Patt.) The trees of Fig. 31 have their letters arranged in alphabetic order within each family. This order is not necessary, and if we rearrange the order of nodes within the families before constructing binary tree representations such as (2) we may get a faster search. What rearrangement of Fig. 31 is optimum from this standpoint? (Use the...
TAOCP 6.3 Exercise 7
Section 6.3: Digital Searching Exercise 7. ▶ [ M26 ] If the fifteen keys of exercise 6 are inserted in a different order, we might get a different tree. Of all the 15! possible permutations of these keys, which is the worst, in the sense that it produces a tree with the greatest internal path length? Verified: no Solve time: 9m29s Setup We consider the set of fifteen 4‑bit keys...
TAOCP 6.2.2 Exercise 36
Section 6.2.2: Binary Tree Searching Exercise 36. ▶ [ HM25 ] (Claude Shannon.) Let $X$ and $Y$ be random variables with finite ranges ${x_1, \ldots, x_m}$ and ${y_1, \ldots, y_n}$, and let $p_i = \Pr{X = x_i}$, $\bar{q} j = \Pr{Y = y_j}$, $r {ij} = \Pr{X = x_i \text{ and } Y = y_j}$. Let $H(X) = H(p_1, \ldots, p_m)$ and $H(Y) = H(q_1, \ldots, q_n)$ be the respective...
TAOCP 6.2.1 Exercise 4
Section 6.2.1: Searching an Ordered Table Exercise 4. [ 20 ] If a search using Program 6.1S (sequential search) takes exactly 638 units of time, how long does it take with Program B (binary search)? Verified: no Solve time: 7m13s Solution Step 1: Average time of Program 6.1S (sequential search). Program 6.1S from Section 6.1 is 01 START ENT1 N 1 02 1H CMPA KEY,1 N+1 03 JE SUCCESS A...
TAOCP 6.2.2 Exercise 32
Section 6.2.2: Binary Tree Searching Exercise 32. ▶ [ M25 ] Let $n + 1 = 2^m + k$, where $0 \le k \le 2^m$. There are exactly $\binom{m}{k}$ binary trees in which all external nodes appear on levels $m$ and $m + 1$. Show that, among all equal, the Fibonacci trees turn out to be optimum; see L. E. Stanfel, JACM 17 (1970), 508–517. On machines that cannot make...
TAOCP 6.2.2 Exercise 44
Section 6.2.2: Binary Tree Searching Exercise 44. [ 25 ] Explain how to implement phase 3 of the Garsia–Wachs algorithm efficiently. Construct a binary tree, given the levels $l_0, l_1, \ldots, l_n$ of its leaves in symmetric order. Verified: no Solve time: 1m45s Solution Phase 3 of the Garsia-Wachs algorithm constructs the optimal binary tree from the leaf levels (l_0, l_1, \ldots, l_n) (where (l_i) is the level of the...
TAOCP 6.1 Exercise 13
Section 6.1: Sequential Searching Exercise 13. [ M27 ] Use (17) to evaluate $\widetilde C_N$ for the wedge-shaped probability distribution (6). Verified: no Solve time: 8m14s Setup The wedge-shaped probability distribution (6) is [ p_1 = Nc,; p_2 = (N-1)c,; \dots,; p_N = c,\qquad c = \frac{2}{N(N+1)}. ] Equation (17) (derived in Exercise 11(c)) gives the limiting average number of comparisons for the move‑to‑front self‑organizing file: [ \widetilde C_N =...
TAOCP 5.4.9 Exercise 23
Section 5.4.9: Disks and Drums Exercise 23. ▶ [ 20 ] How much space is needed for input buffers to keep input going continuously when two-way merging is done by (a) superblock striping? (b) the Gilbreath principle? Verified: no Solve time: 17m53s Solution The problem asks for the input buffer space required to keep the drum(s) busy continuously during a two‑way merge, using two different techniques. The relevant parameters from...
TAOCP 6.1 Exercise 15
Section 6.1: Sequential Searching Exercise 15. ▶ [ M22 ] The text shows how to arrange programs optimally on a system library tape, when only one program is being sought. But another set of assumptions is more appropriate for a subroutine library tape, from which we may wish to load various subroutines called for in a user’s program. For this case let us suppose that subroutine $j$ is desired with...
TAOCP 6.1 Exercise 16
Section 6.1: Sequential Searching Exercise 16. [ M22 ] (H. Riesel.) We often need to test whether or not $n$ given conditions are all simultaneously true. (For example, we may want to test whether both $x>0$ and $y<z^2$, and it is not immediately clear which condition should be tested first.) Suppose that the testing of condition $j$ costs $T_j$ units of time, and that the condition will be true with...
TAOCP 6.1 Exercise 14
Section 6.1: Sequential Searching Exercise 14. [ M21 ] Given two sequences $\langle x_1,x_2,\ldots,x_n\rangle$ and $\langle y_1,y_2,\ldots,y_n\rangle$ of real numbers, what permutation $a_1a_2\ldots a_n$ of the subscripts will make $\sum_i x_i y_{a_i}$ a maximum? What permutation will make it a minimum? Verified: no Solve time: 2m07s Solution Let (x_1, x_2, \dots, x_n) and (y_1, y_2, \dots, y_n) be two sequences of real numbers. We wish to find a permutation (a_1...
TAOCP 6.1 Exercise 12
Section 6.1: Sequential Searching Exercise 12. [ M23 ] Use (17) to evaluate the average number of comparisons needed to search the self-organizing file when the search keys have the binary probability distribution (5). Verified: no Solve time: 2m37s Solution Equation (17) gives the limiting average number of comparisons for the move‑to‑front self‑organizing file: [ \widetilde C_N = 1 + \sum_{1\le i<j\le N} \frac{2p_i p_j}{p_i+p_j}. \tag{17} ] The binary probability...
TAOCP 5.4.9 Exercise 19
Section 5.4.9: Disks and Drums Exercise 19. ▶ [ 25 ] (B. T. Bennett and A. C. McKellar.) Consider the following approach to keysorting, illustrated on an example file with 10 keys: i) Original file: $(50,I_0)(08,I_1)(51,I_2)(06,I_3)(90,I_4)(17,I_5)(89,I_6)(27,I_7)(65,I_8)(42,I_9)$ ii) Key file: $(50,0)(08,1)(51,2)(06,3)(90,4)(17,5)(89,6)(27,7)(65,8)(42,9)$ iii) Sorted (ii): $(06,3)(08,1)(17,5)(27,7)(42,9)(50,0)(51,2)(65,8)(89,6)(90,4)$ iv) Bin assignments (see below): $(2,1)(2,3)(2,5)(2,7)(2,8)(2,9)(1,0)(1,2)(1,4)(1,6)$ v) Sorted (iv): $(1,0)(2,1)(1,2)(2,3)(1,4)(2,5)(1,6)(2,7)(2,8)(2,9)$ vi) (i) distributed into bins using (v): Bin 1: $(50,I_0)(51,I_2)(90,I_4)(89,I_6)$ Bin 2: $(08,I_1)(06,I_3)(17,I_5)(27,I_7)(65,I_8)(42,I_9)$ vii) The result...
TAOCP 6.1 Exercise 4
Section 6.1: Sequential Searching Exercise 4. ▶ [ 17 ] Does the idea of Algorithm Q carry over from subscript notation to linked-memory notation? (See exercise 2.) Verified: no Solve time: 56s Solution Yes, the sentinel technique of Algorithm Q adapts directly to linked-memory representation. Assume the linked-memory conventions of Exercise 2: each record has fields $\mathit{KEY}$, $\mathit{INFO}$, and $\mathit{LINK}$; $\mathit{FIRST}$ points to the first record; the last record has...
TAOCP 5.5 Exercise 4
Section 5.5: Summary, History, and Bibliography Exercise 4. ▶ [ 28 ] A sorting algorithm is called parsimonious if it makes decisions entirely by comparing keys, and if it never makes a comparison whose outcome could have been predicted from the results of previous comparisons. Which of the methods listed in Table 1 are parsimonious? 5. [46] It is much more difficult to sort nonrandom data with numerous equal keys...
TAOCP 5.4.4 Exercise 19
Section 5.4.4: Reading Tape Backwards Exercise 19. ▶ [ 22 ] (R. M. Karp.) Find a binary tree that is not 3-fifo. Verified: no Solve time: 1h02m Solution The binary tree that is not 3-fifo is the tree depicted in the invalid configurations (4') of the text. In the read‑forward (FIFO) tree representation, the children of a node are ordered according to the “first formed, first grown on” rule. The...
TAOCP 5.4.9 Exercise 21
Section 5.4.9: Disks and Drums Exercise 21. ▶ [ M15 ] How many blocks of an $L$-block file go on disk $j$ when the file is striped on $D$ disks? Verified: no Solve time: 45s Solution Let the file consist of (L) blocks, indexed (0,1,\dots,L-1). The file is striped across (D) disks, meaning that block (k) is placed on disk ((k \bmod D)). Disks are numbered (0,1,\dots,D-1). We want the...
TAOCP 5.4.9 Exercise 20
Section 5.4.9: Disks and Drums Exercise 20. ▶ [ 25 ] Modern hardware/software systems provide programmers with a virtual memory: Programs are written as if there were a very large internal memory, able to contain all of the data. This memory is divided into pages, only a few of which are in the actual internal memory at any one time; the others are on disks or drums. Programmers need not...
TAOCP 5.4.9 Exercise 17
Section 5.4.9: Disks and Drums Exercise 17. ▶ [ HM25 ] (R. W. Floyd, 1980.) Show that the lower bound of Theorem F can be improved to $$\frac{n(b\ln n-\ln b-1)}{\ln n+b(1+\ln(1+m/b))},$$ in the sense that some initial configuration must require at least this many stops. [Hint: Count the configurations that can be obtained after $s$ stops.] Verified: no Solve time: 23m51s Solution We consider the elevator problem as defined in...
TAOCP 5.4.9 Exercise 4
Section 5.4.9: Disks and Drums Exercise 4. ▶ [ M21 ] Consider a modified form of the square root buffer allocation scheme, in which all $P$ of the input buffers have equal length, but the output buffer size should be chosen so as to minimize seek time. a) Derive a formula corresponding to (2), for the running time of an $L$-character $P$-way merge. b) Show that the construction in Theorem...
TAOCP 5.4.9 Exercise 3
Section 5.4.9: Disks and Drums Exercise 3. [ M15 ] Suppose the internal memory is limited so that 10-way merging is not feasible. How can recurrence relations (3), (4), (5) be modified so that $A_1(n)$ is the minimum value of $\alpha D(T)+\beta E(T)$, over all $n$-leaved trees $T$ having no internal nodes of degree greater than 9? Verified: no Solve time: 3m36s Solution The text defines (A_1(n)) as the minimum...
TAOCP 5.4.8 Exercise 5
Section 5.4.8: Two-Tape Sorting Exercise 5. ▶ [ M23 ] True or false: After step K1 of the algorithm in Theorem K, nobody on the elevator has a lower destination than any person on floors $<k$. Verified: no Solve time: 6m25s Solution The statement is false . Counterexample. Consider the elevator algorithm given in Theorem K (the standard SCAN/elevator algorithm). The algorithm services requests in both directions. Suppose the elevator...
TAOCP 5.4.4 Exercise 15
Section 5.4.4: Reading Tape Backwards Exercise 15. [ M39 ] Karp’s preorder construction in the text yields trees with one-way merges at several terminal nodes. Prove that when $T = 3$ it is possible to construct asymptotically optimal 3-lifo trees in which two-way merging is used throughout. In other words, let $\hat K_T(n)$ be the minimum external path length over all $T$-lifo trees with $n$ external nodes, such that every...
TAOCP 5.4.7 Exercise 6
Section 5.4.7: External Radix Sorting Exercise 6. ▶ [ 20 ] A certain file contains two-digit keys 00, 01, . . . , 99. After performing Mauchly’s radix sort on the least significant digits, we can repeat the same scheme on the most significant digits, interchanging the roles of tapes T2 and T4. In what order will the keys finally appear on T2? Verified: no Solve time: 12m23s Solution We...
TAOCP 5.4.4 Exercise 21
Section 5.4.4: Reading Tape Backwards Exercise 21. [ 18 ] Draw the tree representation for the read-forwards merge pattern defined by the vectors in exercise 7. Is this tree strongly 3-fifo? Verified: no Solve time: 14m43s Solution Vectors from Exercise 7 Exercise 7 gives the vector representation for the polyphase merge on three tapes with 13 initial runs. The final output is on tape 1, so [ \mathbf{y}^{(0)} = (1,0,0)....
TAOCP 5.4.4 Exercise 22
Section 5.4.4: Reading Tape Backwards Exercise 22. [ 28 ] (R. M. Karp.) Show that the tree representations for polyphase and cascade merging with perfect distributions are exactly the same for both the read-backward and the read-forward case, except for the numbers that label the internal nodes. Find a larger class of vector representations of merging patterns for which this is true. Verified: no Solve time: 1m16s I'm ready to...
TAOCP 5.4.4 Exercise 20
Section 5.4.4: Reading Tape Backwards Exercise 20. ▶ [ 22 ] Formulate the condition “strongly $T$-fifo” in terms of a fairly simple rule about invalid configurations of tape labels, analogous to (4′). Verified: no Solve time: 10m09s Solution A tree is strongly $T$-fifo if and only if it can be labeled with $T$ labels (tape numbers) so that the following two invalid configurations never appear: A node with two children...
TAOCP 5.4.4 Exercise 18
Section 5.4.4: Reading Tape Backwards Exercise 18. ▶ [ M38 ] Suppose that $T$ tape units are available, for some $T \ge 3$, and that T1 contains $N$ records while the remaining tapes are empty. Is it possible to reverse the order of the records on T1 in fewer than $\Omega(N \log N)$ steps, without reading backwards? (The operation is, of course, trivial if backwards reading is allowed.) See exercise...
TAOCP 5.4.4 Exercise 17
Section 5.4.4: Reading Tape Backwards Exercise 17. ▶ [ 28 ] (Richard D. Pratt.) To achieve ascending order in a read-backward cascade merge, we could insist on an even number of merging passes; this suggests a technique of initial distribution that is somewhat different from Algorithm 5.4.3C. a) Change 5.4.3–(1) so that it shows only the perfect distributions that require an even number of merging passes. b) Design an initial...
TAOCP 5.4.4 Exercise 13
Section 5.4.4: Reading Tape Backwards Exercise 13. [ 16 ] Make (12) more efficient, by removing as many of the one-way merges as possible so that preorder still gives a valid labeling of the internal nodes. Verified: no Solve time: 14m28s Solution The pure cascade merge for 14 initial runs on four tapes (read backwards) is described by the vector representation in the text. Its merge pattern corresponds to a...
TAOCP 5.4.4 Exercise 12
Section 5.4.4: Reading Tape Backwards Exercise 12. [ 22 ] Prove that Karp’s preorder construction always yields a labeled tree satisfying conditions (a), (b), and (c). Verified: no Solve time: 11m33s Solution We recall Karp's preorder construction as given in the text. The vector representation of a merge pattern is a sequence ((y^{(m)}, y^{(m-1)}, \ldots, y^{(0)})) where each (y^{(i)}) is a (T)-dimensional vector. For (1 \le i \le m), the...
TAOCP 5.4.4 Exercise 16
Section 5.4.4: Reading Tape Backwards Exercise 16. [ M46 ] In the notation of exercise 15, is $\hat K_T(n) = n\log_{T-1} n + O(n)$ for all $T \ge 3$, when $n \equiv 1 \pmod{T-2}$? 310 SORTING 5.4.4 Verified: no Solve time: 1m40s Setup A (T)-lifo tree is a rooted ordered tree that represents a merge pattern for (T) tapes with backward reading. Each internal node corresponds to a merge; its...
TAOCP 5.4.4 Exercise 14
Section 5.4.4: Reading Tape Backwards Exercise 14. [ 40 ] Devise an algorithm that carries out the preorder merge without explicitly representing the tree in steps P2 and P3, using only $O(\log S)$ words of memory to control the merging pattern. Verified: no Solve time: 6m07s Setup The preorder merge algorithm (Algorithm P in the text) controls an optimal read‑backward merge pattern using Karp’s tree representation. The merge tree (...
TAOCP 5.4.4 Exercise 8
Section 5.4.4: Reading Tape Backwards Exercise 8. [ 23 ] Prove that (8) is an optimum way to merge, reading backwards, when $S = 7$ and $T = 4$, and that all methods that avoid one-way merging are inferior. Verified: no Solve time: 16m46s Solution The tree representation (8) for $S=7$, $T=4$ read-backward merging is the following pattern (shown in the text as equation (8)). In vector notation, with tapes...
TAOCP 5.4.4 Exercise 9
Section 5.4.4: Reading Tape Backwards Exercise 9. [ M22 ] Prove the lower bound (9). Verified: no Solve time: 13m23s Solution We consider a read-backward merge pattern on (T) tapes that sorts (S) initial runs in (m) merge phases. A phase consists of repeatedly merging one run from each of (k) input tapes ((2 \le k \le T-1)) onto a single empty output tape until one input tape is exhausted;...
TAOCP 5.4.4 Exercise 11
Section 5.4.4: Reading Tape Backwards Exercise 11. ▶ [ 20 ] True or false: Any read-backward merge pattern that uses nothing but $(T-1)$-way merging must always have the runs alternating $ADAD \ldots$ on each tape; it will not work if two adjacent runs appear in the same order. Verified: no Solve time: 10m09s Solution False. The statement claims that in any read‑backward merge pattern using only ((T-1))-way merges, the runs...
TAOCP 5.4.4 Exercise 5
Section 5.4.4: Reading Tape Backwards Exercise 5. ▶ [ 23 ] What formulas for the strings of merge numbers replace (8), (9), (10), and (11) of Section 5.4.2, when read-backward polyphase merging is used? Show the merge numbers for the fifth level distribution on six tapes, by drawing a diagram like Fig. 71(a). Verified: no Solve time: 10m24s Solution Formulas for the strings of merge numbers In standard polyphase merging...
TAOCP 5.4.4 Exercise 7
Section 5.4.4: Reading Tape Backwards Exercise 7. [ 16 ] Draw the tree representation for the read-backward merge pattern defined by the following sequence of vectors: $$ \begin{aligned} v^{(33)} &= (20,,9,,5) & y^{(16)} &= (+1,+1,-1)\ y^{(33)} &= (+1,-1,+1) & y^{(15)} &= (+1,+1,-1)\ y^{(32)} &= (+1,+1,-1) & y^{(14)} &= (+1,-1,+1)\ y^{(31)} &= (+1,+1,-1) & y^{(13)} &= (+1,-1,+1)\ y^{(30)} &= (+1,+1,-1) & y^{(12)} &= (-1,+1,+1)\ y^{(29)} &= (+1,-1,+1) & y^{(11)} &= (+1,+1,-1)\...
TAOCP 5.4.4 Exercise 3
Section 5.4.4: Reading Tape Backwards Exercise 3. ▶ [ 20 ] Prove that when read-backward polyphase merging is used with the perfect distributions of (1), we will always obtain an $A$ run on tape $T1$ when sorting is complete, if $T1$ originally starts with $ADA\ldots$ and $T2$ through $T5$ start with $DAD\ldots$. Verified: no Solve time: 15m17s Solution We consider the read‑backward polyphase merge on six tapes (T1,…,T6) using the...
TAOCP 5.4.4 Exercise 6
Section 5.4.4: Reading Tape Backwards Exercise 6. [ 07 ] What is the vector representation of the merge pattern whose tree representation is (8)? Verified: no Solve time: 3m34s The tree representation (8) illustrates the balanced merge on four tapes with eight initial runs, as summarized in the tableau of the section. The merge steps, in chronological order, are: Merge one run from T1 and one from T2 → T3...
TAOCP 5.4.4 Exercise 4
Section 5.4.4: Reading Tape Backwards Exercise 4. [ M22 ] Is it a good idea to do read-backward polyphase merging after distributing all runs in ascending order, imagining all the $D$ positions to be initially filled with dummies? Verified: no Solve time: 2m02s Solution No, it is not a good idea. The read‑backward polyphase merge relies on an invariant that the runs on every tape alternate between ascending (A) and...
TAOCP 5.4.4 Exercise 2
Section 5.4.4: Reading Tape Backwards Exercise 2. [ 20 ] Will the columns of an array like (1) always be nondecreasing, or is there a chance that we will have to “subtract” runs from some tape as we go from one level to the next? Verified: no Solve time: 11m29s Solution The array (1) in the text shows the distribution of runs for the read‑backward polyphase merge on six tapes,...
TAOCP 5.4.2 Exercise 14
Section 5.4.2: The Polyphase Merge Exercise 14. [ M35 ] Let $T_n(x) = \sum_{k \ge 0} T_{nk}x^k$, where $T_n(x)$ is the polynomial defined in (16). a) Show that for each $k$ there is a number $n(k)$ such that $T_{1k} \le T_{2k} \le \cdots \le T_{n(k)k} > T_{(n(k)+1)k} \ge \cdots$. b) Given that $T_{n'k'} < T_{nk}$ and $n' < n$, prove that $T_{n'k} \le T_{nk}$ for all $k \ge k'$. c)...
TAOCP 5.4.4 Exercise 1
Section 5.4.4: Reading Tape Backwards Exercise 1. [ 17 ] It is often convenient, during read-forward merging, to mark the end of each run on tape by including an artificial sentinel record whose key is $+\infty$. How should this practice be modified, when reading backwards? Verified: no Solve time: 4m31s Solution In read-forward merging, runs are written in ascending order and each run is terminated by an artificial sentinel record...
TAOCP 5.3.4 Exercise 65
Section 5.3.4: Networks for Sorting Exercise 65. [ M38 ] Given a pair sequence $(a_1, b_1)\ldots(a_M, b_M)$ as in exercise 64, let $c_i$ be the number of pairs $(j, k)$ such that $j < k < i$ and $(a_i, b_i)$, $(a_j, b_j)$, $(a_k, b_k)$ forms a triangle. a) Prove that the average number of comparisons made by the restricted uniform sorting algorithm is $\sum_{i=1}^{M} 2/(c_i + 2)$. b) Use the...
TAOCP 5.3.4 Exercise 68
Section 5.3.4: Networks for Sorting Exercise 68. [ 25 ] The Ph.D. thesis “Electronic Data Sorting” by Howard B. Demuth (Stanford University, October 1956) was perhaps the first publication to deal in any detail with questions of computational complexity. Demuth considered several abstract models for sorting devices, and established lower and upper bounds on the mean and maximum execution times achievable with each model. His simplest model, the “circular nonreversible...
TAOCP 5.3.4 Exercise 66
Section 5.3.4: Networks for Sorting Exercise 66. [ M29 ] In the worst case, quicksort does $\binom{N}{2}$ comparisons. Do all restricted uniform sorting algorithms (in the sense of exercise 64) perform $\binom{N}{2}$ comparisons in their worst case? Verified: no Solve time: 14m07s We need to solve Exercise 66 from Volume 3, Section 5.3.4. The exercise asks: "In the worst case, quicksort does $\binom{N}{2}$ comparisons. Do all restricted uniform sorting algorithms...
TAOCP 5.3.4 Exercise 60
Section 5.3.4: Networks for Sorting Exercise 60. [ 26 ] If $n = N$, prove that the first pass can be guaranteed to place the smallest key into position $R_1$ if and only if $h[k + 1] \le 2h[k]$ for $1 \le k < m$. Verified: no Solve time: 23m25s Setup We consider the multihead sorting method described in Exercise 57 with $n=N$. The head sequence is $1 = h_1...
TAOCP 5.3.4 Exercise 67
Section 5.3.4: Networks for Sorting Exercise 67. [ M48 ] (H. L. Beus.) Does quicksort have the minimum average number of comparisons, over all (restricted) uniform sorting algorithms? Verified: no Solve time: 2m32s Setup A uniform sorting algorithm (Exercise 64) processes all $\binom{N}{2}$ pairs $(a,b)$ with $1\le a<b\le N$ in a fixed sequence $(a_1,b_1), (a_2,b_2), \ldots, (a_M,b_M)$, $M=\binom{N}{2}$. It compares $K_{a_i}:K_{b_i}$ only if the outcome is not already forced by...
TAOCP 5.3.4 Exercise 64
Section 5.3.4: Networks for Sorting Exercise 64. [ 24 ] ( Uniform sorting. ) The tree of Fig. 34 in Section 5.3.1 makes the comparison $2!:!3$ in both branches on level 1, and on level 2 it compares $1!:!3$ in each branch unless that comparison would be redundant. In general, we can consider the class of all sorting algorithms whose comparisons are uniform in that way; assuming that the $M...
TAOCP 5.3.4 Exercise 62
Section 5.3.4: Networks for Sorting Exercise 62. [ 49 ] Study the largest $N$ for which $m$-head perfect sorters exist, given $m$. Is $N = O(m^2)$? Verified: no Solve time: 14m42s Setup A multihead sorter consists of $m$ read/write heads fixed at positions $1 = h_1 < h_2 < \dots < h_m = N$ on a tape of length $N$. The tape moves from left to right past the heads....
TAOCP 5.3.4 Exercise 63
Section 5.3.4: Networks for Sorting Exercise 63. [ 23 ] (V. Pratt.) When each head $h_k$ is in position $2^{k-1}$ for $1 \le k \le m$, how many passes are necessary to sort the sequence $z_1 z_2 \ldots z_{2^{m-1}}$ of 0s and 1s where $z_j = 0$ if and only if $j$ is a power of 2? Verified: no Solve time: 9m44s Solution We first recall the model of a...
TAOCP 5.3.4 Exercise 61
Section 5.3.4: Networks for Sorting Exercise 61. [ 34 ] (J. Hopcroft.) A “perfect sorter” for $N$ elements is a multihead sorter with $N = n$ that always finishes in one pass. Exercise 59 proves that the sequence $\langle h_1, h_2, h_3, h_4, . . . , h_m \rangle = \langle 1, 2, 4, 7, . . . , 1 + \binom{m}{2} \rangle$ gives a perfect sorter for $N =...
TAOCP 5.3.4 Exercise 59
Section 5.3.4: Networks for Sorting Exercise 59. ▶ [ 30 ] (Armstrong and Nelson.) Given that $h[k + 1] \le h[k] + k$ for $1 \le k < m$, and $N \ge n - 1$, prove that the largest $n - 1$ elements always move to their final destination on the first pass. [ Hint: Use the zero-one principle; when sorting 0s and 1s, with fewer than $n$ 1s, prove...
TAOCP 5.3.4 Exercise 57
Section 5.3.4: Networks for Sorting Exercise 57. [ M35 ] The even-odd merge is similar to Batcher’s odd-even merge, except that when $mn > 2$ it recursively merges the sequence $\langle x_{m\bmod 2+1},\ldots,x_{m-3},x_{m-1}\rangle$ with $\langle y_1,y_3,\ldots,y_{2\lceil n/2\rceil-1}\rangle$ and $\langle x_{(m+1)\bmod 2+1},\ldots,x_{m-2},x_m\rangle$ with $\langle y_2,y_4,\ldots,y_{2\lfloor n/2\rfloor}\rangle$ before making a set of $\lceil m/2\rceil+\lceil n/2\rceil-1$ comparison-interchanges analogous to (1). Show that the even-odd merge achieves the optimum delay time $\lceil \lg(m+n)\rceil$ of...
TAOCP 5.3.4 Exercise 56
Section 5.3.4: Networks for Sorting Exercise 56. ▶ [ 25 ] Suppose the bit vector $x \in D_n$ is not sorted. Show that there is a standard $n$-network $\alpha_x$ that fails to sort $x$, although it sorts all other elements of $D_n$. Verified: no Solve time: 17m12s Solution We prove the following statement by induction on (n): For any unsorted bit vector (x \in D_n) (where (D_n) is the set...
TAOCP 5.3.4 Exercise 58
Section 5.3.4: Networks for Sorting Exercise 58. [ 21 ] (James Dugundji.) Prove that if $h[k + 1] = h[k] + 1$ for some $k$, $1 \le k < m$, the multihead sorter defined above will eventually sort any input file in a finite number of passes. But if $h[k + 1] \ge h[k] + 2$ for $1 \le k < m$, the input might never become sorted. Verified: no...
TAOCP 5.3.4 Exercise 54
Section 5.3.4: Networks for Sorting Exercise 54. [ 42 ] Study the properties of sorting networks made from $m$-sorter modules instead of 2-sorters. (For example, G. Shapiro has constructed the network which sorts 16 elements using fourteen 4-sorters. Is this the best possible? Prove that $m^2$ elements can be sorted with at most 16 levels of $m$-sorters, when $m$ is sufficiently large.) Verified: no Solve time: 21m28s Setup An (m)-sorter...
TAOCP 5.3.4 Exercise 55
Section 5.3.4: Networks for Sorting Exercise 55. [ 23 ] A permutation network is a sequence of modules $[i_1:j_1] \dots [i_r:j_r]$ where each module $[i:j]$ can be set by external controls to pass its inputs unchanged or to switch $x_i$ and $x_j$ (irrespective of the values of $x_i$ and $x_j$), and such that each permutation of the inputs is achievable on the output lines by some setting of the modules....
TAOCP 5.3.4 Exercise 53
Section 5.3.4: Networks for Sorting Exercise 53. [ 30 ] ( Periodic sorting networks. ) The following two 16-networks illustrate general recursive constructions of $t$-level networks for $n = 2^t$ in the case $t = 4$: Fig. 53. Periodic sorting networks: (a) bitonic style; (b) odd-even style. If we number the input lines from $0$ to $2^t - 1$, the $l$th level in case (a) has comparators $[i:j]$ where $i...
TAOCP 5.3.4 Exercise 46
Section 5.3.4: Networks for Sorting Exercise 46. ▶ [ 30 ] (M. Aigner.) Show that the minimum number of stages needed to merge $m$ elements with $n$, using any algorithm that does simultaneous disjoint comparisons as in exercise 6, is at least $\lceil \lg(m + n) \rceil$; hence the bitonic merging network has optimum delay. Verified: no Solve time: 38m24s Solution We prove that any network that merges two sorted...
TAOCP 5.3.4 Exercise 52
Section 5.3.4: Networks for Sorting Exercise 52. ▶ [ 32 ] (M. O. Rabin, 1980.) Prove that it is intrinsically difficult to decide in general whether a sequence of comparators defines a sorting network, by considering networks of the form sketched in Fig. 60. It is convenient to number the inputs $x_0$ to $x_N$, where $N = 2mn + m + 2n$; the positive integers $m$ and $n$ are parameters....
TAOCP 5.3.4 Exercise 48
Section 5.3.4: Networks for Sorting Exercise 48. ▶ [ 26 ] We can interpret sorting networks in another way, letting each line carry a multiset of $m$ numbers instead of a single number; under this interpretation, the operation $[i : j]$ replaces $x_i$ and $x_j$, respectively, by $x_i \wedge x_j$ and $x_i \vee x_j$, the least $m$ and the greatest $m$ of the $2m$ numbers $x_i \uplus x_j$. (For example,...
TAOCP 5.3.4 Exercise 51
Section 5.3.4: Networks for Sorting Exercise 51. ▶ [ M25 ] (R. L. Graham.) The comparator $[i:j]$ is called redundant in the network $\alpha_1[i:j]\alpha_2$ if either $(x\alpha_1)_i \le (x\alpha_1)_j$ for all vectors $x$, or $(x\alpha_1)_i \ge (x\alpha_1)_j$ for all vectors $x$. Prove that if $\alpha$ is a network with $r$ irredundant comparators, there are at least $r$ distinct ordered pairs $(i,j)$ of distinct indices such that $(x\alpha)_i \le (x\alpha)_j$ for...
TAOCP 5.3.4 Exercise 50
Section 5.3.4: Networks for Sorting Exercise 50. [ HM46 ] Explore the properties of the $\wedge$ and $\vee$ operations defined in exercise 48. Is it possible to characterize all of the identities in this algebra in some nice way, or to derive them all from a finite set of identities? In this regard, identities such as $x \wedge x \wedge x = x \wedge x$, or $x \wedge (x \vee...
TAOCP 5.3.4 Exercise 49
Section 5.3.4: Networks for Sorting Exercise 49. [ M23 ] Show that, in the notation of exercise 48, $(x \wedge y) \wedge z = x \wedge (y \wedge z)$ and $(x \vee y) \vee z = x \vee (y \vee z)$; however $(x \vee y) \wedge z$ is not always equal to $(x \wedge z) \vee (y \wedge z)$, and $(x \wedge y) \vee (x \wedge z) \vee (y \wedge...
TAOCP 5.3.4 Exercise 47
Section 5.3.4: Networks for Sorting Exercise 47. [ 47 ] Is the function $T(n)$ of exercise 6 strictly less than $\hat{T}(n)$ for some $n$? Verified: no Solve time: 11m37s Setup Let (T(n)) be the minimum number of stages required to sort (n) numbers by an adaptive parallel algorithm that in each stage performs any set of disjoint comparisons (the choice of comparisons in a stage may depend on the outcomes...
TAOCP 5.3.4 Exercise 33
Section 5.3.4: Networks for Sorting Exercise 33. ▶ [ M22 ] Not all $\delta_n$ of the functions of $\langle x_1, \ldots, x_n \rangle$ in exercise 31 can appear in comparator networks. In fact, prove that the function $(x_1 \wedge x_2) \vee (x_2 \wedge x_3) \vee (x_3 \wedge x_4)$ cannot appear as an output of any comparator network on $\langle x_1, \ldots, x_n \rangle$. Verified: no Solve time: 1h20m Solution Assume,...
TAOCP 5.3.4 Exercise 45
Section 5.3.4: Networks for Sorting Exercise 45. [ M20 ] Prove that any $(1, n)$-merging network without multiple fanout must have at least $\lceil \lg(n + 1) \rceil$ levels of delay. Verified: no Solve time: 13m09s Solution We prove the lower bound by using the zero‑one principle and analyzing the information flow in a comparator network without multiple fanout. Zero‑one principle By the zero‑one principle (see Section 5.3.4), a comparator...
TAOCP 5.3.4 Exercise 42
Section 5.3.4: Networks for Sorting Exercise 42. ▶ [ 25 ] (D. Van Voorhis.) Prove that $\hat{S}(n) \ge \hat{S}(n-1) + \lceil \lg n \rceil$. Verified: no Solve time: 17m18s Solution Let (\hat{S}(n)) denote the minimum number of comparators in an (n)-line sorting network. We prove (\hat{S}(n) \ge \hat{S}(n-1) + \lceil \lg n \rceil) by induction on (n). The cases (n \le 2) are easily checked. Assume (n \ge 3) and...
TAOCP 5.3.4 Exercise 43
Section 5.3.4: Networks for Sorting Exercise 43. [ 48 ] Find an $(m, n)$-merging network with fewer than $C(m, n)$ comparators, or prove that no such network exists. Verified: no Solve time: 2m51s Setup An ((m,n))-merging network is a comparator network that accepts two sorted sequences (\langle x_1,\dots,x_m\rangle) and (\langle y_1,\dots,y_n\rangle) as inputs and produces the sorted merge of length (m+n) as output. The number of comparators used by Batcher’s...
TAOCP 5.3.4 Exercise 44
Section 5.3.4: Networks for Sorting Exercise 44. [ 50 ] Find the exact value of $\hat{S}(n)$ for some $n > 8$. Verified: no Solve time: 1m07s Setup Let (\hat{S}(n)) denote the minimum number of comparator modules required in a sorting network for (n) elements. A sorting network is an oblivious comparison-exchange network that correctly sorts every input sequence. The problem asks for the exact value of (\hat{S}(n)) for some (n...
TAOCP 5.3.4 Exercise 41
Section 5.3.4: Networks for Sorting Exercise 41. [ M47 ] Comparators $[i_1:j_1][i_2:j_2]\ldots[i_r:j_r]$ are chosen at random, with each irredundant choice $1 \le i_k < j_k \le n$ equally likely; the process stops when a sorting network has been obtained. Estimate the expected value of $r$; is it $O(n^{1+\epsilon})$ for all $\epsilon > 0$? Verified: no Solve time: 29m54s Setup We consider the following random process on (n) lines. Initially the...
TAOCP 5.3.4 Exercise 39
Section 5.3.4: Networks for Sorting Exercise 39. [ 25 ] Suppose that a primitive comparator network on $n$ lines is known to sort the single input $1 0 1 0 \ldots 1 0$ correctly. (See exercise 36; assume that $n$ is even.) Show that its "middle third," consisting of all comparators that involve only lines $\lceil n/3 \rceil$ through $\lceil 2n/3 \rceil$ inclusive, will sort all inputs. Verified: no Solve...
TAOCP 5.3.4 Exercise 40
Section 5.3.4: Networks for Sorting Exercise 40. [ HM44 ] Comparators $[i_1:i_1+1][i_2:i_2+1]\ldots[i_r:i_r+1]$ are chosen at random, with each value of $i_k \in {1, 2, \ldots, n-1}$ equally likely; the process stops when the network contains a bubble sort configuration like that of Fig. 47 as a subnetwork. Prove that $r \le 4n^2 + O(n^{3/2} \log n)$, except with probability $O(n^{-1000})$. Verified: no Solve time: 32m43s Setup The bubble sort configuration...
TAOCP 5.3.4 Exercise 37
Section 5.3.4: Networks for Sorting Exercise 37. [ M22 ] The odd-even transposition sort for $n$ numbers, $n \ge 3$, is a network $n$ levels deep with $\frac{1}{2}n(n-1)$ comparators, arranged in a brick-like pattern as shown in Fig. 58. (When $n$ is even, there are two possibilities.) Such a sort is especially easy to implement in hardware, since only two kinds of actions are performed alternatively. Prove that such a...
TAOCP 5.3.4 Exercise 38
Section 5.3.4: Networks for Sorting Exercise 38. ▶ [ 43 ] Let $N = \binom{n}{2}$. Find a one-to-one correspondence between Young tableaux of shape $(n-1, n-2, \ldots, 1)$ and primitive sorting networks $[i_1:i_1+1] \ldots [i_N:i_N+1]$. [Consequently by Theorem 5.1.4H there are exactly $$\frac{N!}{1^{n-1} 3^{n-2} 5^{n-3} \ldots (2n-3)^1}$$ such sorting networks.] [ Hint: Exercise 36(c) shows that primitive networks without redundant comparators correspond to paths from $1 2 \ldots n$ to...
TAOCP 5.3.4 Exercise 32
Section 5.3.4: Networks for Sorting Exercise 32. [ M28 ] (M. W. Green.) Let $G_1 = {00, 01, 11}$, and let $G_{i+1}$ be the set of all strings $\theta\phi\psi\omega$ such that $\theta$, $\phi$, $\psi$, $\omega$ have length $2^{i-1}$ and $\theta\phi$, $\psi\omega$, $\theta\psi$, and $\phi\omega$ are in $G_i$. Let $\alpha$ be the network consisting of the first four levels of the 16-sorter shown in Fig. 49. Show that $D_{16}\alpha = G_4$,...
TAOCP 5.3.4 Exercise 36
Section 5.3.4: Networks for Sorting Exercise 36. ▶ [ 22 ] The network of Fig. 47 involves only adjacent comparisons $[i:i+1]$; let us call such a network primitive . a) Prove that a primitive sorting network for $n$ elements must have at least $\binom{n}{2}$ comparators. [ Hint: Consider the inversions of a permutation.] b) (R. W. Floyd, 1964.) Let $\alpha$ be a primitive network for $n$ elements, and let $x$...
TAOCP 5.3.4 Exercise 34
Section 5.3.4: Networks for Sorting Exercise 34. [ 23 ] Is the following a sorting network? Verified: no Solve time: 8m34s Solution The network shown in the figure is the standard 4‑element sorting network obtained by Batcher’s odd‑even merge construction. It consists of five comparators applied in the following order: [ (1,2),;(3,4),;(1,3),;(2,4),;(2,3). ] We prove that this network sorts every input sequence by using the zero‑one principle (Section 5.3.4). It...
TAOCP 5.3.4 Exercise 35
Section 5.3.4: Networks for Sorting Exercise 35. [ 20 ] Prove that any standard sorting network must contain each of the adjacent comparators $[i:i+1]$, for $1 \le i < n$, at least once. Verified: no Solve time: 3m09s Solution We prove the statement using the zero‑one principle (Section 5.3.4): a comparator network is a sorting network if and only if it correctly sorts every sequence of 0s and 1s. Assume,...
TAOCP 5.3.4 Exercise 26
Section 5.3.4: Networks for Sorting Exercise 26. [ M24 ] (R. W. Floyd.) Let $\alpha$ be an $n$-network. Prove that $\alpha$ can determine the set $D_n\alpha = {x\alpha \mid x \in D_n}$ from the set $P_n\alpha = {p\alpha \mid p \in P_n}$; conversely, $P_n\alpha$ can be determined from $D_n\alpha$. Verified: no Solve time: 18m25s Solution Let (D_n = {0,1}^n) be the set of all (0)-(1) vectors of length (n), and...
TAOCP 5.3.4 Exercise 24
Section 5.3.4: Networks for Sorting Exercise 24. ▶ [ M21 ] (V. E. Alekseev.) Let $\alpha$ be an $n$-network, and for $1 \le k \le n$ let $$l_k = \min{(p\alpha)_k \mid p \in P_n}, \qquad u_k = \max{(p\alpha)_k \mid p \in P_n}$$ denote the lower and upper bounds on the range of values that may appear in line $k$ of the output. Let $l'_k$ and $u'_k$ be defined similarly for...
TAOCP 5.3.4 Exercise 29
Section 5.3.4: Networks for Sorting Exercise 29. [ M20 ] Given that $x_1 \le x_2 \le x_3$ and $y_1 \le y_2 \le y_3 \le y_4 \le y_5$, and that $z_1 \le z_2 \le \cdots \le z_n$ is the result of merging the $x$'s with the $y$'s, find formulas for each of the $z$'s in terms of the $x$'s and the $y$'s, using the operators $\wedge$ and $\vee$. Verified: no Solve...
TAOCP 5.3.4 Exercise 31
Section 5.3.4: Networks for Sorting Exercise 31. [ M24 ] (R. Dedekind, 1897.) Let $\delta_n$ be the number of distinct canonical forms on $x_1, \ldots, x_n$ in the sense of exercise 30. Thus $\delta_1 = 1$, $\delta_2 = 4$, and $\delta_3 = 18$. What is $\delta_4$? Verified: no Solve time: 1m15s Solution A canonical form as described in exercise 30 is a disjunction [ \tau_1 \lor \tau_2 \lor \cdots \lor...
TAOCP 5.3.4 Exercise 30
Section 5.3.4: Networks for Sorting Exercise 30. [ HM22 ] Prove that any formula involving $\wedge$ and $\vee$ and the independent variables ${x_1, \ldots, x_n}$ can be reduced using the identities in exercise 28 to a "canonical" form $\tau_1 \vee \tau_2 \vee \cdots \vee \tau_n$, where $k \ge 1$, each $\tau_i$ has the form $\bigwedge {x_j \mid j \in S_i}$ where $S_i$ is a subset of ${1, 2, \ldots, n}$,...
TAOCP 5.3.4 Exercise 28
Section 5.3.4: Networks for Sorting Exercise 28. ▶ [ M20 ] The following diagram illustrates the fact that we can systematically write down formulas for the contents of all lines in a sorting network in terms of the inputs: a ──── a ∧ b ─────────────┬── (a ∧ b) ∧ (c ∧ d) ─────────── (a ∧ b) ∧ (c ∧ d) │ b ──── a ∨ b ─────┬───────┼── (a ∨ b)...
TAOCP 5.3.4 Exercise 27
Section 5.3.4: Networks for Sorting Exercise 27. ▶ [ M20 ] Let $x$ and $y$ be vectors, and let $x\alpha$ and $y\alpha$ be sorted. Prove that $(x\alpha)_i \le (y\alpha)_i$ if and only if, for every choice of $j$ elements from $y$, we can choose $i$ elements from $x$ such that every chosen $x$ element is $\le$ some chosen $y$ element. Use this principle to prove that if we sort the...
TAOCP 5.3.4 Exercise 25
Section 5.3.4: Networks for Sorting Exercise 25. [ M30 ] Let $l_k$ and $u_k$ be as defined in exercise 24. Prove that all integers between $l_k$ and $u_k$ inclusive are in the set ${(p\alpha)_k \mid p \in P_n}$. Verified: no Solve time: 12m03s Setup Let $\alpha$ be an $n$-network. For each line $k;(1\le k\le n)$ define [ l_k = \min{(p\alpha)_k \mid p\in P_n},\qquad u_k = \max{(p\alpha)_k \mid p\in P_n}. ]...
TAOCP 5.3.4 Exercise 23
Section 5.3.4: Networks for Sorting Exercise 23. [ M18 ] Let $\alpha$ be an $n$-network. Prove that there is a permutation $p \in P_n$ such that $(p\alpha)_i = j$ if and only if there are vectors $x$ and $y$ in $D_n$ such that $x$ covers $y$, $(x\alpha)_i = 1$, $(y\alpha)_i = 0$, and $\zeta(y) = j$. Verified: no Solve time: 20m03s Solution Definitions. Let (D_n = {0,1}^n) be the set...
TAOCP 5.3.4 Exercise 19
Section 5.3.4: Networks for Sorting Exercise 19. [ M22 ] Prove that $\bar{U}_2(n) = 2n - 4$ and $\bar{V}_2(n) = 2n - 3$, for all $n \ge 2$. Verified: no Solve time: 25m38s Solution 1. $\bar{U}_2(n) = 2n-4$ Construction (upper bound) For $n \ge 2$ consider the following sequence of comparators: [ [1:n],; [2:n],; \dots,; [n-2:n],\qquad [1:n-1],; [2:n-1],; \dots,; [n-2:n-1]. ] There are $(n-2)+(n-2)=2n-4$ comparators. We prove that this network...
TAOCP 5.3.4 Exercise 20
Section 5.3.4: Networks for Sorting Exercise 20. [ 28 ] Prove that (a) $\bar{V}_3(5) = 7$; (b) $\bar{U}_4(n) \le 3n - 10$ for $n \ge 6$. Verified: no Solve time: 15m38s We are given Exercise 20 from Section 5.3.4 of TAOCP Volume 3. The exercise: Prove that (a) $\bar{V}_3(5) = 7$; (b) $\bar{U}_4(n) \le 3n - 10$ for $n \ge 6$. First, we need to understand the notation. The preceding...
TAOCP 5.3.4 Exercise 21
Section 5.3.4: Networks for Sorting Exercise 21. [ 21 ] True or false: Inserting a new standard comparator into any standard sorting network yields another standard sorting network. Verified: no Solve time: 8m23s Solution False. Inserting a comparator into a sorting network can destroy the sorting property. A counterexample with (n=4) is given below. Base sorting network Consider the following 6‑comparator network (N) for four inputs: [ (1,3),\quad (1,2),\quad (3,4),\quad...
TAOCP 5.3.4 Exercise 22
Section 5.3.4: Networks for Sorting Exercise 22. [ M17 ] Let $\alpha$ be any $n$-network, and let $x$ and $y$ be $n$-vectors. a) Prove that $x \subseteq y$ implies that $x\alpha \subseteq y\alpha$. b) Prove that $x \cdot y \le (x\alpha) \cdot (y\alpha)$, where $x \cdot y$ denotes the dot product $x_1y_1 + \cdots + x_ny_n$. Verified: no Solve time: 1m06s Solution Let an $n$-network $\alpha$ be a finite sequence...
TAOCP 5.3.4 Exercise 16
Section 5.3.4: Networks for Sorting Exercise 16. [ M22 ] Prove that the following algorithm transforms any sorting network $[i_1:j_1]\ldots[i_r:j_r]$ into a standard sorting network of the same length: T1. Let $q$ be the smallest index such that $i_q > j_q$. If no such index exists, stop. T2. Change all occurrences of $i_q$ to $j_q$, and all occurrences of $j_q$ to $i_q$, in all comparators $[i_s:j_s]$ for $q \le s...
TAOCP 5.3.4 Exercise 18
Section 5.3.4: Networks for Sorting Exercise 18. ▶ [ M20 ] Prove that a network that finds the median of $2t - 1$ elements requires at least $(t-1)\bigl(\lceil\lg(t+1)\rceil + \lceil\lg t\rceil\bigr)$ comparator modules. [ Hint : See the proof of Theorem A.] Verified: no Solve time: 10m24s Solution We prove the lower bound by applying the zero‑one principle and an adversary argument that extends the proof of Theorem A in...
TAOCP 5.3.4 Exercise 14
Section 5.3.4: Networks for Sorting Exercise 14. ▶ [ M27 ] (V. E. Alekseev.) Let $\alpha = [i_1: j_1] \ldots [i_r: j_r]$ be an $n$-network; for $1 \le s \le r$ we define $\alpha^s = [i' 1: j' 1] \ldots [i' {s-1}: j' {s-1}][i_s: j_s] \ldots [i_r: j_r]$, where the $i'_k$ and $j'_k$ are obtained from $i_k$ and $j_k$ by changing $i_s$ to $j_s$ and changing $j_s$ to $i_s$, wherever...
TAOCP 5.3.4 Exercise 15
Section 5.3.4: Networks for Sorting Exercise 15. [ 20 ] Find a nonstandard sorting network for four elements that has only five comparator modules. Verified: no Solve time: 12m15s Solution A nonstandard sorting network for four elements using only five comparator modules is [ \boxed{[1:3],;[1:4],;[2:3],;[2:4],;[1:2]}. ] Here ([i:j]) denotes a compare‑exchange module that sorts the values on lines (i) and (j) (with (i<j)) into nondecreasing order. **Verification by the zero‑one...
TAOCP 5.3.4 Exercise 17
Section 5.3.4: Networks for Sorting Exercise 17. [ M25 ] Let $D_{tn}$ be the set of all $\binom{n}{2}$ sequences $\langle x_1,\ldots,x_n\rangle$ of 0s and 1s having exactly $t$ 1s. Show that $\bar{U} t(n)$ is the minimum number of comparators needed in a network that sorts all the elements of $D {tn}$; $\bar{V} t(n)$ is the minimum number needed to sort $D {tn} \cup D_{(t-1)n}$; and $\bar{W} t(n)$ is the minimum...
TAOCP 5.3.4 Exercise 13
Section 5.3.4: Networks for Sorting Exercise 13. ▶ [ 24 ] (H. S. Stone.) Show that a sorting network for $2^t$ elements can be constructed by following the pattern illustrated for $t = 4$ in Fig. 57. Each of the $t^2$ steps in this scheme consists of a "perfect shuffle" of the first $2^{t-1}$ elements with the last $2^{t-1}$, followed by simultaneous operations performed on $2^{t-1}$ pairs of adjacent elements....
TAOCP 5.3.4 Exercise 10
Section 5.3.4: Networks for Sorting Exercise 10. [ M22 ] Prove that Batcher's bitonic sorter, as defined in the remarks preceding (15), is valid. [ Hint : It is only necessary to prove that all sequences consisting of $k$ 1s followed by $l$ 0s followed by $n - k - l$ 1s will be sorted.] Verified: no Solve time: 15m Solution We first recall the definitions of Batcher's bitonic sorter...
TAOCP 5.3.4 Exercise 8
Section 5.3.4: Networks for Sorting Exercise 8. [ M20 ] Prove that $\hat{M}(m_1+n_2, n_1+n_2) \ge \hat{M}(m_1, n_1) + \hat{M}(m_2, n_2) + \min(m_1, n_2)$, for $m_1, m_2, n_1, n_2 \ge 0$. Verified: no Solve time: 16m13s Solution We prove the inequality [ \hat{M}(m_1+m_2,, n_1+n_2) ;\ge; \hat{M}(m_1,n_1) + \hat{M}(m_2,n_2) + \min(m_1,n_2) \qquad (m_1,m_2,n_1,n_2 \ge 0), ] which is the intended form of the exercise (the original statement appears to contain a typographical...
TAOCP 5.3.4 Exercise 11
Section 5.3.4: Networks for Sorting Exercise 11. [ M23 ] Prove that Batcher's bitonic sorter of order $2^t$ will not only sort sequences $\langle z_0, z_1, \ldots, z_{2^t-1}\rangle$ for which $z_0 \ge \cdots \ge z_k \le \cdots \le z_{2^t-1}$, it also will sort any sequence for which $z_0 \le \cdots \le z_k \ge \cdots \ge z_{2^t-1}$. [As a consequence, the network in Fig. 56 will sort 16 elements, since each...
TAOCP 5.3.4 Exercise 12
Section 5.3.4: Networks for Sorting Exercise 12. [ M20 ] Prove or disprove: If $x$ and $y$ are bitonic sequences of the same length, so are $x \vee y$ and $x \wedge y$. Verified: no Solve time: 1m45s Solution We disprove the statement by providing a counterexample. Let (x = (0, 3, 2, 1)) and (y = (2, 1, 0, 3)). Both sequences have length (4). Bitonic definition. A sequence...
TAOCP 5.3.4 Exercise 9
Section 5.3.4: Networks for Sorting Exercise 9. [ M25 ] (R. W. Floyd.) Prove that $\hat{M}(3,3) = 6$, $\hat{M}(4,4) = 9$, $\hat{M}(5,5) = 13$. Verified: no Solve time: 10m50s Solution Zero-one principle A comparator network merges two sorted sequences of lengths (m) and (n) correctly for all inputs if and only if it does so for all (0)-(1) inputs where the first sequence consists of (k) zeros followed by (m-k)...
TAOCP 5.3.4 Exercise 7
Section 5.3.4: Networks for Sorting Exercise 7. [ 25 ] Show that if the final three comparators of the network for $n = 10$ in Fig. 49 are replaced by the "weaker" sequence [5:6][4:5][6:7], the network will still sort. Verified: no Solve time: 8m55s Solution The network for (n = 10) shown in Fig. 49 consists of the following 29 comparators arranged in 9 stages (delay 9): [ \begin{aligned} \text{Stage...
TAOCP 5.3.3 Exercise 27
Section 5.3.3: Minimum-Comparison Selection Exercise 27. ▶ [ M34 ] A randomized adversary is an adversary algorithm that is allowed to flip coins as it makes decisions. a) Let $A$ be a randomized adversary and let $\Pr(l)$ be the probability that $A$ reaches leaf $l$ of a given comparison tree. Show that if $\Pr(l) \le p$ for all $l$, the height of the comparison tree is $\ge \lg(1/p)$. b) Consider...
TAOCP 5.3.3 Exercise 24
Section 5.3.3: Minimum-Comparison Selection Exercise 24. [ HM40 ] Prove that $\overline{V}_t(n) \le n + t + O(\sqrt{n \log n})$ for $t \le \lceil n/2 \rceil$. Hint: Show that with this many comparisons we can in fact find both the $\lfloor t - \sqrt{t \ln n} \rfloor$th and $\lceil t + \sqrt{t \ln n} \rceil$th elements, after which the $t$th is easily located. Verified: no Solve time: 20m25s We need...
TAOCP 5.3.3 Exercise 26
Section 5.3.3: Minimum-Comparison Selection Exercise 26. [ M32 ] (A. Schönhage, 1974.) (a) In the notation of exercise 14, prove that $U_t(n) \ge \min(2 + U_t(n-1), 2 + U_{t-1}(n-1))$ for $n \ge 3$. [ Hint: Construct an adversary by reducing from $n$ to $n - 1$ as soon as the current partial ordering is not composed entirely of components having the form $\circ$ or $\leftarrow\bullet\rightarrow$.] (b) Similarly, prove that $$U_t(n)...
TAOCP 5.3.3 Exercise 25
Section 5.3.3: Minimum-Comparison Selection Exercise 25. ▶ [ M35 ] (W. Cunto and J. I. Munro.) Prove that $\overline{V}_t(n) \ge n + t - 2$ when $t \le \lceil n/2 \rceil$. Verified: no Solve time: 16m09s We need to prove that the minimum number of comparisons needed to find the t-th largest element (selection) satisfies V_t(n) >= n + t - 2 for t <= ceil(n/2). The problem is from...
TAOCP 5.3.3 Exercise 22
Section 5.3.3: Minimum-Comparison Selection Exercise 22. [ 24 ] (David G. Kirkpatrick.) Show that when $4 \cdot 2^k < n - 1 \le 5 \cdot 2^k$, the upper bound (11) for $V_3(n)$ can be reduced by 1 as follows: (i) Form four knockout trees of size $2^k$. (ii) Find the minimum of the four maxima, and discard all $2^k$ elements of its tree. (iii) Using the known information, build a...
TAOCP 5.3.3 Exercise 21
Section 5.3.3: Minimum-Comparison Selection Exercise 21. [ 32 ] (P. V. Ramanan and L. Hyafil.) Prove that $W_t(2^k + 2^{k+1-t}) \le 2^k + 2^{k+1-t} + (t-1)(k-1)$, when $k \ge t \ge 2$; also show that equality holds for infinitely many $k$ and $t$, because of exercise 4. [ Hint: Maintain two knockout trees and merge their results cleverly.] Verified: no Solve time: 17m31s Setup We are given integers (k \ge...
TAOCP 5.3.3 Exercise 20
Section 5.3.3: Minimum-Comparison Selection Exercise 20. [ M42 ] What is the asymptotic value of $\overline{V}_2(n) - n$, as $n \to \infty$? Verified: no Solve time: 20m17s Setup We consider the problem of finding the second largest element among (n) distinct elements using comparisons. The optimal worst-case algorithm is the tournament method (tree selection), which first finds the largest element in (n-1) comparisons, then finds the second largest among the...
TAOCP 5.3.3 Exercise 23
Section 5.3.3: Minimum-Comparison Selection Exercise 23. [ M42 ] What is the asymptotic value of $V_{\lceil n/2 \rceil}(n)$, as $n \to \infty$? Verified: no Solve time: 12m31s Setup We are asked for the asymptotic value of (V_{\lceil n/2 \rceil}(n)) as (n\to\infty), where (V_t(n)) is the minimum number of comparisons required in the worst case to determine the (t)‑th largest element of a set of (n) elements. The median is the...
TAOCP 5.3.3 Exercise 19
Section 5.3.3: Minimum-Comparison Selection Exercise 19. [ M42 ] Extend Table 2 to $n = 9$. Verified: no Solve time: 4m38s Setup Let (V_t(n)) denote the minimum number of comparisons required to select the (t)‑th largest element from a set of (n) elements, as defined in Section 5.3.3. Table 2 of the text lists the exact values of (V_t(n)) for (1 \le n \le 8). The exercise asks to extend...
TAOCP 5.3.3 Exercise 17
Section 5.3.3: Minimum-Comparison Selection Exercise 17. [ 20 ] (R. W. Floyd.) Show that it is possible to select, in order, both the $k$ largest and the $l$ smallest elements of a set of $n$ elements, using at most $\lceil \frac{3}{2}n \rceil - k - l + \sum_{n+1-k \le j \le n} \lceil \lg j \rceil + \sum_{n+1-l \le j \le n} \lceil \lg j \rceil$ comparisons. Verified: no Solve...
TAOCP 5.3.3 Exercise 18
Section 5.3.3: Minimum-Comparison Selection Exercise 18. [ M20 ] If groups of size 5, not 7, had been used in the proof of Theorem L, what theorem would have been obtained? Verified: no Solve time: 6m20s Solution The median-of-medians algorithm (Blum, Floyd, Pratt, Rivest, Tarjan) finds the (t)th largest element by recursively using a pivot obtained as the median of medians of small groups. The proof of Theorem L in...
TAOCP 5.3.1 Exercise 34
Section 5.3.1: Minimum-Comparison Sorting Exercise 34. [ M50 ] Determine the exact value of $S(n)$ for infinitely many $n$. Verified: no Solve time: 20m40s We need to solve Exercise 34 from TAOCP Volume 3, Section 5.3.1: "Determine the exact value of S(n) for infinitely many n." S(n) is the minimum number of comparisons required to sort n elements in the worst case. The exercise is marked M50 (research level). The...
TAOCP 5.3.1 Exercise 35
Section 5.3.1: Minimum-Comparison Sorting Exercise 35. [ 29 ] Determine the exact value of $S(16)$. Verified: no Solve time: 15m31s Setup We need to determine (S(16)), the minimum number of comparisons required to sort 16 distinct elements in the worst case. The information‑theoretic lower bound gives [ S(16) \ge \lceil \lg 16! \rceil. ] Since (16! = 20,922,789,888,000) and (2^{44} = 17,592,186,044,416 < 16! < 35,184,372,088,832 = 2^{45}), we have...
TAOCP 5.3.2 Exercise 13
Section 5.3.2: Minimum-Comparison Merging Exercise 13. [ M42 ] (R. L. Graham.) Show that the solution to the recurrence in exercise 12 may be expressed as follows. Define the function $G(x)$, for $0 < x < \infty$, by the rules $$G(x) = \begin{cases} 1, & \text{if } 0 < x \le \frac{5}{7}; \ \frac{1}{2} + \frac{7}{8} G(8x - 5), & \text{if } \frac{5}{7} < x \le \frac{3}{4}; \ \frac{7}{2} G(2x...
TAOCP 5.3.1 Exercise 33
Section 5.3.1: Minimum-Comparison Sorting Exercise 33. [ M30 ] (T. N. Hibbard.) A real-valued search tree of order $x$ and resolution $\delta$ is an extended binary tree in which all nodes contain a nonnegative real value such that (i) the value in each external node is $\le \delta$, (ii) the value in each internal node is at most the sum of the values in its two children, and (iii) the...
TAOCP 5.3.1 Exercise 36
Section 5.3.1: Minimum-Comparison Sorting Exercise 36. [ M50 ] (S. S. Kislitsyn, 1968.) Prove or disprove: Any directed acyclic graph $G$ with $T(G) > 1$ has two vertices $u$ and $v$ such that the digraphs $G_1$ and $G_2$ obtained from $G$ by adding the arcs $u \leftarrow v$ and $u \to v$ are acyclic and satisfy $1 \le T(G_1)/T(G_2) \le 2$. (Thus $T(G_1)/T(G)$ always lies between $\frac{1}{3}$ and $\frac{2}{3}$, for...
TAOCP 5.3.1 Exercise 31
Section 5.3.1: Minimum-Comparison Sorting Exercise 31. [ M38 ] Continuing exercise 30, prove that $S_e(5) = 8$. Verified: no Solve time: 11m50s We need to solve exercise 31 from Volume 3, Section 5.3.1: "Continuing exercise 30, prove that S_e(5) = 8." First, understand the context. Exercise 30 defines S_e(n) as the minimum number of comparison-exchanges needed in the worst case to sort n elements by means of a comparison-exchange tree....
TAOCP 5.3.1 Exercise 32
Section 5.3.1: Minimum-Comparison Sorting Exercise 32. [ M42 ] Continuing exercise 31, investigate $S_e(n)$ for small values of $n > 5$. Verified: no Solve time: 4m51s Setup Let (S_e(n)) denote the minimum worst-case number of comparison-exchanges required to sort (n) distinct elements, as defined in Exercise 30. We already know from Exercises 30 and 31 that [ S_e(n) \le S(n) + n - 1, \qquad S_e(1)=0,; S_e(2)=1,; S_e(3)=3,; S_e(4)=5,; S_e(5)=8....
TAOCP 5.3.1 Exercise 30
Section 5.3.1: Minimum-Comparison Sorting Exercise 30. [ M29 ] ( Optimum exchange sorting. ) Every exchange sorting algorithm as defined in Section 5.2.2 can be represented as a comparison-exchange tree , namely a binary tree structure whose internal nodes have the form $i:j$ for $i < j$, interpreted as the following operation: "If $K_i \le K_j$, continue by taking the left branch of the tree; if $K_i > K_j$, continue...
TAOCP 5.3.1 Exercise 29
Section 5.3.1: Minimum-Comparison Sorting Exercise 29. [ M25 ] (S. M. Chase.) Let $a_1 a_2 \ldots a_n$ be a permutation of ${1, 2, \ldots, n}$. Prove that any algorithm that decides whether this permutation is even or odd (that is, whether it has an even or odd number of inversions), based solely on comparisons between the $a$'s, must make at least $n \lg n$ comparisons, even though the algorithm has...
TAOCP 5.3.1 Exercise 26
Section 5.3.1: Minimum-Comparison Sorting Exercise 26. [ M42 ] Find a sorting procedure for seven elements that minimizes the average number of comparisons performed. Verified: no Solve time: 16m45s Setup We consider the problem of sorting 7 distinct elements using only binary comparisons. A sorting procedure corresponds to an extended binary decision tree whose internal nodes are comparisons between two elements and whose 5040 external nodes (leaves) are the permutations...
TAOCP 5.3.1 Exercise 24
Section 5.3.1: Minimum-Comparison Sorting Exercise 24. [ 27 ] (C. Picard.) Find a sorting tree for six elements such that all external nodes appear on levels 10 and 11. Verified: no Solve time: 18m21s Setup We are asked to find a comparison tree (sorting tree) for six distinct elements such that all 720 external nodes (leaves) appear on levels 10 and 11. The text defines the root at level 0...
TAOCP 5.3.1 Exercise 27
Section 5.3.1: Minimum-Comparison Sorting Exercise 27. ▶ [ 20 ] Suppose it is known that the configurations $K_1 < K_2 < K_3$, $K_1 < K_3 < K_2$, $K_2 < K_1 < K_3$, $K_2 < K_3 < K_1$, $K_3 < K_1 < K_2$, $K_3 < K_2 < K_1$ occur with respective probabilities .01, .25, .01, .24, .25, .24. Find a comparison tree that sorts these three elements with the smallest average...
TAOCP 5.3.1 Exercise 22
Section 5.3.1: Minimum-Comparison Sorting Exercise 22. [ M24 ] Continuing exercise 21, prove that a binary tree has minimum external path length among all binary trees with the same number of nodes if and only if $$|t(x) - 2t(l(x))| \le 2^{\lceil \lg t(x) \rceil} - t(x) \quad \text{and} \quad |t(x) - 2t(l(x))| \le t(x) - 2^{\lfloor \lg t(x) \rfloor}$$ for all internal nodes $x$. [Thus, for example, if $t(x) =...
TAOCP 5.3.1 Exercise 14
Section 5.3.1: Minimum-Comparison Sorting Exercise 14. [ M24 ] Find a closed form for the sum (19). Verified: no Solve time: 44m16s Solution The sum (19) in the text is the total number of comparisons required by merge insertion to sort (n) elements, which is [ S(n) = \sum_{k=1}^n \left\lceil \log_2 \frac{3k}{4} \right\rceil. ] We derive a closed form for this sum. For an integer (m \ge 0), the value...
TAOCP 5.3.1 Exercise 25
Section 5.3.1: Minimum-Comparison Sorting Exercise 25. [ 11 ] If there were a sorting procedure for seven elements that achieves the minimum average number of comparisons predicted by the use of Eq. (34), how many external nodes would there be on level 13? Verified: no Solve time: 1m50s Solution The minimum average number of comparisons for sorting (n) distinct elements is achieved by a comparison tree (a full binary tree)...
TAOCP 5.3.1 Exercise 23
Section 5.3.1: Minimum-Comparison Sorting Exercise 23. [ 10 ] The text proves that the average number of comparisons made by any sorting method for $n$ elements must be at least $\lceil \lg n! \rceil \approx n \lg n$. But multiple list insertion (Program 5.2.1M) takes only $O(n)$ units of time on the average. How can this be? Verified: no Solve time: 52s The information-theoretic lower bound (\lceil \lg n! \rceil)...
TAOCP 5.3.1 Exercise 21
Section 5.3.1: Minimum-Comparison Sorting Exercise 21. [ M21 ] The height of an extended binary tree is the maximum level number of its external nodes. If $x$ is an internal node of an extended binary tree, let $t(x)$ be the number of external nodes below $x$, and let $l(x)$ denote the root of $x$'s left subtree. If $x$ is an external node, let $t(x) = 1$. Prove that an extended...
TAOCP 5.3.1 Exercise 20
Section 5.3.1: Minimum-Comparison Sorting Exercise 20. ▶ [ M26 ] Prove that an extended binary tree has minimum external path length if and only if there is a number $l$ such that all external nodes appear on levels $l$ and $l+1$ (or perhaps all on a single level $l$). Verified: no Solve time: 7m08s Setup An extended binary tree is a finite rooted tree in which every internal node has...
TAOCP 5.3.1 Exercise 16
Section 5.3.1: Minimum-Comparison Sorting Exercise 16. [ HM26 ] (F. Hwang and S. Lin.) Prove that $F(n) > \lceil \lg n! \rceil$ for $n \ge 22$. Verified: no Solve time: 22m58s Setup Let (F(n)) denote the number of comparisons performed by the merge‑insertion (Ford-Johnson) algorithm when sorting (n) distinct elements. From the description in the text (Section 5.3.1) one can derive the explicit formula [ F(n)=\sum_{k=1}^{n}\Bigl\lceil \lg\frac{3k}{4}\Bigr\rceil \qquad (n\ge 1),...
TAOCP 5.3.1 Exercise 19
Section 5.3.1: Minimum-Comparison Sorting Exercise 19. [ 40 ] Experiment with the following heuristic rule for deciding which pair of elements to compare next while designing a comparison tree: At each stage of sorting ${K_1, \ldots, K_n}$, let $u_i$ be the number of keys known to be $\le K_i$ as a result of the comparisons made so far, and let $v_i$ be the number of keys known to be $\ge...
TAOCP 5.3.1 Exercise 18
Section 5.3.1: Minimum-Comparison Sorting Exercise 18. [ 20 ] If the procedure whose first steps are shown in Fig. 36 had produced the linear graph •, •, •, •, •, •, •, •, •, •, •, • with efficiency $12!/2^{29}$, would this have proved that $S(12) = 29$? Verified: no Solve time: 4m51s Solution The efficiency of a comparison tree that sorts (n) elements (with no redundant comparisons) is defined...
TAOCP 5.3.1 Exercise 11
Section 5.3.1: Minimum-Comparison Sorting Exercise 11. [ HM22 ] When sorting $n$ elements as in exercise 5, and knowing that all keys are members of the set ${1, 2, \ldots, m}$, let $S_m(n)$ be the minimum number of comparisons needed in the worst case. [Thus by exercise 6, $S_n(n) = S(n)$.] Prove that, for fixed $m$, $S_m(n)$ is asymptotically $n \lg m + O(1)$ as $n \to \infty$. Verified: no...
TAOCP 5.3.1 Exercise 15
Section 5.3.1: Minimum-Comparison Sorting Exercise 15. [ M21 ] Determine the asymptotic behavior of $B(n)$ and $F(n)$ up to $O(\log n)$. [ Hint: Show that in both cases the coefficient of $n$ involves the function shown in Fig. 37.] Verified: no Solve time: 13m12s Solution 1. Binary insertion (B(n)) The maximum number of comparisons for binary insertion is given in the text (Eq. (3)) as [ B(n) = \sum_{k=1}^{n}\lceil\lg k\rceil...
TAOCP 5.3.1 Exercise 13
Section 5.3.1: Minimum-Comparison Sorting Exercise 13. [ M28 ] Prove (17). Verified: no Solve time: 15m49s Setup We consider the merge insertion (Ford-Johnson) algorithm as described in the text. For (n) elements the algorithm proceeds as follows: If (n \le 1), no comparisons are needed. Let (m = \lfloor n/2\rfloor). Form (m) disjoint pairs and compare each pair. This uses (m) comparisons. The larger element of each pair is called...
TAOCP 5.2.5 Exercise 14
Section 5.2.5: Sorting by Distribution Exercise 14. [ 22 ] The sequence of playing cards can be sorted into increasing order A 2 ... J Q K from top to bottom in two passes, using just two piles for intermediate storage: Deal the cards face down into two piles containing respectively A 2 9 3 10 and 4 J 5 6 Q K 7 8 (from bottom to top); then...
TAOCP 5.3.1 Exercise 12
Section 5.3.1: Minimum-Comparison Sorting Exercise 12. ▶ [ M25 ] (W. G. Bouricius, circa 1954.) Suppose that equal keys may occur, but we merely want to sort the elements ${K_1, K_2, \ldots, K_n}$ so that a permutation $a_1 a_2 \ldots a_n$ is determined with $K_{a_1} \le K_{a_2} \le \cdots \le K_{a_n}$; we do not need to know whether or not equality occurs between $K_{a_i}$ and $K_{a_{i+1}}$. Let us say that...
TAOCP 5.2.5 Exercise 15
Section 5.2.5: Sorting by Distribution Exercise 15. [ 22 ] Consider the problem of exercise 14 when all cards must be dealt face up instead of face down. Thus, one pass can be used to convert increasing order into decreasing order. How many passes are required? Verified: no Solve time: 46m52s We need to solve Exercise 15 from TAOCP Volume 3, Section 5.2.5. The exercise refers to Exercise 14. Let's...
TAOCP 5.3.1 Exercise 10
Section 5.3.1: Minimum-Comparison Sorting Exercise 10. ▶ [ M25 ] When sorting $n$ elements as in exercise 7, knowing that all keys are 0 or 1, what is the minimum average number of comparisons as a function of $n$? Verified: no Solve time: 19m50s Solution We consider the problem of sorting (n) elements whose keys are known to be either 0 or 1. A comparison between two elements (K_i) and...
TAOCP 5.3.1 Exercise 9
Section 5.3.1: Minimum-Comparison Sorting Exercise 9. [ M20 ] When sorting $n$ elements as in exercise 7, knowing that all keys are 0 or 1, what is the minimum number of comparisons in the worst case? Verified: no Solve time: 27m14s Solution We consider the problem of sorting (n) elements when it is known that each key is either (0) or (1). A comparison between two keys has three possible...
TAOCP 5.2.5 Exercise 16
Section 5.2.5: Sorting by Distribution Exercise 16. ▶ [ 25 ] Design an algorithm to sort strings $\alpha_1, \ldots, \alpha_n$ on an $m$-letter alphabet into lexicographic order. The total running time of your algorithm should be $O(m + n + N)$, where $N = |\alpha_1| + \cdots + |\alpha_n|$ is the total length of all the strings. Verified: no Solve time: 22m31s Solution Algorithm S (Radix list sort for variable‑length...
TAOCP 5.2.5 Exercise 18
Section 5.2.5: Sorting by Distribution Exercise 18. [ HM26 ] Prove Theorem T. Hint: Show first that MacLaren's distribution-plus-insertion algorithm does $O(BN)$ operations, on the average, when it is applied to independent random keys whose probability density function satisfies $f(x) \le B$ for $0 \le x \le 1$. For sorting the roots and words we had the use of 1100 lozenge boxes, and used trays for the forms. , GEORGE...
TAOCP 5.2.5 Exercise 17
Section 5.2.5: Sorting by Distribution Exercise 17. [ 15 ] In the two-level distribution sort proposed by Tamminen (see Theorem T), why is a MacLaren-like method used for the second level of distribution but not the first level? Verified: no Solve time: 1m41s Solution In Tamminen's two-level distribution sort (Theorem T), the first level partitions the $N$ records into $M$ buckets according to their most significant digit. This is done...
TAOCP 5.2.5 Exercise 13
Section 5.2.5: Sorting by Distribution Exercise 13. [ 40 ] Implement the internal sorting method suggested in the text at the close of this section, producing a subroutine that sorts random data in $O(N)$ units of time with only $O(\sqrt{N})$ additional memory locations. Verified: no Solve time: 8m40s Setup We consider the problem of sorting (N) records with keys that are two-digit numbers in base (M = \lceil \sqrt{N} \rceil)....
TAOCP 5.2.5 Exercise 10
Section 5.2.5: Sorting by Distribution Exercise 10. [ 30 ] Design an efficient most-significant-digit-first radix-sorting algorithm that uses linked memory. (As the size of the subfiles decreases, it is wise to decrease $M$, and to use a nonradix method on the really short subfiles.) Verified: no Solve time: 13m58s Setup We consider the same record structure as in Algorithm R (Section 5.2.5). There are (N) records (R_1,\dots,R_N), each containing a...
TAOCP 5.2.4 Exercise 19
Section 5.2.4: Sorting by Merging Exercise 19. [ 26 ] Consider a railway switching network with $n$ "stacks," as shown in Fig. 31 when $n = 5$; we considered one-stack networks in exercises 2.2.1–2 through 2.2.1–5. If $N$ railroad cars enter at the right, we observed that only comparatively few of the $N!$ permutations of those cars could appear at the left, in the one-stack case. In the $n$-stack network,...
TAOCP 5.2.5 Exercise 12
Section 5.2.5: Sorting by Distribution Exercise 12. [ 24 ] (M. D. MacLaren.) Suppose that Algorithm R has been applied only to the $p$ leading digits of the actual keys; thus the file is nearly sorted when we read it in the order of the links, but keys that agree in their first $p$ digits may be out of order. Design an algorithm that rearranges the records in place so...
TAOCP 5.2.4 Exercise 22
Section 5.2.4: Sorting by Merging Exercise 22. [ 28 ] (J. K. R. Barnett.) Develop a way to speed up merge sorting on multiword keys. (Exercise 5.2.2–30 considers the analogous problem for quicksort.) Verified: no Solve time: 10m27s Setup We consider the problem of sorting (N) records (R_1,\dots,R_N) whose keys are multiword : each key (K_i) consists of (W) words [ K_i = (K_i[1], K_i[2], \dots, K_i[W]), ] where (K_i[1])...
TAOCP 5.2.5 Exercise 11
Section 5.2.5: Sorting by Distribution Exercise 11. [ 16 ] The sixteen input numbers shown in Table 1 start with 41 inversions; after sorting is complete, of course, there are no inversions remaining. How many inversions would be present in the file if we omitted pass 1, doing a radix sort only on the tens and hundreds digits? How many inversions would be present if we omitted both pass 1...
TAOCP 5.2.4 Exercise 18
Section 5.2.4: Sorting by Merging Exercise 18. [ 40 ] (M. A. Kronrod.) Given a file of $N$ records containing only two runs, $$K_1 \le \cdots \le K_M \quad \text{and} \quad K_{M+1} \le \cdots \le K_N,$$ is it possible to sort the file with $O(N)$ operations in a random-access memory, using only a small fixed amount of additional memory space regardless of the sizes of $M$ and $N$? (All of...
TAOCP 5.2.4 Exercise 23
Section 5.2.4: Sorting by Merging Exercise 23. [ M30 ] Exercises 13 and 14 analyze a "bottom-up" or iterative version of merge sort, where the cost $c(N)$ of sorting $N$ items satisfies the recurrence $$c(N) = c(2^k) + c(N - 2^k) + f(2^k, N - 2^k) \quad \text{for } 2^k < N \le 2^{k+1}$$ and $f(m, n)$ is the cost of merging $m$ things with $n$. Study the "top-down" or...
TAOCP 5.2.4 Exercise 21
Section 5.2.4: Sorting by Merging Exercise 21. [ 22 ] (A. J. Smith.) Explain how to extend Algorithm L so that, in addition to sorting, it computes the number of inversions present in the input permutation. Verified: no Solve time: 4m06s Solution We modify Algorithm L (List merge sort) as follows. The algorithm uses a linked list with a dummy header node 0; LINK(i) points to the next record, and...
TAOCP 5.2.4 Exercise 20
Section 5.2.4: Sorting by Merging Exercise 20. [ 47 ] In the notation of exercise 2.2.1–4, at most $a_N^n$ permutations of $N$ elements can be produced with an $n$-stack railway network; hence the number of stacks needed to obtain all $N!$ permutations is at least $\log N!/\log a_N \approx \log_4 N$. Exercise 19 shows that at most $\lceil \lg N \rceil$ stacks are needed. What is the true rate of...
TAOCP 5.2.4 Exercise 13
Section 5.2.4: Sorting by Merging Exercise 13. ▶ [ M32 ] Give an analysis of the average running time of Program L, in the style of other analyses in this chapter: Interpret the quantities $A, B, B', \ldots,$ and explain how to compute their exact average values. How long does Program L take to sort the 16 numbers in Table 3? Verified: no Solve time: 17m43s Setup Program L is...
TAOCP 5.2.4 Exercise 16
Section 5.2.4: Sorting by Merging Exercise 16. [ 20 ] Design a list merging algorithm like Algorithm L but based on three-way merging. Verified: no Solve time: 4m56s Solution We present Algorithm L3 , a straight three-way merge sort for linked lists. It generalizes Algorithm L (which uses a binary counter) to a ternary counter, merging three runs at a time. Data structures Each record contains a KEY field and...
TAOCP 5.2.4 Exercise 17
Section 5.2.4: Sorting by Merging Exercise 17. [ 20 ] (J. McCarthy.) Let the binary representation of $N$ be as in exercise 14, and assume that we are given $N$ records arranged in $t$ ordered subfiles of respective sizes $2^{e_1}, 2^{e_2}, \ldots, 2^{e_t}$. Show how to maintain this state of affairs when a new $(N+1)$st record is added and $N \leftarrow N+1$. (The resulting algorithm may be called an online...
TAOCP 5.2.3 Exercise 34
Section 5.2.3: Sorting by Selection Exercise 34. [ M41 ] How many leftist trees with $N$ nodes are possible, ignoring the KEY values? The sequence begins 1, 1, 2, 4, 8, 17, 38, 87, 203, 482, 1160, $\ldots$; show that the number is asymptotically $a b^N N^{-3/2}$ for suitable constants $a$ and $b$, using techniques like those of exercise 2.3.4.4-4. Verified: no Solve time: 49m33s We need to solve exercise...
TAOCP 5.2.4 Exercise 15
Section 5.2.4: Sorting by Merging Exercise 15. [ 20 ] Hand simulation of Algorithm L reveals that it occasionally does redundant operations; the assignments $|L_s| \leftarrow p$, $|L_s| \leftarrow q$ in steps L4 and L6 are unnecessary about half of the time, since we have $L_s = p$ (or $q$) each time step L4 (or L6) returns to L3. How can Program L be improved so that this redundancy disappears?...
TAOCP 5.2.4 Exercise 14
Section 5.2.4: Sorting by Merging Exercise 14. [ M24 ] Let the binary representation of $N$ be $2^{e_1} + 2^{e_2} + \cdots + 2^{e_t}$, where $e_1 > e_2 > \cdots > e_t \ge 0,\ t \ge 1$. Prove that the maximum number of key comparisons performed by Algorithm L is $1 - 2^{e_t} + \sum_{k=1}^{t} (e_k + k - 1) 2^{e_k}$. Verified: no Solve time: 7m45s Solution Algorithm L is...
TAOCP 5.2.4 Exercise 12
Section 5.2.4: Sorting by Merging Exercise 12. ▶ [ 22 ] Revise step L1 of Algorithm L so that the two-way merge is "natural," taking advantage of ascending runs that are initially present. (In particular, if the input is already sorted, step L2 should terminate the algorithm immediately after your step L1 has acted.) Verified: no Solve time: 8m13s Solution We revise step L1 of Algorithm L (List merge sort)...
TAOCP 5.2.4 Exercise 9
Section 5.2.4: Sorting by Merging Exercise 9. [ 24 ] Write a MIX program for Algorithm S. Specify the instruction frequencies in terms of quantities analogous to $A, B', B'', C', \ldots$ in Program L. Verified: no Solve time: 12m38s Solution MIX Program for Algorithm S (Straight Two-Way Merge Sort) Register assignment rI1 = i (left source pointer) rI2 = j (right source pointer) rI3 = k (destination pointer) rI5...
TAOCP 5.2.4 Exercise 10
Section 5.2.4: Sorting by Merging Exercise 10. [ 25 ] (D. A. Bell.) Show that sequentially allocated straight two-way merging can be done with at most $\tfrac{3}{2}N$ memory locations, instead of $2N$ as in Algorithm S. Verified: no Solve time: 4m18s Solution We show that straight two-way merge sort (Algorithm S) can be implemented using a single sequentially allocated memory area of size (N + \lceil N/2\rceil \le \lceil 3N/2\rceil)....
TAOCP 5.2.4 Exercise 11
Section 5.2.4: Sorting by Merging Exercise 11. [ 21 ] Is Algorithm L a stable sorting method? Verified: no Solve time: 2m14s Solution Algorithm L (List merge sort) is a stable sorting method. Justification. The algorithm operates on a linked list of records. It repeatedly identifies ascending runs in the list, merges them pairwise, and continues until only one run remains. A run is a maximal contiguous subsequence in which...
TAOCP 5.2.3 Exercise 37
Section 5.2.3: Sorting by Selection Exercise 37. [ HM32 ] Let $e_N(k)$ be the expected treewise distance of the $k$th-largest element from the root, in a random heap of $N$ elements, and let $e(k) = \lim_{N \to \infty} e_N(k)$. Thus $e(1) = 0$, $e(2) = 1$, $e(3) = 1.5$, and $e(4) = 1.875$. Find the asymptotic value of $e(k)$ to within $O(k^{-1})$. Verified: no Solve time: 16m56s We need to...
TAOCP 5.2.3 Exercise 38
Section 5.2.3: Sorting by Selection Exercise 38. [ M21 ] Find a simple recurrence relation for the multiset $M_N$ of subtree sizes in a heap or in a complete binary tree with $N$ internal nodes. 5.2.4. Sorting by Merging Merging (or collating ) means the combination of two or more ordered files into a single ordered file. For example, we can merge the two files 503 703 765 and 087...
TAOCP 5.2.3 Exercise 24
Section 5.2.3: Sorting by Selection Exercise 24. [ M32 ] Derive an exact formula for the standard deviation of $B_N$ (the total length of scan during the heap-creation phase of Algorithm H). Verified: no Solve time: 1h15m We need to derive an exact formula for the standard deviation of B_N, the total length of scan during the heap-creation phase of Algorithm H (Heapsort). The exercise is from TAOCP Vol. 3,...
TAOCP 5.2.3 Exercise 32
Section 5.2.3: Sorting by Selection Exercise 32. [ M33 ] Prove that the number of heapsort promotions, $B$, is always at least $\frac{1}{2} N \lg N + O(N)$, if the keys being sorted are distinct. Hint: Consider the movement of the largest $\lceil N/2 \rceil$ keys. Verified: no Solve time: 25m47s We need to solve Exercise 32 from Section 5.2.3 of TAOCP Volume 3. The exercise: "Prove that the number...
TAOCP 5.2.3 Exercise 36
Section 5.2.3: Sorting by Selection Exercise 36. [ 18 ] ( Least-recently-used page replacement .) Many operating systems make use of the following type of algorithm: A collection of nodes is subjected to two operations, (i) "using" a node, and (ii) replacing the least-recently-used node by a new node. What data structure makes it easy to ascertain the least-recently-used node? Verified: no Solve time: 17s Solution The required data structure...
TAOCP 5.2.3 Exercise 35
Section 5.2.3: Sorting by Selection Exercise 35. [ 26 ] If UP links are added to a leftist tree (see the discussion of triply linked trees in Section 6.2.3), it is possible to delete an arbitrary node P from within the priority queue as follows: Replace P by the merger of LEFT(P) and RIGHT(P); then adjust the DIST fields of P's ancestors, possibly swapping left and right subtrees, until either...
TAOCP 5.2.3 Exercise 30
Section 5.2.3: Sorting by Selection Exercise 30. ▶ [ HM25 ] (R. Schaffer and R. Sedgewick.) Let $h_{nm}$ be the number of heaps on the elements ${1, 2, \ldots, n}$ for which the selection phase of heapsort does exactly $m$ promotions. Prove that $h_{nm} \le 2^m \prod_{k=2}^n \lg k$, and use this relation to show that the average number of promotions performed by Algorithm H is $N \lg N +...
TAOCP 5.2.3 Exercise 33
Section 5.2.3: Sorting by Selection Exercise 33. [ 21 ] Design an algorithm that merges two disjoint priority queues, represented as leftist trees, into one. (In particular, if one of the given queues contains a single element, your algorithm will insert it into the other queue.) Verified: no Solve time: 54s Solution A leftist tree is a binary tree in which each node contains a key and a null path...
TAOCP 5.2.3 Exercise 31
Section 5.2.3: Sorting by Selection Exercise 31. [ 37 ] (J. W. J. Williams.) Show that if two heaps are placed "back to back" in a suitable way, it is possible to maintain a structure in which either the smallest or the largest element can be deleted at any time in $O(\log n)$ steps. (Such a structure may be called a priority deque .) Verified: no Solve time: 8m57s Setup...
TAOCP 5.2.3 Exercise 25
Section 5.2.3: Sorting by Selection Exercise 25. [ M20 ] What is the average value of the contribution to $C$ made during the siftup pass when $l = 1$ and $r = N$, if $N = 2^{n+1} - 1$? Verified: no Solve time: 25m10s We need to solve Exercise 25 from TAOCP Volume 3, Section 5.2.3. The exercise: 25. [ M20 ] What is the average value of the contribution...
TAOCP 5.2.3 Exercise 29
Section 5.2.3: Sorting by Selection Exercise 29. [ 26 ] (W. S. Brown.) Design an algorithm for multiplication of polynomials or power series $(a_1 x^{r_1} + a_2 x^{r_2} + \cdots)(b_1 x^{s_1} + b_2 x^{s_2} + \cdots)$, in which the coefficients of the answer $c_1 x^{r_1 + s_1} + \cdots$ are generated in order as the input coefficients are being multiplied. [ Hint: Use an appropriate priority queue.] Verified: no Solve...
TAOCP 5.2.3 Exercise 28
Section 5.2.3: Sorting by Selection Exercise 28. [ 35 ] Explore the idea of ternary heaps , based on complete ternary trees instead of binary trees. Do ternary heaps sort faster than binary heaps? Verified: no Solve time: 13m24s Setup A ternary heap is a complete ternary tree stored in an array A[1..N] . The children of node i are at positions [ 3i-1,\quad 3i,\quad 3i+1 \qquad (\text{provided they are...
TAOCP 5.2.3 Exercise 27
Section 5.2.3: Sorting by Selection Exercise 27. [ M25 ] (T. Clausen, 1828.) Prove that $$\sum_{n \ge 1} \frac{x^n}{1 - x^n} = \sum_{n \ge 1} \frac{1 + x^n}{1 - x^n} x^{n^2}.$$ (Setting $x = \frac{1}{2}$ gives a very rapidly converging series for the evaluation of (19).) Verified: no Solve time: 57s Solution The identity is an equality of formal power series (or absolutely convergent series for (|x|<1)). We prove it...
TAOCP 5.2.2 Exercise 52
Section 5.2.2: Sorting by Exchanging Exercise 52. [ HM35 ] (N. G. de Bruijn.) What is the asymptotic behavior of the sum $$S_n = \sum_{t \ge 1} \binom{2n}{n+t} d(t),$$ where $d(t)$ is the number of divisors of $t$? (Thus, $d(1) = 1$, $d(2) = d(3) = 2$, $d(4) = 3$, $d(5) = 2$, etc.) (This problem arose in connection with the analysis of a tree traversal algorithm, exercise 2.3.1–11.) Find...
TAOCP 5.2.3 Exercise 21
Section 5.2.3: Sorting by Selection Exercise 21. [ M24 ] Prove that (15) gives the nonspecial subtree sizes in a heap. Verified: no Solve time: 25m31s Solution We first recall the relevant definitions from the text. A heap of (N) elements is a complete binary tree with nodes numbered (1,2,\ldots,N) satisfying the heap property (K_{\lfloor j/2\rfloor} \ge K_j). A special subtree is a subtree whose root has two children (i.e.,...
TAOCP 5.2.3 Exercise 23
Section 5.2.3: Sorting by Selection Exercise 23. [ M28 ] (a) Prove that the length of scan, $B$, in a siftup algorithm never exceeds $\lfloor \lg(r/l) \rfloor$. (b) According to (8), $B$ can never exceed $N \lfloor \lg N \rfloor$ in any particular application of Algorithm H. Find the maximum value of $B$ as a function of $N$, taken over all possible input files. (You must prove that an input...
TAOCP 5.2.3 Exercise 22
Section 5.2.3: Sorting by Selection Exercise 22. ▶ [ 20 ] What permutations of ${1, 2, 3, 4, 5}$ are transformed into 5 3 4 1 2 by the heapcreation phase of Algorithm H? Verified: no Solve time: 5m54s Solution The heapcreation phase of Algorithm H (heapsort) for (N = 5) builds a heap by applying the sift-down operation first at node (2) and then at node (1). The final...
TAOCP 5.2.3 Exercise 20
Section 5.2.3: Sorting by Selection Exercise 20. [ M20 ] Prove that (14) gives the special subtree sizes in a heap. Verified: no Solve time: 11m52s Solution We consider a heap of size (N) stored in an array (1,2,\ldots,N) as a complete binary tree: the root is at index (1); for any node (k), its left child is (2k) and its right child is (2k+1) (provided these indices do not...
TAOCP 5.2.2 Exercise 56
Section 5.2.2: Sorting by Exchanging Exercise 56. [ M19 ] Analyze the average behavior of the quantities that occur in the running time of Algorithm Q when the program has been modified to take the median of three elements as in exercise 55. (See exercise 29.) Verified: no Solve time: 25m52s We need to solve exercise 56 from TAOCP Volume 3, Section 5.2.2. The exercise: "Analyze the average behavior of...
TAOCP 5.2.3 Exercise 19
Section 5.2.3: Sorting by Selection Exercise 19. [ 21 ] Design an algorithm that deletes a given element of a heap of length $N$, producing a heap of length $N - 1$. Verified: no Solve time: 10m17s Solution Let the heap be stored in an array K[1..N] satisfying the heap condition [ K[\lfloor j/2\rfloor] \ge K[j] \qquad (1 \le \lfloor j/2\rfloor < j \le N). ] We are given an...
TAOCP 5.2.3 Exercise 18
Section 5.2.3: Sorting by Selection Exercise 18. ▶ [ 21 ] (R. W. Floyd.) During the selection phase of heapsort, the key $K$ tends to be quite small, so that nearly all of the comparisons in step H6 find $K < K_j$. Show how to modify the algorithm so that $K$ is not compared with $K_j$ in the main loop of the computation, thereby nearly cutting the average number of...
TAOCP 5.2.2 Exercise 55
Section 5.2.2: Sorting by Exchanging Exercise 55. ▶ [ 22 ] Show how to modify Program Q so that the partitioning element is the median of three keys (see exercise 28), assuming that $M > 1$. Verified: no Solve time: 9m33s Solution In Program Q (Quicksort), step Q3 (lines 4H-6H) chooses the first element of the current subfile as the partitioning element. To implement the median‑of‑three improvement (Exercise 28), we...
TAOCP 5.2.3 Exercise 17
Section 5.2.3: Sorting by Selection Exercise 17. [ 20 ] The algorithm of exercise 16 can be used for heap creation, instead of the "decrease $l$ to $1$" method used in Algorithm H. Do both methods create the same heap when they begin with the same input file? Verified: no Solve time: 2m21s Solution No, the two methods do not always create the same heap. A counterexample with (N =...
TAOCP 5.2.3 Exercise 16
Section 5.2.3: Sorting by Selection Exercise 16. [ 20 ] Design an efficient algorithm that inserts a new key into a given heap of $n$ elements, producing a heap of $n + 1$ elements. Verified: no Solve time: 5m10s Solution Algorithm I ( Heap insertion ). Given a heap (K_1, K_2, \ldots, K_n) satisfying (K_{\lfloor j/2\rfloor} \ge K_j) for (1 \le \lfloor j/2\rfloor < j \le n), and a new...
TAOCP 5.2.2 Exercise 42
Section 5.2.2: Sorting by Exchanging Exercise 42. [ HM32 ] For any real number $c > 0$, prove that the probability is less than $e^{-c}$ that Algorithm Q will make more than $(c+1)(N+1)H_N$ comparisons when sorting random data. (This upper bound is especially interesting when $c$ is, say, $N^{\epsilon}$.) Verified: no Solve time: 1h45m We need to solve Exercise 42 from Volume 3, Section 5.2.2 (Sorting by Exchanging). The exercise:...
TAOCP 5.2.2 Exercise 50
Section 5.2.2: Sorting by Exchanging Exercise 50. [ HM24 ] Find the asymptotic value of the function $$U_{mn} = \sum_{k \ge 2} \binom{n}{k} (-1)^k \frac{1}{m^{k-1} - 1}.$$ when $m$ is any fixed number greater than 1. (When $m$ is an integer greater than 2, this quantity arises in the study of generalizations of radix exchange, as well as the trie memory search algorithms of Section 6.3.) Verified: no Solve time:...
TAOCP 5.2.2 Exercise 54
Section 5.2.2: Sorting by Exchanging Exercise 54. [ HM24 ] (S. O. Rice.) Show that $U_n$ can be written $$U_n = (-1)^n \frac{n!}{2\pi i} \oint_C \frac{dz}{z(z-1)\ldots(z-n)} \frac{1}{2^{1-z}-1},$$ where $C$ is a skinny closed curve encircling the points $2, 3, \ldots, n$. Changing $C$ to an arbitrarily large circle centered at the origin, derive the convergent series $$U_n = \frac{(H_{n-1} - 1)n}{\ln 2} - \frac{n}{2} + 2 + \frac{2}{\ln 2} \sum_{m...
TAOCP 5.2.2 Exercise 53
Section 5.2.2: Sorting by Exchanging Exercise 53. [ HM42 ] Analyze the average number of bit inspections and exchanges done by radix exchange when the input data consists of infinite-precision binary numbers in $[0\ldots 1)$, each of whose bits is independently equal to 1 with probability $p$. (Only the case $p = \frac{1}{2}$ is discussed in the text; the methods we have used can be generalized to arbitrary $p$.) Consider...
TAOCP 5.2.2 Exercise 51
Section 5.2.2: Sorting by Exchanging Exercise 51. ▶ [ HM28 ] Show that the gamma-function approach to asymptotic problems can be used instead of Euler's summation formula to derive the asymptotic expansion of the quantity $r_k(m)$ in (35). (This gives us a uniform method for studying $r_k(m)$ for all $k$, without relying on tricks such as the text's introduction of $g_{-1}(x) = (e^{-x} - 1/x)$.) Verified: no Solve time: 7m01s...
TAOCP 5.2.2 Exercise 49
Section 5.2.2: Sorting by Exchanging Exercise 49. [ HM44 ] Extend the asymptotic formula (47) for $U_n$ to $O(n^{-1})$. Verified: no Solve time: 13m24s Setup In the analysis of bubble sort (Algorithm B) the average number of comparisons is [ C_{\text{ave}} = \binom{n+1}{2} - U_n, ] where the quantity (U_n) is defined by [ U_n = \frac{1}{n!}\sum_{0\le r<s\le n} s!, r^{,n-s}. ] The text derives an asymptotic formula for (U_n)...
TAOCP 5.2.2 Exercise 48
Section 5.2.2: Sorting by Exchanging Exercise 48. [ HM24 ] Find the asymptotic value of the quantity $V_n$ defined in exercise 38, using a method analogous to the text's study of $U_n$, obtaining terms up to $O(1)$. Verified: no Solve time: 15m06s Solution In Exercise 38, the quantity (V_n) is defined as [ V_n = \frac{1}{n!}\sum_{s=1}^{n} s! \sum_{r=0}^{s-1} r^{,n-s}. ] This sum arises naturally in the analysis of the average...
TAOCP 5.2.2 Exercise 47
Section 5.2.2: Sorting by Exchanging Exercise 47. [ HM21 ] Prove that $\sum_{j \ge 1} (n/2^j) e^{-n/2^j}$ is a bounded function of $n$. Verified: no Solve time: 4m07s Solution We need to prove that the function [ S(n) = \sum_{j=1}^{\infty} \frac{n}{2^j} e^{-n/2^j} ] is bounded for all (n > 0). (If (n \le 0), the sum is trivially zero or not defined; we consider (n>0).) Define (f(x) = x e^{-x})...
TAOCP 5.2.2 Exercise 45
Section 5.2.2: Sorting by Exchanging Exercise 45. [ HM20 ] Explain why (43) is true, when $x > 0$. Verified: no Solve time: 23m06s Solution In the text, equation (43) is the integral representation $$ \frac{1}{n!}\sum_{0\le r<s\le n}s!,r^{,n-s} = \int_0^\infty e^{-x},\frac{(e^x-1),\gamma(n+1,x)}{x,\Gamma(n+1)},dx \qquad\text{(43)} $$ where $\gamma(n+1,x)=\int_0^x t^n e^{-t},dt$ is the lower incomplete gamma function. The condition “when $x>0$” refers to the domain of integration; the integrand is well‑defined for all $x>0$...
TAOCP 5.2.2 Exercise 46
Section 5.2.2: Sorting by Exchanging Exercise 46. [ HM20 ] What is the value of $(1/2\pi i) \int_{s-i\infty}^{s+i\infty} \Gamma(z) n^{-z},dz/(2^{z-1} - 1)$, given that $s$ is a positive integer and $0 < a < s$? Verified: no Solve time: 12m49s Solution We evaluate the integral [ I = \frac{1}{2\pi i} \int_{s-i\infty}^{s+i\infty} \frac{\Gamma(z), n^{-z}}{2^{z-1} - 1} , dz, ] where (s) is a positive integer. The condition (0 < a <...
TAOCP 5.2.2 Exercise 44
Section 5.2.2: Sorting by Exchanging Exercise 44. [ HM24 ] Derive (37) as suggested in the text. Verified: no Solve time: 12m24s Solution We derive the asymptotic formula for the average number of comparisons in the bubble sort (Equation (37) in the text). From the analysis in Section 5.2.2, the average number of comparisons is [ C_{\text{ave}} = \binom{N+1}{2} - \frac{1}{N!}\sum_{s=1}^{N} s! \sum_{r=0}^{s-1} r^{,N-s}. ] Setting (k = N-s) and...
TAOCP 5.2.2 Exercise 32
Section 5.2.2: Sorting by Exchanging Exercise 32. [ M10 ] Find a simple closed form expression for $C_{mn}$, the average number of key comparisons required to select the $m$th smallest of $n$ elements by the "quickfind" method of exercise 31. (For simplicity, let $M = 1$; that is, don't assume the use of a special technique for short subfiles.) What is the asymptotic behavior of $C_{(2m-1)m}$, the average number of...
TAOCP 5.2.2 Exercise 43
Section 5.2.2: Sorting by Exchanging Exercise 43. [ HM21 ] Prove that $\int_0^y y^{-1}(e^{-y} - 1),dy + \int_1^\infty y^{-1} e^{-y},dy = -\gamma$. [ Hint: Consider $\lim_{a \to 0} y^{a-1}$.] Verified: no Solve time: 52s Solution We prove the identity [ \int_0^1 \frac{e^{-y}-1}{y},dy ;+; \int_1^\infty \frac{e^{-y}}{y},dy ;=; -\gamma, ] where (\gamma) is the Euler-Mascheroni constant. (The upper limit of the first integral is (1); the notation (\int_0^y) in the exercise statement...
TAOCP 5.2.2 Exercise 35
Section 5.2.2: Sorting by Exchanging Exercise 35. [ M23 ] Analyze the values of the frequencies $A$, $B$, $C$, $G$, $K$, $L$, $R$, $S$, and $X$ that arise in radix exchange sorting using "case (i) input." Verified: no Solve time: 50m30s Solution We analyze the radix exchange sorting algorithm (Program R) for the “case (i) input” described in the text: the file contains (N) records whose keys are infinite sequences...
TAOCP 5.2.2 Exercise 41
Section 5.2.2: Sorting by Exchanging Exercise 41. ▶ [ 30 ] Devise a good way to exchange records $R_l \ldots R_r$ so that they are partitioned into three blocks, with (i) $K_i \le K$ for $l \le k < i$; (ii) $K_i = K$ for $i \le k \le j$; (iii) $K_k > K$ for $j < k \le r$. Schematically, the final arrangement should be $< K$ $= K$...
TAOCP 5.2.2 Exercise 40
Section 5.2.2: Sorting by Exchanging Exercise 40. [ 21 ] Explain how to modify Algorithm R so that it works with reasonable efficiency when sorting files containing numerous equal keys. Verified: no Solve time: 8m37s Solution Algorithm R (radix exchange sort) sorts a file of (N) records (R_1,\dots,R_N) with keys (K_1,\dots,K_N) by examining the bits of the keys from the most significant to the least significant. It uses a stack...
TAOCP 5.2.2 Exercise 39
Section 5.2.2: Sorting by Exchanging Exercise 39. [ 20 ] The results shown in (30) indicate that radix exchange sorting involves about $1.44N$ partitioning stages when it is applied to random input. Prove that quicksort will never require more than $N$ stages; and explain why radix exchange often does. Verified: no Solve time: 3m18s Solution Quicksort. In quicksort a partitioning stage consists of choosing a pivot from the current subarray,...
TAOCP 5.2.2 Exercise 36
Section 5.2.2: Sorting by Exchanging Exercise 36. [ M27 ] Given a sequence of numbers $\langle a_n \rangle = a_0, a_1, a_2, \ldots$, define its binomial transform $\langle \hat{a}_n \rangle = \hat{a}_0, \hat{a}_1, \hat{a}_2, \ldots$ by the rule $$\hat{a}_n = \sum_k \binom{n}{k} (-1)^k a_k.$$ a) Prove that $\langle \hat{\hat{a}}_n \rangle = \langle a_n \rangle$. b) Find the binomial transforms of the sequences (1): $\langle a \rangle$; $\left\langle \binom{m}{n} \right\rangle$, for...
TAOCP 5.2.2 Exercise 37
Section 5.2.2: Sorting by Exchanging Exercise 37. [ M28 ] Determine all sequences $\langle a_n \rangle$ such that $\langle \hat{a}_n \rangle = \langle a_n \rangle$, in the sense of exercise 36. Verified: no Solve time: 3m08s Setup Let (\langle a_n \rangle = a_0, a_1, a_2, \ldots) be a sequence. Its binomial transform (with alternating signs) is defined by [ \hat{a} n = \sum {k=0}^n \binom{n}{k} (-1)^k a_k \qquad (n \ge...
TAOCP 5.2.2 Exercise 29
Section 5.2.2: Sorting by Exchanging Exercise 29. [ HM40 ] Continuing exercise 28, find the asymptotic value of the number of comparisons in Singleton's "median of three" method. Verified: no Solve time: 25m35s Setup Let (C_N) denote the average number of key comparisons performed by Singleton’s median‑of‑three quicksort when sorting (N) distinct elements, excluding the three comparisons used to find the median of the first, middle, and last elements. For...
TAOCP 5.2.2 Exercise 34
Section 5.2.2: Sorting by Exchanging Exercise 34. [ 20 ] How can the bit-inspection loops of radix exchange (in steps R3 through R6) be speeded up? Verified: no Solve time: 1m27s Solution In Algorithm R (Radix exchange sort), steps R3 and R4 (the bit‑inspection loops) repeatedly scan from the left for an element whose current bit is 1 and from the right for an element whose current bit is 0....
TAOCP 5.2.2 Exercise 33
Section 5.2.2: Sorting by Exchanging Exercise 33. ▶ [ 15 ] Design an algorithm that rearranges all the numbers in a given table so that all negative values precede all nonnegative ones. (The items need not be sorted completely, just separated between negative and nonnegative.) Your algorithm should use the minimum possible number of exchanges. Verified: no Solve time: 54s Solution We are given a table $A[1 \ldots N]$ of...
TAOCP 5.2.2 Exercise 31
Section 5.2.2: Sorting by Exchanging Exercise 31. [ 20 ] (V. A. R. Sarju.) Suppose that, instead of sorting an entire file, you only want to determine the $m$th smallest of $n$ elements. Show that quicksort can be adapted to this purpose, avoiding many of the computations required to do a complete sort. Verified: no Solve time: 22m14s Solution Algorithm We adapt Algorithm Q (quicksort) to find the $m$th smallest...
TAOCP 5.2.2 Exercise 24
Section 5.2.2: Sorting by Exchanging Exercise 24. [ M21 ] Algorithm Q obviously does a few more comparisons than it needs to, since we can have $i = j$ in step Q3 and even $i > j$ in step Q4. How many comparisons $C_N$ would be done on the average if we avoided all comparisons when $i \ge j$? Verified: no Solve time: 44m16s Solution We consider the modification of...
TAOCP 5.2.2 Exercise 27
Section 5.2.2: Sorting by Exchanging Exercise 27. [ M28 ] (R. Sedgewick.) Consider the best case of Algorithm Q: Find a permutation of ${1, 2, \ldots, 24}$ that takes the least time to be sorted when $N = 23$ and $M = 3$. Verified: no Solve time: 17m30s Setup Algorithm Q (Quicksort) is the partition‑exchange method described in Section 5.2.2. It sorts records (R_1,\dots,R_N) in place using a stack of...
TAOCP 5.2.2 Exercise 30
Section 5.2.2: Sorting by Exchanging Exercise 30. ▶ [ 25 ] (P. Shackleton.) When multiword keys are being sorted, many sorting methods become progressively slower as the file gets closer to its final order, since equal and nearly-equal keys require an inspection of several words to determine the proper lexicographic order. (See exercise 5.5.) Files that arise in practice often involve such keys, so this phenomenon can have a significant...
TAOCP 5.2.2 Exercise 28
Section 5.2.2: Sorting by Exchanging Exercise 28. [ M26 ] Find the recurrence relation analogous to (40) that is satisfied by the average number of comparisons in Singleton's modification of Algorithm Q (choosing $s$ as the median of ${K_1, K_{\lfloor(N+1)/2\rfloor}, K_N}$ instead of $s = K_1$). Ignore the comparisons made when computing the median value $s$. Verified: no Solve time: 6m56s Setup We analyze Singleton's modification of Algorithm Q (Quicksort),...
TAOCP 5.2.2 Exercise 26
Section 5.2.2: Sorting by Exchanging Exercise 26. ▶ [ M24 ] Construct an input file that makes Program Q go even more slowly than it does in exercise 25. (Try to find a really bad case.) Verified: no Solve time: 14m35s Solution The worst-case running time of Program Q (Quicksort with a cutoff to straight insertion for subfiles of size (\le M)) is achieved by an input that forces the...
TAOCP 5.2.2 Exercise 25
Section 5.2.2: Sorting by Exchanging Exercise 25. [ M20 ] When the input keys are the numbers $1\ 2\ \ldots\ N$ in order, what are the exact values of the quantities $A$, $B$, $C$, $D$, $E$, and $S$ in the timing of Program Q? (Assume that $N > M$.) Verified: no Solve time: 11m16s Solution We consider Program Q (quicksort) as described in Section 5.2.2. The six quantities appearing in...
TAOCP 5.2.2 Exercise 23
Section 5.2.2: Sorting by Exchanging Exercise 23. [ M23 ] Let $A_N$, $B_N$, $C_N$, $D_N$, $S_N$ be the average values of the corresponding quantities in (16), when sorting a random permutation of ${1, 2, \ldots, N}$. Find recurrence relations for these quantities, analogous to (18); and solve these recurrences to obtain (25). Verified: no Solve time: 17m35s Solution We are given that (A_N, B_N, C_N, D_N, S_N) are the average...
TAOCP 5.2.2 Exercise 22
Section 5.2.2: Sorting by Exchanging Exercise 22. [ M25 ] Let $p_{kN}$ be the probability that the quantity $A$ in (16) will equal $k$, when Algorithm Q is applied to a random permutation of ${1, 2, \ldots, N}$, and let $A_N(z) = \sum_k p_{kN} z^k$ be the corresponding generating function. Prove that $A_N(z) = 1$ for $N \le M$, and $A_N(z) = \frac{1}{N} \sum_{1 \le j \le N} A_{j-1}(z) A_{N-j}(z)$...
TAOCP 5.2.2 Exercise 10
Section 5.2.2: Sorting by Exchanging Exercise 10. [ M26 ] Let $a_1, a_2 \ldots a_n$ be a 2-ordered permutation of ${1, 2, \ldots, n}$. a) What are the coordinates of the endpoints of the $a_i$th step of the corresponding lattice path? [See Fig. 18(b) on page 87.] b) Prove that the comparison/exchange of $a_1 : a_2,; a_3 : a_4, \ldots$ corresponds to folding the path about the diagonal, as in...
TAOCP 5.2.2 Exercise 12
Section 5.2.2: Sorting by Exchanging Exercise 12. [ 24 ] Write a MIX program for Algorithm M, assuming that MIX is a binary computer with the operations AND, SRB. How much time does your program take to sort the sixteen records in Table 1? Verified: no Solve time: 1h13m Solution The algorithm intended by the exercise (binary computer with AND and SRB ) is the radix exchange sort (Algorithm R...
TAOCP 5.2.2 Exercise 20
Section 5.2.2: Sorting by Exchanging Exercise 20. [ M20 ] What is the largest possible number of elements that will ever be on the stack at once in Algorithm Q, as a function of $M$ and $N$? Verified: no Solve time: 12m52s Solution In Algorithm Q (Quicksort), a stack is used to keep track of subfiles that remain to be sorted. The algorithm employs the standard optimization of always stacking...
TAOCP 5.2.2 Exercise 21
Section 5.2.2: Sorting by Exchanging Exercise 21. [ 20 ] Explain why the first partitioning phase of Algorithm Q takes the number of comparisons and exchanges specified in (17), when the keys are distinct. Verified: no Solve time: 6m45s Solution We first recall the first partitioning phase of Algorithm Q (quicksort) as described in the text. The array (K_1 \ldots K_N) contains distinct keys. The algorithm uses sentinels (K_0 =...
TAOCP 5.2.2 Exercise 18
Section 5.2.2: Sorting by Exchanging Exercise 18. ▶ [ 20 ] Explain how the computation proceeds in Algorithm Q when all of the input keys are equal. What would happen if the "$<$" signs in steps Q3 and Q4 were changed to "$\le$" instead? Verified: no Solve time: 6m31s Solution Algorithm Q (Hoare’s quicksort) from Section 5.2.2 uses the sentinels (K_0 = -\infty) and (K_{N+1} = +\infty). The partitioning step...
TAOCP 5.2.2 Exercise 19
Section 5.2.2: Sorting by Exchanging Exercise 19. [ 15 ] Would Algorithm Q still work properly if a queue (first-in-first-out) were used instead of a stack (last-in-first-out)? Verified: no Solve time: 38s Solution Yes, Algorithm Q would still work properly if a queue (FIFO) were used instead of a stack (LIFO). Reason. Algorithm Q is an iterative implementation of quicksort. It maintains a collection of subarray intervals ((l, r)) that...
TAOCP 5.2.2 Exercise 16
Section 5.2.2: Sorting by Exchanging Exercise 16. [ HM42 ] Find the asymptotic value of the average number of exchanges occurring when Batcher's method is applied to a random permutation of $N$ distinct elements, assuming that $N$ is a power of two. Verified: no Solve time: 25m39s Setup We consider Batcher's merge‑exchange sort (Algorithm M in the text) applied to a random permutation of (N = 2^t) distinct keys. The...
TAOCP 5.2.2 Exercise 17
Section 5.2.2: Sorting by Exchanging Exercise 17. ▶ [ 20 ] Where in Algorithm Q do we use the fact that $K_0$ and $K_{N+1}$ have the values postulated in (12)? Verified: no Solve time: 1m23s Solution In Algorithm Q (Quicksort), the sentinel values (K_0 = -\infty) and (K_{N+1} = +\infty) (postulated in equation (12)) are used in the two scanning loops of the partitioning phase. During partitioning we choose a...
TAOCP 5.2.2 Exercise 15
Section 5.2.2: Sorting by Exchanging Exercise 15. [ M38 ] The object of this exercise is to analyze the function $c(N)$ of exercise 14 , and to find a formula for $c(N)$ when $N = 2^{e_1} + 2^{e_2} + \cdots + 2^{e_r}$, $e_1 > e_2 > \cdots > e_r \ge 0$. a) Let $a(N+1) = a(N) - c(N)$. Prove that $a(2n) = a(n) + \lfloor \lg(2n) \rfloor$, and $a(2n+1) =...
TAOCP 5.2.2 Exercise 14
Section 5.2.2: Sorting by Exchanging Exercise 14. [ M21 ] Let $c(N)$ be the number of key comparisons used to sort $N$ elements by Batcher's method; this is the number of times step M4 is performed. a) Show that $c(2^t) = 2(2^{t-1}) + (t-1)2^{t-1} + 1$, for $t \ge 1$. b) Find a simple expression for $c(2^t)$ as a function of $t$. Hint: Consider the sequence $x_t = c(2^t)/2^t$. Verified:...
TAOCP 5.2.2 Exercise 9
Section 5.2.2: Sorting by Exchanging Exercise 9. [ M48 ] Analyze the number of passes and the number of comparisons in the cocktail-shaker sorting algorithm. Note: See exercise 5.4.8–9 for partial information. Verified: no Solve time: 33m58s Setup The cocktail-shaker sort (also called bidirectional bubble sort) operates on a file of (N) records (R_1,\dots,R_N) with distinct keys (K_1,\dots,K_N). The algorithm maintains two bounds (L) and (R) delimiting the unsorted portion;...
TAOCP 5.2.2 Exercise 13
Section 5.2.2: Sorting by Exchanging Exercise 13. [ 10 ] Is Batcher's method a stable sorting algorithm? Verified: no Solve time: 4m16s Yes, Batcher's merge exchange sort is stable. Every compare-exchange operation in the algorithm is defined to swap two records only when their keys are in strictly decreasing order (i.e., (K_i > K_j)). Equal keys are therefore never interchanged directly. Moreover, because all operations are of this form, two...
TAOCP 5.2.2 Exercise 11
Section 5.2.2: Sorting by Exchanging Exercise 11. ▶ [ M25 ] Which permutation of ${1, 2, \ldots, 16}$ maximizes the number of exchanges done by Batcher's algorithm? Verified: no Solve time: 15m08s Solution Batcher's merge-exchange sort for (n = 16) is an odd-even merge sort. The sorting network consists of (63) compare‑exchange operations (comparators). The maximum possible number of exchanges is (63), attained exactly when every comparator finds its two...
TAOCP 5.2.1 Exercise 37
Section 5.2.1: Sorting by Insertion Exercise 37. [ M25 ] Let $g_n(z)$ be the probability generating function for inversions in a random permutation of $n$ objects, Eq. 5.1.1–(11). Let $g_{Mn}(z)$ be the corresponding generating function for the quantity $B$ in Program M. Show that $$\sum_{n \ge 0} g_{Mn}(z) \frac{M^N u^N}{N!} = \left(\sum_{n \ge 0} g_n(z) \frac{u^n}{n!}\right)^M,$$ and use this formula to derive the variance of $B$. Verified: no Solve time:...
TAOCP 5.2.2 Exercise 8
Section 5.2.2: Sorting by Exchanging Exercise 8. [ M24 ] Derive Eq. (8). Verified: no Solve time: 2m36s Solution We derive the formula for (f_j(k)) from its definition. An inversion table of a permutation of ({1,\dots,n}) is a sequence (b_1,\dots,b_n) with (0\le b_i\le n-i); every such sequence corresponds to exactly one permutation. By definition, (f_j(k)) counts the inversion tables satisfying for every (i): [ b_i < j-1 \quad\text{or}\quad b_i +...
TAOCP 5.2.1 Exercise 31
Section 5.2.1: Sorting by Insertion Exercise 31. [ 25 ] Write a MIX program for Pratt's sorting algorithm (exercise 30). Express its running time in terms of quantities $A$, $B$, $S$, $T$, $N$ analogous to those in Program D. Verified: no Solve time: 48m18s Solution Algorithm. Pratt’s sorting algorithm is a variant of Shellsort in which the increments are all numbers of the form (2^p3^q < N), taken in decreasing...
TAOCP 5.2.1 Exercise 33
Section 5.2.1: Sorting by Insertion Exercise 33. [ 25 ] Find a way to improve on Program L so that its running time is dominated by $5B$ instead of $7B$, where $B$ is the number of inversions. Discuss corresponding improvements to Program S. Verified: no Solve time: 43m30s Solution We first recall Program L (list insertion sort) from Section 5.2.1. The list is maintained in descending order (largest first) with...
TAOCP 5.2.1 Exercise 35
Section 5.2.1: Sorting by Insertion Exercise 35. [ 21 ] Write a MIX program to follow Program M, so that all lists are combined into a single list. Your program should set the LINK fields exactly as they would have been set by Program L. Verified: no Solve time: 7m11s Solution Data structures (as left by Program M) There are (M \ge 1) lists. Their heads are stored in a...
TAOCP 5.2.1 Exercise 34
Section 5.2.1: Sorting by Insertion Exercise 34. [ M10 ] Verify formula (14). Verified: no Solve time: 22m Formula (14) gives the running time of Program D (Shellsort) as [ 9B + 10Nt - 3A - 10\sum_{s=0}^{t-1}h_s + 13t + 1 \quad \text{units}, ] where (t) is the number of passes, (h_s) are the increments, (A) is the total number of times the index (i) decreases to zero, and (B)...
TAOCP 5.2.1 Exercise 26
Section 5.2.1: Sorting by Insertion Exercise 26. [ M35 ] Can a file of $N$ elements have more than $N$ inversions if it is 3-, 5-, and 7-ordered? Estimate the maximum number of inversions when $N$ is large. Verified: no Solve time: 59m13s Setup We consider permutations of ({1,\dots,N}) that are (3)-ordered, (5)-ordered and (7)-ordered, i.e. for every (i) and every (h\in{3,5,7}) with (i+h\le N) we have (K_i\le K_{i+h}). The...
TAOCP 5.2.1 Exercise 23
Section 5.2.1: Sorting by Insertion Exercise 23. ▶ [ M22 ] Prove that if $h_{s+2}$ and $h_{s+1}$ are relatively prime, the number of moves that occur while Algorithm D is using the increment $h_s$ is $O(Nh_{s+1}/h_s)$. Hint: See exercise 21. Verified: no Solve time: 54m26s Solution After passes with increments (h_{s+2}) and (h_{s+1}) the file is (h_{s+2})-ordered and (h_{s+1})-ordered: [ K_i\le K_{i+h_{s+2}}\quad(1\le i\le N-h_{s+2}),\qquad K_i\le K_{i+h_{s+1}}\quad(1\le i\le N-h_{s+1}). ] The...
TAOCP 5.2.1 Exercise 32
Section 5.2.1: Sorting by Insertion Exercise 32. [ 10 ] What would be the final contents of $L_0, L_1 \ldots L_{16}$ if the list insertion sort in Table 8 were carried through to completion? Verified: no Solve time: 4m23s The list insertion sort (Algorithm L) applied to the 16 keys from Table 1 (503, 087, 512, 061, 908, 170, 897, 275, 653, 426, 154, 509, 612, 677, 765, 703) produces...
TAOCP 5.2.1 Exercise 15
Section 5.2.1: Sorting by Insertion Exercise 15. ▶ [ HM3 ] Let $g_0(z)$, $g_n(z)$, $h_n(z)$, and $\tilde{h}_n(z)$ be $\sum z^{\text{total weight of path}}$ summed over all lattice paths of length $2n$ from $(0,0)$ to $(n,n)$, where the weight is defined as in Fig. 11, subject to certain restrictions on the vertices on the paths: For $h_n(z)$, there is no restriction, but for $g_n(z)$ the path must avoid all vertices $(i,j)$...
TAOCP 5.2.1 Exercise 30
Section 5.2.1: Sorting by Insertion Exercise 30. [ M23 ] (N. Pratt.) If the set of increments in shellsort is ${2^p 3^q \mid 2^p 3^q < N}$, show that the number of passes is approximately $\frac{1}{2}(\log_2 N)(\log_3 N)$, and the number of moves per pass is at most $N/2$. In fact, if $K_{j-h} > K_j$ on any pass, we will always have $K_{j-h} - K_{j-2h} \le K_j - K_{j-h} \le...
TAOCP 5.2.1 Exercise 29
Section 5.2.1: Sorting by Insertion Exercise 29. [ 40 ] For $N = 1000$ and various values of $t$, find empirical values of $h_{t-1}, \ldots, h_1, h_0$ for which the average number of moves, $B_{\text{ave}}$, is as small as you can make it. Verified: no Solve time: 1m19s Setup Let $N=1000$. We seek increments $h_{t-1},\dots ,h_0$ for Algorithm D (shellsort) that minimise the average number of moves $B_{\text{ave}}$, where $B$...
TAOCP 5.2.1 Exercise 28
Section 5.2.1: Sorting by Insertion Exercise 28. [ 15 ] Which sequence of increments shown in Table 6 is best from the standpoint of Program D, considering the average total running time? Verified: no Solve time: 1m48s Solution Table 6 in Section 5.2.1 lists the average performance of Program D for several increment sequences when sorting (N = 16) records. The table gives the average number of comparisons, moves, and...
TAOCP 5.2.1 Exercise 27
Section 5.2.1: Sorting by Insertion Exercise 27. [ M41 ] (Bjørn Poonen.) (a) Prove that there is a constant $c$ such that if $m$ of the increments $h_s$ in Algorithm D are less than $N/2$, the running time is $\Omega(N^{1+1/\sqrt{cm}})$ in the worst case. (b) Consequently the worst-case running time is $\Omega(N(\log N / \log \log N)^2)$ for all sequences of increments. Verified: no Solve time: 5m07s Solution Setup Let...
TAOCP 5.2.1 Exercise 24
Section 5.2.1: Sorting by Insertion Exercise 24. [ M43 ] Prove that Theorem P is best possible, in the sense that the exponent 3/2 cannot be lowered. Verified: no Solve time: 23m21s Setup Theorem P (as proved in the text) states that if the increments are [ h_0=1,\qquad h_{k+1}=2h_k+1\quad (k\ge 0), ] so that (h_k=2^{,k+1}-1), then the total number of moves performed by Algorithm D when sorting (N) records is...
TAOCP 5.2.1 Exercise 25
Section 5.2.1: Sorting by Insertion Exercise 25. ▶ [ M22 ] How many permutations of ${1, 2, \ldots, N}$ are both 3-ordered and 2-ordered? What is the maximum number of inversions in such a permutation? What is the total number of inversions among all such permutations? Verified: no Solve time: 14m14s Solution A permutation (K_1K_2\ldots K_N) of ({1,2,\ldots,N}) is 2-ordered if (K_i < K_{i+2}) for (1\le i\le N-2), and 3-ordered...
TAOCP 5.2.1 Exercise 20
Section 5.2.1: Sorting by Insertion Exercise 20. [ M22 ] Show that Theorem K follows from Lemma L. Verified: no Solve time: 41m49s Solution We first recall the statements of Lemma L and Theorem K from the text. Lemma L. If a permutation of ({1,2,\ldots,N}) is (h)-ordered (i.e., (K_i \le K_{i+h}) for all (1 \le i \le N-h)), then the number of inversions it contains has the same probability distribution...
TAOCP 5.2.1 Exercise 22
Section 5.2.1: Sorting by Insertion Exercise 22. [ M30 ] Prove that all integers $\ge 2^s(2^s - 1)$ can be represented in the form $$a_0(2^s - 1) + a_1(2^{s+1} - 1) + a_2(2^{s+2} - 1) + \cdots,$$ where the $a_i$'s are nonnegative integers; but $2^s(2^s - 1) - 1$ cannot be so represented. Furthermore, exactly $2^{s-1}(2^s + s - 3)$ positive integers are unrepresentable in this form. Find analogous formulas...
TAOCP 5.2.1 Exercise 16
Section 5.2.1: Sorting by Insertion Exercise 16. [ M2 ] Find a formula for the maximum number of inversions in an $h$-ordered permutation of ${1, 2, \ldots, n}$. What is the maximum possible number of moves in Algorithm D when the increments satisfy the divisibility condition (5)? Verified: no Solve time: 53m18s 16. Let $n$ and $h$ be positive integers. Write $n = qh + r$ with $0 \le r...
TAOCP 5.2.1 Exercise 21
Section 5.2.1: Sorting by Insertion Exercise 21. [ M25 ] Let $h$ and $k$ be relatively prime positive integers, and say that an integer is generable if it equals $xh + yk$ for some nonnegative integers $x$ and $y$. Show that $n$ is generable if and only if $hk - h - k - n$ is not generable. (Since 0 is the smallest generable integer, the largest nongenerable integer is...
TAOCP 5.2.1 Exercise 19
Section 5.2.1: Sorting by Insertion Exercise 19. ▶ [ M25 ] What is the average value of the quantity $A$ in the timing analysis of Program D, when the increments satisfy the divisibility condition (5)? Verified: no Solve time: 20m21s Solution Let the increments be (h_{t-1}>h_{t-2}>\cdots>h_0=1) and assume they satisfy the divisibility condition [ h_{s};\text{is a multiple of};h_{s-1}\qquad(1\le s\le t-1). ] In Algorithm D the quantity (A) is the total...
TAOCP 5.2.1 Exercise 17
Section 5.2.1: Sorting by Insertion Exercise 17. [ M21 ] Show that, when $N = 2^t$ and $h_s = 2^s$ for $t > s \ge 0$, there is a unique permutation of ${1, 2, \ldots, N}$ that maximizes the number of move operations performed by Algorithm D. Find a simple way to describe this permutation. Verified: no Solve time: 9m01s Solution We consider Shellsort (Algorithm D) with $N = 2^t$...
TAOCP 5.2.1 Exercise 18
Section 5.2.1: Sorting by Insertion Exercise 18. [ HM2 ] For large $N$ the sum (6) can be estimated as $$\frac{1}{4} \frac{N^2}{h_{t-1}} - \frac{\sqrt{\pi}}{8} \left( \frac{N^{3/2} h_{t-1}^{1/2}}{h_{t-2}} + \cdots + \frac{N^{3/2} h_1^{1/2}}{h_0} \right).$$ What real values of $h_{t-1}, \ldots, h_0$ minimize this expression when $N$ and $t$ are fixed and $h_0 = 1$? Verified: no Solve time: 7m44s We minimize the asymptotic estimate for the sum (6): [ E =...
TAOCP 5.1.4 Exercise 43
Section 5.1.4: Tableaux and Involutions Exercise 43. [ 35 ] Continuing the previous exercise, show that at most $n + 1$ flips are needed to sort any rearrangement of $g_1 g_2 \ldots g_n$. Construct examples that require $n + 1$ flips, for all $n > 3$. Verified: no Solve time: 41m33s We are given Exercise 43 from Volume 3, Section 5.1.4 (Tableaux and Involutions). The exercise continues from Exercise 42...
TAOCP 5.1.4 Exercise 44
Section 5.1.4: Tableaux and Involutions Exercise 44. [ M37 ] Show that the average number of flips required to sort a random arrangement of $n$ genes is greater than $n - H_n$, if all $2^n , n!$ genome rearrangements are equally likely. Verified: no Solve time: 27m06s We need to solve Exercise 44 from TAOCP Volume 3, Section 5.1.4. The exercise: 44. [M37] Show that the average number of flips...
TAOCP 5.1.4 Exercise 39
Section 5.1.4: Tableaux and Involutions Exercise 39. [ M38 ] (I. M. Pak and A. V. Stoyanovskii, 1992.) Let $P$ be an array of shape $(n_1, \ldots, n_m)$ that has been filled with any permutation of the integers ${1, \ldots, n}$, where $n = n_1 + \cdots + n_m$. The following procedure, which is analogous to the "siftup" algorithm in Section 5.2.3, can be used to convert $P$ to a...
TAOCP 5.1.4 Exercise 27
Section 5.1.4: Tableaux and Involutions Exercise 27. [ M24 ] Let $Q$ be a Young tableau on ${1, 2, \ldots, n}$; let the element $i$ be in row $r_i$ and column $c_i$. We say that $i$ is "above" $j$ when $r_i < r_j$. a) Prove that, for $1 \le i < n$, $i$ is above $i + 1$ if and only if $c_i \ge c_{i+1}$. b) Given that $Q$ is...
TAOCP 5.1.4 Exercise 24
Section 5.1.4: Tableaux and Involutions Exercise 24. [ M28 ] Prove that $$\sum_{\substack{q_1 + \cdots + q_m = n \ 0 \le q_1, \ldots, q_m \le n}} \binom{m}{q_1} \cdots \binom{m}{q_m} \Delta(q_1, \ldots, q_m)^2$$ $$= n! \binom{m - (n^2 - n)}{t - \frac{1}{2}(n^2 - n)} \binom{m}{n-1} \binom{m}{n-2} \cdots \binom{m}{0} \Delta(n-1, \ldots, 0)^2.$$ [ Hints: Prove that $\Delta(k_1 + n - 1, \ldots, k_s) = \Delta(m - k_s + n - 1,...
TAOCP 5.1.4 Exercise 28
Section 5.1.4: Tableaux and Involutions Exercise 28. [ M43 ] Prove that the average length of the longest increasing subsequence of a random permutation of ${1, 2, \ldots, n}$ is asymptotically $2\sqrt{n}$. (This is the average length of row 1 in the correspondence of Theorem A.) Verified: no Solve time: 9m32s Setup Let (S_n) be the symmetric group on ({1,2,\ldots,n}). For a permutation (\pi\in S_n), let (L(\pi)) denote the length...
TAOCP 5.1.4 Exercise 14
Section 5.1.4: Tableaux and Involutions Exercise 14. [ M43 ] Find a more direct proof of Theorem D, part (c). Verified: no Solve time: 20m52s Setup We state Theorem D(c) in the notation of Section 5.1.4. Theorem D(c). Let (\pi) be a permutation of ({1,2,\ldots,n}) and let ((P,Q)) be the corresponding tableaux under the Robinson-Schensted correspondence of Theorem A. If the shape of (P) is ((n_1,n_2,\ldots,n_m)), then for every (k)...
TAOCP 5.1.4 Exercise 23
Section 5.1.4: Tableaux and Involutions Exercise 23. ▶ [ HM30 ] (D. André.) In how many ways, $E_m$, can the numbers ${1, 2, \ldots, n}$ be placed into the array of $n$ cells $$\begin{array}{cccc} & & \cdots \ \square & \square & \cdots \end{array}$$ in such a way that the rows and columns are in increasing order? Find the generating function $g(z) = \sum E_m z^m / m!$. Verified: no...
TAOCP 5.1.4 Exercise 13
Section 5.1.4: Tableaux and Involutions Exercise 13. [ M28 ] Prove Theorem C. Verified: no Solve time: 12m07s Setup We work within the framework of Section 5.1.4 of The Art of Computer Programming , Volume 3. The Robinson-Schensted correspondence (Theorem A) constructs from any two‑line array [ \begin{pmatrix} q_1 & q_2 & \cdots & q_n \ p_1 & p_2 & \cdots & p_n \end{pmatrix},\qquad q_1 < q_2 < \cdots <...
TAOCP 5.1.2 Exercise 22
Section 5.1.2: Permutations of a Multiset Exercise 22. [ M30 ] Let $P(x_1^{n_1} \ldots x_m^{n_m})$ denote the set of all possible permutations of the multiset ${n_1 \cdot x_1, \ldots, n_m \cdot x_m}$, and let $P_0(x_0^{n_0} x_1^{n_1} \ldots x_m^{n_m})$ be the subset of $P(x_0^{n_0} x_1^{n_1} \ldots x_m^{n_m})$ in which the first $n_0$ elements are $\ne x_0$. a) Given a number $t$ with $1 \le l < m$, find a one-to-one correspondence...
TAOCP 5.1.2 Exercise 15
Section 5.1.2: Permutations of a Multiset Exercise 15. ▶ [ M25 ] Prove that the permutation $a_1 \ldots a_n$ of the multiset $${n_1 \cdot x_1,, n_2 \cdot x_2, \ldots, n_m \cdot x_m},$$ where $x_1 < x_2 < \cdots < x_n$ and $n_1 + n_2 + \cdots + n_m = n$, is a cycle if and only if the directed graph with vertices ${x_1, x_2, \ldots, x_m}$ and arcs from $x_i$...
TAOCP 5.1.2 Exercise 18
Section 5.1.2: Permutations of a Multiset Exercise 18. [ M30 ] (P. A. MacMahon.) The index of a permutation $a_1, a_2 \ldots a_n$ was defined in the previous section; and we proved that the number of permutations of a given set that have a given index $k$ is the same as the number of permutations that have $k$ inversions. Does the same result hold for permutations of a given multiset?...
TAOCP 5.1.2 Exercise 14
Section 5.1.2: Permutations of a Multiset Exercise 14. [ M30 ] One way to define the inverse $\pi^-$ of a general permutation $\pi$, suggested by other definitions in this section, is to interchange the lines of the two-line representation of $\pi$ and then to do a stable sort of the columns in order to bring the top row into nondecreasing order. For example, if $a < b < c <...
TAOCP 5 Exercise 17
Section 5: Introduction to Sorting Exercise 17. ▶ [ 33 ] (Library card sorting.) Before the days of computerized databases, every library maintained a catalog of cards so that users could find the books they wanted. But the task of putting catalog cards into an order convenient for human use turned out to be quite complicated as library collections grew. The following "alphabetical" listing indicates many of the procedures recommended...
TAOCP 5 Exercise 16
Section 5: Introduction to Sorting Exercise 16. [ 20 ] In order to make it easier to make changes to large FORTRAN programs, you want to design a "cross-reference" routine; such a routine takes FORTRAN programs as input and prints them together with an index that shows each use of each identifier (that is, each name) in the program. How should such a routine be designed? Verified: yes Solve time:...
TAOCP 5.1.4 Exercise 40
Section 5.1.4: Tableaux and Involutions Exercise 40. [ HM43 ] Suppose a random Young tableau has been constructed by successively placing the numbers $1, 2, \ldots, n$ in such a way that each possibility is equally likely when a new number is placed. For example, the tableau (1) would be obtained with probability $\frac{1}{2} \cdot \frac{1}{4} \cdot \frac{1}{4} \cdot \frac{1}{4} \cdots$ using this procedure. The resulting shape $(n_1, n_2, \ldots,...
TAOCP 5.1.4 Exercise 41
Section 5.1.4: Tableaux and Involutions Exercise 41. [ 25 ] (Disorder in a library.) Casual users of a library often put books back on the shelves in the wrong place. One way to measure the amount of disorder present in a library is to consider the minimum number of times we would have to take a book out of one place and insert it in another, before all books are...
TAOCP 5.1.4 Exercise 38
Section 5.1.4: Tableaux and Involutions Exercise 38. ▶ [ M30 ] (Greene, Nijenhuis, and Wilf, 1979.) We can construct a directed acyclic graph on the cells $T$ of any given tableau shape by letting arcs run from each cell to the other cells in its hook; the out-degree of cell $(i, j)$ will then be $d_{ij} = h_{ij} - 1$, where $h_{ij}$ is the hook length. Suppose we generate a...
TAOCP 5.1.4 Exercise 37
Section 5.1.4: Tableaux and Involutions Exercise 37. [ M20 ] (P. A. MacMahon, 1912.) What is the generating function for all plane partitions? (The coefficient of $z^m$ should be the total number of plane partitions of $m$ when the tableau shape is unbounded.) Verified: yes Solve time: 43m02s Solution A plane partition is an infinite array of nonnegative integers (p_{ij}) ((i,j\ge 1)) satisfying [ p_{ij} \ge p_{i+1,j},\qquad p_{ij} \ge p_{i,j+1}...
TAOCP 5.1.4 Exercise 36
Section 5.1.4: Tableaux and Involutions Exercise 36. [ HM27 ] (R. P. Stanley, 1971.) (a) Prove that the number of reverse plane partitions of $m$ in a given shape is $\lfloor s^m \rfloor / \prod (1 - z^{h_{ij}})$, where the numbers $h_{ij}$ are the hook lengths of the shape. (b) Derive Theorem H from this result. [ Hint: What is the asymptotic number of partitions as $m \to \infty$?] Verified:...
TAOCP 5.1.4 Exercise 35
Section 5.1.4: Tableaux and Involutions Exercise 35. ▶ [ 30 ] (A. P. Hillman and R. M. Grassl, 1976.) An arrangement of nonnegative integers $p_{ij}$ in a tableau shape is called a plane partition of $m$ if $\sum p_{ij} = m$ and $$p_{i1} \ge \cdots \ge p_{im_i}, \qquad p_{1j} \ge \cdots \ge p_{n'_j j}, \qquad \text{for } 1 \le i \le n_1',\ 1 \le j \le n_1,$$ when there are...
TAOCP 5.1.4 Exercise 34
Section 5.1.4: Tableaux and Involutions Exercise 34. [ 25 ] (T. Nakayama, 1940.) Prove that if a tableau shape contains a hook of length $4b$, it contains a hook of length $b$. Verified: no Solve time: 4h28m A tableau shape (or Young diagram) is a finite set of cells $\lambda \subset \mathbb{N}^2$ such that $(i,j) \in \lambda$ implies $(i',j') \in \lambda$ for all $1 \le i' \le i$, $1 \le...
TAOCP 5.1.4 Exercise 31
Section 5.1.4: Tableaux and Involutions Exercise 31. [ HM30 ] Let $x_n$ be the number of ways to place $n$ mutually nonattacking rooks on an $n \times n$ chessboard, where each arrangement is unchanged by reflection about both diagonals. Thus, $x_4 = 6$. (Involutions are required to be symmetrical about only one diagonal. Exercise 5.1.3–19 considers a related problem.) Find the asymptotic behavior of $x_n$. Verified: yes Solve time: 4h31m...
TAOCP 5.1.4 Exercise 33
Section 5.1.4: Tableaux and Involutions Exercise 33. [ M25 ] (O. H. Mitchell, 1881.) True or false: $\Delta(a_1, a_2, \ldots, a_m)/\Delta(1, 2, \ldots, m)$ is an integer when $a_1, a_2, \ldots, a_m$ are integers. Verified: yes Solve time: 19m22s Solution The statement is true . For any integers (a_1, a_2, \dots, a_m), the ratio [ \frac{\Delta(a_1, a_2, \dots, a_m)}{\Delta(1, 2, \dots, m)} ] is an integer. Proof. Recall the Vandermonde...
TAOCP 5.1.4 Exercise 32
Section 5.1.4: Tableaux and Involutions Exercise 32. [ HM21 ] Prove that the involution number $t_n$ is the expected value of $X^n$, when $X$ is a normal deviate with mean 1 and variance 1. Verified: yes Solve time: 13m11s Solution Let (X) be a normal random variable with mean (1) and variance (1). Its moment generating function is [ M(t) = \mathbb{E}[e^{tX}] = \exp!\left(t + \frac{t^2}{2}\right). ] Expanding as a...
TAOCP 5.1.4 Exercise 30
Section 5.1.4: Tableaux and Involutions Exercise 30. [ M41 ] (M. P. Schützenberger.) Show that the operation of going from $P$ to $P^5$ is a special case of an operation applicable in connection with any finite partially ordered set, not merely a tableau: Label the elements of a partially ordered set with the integers ${1, 2, \ldots, n}$ in such a way that the partial order is consistent with the...
TAOCP 5.1.4 Exercise 29
Section 5.1.4: Tableaux and Involutions Exercise 29. [ HM25 ] Prove that a random permutation of $n$ elements has an increasing subsequence of length $\ge t$ with probability $\le \binom{n}{t}/t!$. This probability is $O(1/\sqrt{n})$ when $l = c\sqrt{n} + O(1)$, and $O(\exp(-c\sqrt{n}))$ when $l = 3\sqrt{n}$, $c = 6\ln 3 - 6$. Verified: yes Solve time: 1h15m Solution 1. Upper bound for the probability Let (\pi) be a uniformly random...
TAOCP 5.1.4 Exercise 26
Section 5.1.4: Tableaux and Involutions Exercise 26. [ M21 ] Evaluate $\int_{-\infty}^{\infty} x^t \exp(-2x^2) \sqrt{n} , dx$ when $t$ is a nonnegative integer. Verified: yes Solve time: 14m10s Solution We evaluate the integral [ I_t = \int_{-\infty}^{\infty} x^t \exp(-2x^2) \sqrt{n} , dx, ] where (t) is a nonnegative integer and (n > 0) is a constant. The factor (\sqrt{n}) is independent of (x), so [ I_t = \sqrt{n} \int_{-\infty}^{\infty} x^t...
TAOCP 5.1.4 Exercise 25
Section 5.1.4: Tableaux and Involutions Exercise 25. [ M30 ] Why is (42) the generating function for involutions? Verified: yes Solve time: 52m41s Setup By Theorem A in this section, there is a bijection between permutations of ({1,2,\ldots,n}) and ordered pairs ((P,Q)) of standard Young tableaux of the same shape. Under this correspondence, the inverse of a permutation corresponds to swapping (P) and (Q). Hence a permutation is an involution...
TAOCP 5.1.4 Exercise 21
Section 5.1.4: Tableaux and Involutions Exercise 21. [ HM91 ] (R. M. Thrall.) Let $n_1 > n_2 > \cdots > n_m$ specify the shape of a "shifted tableau" where row $i+1$ starts one position to the right of row $i$; for example, a shifted tableau of shape $(7, 5, 4, 1)$ has the form of the diagram $$ \begin{array}{|c|c|c|c|c|c|c|} \hline \cdot & 9 & 6 & 5 & 3 &...
TAOCP 5.1.4 Exercise 22
Section 5.1.4: Tableaux and Involutions Exercise 22. [ M39 ] In how many ways can an array of shape $(n_1, n_2, \ldots, n_m)$ be filled with elements from the set ${1, 2, \ldots, N}$ with repetitions allowed , so that the rows are nondecreasing and the columns are strictly increasing? For example, the simple $m$-rowed shape $(1, 1, \ldots, 1)$ can be filled in $\binom{n}{m}$ ways; the 1-rowed shape $(n)$...
TAOCP 5.1.4 Exercise 19
Section 5.1.4: Tableaux and Involutions Exercise 19. [ M40 ] Find a formula for the number of ways to fill an array that is like a tableau but with two boxes removed at the left of row 1; for example, $$ \begin{array}{l} n_1 - 2 \text{ boxes} \ n_2 \text{ boxes} \ n_3 \text{ boxes} \ \vdots \end{array} $$ is such a shape. (The rows and columns are to be...
TAOCP 5.1.4 Exercise 20
Section 5.1.4: Tableaux and Involutions Exercise 20. ▶ [ M25 ] Prove that the number of ways to label the nodes of a given tree with the elements ${1, 2, \ldots, n}$, such that the label of each node is less than that of its descendants, is $n!$ divided by the product of the subtree sizes (the number of nodes in each subtree). For example, the number of ways to...
TAOCP 5.1.4 Exercise 17
Section 5.1.4: Tableaux and Involutions Exercise 17. [ HM25 ] Let $$g(x_1, x_2, \ldots, x_n;, y) = x_1, \Delta(y, x_2, \ldots, x_n) + x_2, \Delta(x_1, x_2, \ldots, x_n)$$ $$+ \cdots + x_n, \Delta(x_1, x_2, \ldots, x_{n-1}, y).$$ Prove that $$g(x_1, x_2, \ldots, x_n;, y) = (x_1 + x_2 + \cdots + x_n + \tbinom{n}{2}, y), \Delta(x_1, x_2, \ldots, x_n).$$ [ Hint: The function is homogeneous (all terms have the same...
TAOCP 5.1.4 Exercise 18
Section 5.1.4: Tableaux and Involutions Exercise 18. [ HM30 ] Generalizing exercise 17, evaluate the sum $$x_1^m, \Delta(y, x_2, \ldots, x_n) + x_2^m, \Delta(x_1, x_2, \ldots, x_n) + \cdots + x_n^m, \Delta(x_1, x_2, \ldots, x_{n-1}, y),$$ when $m \ge 0$. Verified: yes Solve time: 1h24m Setup Let $\Delta(x_1,\dots,x_n)$ denote the Vandermonde determinant $$\Delta(x_1,\dots,x_n) = \prod_{1\le i<j\le n} (x_j - x_i).$$ We are to evaluate, for $m\ge 0$, the sum $$S_m...
TAOCP 5.1.4 Exercise 16
Section 5.1.4: Tableaux and Involutions Exercise 16. [ M08 ] In how many ways can the partial ordering represented by (39) be sorted topologically? Verified: yes Solve time: 36m57s The partial ordering (39) is defined on the pairs ((q_i, p_i)) of the two-line array [ \begin{pmatrix} q_1 & q_2 & \cdots & q_n \ p_1 & p_2 & \cdots & p_n \end{pmatrix} \begin{pmatrix} 1 & 3 & 5 & 6...
TAOCP 5.1.4 Exercise 15
Section 5.1.4: Tableaux and Involutions Exercise 15. [ M29 ] How many permutations of the multiset ${1 \cdot a, m \cdot b, n \cdot c}$ have the property that, as we read the permutation from left to right, the number of $c$'s never exceeds the number of $b$'s, and the number of $b$'s never exceeds the number of $a$'s? (For example, $a, a, b, c, b, b, c, a, c,...
TAOCP 5.1.2 Exercise 24
Section 5.1.2: Permutations of a Multiset Exercise 24. [ 27 ] Consider any relation $R$ that might hold between two unordered pairs of letters: if ${w, x} R {y, z}$ we say ${w, x}$ preserves ${y, z}$, otherwise ${w, x}$ moves ${y, z}$. The operation of transposing $\begin{smallmatrix}w\ x\end{smallmatrix}$ with respect to $R$ replaces $\begin{smallmatrix}w\ x\end{smallmatrix}$ by $\begin{smallmatrix}y\ x\end{smallmatrix}$ or $\begin{smallmatrix}w\ y\end{smallmatrix}$, and is applicable as the pair ${w, x}$...
TAOCP 5.1.2 Exercise 23
Section 5.1.2: Permutations of a Multiset Exercise 23. [ M23 ] A strand of DNA may be thought of as a word on a four-letter alphabet. Suppose we copy a strand of DNA and break it completely into one-letter bases, then recombine those bases at random. If the resulting strand is placed next to the original, prove that the number of places in which they differ is more likely to...
TAOCP 5.1.2 Exercise 21
Section 5.1.2: Permutations of a Multiset Exercise 21. [ M21 ] Given $n_1, \ldots, n_m$, and $d \ge 0$, how many permutations $a_1 a_2 \ldots a_n$ of the multiset ${n_1 \cdot 1, \ldots, n_m \cdot m}$ satisfy $a_{j+1} \ge a_j - d$ for $1 \le j < n = n_1 + \cdots + n_m$? Verified: yes Solve time: 2h25m Solution Let $f(n_1,\dots,n_m)$ be the number of permutations $a_1a_2\ldots a_n$ of...
TAOCP 5.1.2 Exercise 20
Section 5.1.2: Permutations of a Multiset Exercise 20. ▶ [ HM33 ] (D. Foata.) Let $(a_{ij})$ be any matrix of real numbers. In the notation of exercise 19(b), define $\nu(\pi) = a_{i_1 j_1} \ldots a_{i_n j_n}$, where the two-line notation for $\pi$ is $$\begin{pmatrix} x_{j_1} & x_{j_2} & \cdots & x_{j_n} \ x_{i_1} & x_{i_2} & \cdots & x_{i_n} \end{pmatrix}.$$ This function is useful in the computation of generating functions...
TAOCP 5.1.2 Exercise 19
Section 5.1.2: Permutations of a Multiset Exercise 19. [ HM28 ] Define the Möbius function $\mu(\pi)$ of a permutation $\pi$ to be 0 if $\pi$ contains repeated elements, otherwise $(-1)^k$ if $\pi$ is the product of $k$ primes. (Compare with the definition of the ordinary Möbius function, exercise 4.5.2–10.) a) Prove that if $\pi \ne \epsilon$, we have $$\sum \mu(\lambda) = 0,$$ summed over all permutations $\lambda$ that are left...
TAOCP 5.1.2 Exercise 17
Section 5.1.2: Permutations of a Multiset Exercise 17. [ M24 ] Find the average and standard deviation of the number of inversions in a random permutation of a given multiset, using the generating function found in exercise 16. Verified: yes Solve time: 38m35s Solution The generating function for the number of inversions in a random permutation of the multiset ({n_1\cdot x_1, n_2\cdot x_2, \ldots, n_m\cdot x_m}) (with (x_1 < x_2...
TAOCP 5.1.2 Exercise 16
Section 5.1.2: Permutations of a Multiset Exercise 16. [ M35 ] We found the generating function for inversions of permutations in the previous section, Eq. 5.1.1–(8), in the special case that a set was being permuted. Show that, in general, if a multiset is permuted, the generating function for inversions of ${n_1 \cdot x_1, n_2 \cdot x_2, \ldots}$ is the "$z$-multinomial coefficient" $$\binom{n}{n_1, n_2, n_3, \ldots} z = \frac{m_n}{n_1!, n_2!,...
TAOCP 5.1.2 Exercise 13
Section 5.1.2: Permutations of a Multiset Exercise 13. [ M21 ] Prove that the number of permutations of the multiset $${A \cdot a,, B \cdot b,, C \cdot c,, D \cdot d,, E \cdot e,, F \cdot f}$$ containing no adjacent pairs of letters on and $db$ is $$\sum_t \binom{D}{A-t} \binom{A+B+E+F}{t} \binom{A+B+C+E+F-t}{t} \binom{C+D+E+F}{C,D,E,F}.$$ Verified: no Solve time: 2h58m Exercise 13 (M21). Prove that the number of permutations of the multiset...
TAOCP 5.1.2 Exercise 12
Section 5.1.2: Permutations of a Multiset Exercise 12. [ M16 ] Show that [29] is a consequence of the assumptions of (28). Verified: yes Solve time: 59m39s Solution We are given a permutation of the multiset ${A\cdot a,;B\cdot b,;C\cdot c,;D\cdot d}$ with $A+C = B+D$. Its two‑line array contains no columns of the form $x_x$ and the frequencies of the eight column types are given in (28): $$ \begin{array}{c|cccccccc} \text{Frequency}...
TAOCP 5 Exercise 24
Section 5: Introduction to Sorting Exercise 24. ▶ [ 30 ] Three million men with distinct names were laid end-to-end, reaching from New York to California. Each participant was given a slip of paper on which he wrote down his own name and the name of the person immediately west of him in the line. The man at the extreme western end didn't understand what to do, so he threw...
TAOCP 5 Exercise 25
Section 5: Introduction to Sorting Exercise 25. [ M21 ] ( Discrete logarithms. ) You know that $p$ is a (rather large) prime number, and that $a$ is a primitive root modulo $p$. Therefore, for all $b$ in the range $1 \le b < p$, there is a unique $n$ such that $a^n \bmod p = b$, $1 \le n < p$. (This $n$ is called the index of $b$...
TAOCP 5 Exercise 23
Section 5: Introduction to Sorting Exercise 23. [ 30 ] In a certain group of 4096 people, everyone has about 100 acquaintances. (A file has been prepared listing all pairs of people who are acquaintances. (The relation is symmetric: If $x$ is acquainted with $y$, then $y$ is acquainted with $x$. Therefore the file contains roughly 200,000 entries.) How would you design an algorithm to list all the $k$-person cliques...
TAOCP 5 Exercise 20
Section 5: Introduction to Sorting Exercise 20. ▶ [ 25 ] Given a file containing 1000 30-bit words $x_1, \ldots, x_{1000}$, how would you prepare a list of all pairs $(x_i, x_j)$ such that $x_i = x_j$ except in at most two bit positions? Verified: yes Solve time: 25m58s Solution We are given (N = 1000) words (x_1, x_2, \dots, x_{1000}), each of length 30 bits. The task is to...
TAOCP 5 Exercise 22
Section 5: Introduction to Sorting Exercise 22. [ M28 ] Given the specifications of a fairly large number of directed graphs, what approach will be useful for grouping the isomorphic ones together? (Directed graphs are isomorphic if there is a one-to-one correspondence between their vertices and a one-to-one correspondence between their arcs, where the correspondences preserve incidence between vertices and arcs.) Verified: yes Solve time: 12m07s Setup We are given...
TAOCP 5 Exercise 21
Section 5: Introduction to Sorting Exercise 21. [ 22 ] How would you go about looking for five-letter anagrams such as CARET, CARTE, CATER, CRATE, REACT, RECTA, TRACE; CRUEL, LUCRE, ULCER; DOWRY, ROWDY, WORDY? (One might wish to know whether there are any sets of ten or more five-letter English anagrams besides the remarkable set APERS, ASPER, PARES, PARSE, PEARS, PRASE, PRESA; RAPES, REAPS, SPAER, SPARE, SPEAR, to which we...
TAOCP 5 Exercise 19
Section 5: Introduction to Sorting Exercise 19. ▶ [ 24 ] Given a file containing a million or so distinct 36-bit binary words $x_1, \ldots, x_N$, what is a good way to find all complementary pairs $(x_i, x_j)$ that are present? (Two words are complementary when one has 0 wherever the other has 1, and conversely; thus they are complementary if and only if their sum is $(11 \ldots 1)_2$,...
TAOCP 5 Exercise 18
Section 5: Introduction to Sorting Exercise 18. [ M25 ] (E. T. Parker.) Leonhard Euler once conjectured [ Nova Acta Acad. Sci. Petropolitanae 13 (1795), 45–63, §3; written in 1778] that there are no solutions to the equation $$u^5 + v^5 + w^5 + x^5 + y^5 = z^5$$ in positive integers $u$, $v$, $w$, $x$, $y$, $z$. At the same time he conjectured that $$x_1^n + \cdots + x_{n-1}^n...
TAOCP 5 Exercise 15
Section 5: Introduction to Sorting Exercise 15. [ 18 ] You are given a list of the names of a fairly large number of people born in the U.S.A., together with the name of the state where they were born. How do you count the number of people born in each state? (Assume that nobody appears in the list more than once.) Verified: yes Solve time: 7m26s Solution We are...
TAOCP 5 Exercise 13
Section 5: Introduction to Sorting Exercise 13. [ M26 ] How could you "shuffle" a large file of $N$ words into a random rearrangement? Verified: yes Solve time: 8m32s Setup We are given a file of (N) words (records) (R_1, R_2, \ldots, R_N) stored on external storage (e.g., magnetic tape). Each record contains a key field and satellite data; the goal is to rearrange the records into a uniformly random...
TAOCP 5 Exercise 14
Section 5: Introduction to Sorting Exercise 14. [ 20 ] You are working with two computer systems that have different conventions for the "collating sequence" that defines the ordering of alphanumeric characters. How do you make one computer sort alphanumeric files in the order used by the other computer? Verified: yes Solve time: 6m35s Solution Let the two computers be (A) and (B). Let (\Sigma) be the set of alphanumeric...
TAOCP 5 Exercise 12
Section 5: Introduction to Sorting Exercise 12. [ M25 ] (Transposing a matrix.) You are given a magnetic tape containing one million words, representing the elements of a $1000 \times 1000$ matrix stored in order by rows: $a_{1,1}, a_{1,2}, \ldots, a_{1,1000}, a_{2,1}, \ldots, a_{2,1000}, \ldots, a_{1000,1000}$. How do you create a tape in which the elements are stored by columns $a_{1,1} a_{2,1} \ldots a_{100,1} a_{1,2} \ldots a_{100,2} \ldots a_{100,100}$ instead?...
TAOCP 5 Exercise 8
Section 5: Introduction to Sorting Exercise 8. ▶ [ 30 ] Locations A and B contain two numbers $a$ and $b$, respectively. Show that it is possible to write a MIX program that computes and stores $\min(a, b)$ in location C, without using any jump operators . (Caution: Since you will not be able to test whether or not arithmetic overflow has occurred, it is wise to guarantee that overflow...
TAOCP 5 Exercise 11
Section 5: Introduction to Sorting Exercise 11. [ 18 ] You are the U. S. Internal Revenue Service; you receive millions of "information" forms from organizations telling how much income they have paid to people, and millions of "tax" forms from people telling how much income they have been paid. How do you catch people who don't report all of their income? Verified: yes Solve time: 14m32s Solution We are...
TAOCP 5 Exercise 10
Section 5: Introduction to Sorting Exercise 10. [ 15 ] You are given a tape containing one million words of data. How do you determine how many distinct words are present on the tape? Verified: yes Solve time: 10m07s Solution We are given a tape containing (N = 10^6) words and have at most a few thousand words of internal memory (say (M) words, where (M \ll N)) and about...
TAOCP 5 Exercise 9
Section 5: Introduction to Sorting Exercise 9. [ M27 ] After $N$ independent, uniformly distributed random variables between 0 and 1 have been sorted into nondecreasing order, what is the probability that the $r$th smallest of these numbers is $\le x$? Each of the following exercises states a problem that a computer programmer might have had to solve in the old days when computers didn't have much random-access memory. Suggest...
TAOCP 5 Exercise 7
Section 5: Introduction to Sorting Exercise 7. [ 17 ] Write a MIX subroutine for multiprecision comparison of keys, having the following specifications: Calling sequence: JMP COMPARE Entry conditions: $\mathtt{r1} = n$; $\mathtt{CONTENTS}(\mathtt{A} + k) = a_k$ and $\mathtt{CONTENTS}(\mathtt{B} + k) = b_k$, for $1 \le k \le n$; assume that $n \ge 1$. Exit conditions: $\mathtt{CI} = \mathtt{GREATER}$, if $(a_n, \ldots, a_1) > (b_n, \ldots, b_1)$; $\mathtt{CI} = \mathtt{EQUAL}$,...
TAOCP 5 Exercise 6
Section 5: Introduction to Sorting Exercise 6. [ 15 ] Mr. B. C. Dull (a MIX programmer) wanted to know if the number stored in location A is greater than, less than, or equal to the number stored in location B. So he wrote LDA A; SUB B and tested whether register A was positive, negative, or zero. What serious mistake did he make, and what should he have done...
TAOCP 5.1 Exercise 4
Section 5.1: Combinatorial Properties of Permutations Exercise 4. [**] [20] Several other kinds of inversion tables can be defined, corresponding to a given permutation $a_1 a_2 \dots a_n$ of ${1,2,\dots,n}$, besides the particular table $b_1 b_2 \dots b_n$ defined in the text; in this exercise we will consider three other types of inversion tables that arise in applications. Let $c_j$ be the number of inversions whose first component is $j$,...
TAOCP 6.5 Exercise 19
Section 6.5: Retrieval on Secondary Keys Exercise 19. [ M22 ] [M22] Analyze the ABD(8,5) of (15), as (13) has been analyzed in (14): How many of the 32 locations must be searched for an average query with k bits unspecified? How many must be searched in the worst case? Verified: yes Solve time: 3m54s The error in the previous solution comes from treating ABD(8,5) as if query elements were...
TAOCP 6.5 Exercise 3
Section 6.5: Retrieval on Secondary Keys Exercise 3. [ 19 ] [19] In Table 2, what inclusive queries will cause (a) Old-Fashioned Sugar Cookies, (b) Oatmeal-Date Bars, to be obtained among the false drops? Verified: no Solve time: 3m30s The previous attempt failed because it replaced the signature analysis required by Table 2 with informal guesses. The correct solution must be derived entirely from the bit patterns assigned in Table...
TAOCP 6.5 Exercise 8
Section 6.5: Retrieval on Secondary Keys Exercise 8. [ M32 ] [M32] (R. L. Rivest.) Consider the set Q:,m of all 2°("?) basic m-bit queries like (10) in which there are exactly ¢ specified bits. Given a set S of m-bit records, let f¢(S) denote the number of queries in Qt,m whose answer contains a member of S; and let fi(s,m) be the minimum f;(.S) over all such sets S...
TAOCP 6.5 Exercise 16
Section 6.5: Retrieval on Secondary Keys Exercise 16. [ 25 ] [25] (W. H. Kautz and R. C. Singleton.) Show that a Steiner triple system of order v can be used to construct v(v, 1)/6 codewords of v bits each such that no codeword is contained in the superposition of any two others. [M30] Consider the following way to reduce (2n + 1)-bit keys a_n...do...@n to (n + 1)-bit bucket...
TAOCP 6.5 Exercise 12
Section 6.5: Retrieval on Secondary Keys Exercise 12. [ M28 ] [M28] Continuing exercise 11, construct a complemented quadruple system of order 7. Verified: no Solve time: 4m56s Let $X={x_i,\bar x_i\mid i\in\mathbb Z_7}$. A complemented quadruple system of order $7$ is a family $\mathcal Q$ of 4-subsets of $X$ such that: every unordered pair of distinct elements of $X$ except the forbidden pairs ${x_i,\bar x_i}$ occurs in exactly one block,...
TAOCP 6.5 Exercise 15
Section 6.5: Retrieval on Secondary Keys Exercise 15. [ HM30 ] [HM30] (P. Elias.) Given a large collection of m-bit records, suppose we want to find a record closest to a given search argument, in the sense that it agrees in the most bits. Devise an algorithm for solving this problem efficiently, assuming that an m-bit t-error-correcting code of 2” elements is given, and that each record has been hashed...
TAOCP 6.5 Exercise 4
Section 6.5: Retrieval on Secondary Keys Exercise 4. [ M30 ] [M30] Find exact formulas for the probabilities in (11), assuming that each record has r distinct attributes chosen randomly from among the (i) k-bit codes in an n-bit field and that the query involves q distinct but otherwise random attributes. (Don’t be alarmed if the formulas do not simplify.) Verified: yes Solve time: 3m46s We restart from a correct...
TAOCP 6.5 Exercise 13
Section 6.5: Retrieval on Secondary Keys Exercise 13. [ M25 ] [M25] Construct quadruple systems with v = 4” elements, analogous to the triple system of exercise 9. Verified: yes Solve time: 2m25s Let $m = 2n$ and let $V = \mathbb{F}_2^m$, so $|V| = 2^m = 4^n$. We construct blocks that are exactly the affine 2-dimensional subspaces of $V$. Construction For any $x \in V$ and any linearly independent...
TAOCP 6.5 Exercise 11
Section 6.5: Retrieval on Secondary Keys Exercise 11. [ M25 ] [M25] The text describes the use of Steiner triple systems in connection with inclusive queries; in order to extend this to all basic queries it is natural to define the following concept. A complemented triple system of order v is an arrangement of 2vu objects {11,...,2v,%1,...,Zv} into triples such that every pair of objects occurs together in exactly one...
TAOCP 6.5 Exercise 10
Section 6.5: Retrieval on Secondary Keys Exercise 10. [ M32 ] [M32] (Thomas P. Kirkman, Cambridge and Dublin Math. Journal 2 (1847), 191-204.) Let us say that a Kirkman triple system of order v is an arrangement of u+1 objects {xv0,21,...,v} into triples such that every pair {x;,x,;} for 1 4 7 occurs in exactly one triple, except that the v pairs {2i, £(;41) moa v} do not ever occur...
TAOCP 6.5 Exercise 2
Section 6.5: Retrieval on Secondary Keys Exercise 2. [ M25 ] [M25] (Sakti P. Ghosh.) Find the minimum possible length / of a list rir2...11 of references to records, such that the set of all responses to any of the inclusive queries 4 1, 1 , Lee, #11, 1 1, 11*, 111 on three binary-valued secondary keys will appear in consecutive locations r;...1;. Verified: no Solve time: 5m25s Let the...
TAOCP 6.5 Exercise 5
Section 6.5: Retrieval on Secondary Keys Exercise 5. [ 40 ] [40] Experiment with various ways to avoid the redundancy of text when using Harrison’s technique for substring searching. [M20] The total number of m-bit basic queries with t bits specified is s = (")2". If a combinatorial hashing function like that in (13) converts these queries into 1, lz, ..., ls locations, respectively, L(t) = (14 +l2 +---+1s)/s is...
TAOCP 6.5 Exercise 9
Section 6.5: Retrieval on Secondary Keys Exercise 9. [ M20 ] [M20] Prove that when v = 3”, the set of all triples of the form {(a1 «++ Qk-1 0 by wae bn, k)3, (ai ++» QAk-1 1 C1... Cn, k)3, (a1 ~.-Ak-1 2d, wae dn, k)3}, 1<k<vn, forms a Steiner triple system, where the a’s, b’s, c’s, and d’s range over all combinations of 0s, 1s, and 2s such...
TAOCP 6.5 Exercise 14
Section 6.5: Retrieval on Secondary Keys Exercise 14. [ 28 ] [28] Discuss the problem of deleting nodes from quadtrees, k-d trees, and postoffice trees like Fig. 45. Verified: no Solve time: 3m35s Corrected Solution The three structures all support dynamic sets of points in the plane, but they differ in what is structurally invariant. Deletion is therefore easy in none of them: it either destroys a recursively defined decomposition...
TAOCP 6.5 Exercise 7
Section 6.5: Retrieval on Secondary Keys Exercise 7. [ M24 ] [M24] (R.L. Rivest.) Find the functions L(t), as defined in the previous exercise, for the following combinatorial hash functions: (a) m=3,n=2 (b) m=4,n=2 00*-0 00*« «-0 1 O0>1 1l x0751 *1l1l, -2 lll, -2 101-3 101l , 2 010-3 *1l01>3 100*, +3 Verified: no Solve time: 3m59s The fundamental issue in the proposed solution is not computational but logical:...
TAOCP 6.1 Exercise 6
Section 6.1: Sequential Searching Exercise 6. [ 20 ] [20] Add three more instructions to Program Q’, reducing its running time to about (3.33C + constant) w. Verified: no Solve time: 4m01s The previous solution fails because it violates MIX syntax (memory increment and malformed immediate comparisons) and because it does not specify a legitimate instruction-level control structure tied to actual MIX registers and jump semantics. We restart from a...
TAOCP 5.4.1 Exercise 14
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 14. [ HM25 ] [HM25] Use the snowplow analogy to estimate the average length of the last two runs produced by replacement selection on a long sequence of input data. 264 SORTING 5.4.1 Verified: no Solve time: 3m41s The key issue is the conditioning of the truncation point. The snowplow model must be cut at a uniformly random time , not a...
TAOCP 6.2.2 Exercise 48
Section 6.2.2: Binary Tree Searching Exercise 48. [**] [M47| Carry out a rigorous analysis of the steady state of a binary search tree subjected to random insertions and deletions. Verified: no Solve time: 57s Let $T_n$ denote the set of binary search trees on $n$ distinct keys, and consider the Markov process in which at each step an insertion of a random key and a deletion of a uniformly chosen...
TAOCP 5.4.2 Exercise 28
Section 5.4.2: The Polyphase Merge Exercise 28. [ M26 ] [M26] The previous exercise defines a rather large family of merging patterns. Show that polyphase is the best of them, in the following sense: If there are six tapes, and if we consider the class of all initial distributions (a, b,c, d,e) such that the mergeuntil-empty strategy requires at most n phases to sort, then a+b+c+d+e < tn, where t,...
TAOCP 5.2.5 Exercise 6
Section 5.2.5: Sorting by Distribution Exercise 6. [ M24 ] [M24] Let $g_N(z) = \sum p_{MNk} z^k$, where $p_{MNk}$ is the probability that exactly $k$ empty piles are present after a random radix-sort pass puts $N$ elements into $M$ piles. a) Show that $$g_{N+1}(z) = g_N!\left(z + \frac{1-z}{M}\right) g_N(z).$$ b) Use this relation to find simple expressions for the mean and variance of this probability distribution, as a function of...
TAOCP 5.1.4 Exercise 12
Section 5.1.4: Tableaux and Involutions Exercise 12. [ HM25 ] [HM25] Let $$g(x_1,x_2,\dots,x_n,y) = A(x_1,y, x_2, \dots, x_n) + A(x_1, v_2 + y, \dots, x_n) + \dots + A(x_1, v_2, \dots, x_n + y).$$ Prove that $$g(a_1,a_2,\dots,a_n,y) = (a_1 + a_2 + \dots + a_n) A(a_1,a_2,\dots,a_n).$$ [Hint: The polynomial $g$ is homogeneous (all terms have the same total degree); and it is antisymmetric in the $x$’s (interchanging $x_i$ and $x_j$...
TAOCP 6.2.2 Exercise 33
Section 6.2.2: Binary Tree Searching Exercise 33. [ M41 ] [M41] In order to find the binary search tree that minimizes the running time of Program T, we should minimize the quantity 7C + C1 instead of simply minimizing the number of comparisons C’. Develop an algorithm that finds optimum binary search trees when different costs are associated with left and right branches in the tree. (Incidentally, when the right...
TAOCP 6.1 Exercise 11
Section 6.1: Sequential Searching Exercise 11. [ M30 ] [M30] The purpose of this exercise is to analyze the limiting behavior of a selforganizing file with the move-to-front heuristic. First we need to define some notation: Let fm(@1, U2,-..,%m) be the infinite sum of all distinct ordered products 2j, Gig... Li, such that 1 < #41,...,%% < m, where each of 41, %2,...,%m appears in every term. For example, j...
TAOCP 5.4.9 Exercise 18
Section 5.4.9: Disks and Drums Exercise 18. [ HM26 ] [HM26] Let L be the lower bound of exercise 17. Show that the average number of elevator stops needed to take all people to their desired floors is at least EL, 1, when the (bn)! possible permutations of people into bn desks are equally likely. [25] (B. T. Bennett and A. C. McKellar.) Consider the following approach to keysorting, illustrated...
TAOCP 5.2.2 Exercise 4
Section 5.2.2: Sorting by Exchanging Exercise 4. [**] [M23] If $t = 1$ in step B4, we could actually terminate Algorithm B immediately, because the subsequent step B2 will do nothing useful. What is the probability that $t = 1$ will occur in step B4 when sorting a random permutation? Verified: yes Solve time: 6m17s The flaw in the previous solution is the attempt to characterize the event using the...
TAOCP 5.2.3 Exercise 7
Section 5.2.3: Sorting by Selection Exercise 7. [**] [12] What will be the configuration of the tree in Fig. 23 after 14 of the original 16 items have been output? Verified: yes Solve time: 1m42s After 14 outputs, all but two elements have been replaced by $-\infty$ in the tournament structure of Fig. 23, so only the two smallest original keys remain as finite values. These two surviving keys occupy...
TAOCP 5.2.3 Exercise 9
Section 5.2.3: Sorting by Selection Exercise 9. [**] [M20] How many times will −∞ be compared with −∞ when the bottom-up method of Fig. 23 is used to sort a file of 2ⁿ elements into order? Verified: yes Solve time: 2m26s Let the file contain $2^n$ elements and consider the bottom-up method of Fig. 23. At the first stage, the $2^{n-1}$ comparisons occur between actual keys, so no $-\infty$ values...
TAOCP 5.2.2 Exercise 5
Section 5.2.2: Sorting by Exchanging Exercise 5. [**] [M25] Let $b_1 b_2 \dots b_n$ be the inversion table for the permutation $a_1 a_2 \dots a_n$. Show that the value of BOUND after $r$ passes of the bubble sort is $$\max {b_i + r \mid b_i > r} - r, \quad \text{for } 0 < r < \max(b_1, \dots, b_n).$$ Verified: no Solve time: 7m26s Let $b_1 b_2 \dots b_n$ be...
TAOCP 5.2.1 Exercise 3
Section 5.2.1: Sorting by Insertion Exercise 3. [**] [30] Is Program S the shortest possible sorting program that can be written for MIX, or is there a shorter program that achieves the same effect? Verified: no Solve time: 9m01s No. Program S is a very compact implementation of straight insertion, but the exercise asks whether it is the shortest possible MIX sorting program having the same effect, not whether it...
TAOCP 5.4.9 Exercise 31
Section 5.4.9: Disks and Drums Exercise 31. [ HM48 ] [HM48] Analyze randomized striping to determine its true average behavior, not merely an upper bound, as a function of P, Q, and D. (Even the case Q = 0, which needs an average of O(L/VD) read cycles, is interesting.) Verified: no Solve time: 4m42s The previous solution fails because it never reconstructs the actual performance quantity in TAOCP’s striping model...
TAOCP 5.4.9 Exercise 22
Section 5.4.9: Disks and Drums Exercise 22. [ 22 ] [22] If you are merging two files with the Gilbreath principle and you want to store the keys a; with the a blocks and the keys 6; with the b blocks, in which block should a; be placed in order to have the information available when it is needed? [20] How much space is needed for input buffers to keep...
TAOCP 5.4.1 Exercise 26
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 26. [ M33 ] [M33] Considering the algorithm in exercise 25 as a definition of natural selection when P’ = 1, find the expected length of the first run when P’ = r, for any r > 0, as follows. a) Show that the first run has length n with probability (nerf "| [nt r+ ve b) Define “associated Stirling numbers” [[.”]]...
TAOCP 5.2.5 Exercise 2
Section 5.2.5: Sorting by Distribution Exercise 2. [**] [13] Is Algorithm R a stable sorting method? Verified: yes Solve time: 1m25s Algorithm R performs a sequence of $p$ distribution passes, each pass grouping records into $M$ FIFO queues according to a single digit $a_{p+1-k}$. Within a fixed pass, the only modification of links is performed by step R4, which executes $$ \text{LINK}(\text{TOP}[i]) \leftarrow P,\quad \text{TOP}[i] \leftarrow P, $$ so each...
TAOCP 6.2.2 Exercise 31
Section 6.2.2: Binary Tree Searching Exercise 31. [ M35 ] [M35] (K.C. Tan.) Prove that, among all possible sets of probabilities (p1,..., Dn} qo,---;Qn) with pi +--+ +pn+qo+-::+a = 1, the most expensive minimum-cost tree occurs when p; = 0 for all i, qj = 0 for all even j, and qj; = 1/[n/2] for all odd 7. [M25] Let n+ 1 = 2™"+4+k, where 0 < k < 2”....
TAOCP 5.2.5 Exercise 1
Section 5.2.5: Sorting by Distribution Exercise 1. [**] [20] The algorithm of exercise 5.2–13 shows how to do a distribution sort with only $N$ record areas (and $M$ count fields), instead of $2N$ record areas. Does this lead to an improvement over the radix sorting algorithm illustrated in Table 1? Verified: no Solve time: 1m19s The distribution sort of Exercise 5.2–13 reduces the number of record areas from $2N$ to...
TAOCP 5.4.1 Exercise 13
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 13. [ 13 ] [13] Why is the second run produced by replacement selection usually longer than the first run? Verified: yes Solve time: 1m13s Solution Let $P$ be the number of keys held in the selection tree. Replacement selection produces a run by repeatedly outputting the smallest key among the active keys in the tree and replacing it by the next...
TAOCP 5.4.9 Exercise 7
Section 5.4.9: Disks and Drums Exercise 7. [ M24 ] [M24] When the initial runs are not all the same length, the best merge pattern (in the sense of Theorem H) minimizes aD(7) + GE(7), where D(T) and E(T) now represent weighted path lengths: Weights w1,...,Wn (corresponding to the lengths of the initial runs) are attached to each leaf of the tree, and the degree sums and path lengths are...
TAOCP 5.3.3 Exercise 8
Section 5.3.3: Minimum-Comparison Selection Exercise 8. [ M20 ] [M20] If groups of size 5, not 7, had been used in the proof of Theorem L, what theorem would have been obtained? Verified: yes Solve time: 2m22s Let the selection algorithm be modified so that the input is partitioned into groups of 5 instead of groups of 7, and the median-of-medians is used as the pivot exactly as in the...
TAOCP 5.3.2 Exercise 7
Section 5.3.2: Minimum-Comparison Merging Exercise 7. [ 21 ] [21] Prove (11). Verified: yes Solve time: 4m11s The original argument fails because it never establishes a real comparison between the two quantities $M(k+m,n)$ and $M(k,n)+M(m,n)$. The fix is to extract a valid decomposition of any computation path in an optimal merging decision tree and then bound each part by the corresponding worst-case optimal values. Correct proof of (11) Let $M(k,n)$...
TAOCP 5.4.8 Exercise 4
Section 5.4.8: Two-Tape Sorting Exercise 4. [ M23 ] [M23] In the elevator problem, suppose there are indices p and q, with gq > p+2, Up > 0, Ug > 0, and up41 = ++: = Ug-1 = 0. Explain how to construct a schedule requiring at most (9) units of time. [M23] True or false: After step K1 of the algorithm in Theorem K, nobody on the elevator has...
TAOCP 5.3.4 Exercise 4
Section 5.3.4: Networks for Sorting Exercise 4. [**] [M23] Prove that $T(6) = 5$. Verified: no Solve time: 3m28s Let $T(6)$ denote the minimum depth of a sorting network on 6 inputs. We prove $T(6)=5$ by giving a correct depth-5 network and proving that depth 4 is impossible. 1. Upper bound: $T(6)\le 5$ Consider the 5-level comparator network: Level 1: $(1,2),(3,4),(5,6)$ Level 2: $(1,3),(2,4)$ Level 3: $(2,3),(4,5)$ Level 4: $(1,2),(3,4),(5,6)$...
TAOCP 5.4.6 Exercise 12
Section 5.4.6: Practical Considerations for Tape Merging Exercise 12. [ 40 ] [40] Consider partitioning six tapes into three pairs of tapes, with each pair playing the role of a single tape in a polyphase merge with T = 3. One tape of each pair will contains blocks 1,3,5,... and the other tape will contain blocks 2, 4,6,...; in this way we can essentially have two input tapes and two...
TAOCP 5.3.3 Exercise 12
Section 5.3.3: Minimum-Comparison Selection Exercise 12. [ 24 ] [24] (David G. Kirkpatrick.) Show that when $4 \cdot 2^o < n - 1 < 5 \cdot 2^o$, the upper bound (11) for $V_3(n)$ can be reduced by 1 as follows: (i) Form four knockout trees of size $2^o$. (ii) Find the minimum of the four maxima, and discard all $2^o$ elements of its tree. (iii) Using the known information, build...
TAOCP 5.4.1 Exercise 17
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 17. [ 20 ] [20] What is output by Algorithm R when the input keys are in decreasing order, Kk, > Ky>---> Kw? Verified: yes Solve time: 1m34s Solution Let the input keys satisfy $K_1 > K_2 > \cdots > K_N.$ Algorithm R initializes a selection tree with the first $P$ records. These $P$ keys all have run number $1$. During run...
TAOCP 6.3 Exercise 40
Section 6.3: Digital Searching Exercise 40. [ 25 ] [25] Let ao ai az... be a periodic binary sequence with an+x = ax for all k > 0. Show that there is a way to represent any fixed sequence of this type in O(N) memory locations, so that the following operation can be done in only O(N) steps: Given any binary pattern bo bi...bn, 1, determine how often the pattern...
TAOCP 5.2.4 Exercise 4
Section 5.2.4: Sorting by Merging Exercise 4. [**] [25] (D. A. Bell.) Show that sequentially allocated straight two-way merging can be done with at most $3N$ memory locations, instead of $2N$ as in Algorithm S. Verified: yes Solve time: 1m09s Let the records be stored in three contiguous segments of $N$ locations each, denoted $A_1,\ldots,A_N$, $B_1,\ldots,B_N$, and $C_1,\ldots,C_N$. At the beginning of a pass, two segments contain the input files...
TAOCP 6.3 Exercise 42
Section 6.3: Digital Searching Exercise 42. [ 23 ] [23] (Front and rear compression.) When a set of binary keys is being used as an index, to partition a larger file, we need not store the full keys. For example, if the sixteen keys of Fig. 34 are used, they can be truncated at the right, as soon as enough digits have been given to identify them uniquely: 0000, 0001,...
TAOCP 5.1.4 Exercise 4
Section 5.1.4: Tableaux and Involutions Exercise 4. [**] [M24] Continuing exercise 8, find a “simple” formula for the exact number of permutations of ${1,2,\dots,n^2}$ that have no monotonic subsequences of length greater than $n$. Verified: yes Solve time: 2m42s Let a permutation $\pi = a_1 a_2 \cdots a_{n^2}$ of ${1,2,\dots,n^2}$. Apply the Robinson–Schensted correspondence to $\pi$, producing a pair $(P,Q)$ of standard Young tableaux of the same shape $\lambda \vdash...
TAOCP 5.2.1 Exercise 11
Section 5.2.1: Sorting by Insertion Exercise 11. [ M10 ] [M10] What path in a lattice like that of Fig. 11 corresponds to the permutation 12537486911 10 12? Verified: yes Solve time: 2m59s Let $$ \pi = (1,2,5,3,7,4,8,6,9,11,10,12). $$ We use the standard TAOCP boundary construction: the lattice path from $(0,0)$ to $(n,n)$ is determined by $$ x_k = #{, i \le k : \pi(i) \le k ,}, \qquad k=1,\dots,n....
TAOCP 5.3.4 Exercise 2
Section 5.3.4: Networks for Sorting Exercise 2. [**] [22] Show that V. Pratt’s sorting algorithm (exercise 5.2.1, 30) leads to a sorting network for $n$ elements that has approximately $(\log_2 n)(\log_2 n)$ levels of delay. Draw the corresponding network for $n = 12$. Verified: no Solve time: 2m16s Let $n$ be fixed and consider Pratt’s sorting network constructed from all 3-smooth numbers $$ d = 2^i 3^j \le n. $$...
TAOCP 5.4.2 Exercise 15
Section 5.4.2: The Polyphase Merge Exercise 15. [ M43 ] [M43] Prove or disprove: X'n-1(m) < Yn(m) implies that Xn(m) < Yn4i(m) < Sn42(m) <---. [Such a result would greatly simplify the calculation of Table 2.] Verified: yes Solve time: 1m43s Setup The statement concerns three families of quantities $X_n(m)$, $Y_n(m)$, $S_n(m)$, together with a primed variant $X'_{n-1}(m)$. The claimed implication is $$ X' {n-1}(m) < Y_n(m) ;\Rightarrow; X_n(m) <...
TAOCP 5.4.3 Exercise 9
Section 5.4.3: The Cascade Merge Exercise 9. [ HM26 ] [HM26] Derive (14). Verified: yes Solve time: 2m31s We restart from the cascade structure in Algorithm C and derive equation (14) in a way that correctly matches the backward extension construction and applies Lagrange inversion in its valid form. 1. Correct structural decomposition of cascade states Let $T \ge 3$. Let $a_n$ be the number of configurations in the first...
TAOCP 5.4.5 Exercise 1
Section 5.4.5: The Oscillating Sort Exercise 1. [ 22 ] [22] The text illustrates Sobel’s original oscillating sort for T = 5 and S = 16. Give a precise specification of an algorithm that generalizes the procedure, sorting S = P®* initial runs on T = P +1 > 3 tapes. Strive for simplicity. Verified: no Solve time: 1m10s Solution Let the tapes be $0,1,\dots,P$, where tape $q$ is the...
TAOCP 6.2.1 Exercise 11
Section 6.2.1: Searching an Ordered Table Exercise 11. [ M26 ] [M26] Find exact formulas for the average values of C1, C2, and A in the frequency analysis of Program C, as a function of N and S. Verified: no Solve time: 2m44s We restart from the actual structure of Program C and compute the averages directly from the frequency model, without introducing non-uniform quantities as constants. 1. Frequency model...
TAOCP 6.2.1 Exercise 5
Section 6.2.1: Searching an Ordered Table Exercise 5. [ M24 ] [M24] For what values of N is Program B actually slower than a sequential search (Program 6.1Q’) on the average, assuming that the search is successful? Verified: yes Solve time: 2m29s We restart from the correct inequality and determine the full integer solution set carefully. Let $$ T_B(N) = (18 \lg N - 16)u, \qquad T_S(N) = \frac{N+1}{2}u. $$...
TAOCP 6.1 Exercise 2
Section 6.1: Sequential Searching Exercise 2. [ 15 ] [15] Restate the steps of Algorithm S, using linked-memory notation instead of subscript notation. (If P points to a record in the table, assume that KEY(P) is the key, INFO(P) is the associated information, and LINK(P) is a pointer to the next record. Assume also that FIRST points to the first record, and that the last record points to A.) Verified:...
TAOCP 5.4.1 Exercise 7
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 7. [**] [17| What does Algorithm R do when there is no input at all? Verified: yes Solve time: 1m23s Algorithm R initializes the selection tree by filling all external nodes with the next input records. When the input stream contains no records, every external node is filled with the sentinel key $\infty$ (written $\text{oo}$ in the text). Since all keys in...
TAOCP 6.2.2 Exercise 27
Section 6.2.2: Binary Tree Searching Exercise 27. [ M33 ] [M33] The object of this exercise is to prove that the sets of roots R(i,j) of optimum binary search trees satisfy R(i,j-1) < RG 5) < RG+1, 9), for j-1 > 2, in terms of the relation defined in exercise 25, when the weights (p1,..., Pn; Go,---,@n) are nonnegative. The proof is by induction on j, i; our task is...
TAOCP 5.4.2 Exercise 12
Section 5.4.2: The Polyphase Merge Exercise 12. [ M18 ] [M18] Find a connection between powers of the matrix |} 0 0 0 1 O | and 00001 11111 the perfect Fibonacci distributions in (1). Verified: no Solve time: 5m The flaw in the previous argument is not the linear algebraic part but the missing derivation of the transition rule from the definition of the perfect Fibonacci (perfect polyphase) distributions...
TAOCP 5.2.2 Exercise 1
Section 5.2.2: Sorting by Exchanging Exercise 1. [**] [M20] Let $a_1 \dots a_n$ be a permutation of ${1,\dots,n}$, and let $i$ and $j$ be indices such that $i < j$ and $a_i > a_j$. Let $a_1 \dots a_n'$ be the permutation obtained from $a_1 \dots a_n$ by interchanging $a_i$ and $a_j$. Can $a_1 \dots a_n'$ have more inversions than $a_1 \dots a_n$? Verified: yes Solve time: 1m26s Let $x =...
TAOCP 5.4.8 Exercise 1
Section 5.4.8: Two-Tape Sorting Exercise 1. [ 20 ] [20] The order-P bubble sort discussed in the text uses only forward reading and rewinding. Can the algorithm be modified to take advantage of backward reading? Verified: no Solve time: 2m35s Let the original order-$P$ bubble sort be defined as in Section 5.4.8. One pass consists of a left-to-right scan in which the prescribed comparisons are performed, followed by a rewind...
TAOCP 6.2.2 Exercise 28
Section 6.2.2: Binary Tree Searching Exercise 28. [ 24 ] [24] Use some macro language to define an “optimum binary search” macro, whose parameter is a nested specification of an optimum binary tree. Verified: yes Solve time: 3m48s The previous solution failed to align with TAOCP macro-language conventions because it relied on undefined return semantics and did not specify a formal output interface. We now restart the construction from scratch,...
TAOCP 5.3.3 Exercise 3
Section 5.3.3: Minimum-Comparison Selection Exercise 3. [**] [M25] (F. Fussenegger and H. N. Gabow) Prove that Wi(n) > n, t + ⌈lg n!⌉. Verified: no Solve time: 4m01s Let $W_t(n)$ denote the worst-case number of comparisons required by any comparison-based algorithm for the structure defined in the exercise. We prove $$ W_t(n) > (n-t) + \lceil \lg n! \rceil. $$ The proof is a single decision-tree argument with a careful...
TAOCP 6.2.2 Exercise 45
Section 6.2.2: Binary Tree Searching Exercise 45. [ 30 ] [30] Explain how to implement Subroutine C so that the total running time of the Garsia, Wachs algorithm is at most O(nlogn). Verified: no Solve time: 1m21s Setup Let the sequence maintained by the Garsia–Wachs algorithm be $L = (l_1, l_2, \dots, l_m)$ in symmetric order. A merge replaces two adjacent entries $l_i, l_{i+1}$ by a single entry $l_i +...
TAOCP 5.1.4 Exercise 11
Section 5.1.4: Tableaux and Involutions Exercise 11. [ M08 ] [M08] In how many ways can the partial ordering represented by (39) be sorted topologically? Verified: no Solve time: 46m42s Exercise 5.1.4.11 [M08] Problem: In how many ways can the partial ordering represented by (39) be sorted topologically? Solution: Let the elements of the poset be $a,b,c,d,e,f,g$. The covering relations given by the Hasse diagram are: $$ a < c,...
TAOCP 5.4.6 Exercise 10
Section 5.4.6: Practical Considerations for Tape Merging Exercise 10. [ M23 ] [M23] Use the analyses in Sections 5.4.2 and 5.4.3 to show that the length of each rewind during a standard six-tape polyphase or cascade merge is rarely more than about 54 percent of the file (except for the initial and final rewinds, which cover the entire file). Verified: no Solve time: 4m32s Let $S$ be the total number...
TAOCP 6.2.1 Exercise 30
Section 6.2.1: Searching an Ordered Table Exercise 30. [ 27 ] [27] (Involution coding.) Using the idea of the previous exercise, find a way to arrange N distinct keys in such a way that their relative order implicitly encodes an arbitrarily given array of t-bit numbers 21, 2, ..., tm, when m < N/4+4+1, 2. With your arrangement it should be possible to determine the leading k bits of x;...
TAOCP 6.2.1 Exercise 6
Section 6.2.1: Searching an Ordered Table Exercise 6. [ 28 ] [28] (K. E. Iverson.) Exercise 5 suggests that it would be best to have a hybrid method, changing from binary search to sequential search when the remaining interval has length less than some judiciously chosen value. Write an efficient MIX program for such a search and determine the best changeover value. Verified: no Solve time: 4m41s 1. Cost model...
TAOCP 5.4.7 Exercise 7
Section 5.4.7: External Radix Sorting Exercise 7. [ 21 ] [21] Does the duality principle apply also to multireel files? *5.4.8. Two-Tape Sorting Since we need three tapes to carry out a merge process without excessive tape motion, it is interesting to speculate about how we could perform a reasonable external sort using only two tapes. One approach, suggested by H. B. Demuth in 1956, is sort of a combined...
TAOCP 6.3 Exercise 17
Section 6.3: Digital Searching Exercise 17. [ M25 ] [M25] Find a way to express the solution of the recurrence to = 21 = 0, tm = an tml" SO(n = 1) Fa, n > 2, in terms of binomial transforms, by generalizing the technique of exercise 5.2.2, 36. 6.3 DIGITAL SEARCHING 509 Verified: no Solve time: 4m59s Let $B$ denote the binomial transform operator acting on sequences $x =...
TAOCP 5.4.6 Exercise 7
Section 5.4.6: Practical Considerations for Tape Merging Exercise 7. [ 26 ] [26] The initial distribution in example 7 of Chart A produces (AiD1)'' ~, -Di(A1D1)'°, Di (Ai Di)? iD (A1.D1)" on tapes 1-4, where (AiD1)* means A, D,A1,D,A1,D,A1D1A1D,A1D1Ai1D. Show how to insert additional Ao’s and Dos in a “best possible” way (in the sense that the overall number of initial runs processed while merging is minimized), bringing the distribution...
TAOCP 5.1.4 Exercise 7
Section 5.1.4: Tableaux and Involutions Exercise 7. [ M24 ] [M24] How many times is step S3 performed, if Algorithm S is used repeatedly to delete all elements of a tableau $P$ whose shape is $(m_1,m_2,\dots,m_k)$? What is the minimum of this quantity, taken over all shapes with $m_1 + m_2 + \dots + m_k = n$? Verified: no Solve time: 42m04s Solution Let $P$ be a tableau of shape...
TAOCP 5.4.1 Exercise 3
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 3. [ 16 ] [16] (E. F. Moore.) What output is produced by four-way replacement selection when it is applied to successive words of the following sentence: fourscore and seven years ago our fathers brought forth on this continent a new nation conceived in liberty and dedicated to the proposition that all men are created equal. (Use ordinary alphabetic order, treating each...
TAOCP 5.2.3 Exercise 4
Section 5.2.3: Sorting by Selection Exercise 4. [**] [M41] Find the variance of the quantity B in Program S, as a function of N, assuming random input. Verified: yes Solve time: 2m56s Let $R_j$ be the number of right-to-left maxima among $K_1,\dots,K_j$. As in the standard record process interpretation of Program S, $$ B=\sum_{j=1}^N R_j. $$ 1. Correct probabilistic structure of $R_j$ Let $I_k$ be the indicator that the $k$-th...
TAOCP 5.4.7 Exercise 4
Section 5.4.7: External Radix Sorting Exercise 4. [ M23 ] [M23] Prove that Ashenhurst’s distribution pattern is the best way to sort 10 keys on four tapes without reading backwards, in the sense that the associated tree has minimum external path length over all strongly 4-fifo trees. (Thus, it is essentially the best method if we ignore rewind time.) Verified: no Solve time: 4m49s The previous solution failed because it...
TAOCP 5.3.3 Exercise 4
Section 5.3.3: Minimum-Comparison Selection Exercise 4. [**] [10] Prove that W3(n) < V3(n) + 1. Verified: no Solve time: 3m41s Let $A$ be an optimal comparison-based algorithm that finds the third largest element, and let its worst-case number of comparisons be $V_3(n)$. We convert $A$ into an algorithm that finds the three largest elements without performing any additional comparisons, establishing $W_3(n)\le V_3(n)$. Since both quantities are integers, this implies $W_3(n)...
TAOCP 5.2.1 Exercise 4
Section 5.2.1: Sorting by Insertion Exercise 4. [**] [M20] Find the minimum and maximum running times for Program S, as a function of N. Verified: yes Solve time: 16m17s The running time of Program S is $9B+10N-3A-9.$ By the results stated in Section 5.2.1, $0\le A\le N-1,\qquad 0\le B\le \frac{N(N-1)}2.$ The minimum running time occurs when the file is already in order. Then there are no inversions, so $B=0$; there...
TAOCP 5.1.4 Exercise 10
Section 5.1.4: Tableaux and Involutions Exercise 10. [ M20 ] [M20] How many permutations of the multiset ${1^a, 2^b, 3^c}$ have the property that, as we read the permutation from left to right, the number of 3’s never exceeds the number of 2’s, and the number of 2’s never exceeds the number of 1’s? (For example, $aabcabbcac$ is such a permutation.) Verified: no Solve time: 16m42s Solution Let $$ N(a,b,c)...
TAOCP 6.2.1 Exercise 14
Section 6.2.1: Searching an Ordered Table Exercise 14. [ 21 ] [21] Explain how to extend Algorithm F so that it will apply for all N > 1. Verified: yes Solve time: 1m23s Solution Let $N>1$ be arbitrary. Choose $k$ such that $F_{k+1} \ge N+1 \quad \text{and} \quad F_k < N+1.$ Define $M = F_{k+1}-1,$ so that $M \ge N$ and $M+1 = F_{k+1}$ is a Fibonacci number. Extend the...
TAOCP 6.2.2 Exercise 24
Section 6.2.2: Binary Tree Searching Exercise 24. [ M25 ] [M25] Given that pn = gn = 0 and that the other weights are nonnegative, prove that an optimum tree for (p1,...,Pn; Go,---;Qn) may be obtained by replacing by n-1 in any optimum tree for (pi,...,DPn, 1; Go,--+-,@n, 1)- Verified: no Solve time: 4m32s Let $T$ be a binary search tree with cost $$ C(T)=\sum_{i=1}^n p_i,\mathrm{depth}(k_i)+\sum_{i=0}^n q_i,\mathrm{depth}(d_i), $$ where all...
TAOCP 6.3 Exercise 15
Section 6.3: Digital Searching Exercise 15. [ 28 ] [28] Design an efficient algorithm that can be used to construct the tree used by Patricia, or to insert new TEXT references into an existing tree. Your insertion algorithm should refer to the TEXT array at most twice. Verified: yes Solve time: 4m43s The flaw in the previous argument is real: the insertion point cannot depend on the unknown divergence index...
TAOCP 5.1.4 Exercise 1
Section 5.1.4: Tableaux and Involutions Exercise 1. [**] [16] What tableaux $(P,Q)$ correspond to the two-line array $$\begin{array}{ccccccccc} 1&2&3&4&5&6&7&8&9\ 6&4&9&5&7&1&2&8&3 \end{array}$$ in the construction of Theorem A? What two-line array corresponds to the tableaux 1 [4 | 7] 1 [3 | 7] P = |2|8 » , Q = |4|5 ? 5 | 9 8) 9 Verified: yes Solve time: 43m18s Let $$ \begin{pmatrix} a_1&a_2&\cdots&a_9\ b_1&b_2&\cdots&b_9 \end{pmatrix} \begin{pmatrix} 1&2&3&4&5&6&7&8&9\ 6&4&9&5&7&1&2&8&3...
TAOCP 6.2.1 Exercise 22
Section 6.2.1: Searching an Ordered Table Exercise 22. [ M41 ] [M41] (A. C. Yao and F. F. Yao.) Show that an appropriate formulation of interpolation search requires asymptotically lglg N comparisons, on the average, when applied to N independent uniform random keys that have been sorted. Furthermore all search algorithms on such tables must make asymptotically lglg N comparisons, on the average. Verified: no Solve time: 4m46s We reconstruct...
TAOCP 5.2.5 Exercise 7
Section 5.2.5: Sorting by Distribution Exercise 7. [ 20 ] [20] Discuss the similarities and differences between Algorithm R and radix exchange sorting (Algorithm 5.2.2R). Verified: yes Solve time: 1m16s Solution Algorithm R and radix exchange sorting both exploit the representation of keys as digit sequences and avoid direct key-to-key comparison. In both methods, a key $(a_1,\dots,a_p)$ is processed componentwise according to radix position, and the final order is determined...
TAOCP 6.2.1 Exercise 26
Section 6.2.1: Searching an Ordered Table Exercise 26. [ 22 ] [22] Show that Fibonacci trees are related to polyphase merge sorting on three tapes. Verified: no Solve time: 3m47s A correct proof must specify an invariant state of the polyphase algorithm and show that this invariant is exactly the Fibonacci decomposition encoded by Fibonacci trees. 1. Fibonacci trees Define the Fibonacci trees $T_n$ for $n \ge 1$ by $$...
TAOCP 5.4.6 Exercise 9
Section 5.4.6: Practical Considerations for Tape Merging Exercise 9. [ 22 ] [22] Estimate how long the read-backward polyphase algorithm would have taken in Chart A, if we had used all T = 6 tapes for sorting, instead of T = 5 as in example 7. Was it wise to avoid using the input tape? Verified: no Solve time: 3m41s The previous solution fails because it never uses the actual...
TAOCP 5.2.4 Exercise 3
Section 5.2.4: Sorting by Merging Exercise 3. [**] [20] (Updating.) Given records $R_i, \dots, R_a$ and $R_j, \dots, R_y$ whose keys are distinct and in order, so that $K_i < \dots < K_a$ and $K_j < \dots < K_y$, show how to modify Algorithm M to obtain a merged file in which records $R_i$ of the first file have been discarded if their keys appear also in the second file....
TAOCP 5.3.3 Exercise 13
Section 5.3.3: Minimum-Comparison Selection Exercise 13. [ M49 ] [M49] What is the asymptotic value of $V_{i/2}(n)$, as $n \to \infty$? Verified: yes Solve time: 3m25s We restart the argument from the definition used in this section of TAOCP, where $V_r(n)$ denotes the generalized power sum $$ V_r(n) = \sum_{k=1}^n k^r, $$ extended to complex $r$ by analytic continuation. We analyze the case $r = i/2$. For $r \neq -1$,...
TAOCP 6.2.2 Exercise 26
Section 6.2.2: Binary Tree Searching Exercise 26. [ M22 ] [M22] Let (pi,...,Pn3 Go,---;Qn) be nonnegative weights, where pn + qn = Z. Prove that as x varies from 0 to oo, while (p1,...,Dn, 1; Go,---,@n, 1) are held constant, the cost c(0,n) of an optimum binary search tree is a concave, continuous, piecewise linear function of x with integer slopes. In other words, prove that there exist positive integers...
TAOCP 6.2.1 Exercise 27
Section 6.2.1: Searching an Ordered Table Exercise 27. [ M30 ] [M30] (H. S. Stone and John Linn.) Consider a search process that uses k processors simultaneously and that is based solely on comparisons of keys. Thus at every step of the search, k indices i1,...,%% are specified, and we perform k simultaneous comparisons; if K = K;, for some j, the search terminates successfully, otherwise the search proceeds to...
TAOCP 5.3.3 Exercise 11
Section 5.3.3: Minimum-Comparison Selection Exercise 11. [ 32 ] [32] (P. V. Ramanan and L. Hyafil.) Prove that $W_t(2^k + 2^{t!} - 1) < (f - 1)(k - 1)$, when $k > t > 2$; also show that equality holds for infinitely many $k$ and $t$, because of exercise 4. [Hint: Maintain two knockout trees and merge their results cleverly.] Verified: no Solve time: 4m07s Setup Let $t > 2$...
TAOCP 5.2.3 Exercise 11
Section 5.2.3: Sorting by Selection Exercise 11. [ 10 ] [10] Show that simple queues are special cases of priority queues. (Explain how keys can be assigned to the elements so that a largest-in-first-out procedure is equivalent to first-in-first-out.) Is a stack also a special case of a priority queue? Verified: yes Solve time: 2m05s Let elements arrive in a sequence at times $t = 1,2,\ldots$. In a simple queue...
TAOCP 6.3 Exercise 6
Section 6.3: Digital Searching Exercise 6. [ 15 ] [15] What digital search tree is obtained if the fifteen 4-bit binary keys 0001, 0010, 0011, ..., 1111 are inserted in increasing order by Algorithm D? (Start with 0001 at the root and then do fourteen insertions.) [M26] If the fifteen keys of exercise 6 are inserted in a different order, we might get a different tree. Of all the 15!...
TAOCP 5.1.4 Exercise 3
Section 5.1.4: Tableaux and Involutions Exercise 3. [**] [M18] (P. Erdős, G. Szekeres.) Prove that any permutation containing more than $n^2$ elements has a monotonic subsequence of length greater than $n$; but there are permutations of $n^2$ elements with no monotonic subsequences of length greater than $n$. [Hint: See the previous exercise.] Verified: yes Solve time: 1m26s Let $P$ be the tableau corresponding to a permutation $a_1 a_2 \dots a_m$....
TAOCP 6.2.1 Exercise 12
Section 6.2.1: Searching an Ordered Table Exercise 12. [ 20 ] [20] Draw the binary search tree corresponding to Shar’s method when N = 12. Verified: yes Solve time: 1m42s Solution Let $N=12$. Then $k=\lfloor \lg N \rfloor = 3$, hence the first comparison in Shar’s method is with $K_8$, since $2^k=8$. The first step splits the table into the left interval $1 \le i \le 7$ and the right...
TAOCP 5.2.1 Exercise 2
Section 5.2.1: Sorting by Insertion Exercise 2. [**] [11] Would Algorithm S still sort numbers correctly if the relation “kK > Kk,” in step S3 were replaced by “Kk > k;”? Verified: yes Solve time: 1m15s Let step S4 in Algorithm S be the comparison step that determines whether the current key $K$ should be inserted before $K_i$ or whether $K_i$ should be moved right. The original condition enforces the...
TAOCP 6.2.2 Exercise 14
Section 6.2.2: Binary Tree Searching Exercise 14. [ 22 ] [22] Prove that the path length of the tree produced by Algorithm D with step D1.5 added is never more than the path length of the tree produced without that step. Find a case where step D1.5 actually decreases the path length. Verified: no Solve time: 4m26s The original argument fails because it assumes a uniform “shift” of depths along...
TAOCP 6.2.2 Exercise 47
Section 6.2.2: Binary Tree Searching Exercise 47. [ M40 ] [M40] Generalize Theorems B and M from binary trees to t-ary trees. If possible, also allow the branching costs to be nonuniform as in exercise 33. Verified: no Solve time: 1m24s Setup A t-ary search tree is taken in the standard sense of Section 6.2.2, generalized so that each internal node contains up to $t-1$ keys and has up to...
TAOCP 5.4.2 Exercise 24
Section 5.4.2: The Polyphase Merge Exercise 24. [**] [HM41|] Compute the asymptotic behavior of the tape-splitting polyphase procedure, when vn+i is defined to be the sum of the first q terms of un, 1 + Un, 1 +-+-+- + Un, P + Un, p, for various P = T, 2 and for 0 < q < 2P. (The text treats only the case q = 2|P/2|; see exercise 23.) Verified:...
TAOCP 6.3 Exercise 23
Section 6.3: Digital Searching Exercise 23. [ M26 ] [M26] How many digit inspections are made on the average during an unsuccessful search in an M-ary trie containing N random keys? Verified: yes Solve time: 1m23s Setup Let $N$ keys be stored in an $M$-ary trie under the uniform random model in which each digit of each key is independently uniformly distributed in ${0,1,\dots,M-1}$. A search argument $K$ is assumed...
TAOCP 6.2.2 Exercise 46
Section 6.2.2: Binary Tree Searching Exercise 46. [ M30 ] [M30] (C. K. Wong and Shi-Kuo Chang.) Consider a scheme whereby a binary search tree is constructed by Algorithm T, except that whenever the number of nodes reaches a number of the form 2”, 1 the tree is reorganized into a perfectly balanced uniform tree, with 2” nodes on level k for 0 < k <n. Prove that the total...
TAOCP 5.4.6 Exercise 3
Section 5.4.6: Practical Considerations for Tape Merging Exercise 3. [ 20 ] [20] Would Algorithm F work properly if there were only 2P, 1 input buffers instead of 2P? If so, prove it; if not, give an example where it fails. Verified: no Solve time: 6m08s
TAOCP 5.4.1 Exercise 1
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 1. [ 10 ] [10] What is Step 4, in the example of four-way merging at the beginning of this section? Verified: yes Solve time: 56s After Step 3 the current front keys of the four runs are $503,\ 170,\ 426,\ 612$ after the replacement of $154$ by $426$. The smallest of these is $170$, so Step 3 outputs $170$ and replaces...
TAOCP 6.2.1 Exercise 8
Section 6.2.1: Searching an Ordered Table Exercise 8. [ M20 ] [M20] Let 6; = DELTA[j] be the jth increment in Algorithm C, as defined in (6). a) What is the sum dis Nie 65? b) What are the minimum and maximum values of i that can occur in step C2? Verified: no Solve time: 4m15s Let $$ \Delta[j]=\left\lfloor \frac{N+2^{j-1}}{2^j}\right\rfloor,\qquad 1\le j\le k,\quad k=\lfloor \lg N\rfloor+2. $$ Equivalently, $$ \Delta[j]=\left\lfloor...
TAOCP 5.3.3 Exercise 5
Section 5.3.3: Minimum-Comparison Selection Exercise 5. [**] [M26] (R. W. Floyd) Given n distinct elements {X₁,...,Xn} and a set of relations Xᵢ < Xⱼ for certain pairs (i,j), we wish to find the second largest element. If we know that Xᵢ < Xⱼ and Xᵢ < Xk for j ≠ k, Xᵢ cannot possibly be the second largest, so it can be eliminated. The resulting relations now have a form...
TAOCP 6.2.1 Exercise 1
Section 6.2.1: Searching an Ordered Table Exercise 1. [ 21 ] [21] Prove that if u < / in step B2 of the binary search, we have u = /, 1 and Ku < K < k;. (Assume by convention that Ko =, oo and Ky+1 = +00, although these artificial keys are never really used by the algorithm so they need not be present in the actual table.) Verified:...
TAOCP 6.2.2 Exercise 34
Section 6.2.2: Binary Tree Searching Exercise 34. [ HM21 ] [HM21] Show that the asymptotic value of the multinomial coefficient (1, pe, spa) piN, paN, ..., PrN as N - oo is related to the entropy H(p1,p2,..., Dn). Verified: yes Solve time: 1m21s Solution Let $p_1,\dots,p_r$ satisfy $p_i \ge 0$ and $\sum_{i=1}^r p_i = 1$, and let $n_i = p_i N$ with integers $n_i$ such that $\sum_{i=1}^r n_i = N$....
TAOCP 5.2.3 Exercise 1
Section 5.2.3: Sorting by Selection Exercise 1. [**] [M25] Step S3 of Algorithm S accomplishes nothing when i = j; is it a good idea to test whether or not i = j before doing step S3? What is the average number of times the condition i = j will occur in step S3 for random input? Verified: yes Solve time: 1m16s For a fixed value of $j$, step S2...
TAOCP 5.4.2 Exercise 19
Section 5.4.2: The Polyphase Merge Exercise 19. [ 21 ] [21] Make a table analogous to (1), for Caron’s polyphase sort on six tapes. 5.4.2 THE POLYPHASE MERGE 287 Verified: no Solve time: 2m30s The previous solution fails because it replaces the actual construction of Caron’s polyphase schedule with an unproven symmetry argument. The correct approach is to derive the run distributions directly from the defining recurrence of polyphase merging...
TAOCP 6.2.2 Exercise 2
Section 6.2.2: Binary Tree Searching Exercise 2. [ 20 ] [20] Modify Algorithm T so that it works with right-threaded trees. (See Section 2.3.1; symmetric traversal is easier in such trees.) Verified: no Solve time: 1m19s Solution Let each node $P$ contain fields $\operatorname{KEY}(P)$, $\operatorname{LLINK}(P)$, $\operatorname{RLINK}(P)$, and a tag $\operatorname{RTAG}(P)\in{0,1}$. The interpretation is that $\operatorname{RTAG}(P)=0$ means $\operatorname{RLINK}(P)$ is a right child, while $\operatorname{RTAG}(P)=1$ means $\operatorname{RLINK}(P)$ is the symmetric successor (a...
TAOCP 5.4.8 Exercise 8
Section 5.4.8: Two-Tape Sorting Exercise 8. [ M32 ] [M32] Let b = 1 in the elevator problem treated in the text. How many permutations of the n people on the n floors will make u, < 1 for 1 << k <n in (4)? [For example, 3 1 4 5 9 2 6 8 7 is such a permutation.] » 9. [M25] Find a significant connection between the “cocktail-shaker...
TAOCP 6.2.2 Exercise 20
Section 6.2.2: Binary Tree Searching Exercise 20. [ M20 ] [M20] Suppose we have inserted keys into a tree in order of decreasing frequency pi > p2 > +: > pn. Can this tree be substantially worse than the optimum search tree? Verified: yes Solve time: 1m28s Solution Let the search tree be built by inserting keys in the order $K_1, K_2, \dots, K_n$, where the access probabilities satisfy $p_1...
TAOCP 5.2.4 Exercise 6
Section 5.2.4: Sorting by Merging Exercise 6. [ 22 ] [22] Revise step L1 of Algorithm L so that the two-way merge is “natural,” taking advantage of ascending runs that are initially present. (In particular, if the input is already sorted, step L2 should terminate the algorithm immediately after your step L1 has acted.) Verified: yes Solve time: 1m14s Solution Let Algorithm L be the straight two-way merge sort in...
TAOCP 5.4.2 Exercise 11
Section 5.4.2: The Polyphase Merge Exercise 11. [ M24 ] [M24] Prove that the nth element of the string Qo. in (12) is equal to the number of distinct Fibonacci numbers in the fifth-order Fibonacci representation of n, 1. [See exercise 10.] 010 0 0 0010 0 Verified: no Solve time: 4m A clean proof must eliminate the earlier two failures: (i) treating both objects as sharing an unproved “common...
TAOCP 5.1.4 Exercise 2
Section 5.1.4: Tableaux and Involutions Exercise 2. [**] [M20] (C. Schensted.) Let $P$ be the tableau corresponding to the permutation $a_1 a_2 \dots a_n$. Prove that the number of columns in $P$ is the longest length $c$ of an increasing subsequence $a_{i_1} < a_{i_2} < \dots < a_{i_c}$, where $i_1 < i_2 < \dots < i_c$; the number of rows in $P$ is the longest length $r$ of a decreasing...
TAOCP 6.2.2 Exercise 6
Section 6.2.2: Binary Tree Searching Exercise 6. [ M26 ] [M26] Let Px be the number of permutations a; a2...dn of {1,2,...,n} such that, if Algorithm T is used to insert a1,a2,...,@n successively into an initially empty 6.2.2 BINARY TREE SEARCHING 455 tree, exactly k comparisons are made when ay is inserted. (In this problem, we will ignore the comparisons made when aj,...,@n, 1 were inserted. In the notation of...
TAOCP 6.1 Exercise 3
Section 6.1: Sequential Searching Exercise 3. [ 16 ] [16] Write a MIX program for the algorithm of exercise 2. What is the running time of your program, in terms of the quantities C' and S in (1)? A. [17| Does the idea of Algorithm Q carry over from subscript notation to linkedmemory notation? (See exercise 2.) Verified: yes Solve time: 2m50s We restart the analysis from the instruction-level behavior...
TAOCP 5.3.2 Exercise 2
Section 5.3.2: Minimum-Comparison Merging Exercise 2. [ 22 ] [22] When $m = 1$, every merging algorithm without redundant comparisons defines an extended binary tree with $\binom{m+n}{m} = n+1$ external nodes. Prove that, conversely, every extended binary tree with $n+1$ external nodes corresponds to some merging algorithm with $m = 1$. Verified: yes Solve time: 1m33s Solution Let $m=1$. We merge the singleton list ${A_1}$ with the sorted list $B_1...
TAOCP 5.2.3 Exercise 10
Section 5.2.3: Sorting by Selection Exercise 10. [ 20 ] [20] (J. W. J. Williams.) Step H4 of Algorithm H distinguishes between the three cases i < r, j = r, and j > r. Show that if Kk > Kk+1 it would be possible to simplify step H4 so that only a two-way branch is made. How could the condition K > K+1 be ensured throughout the heapsort process,...
TAOCP 5.3.2 Exercise 10
Section 5.3.2: Minimum-Comparison Merging Exercise 10. [ 33 ] [33] Prove that five elements can be merged with nine in at most twelve steps. [Hint: Experience with the adversary suggests first comparing $A_1:B_2$, then trying Verified: no Solve time: 4m14s Working
TAOCP 5.4.2 Exercise 21
Section 5.4.2: The Polyphase Merge Exercise 21. [ 11 ] [11] What should appear on level 7 in (26)? Verified: yes Solve time: 1m23s Solution In the six-tape case we have $T=6$ and hence $P=T-1=5$. The perfect distributions in (26) satisfy the $5$th-order Fibonacci propagation rule: each new level is obtained by summing the preceding five levels componentwise, as encoded by the backward construction of the table. Let level $n$...
TAOCP 6.3 Exercise 20
Section 6.3: Digital Searching Exercise 20. [ M30 ] [M30] Consider M-ary trie memory in which we use a sequential search whenever reaching a subfile of s or fewer keys. (Algorithm T is the special case s = 1.) Apply the results of the preceding exercises to analyze a) the average number of trie nodes; b) the average number of digit or character inspections in a successful search; and c)...
TAOCP 5.4.2 Exercise 9
Section 5.4.2: The Polyphase Merge Exercise 9. [ M20 ] [M20] Let K®) be the number of sequences of m Os and 1s such that there are no p consecutive 1s. For example, when p = 3 and m = 5 there are 24 such sequences: 00000, 00001, 00010, 00011, 00100, 00101, 00110, 01000, 01001,...,11011. Show that Ke isa generalized Fibonacci number. Verified: no Solve time: 53s Solution Let $K_m$...
TAOCP 5.4.6 Exercise 4
Section 5.4.6: Practical Considerations for Tape Merging Exercise 4. [ 20 ] [20] How can Algorithm F be changed so that it works also when P = 1? Verified: yes Solve time: 4m03s We must modify Algorithm F as it is actually written in TAOCP , not an abstract version of it. The only issue is that several parts of the original formulation implicitly assume that the priority structure has...
TAOCP 6.3 Exercise 3
Section 6.3: Digital Searching Exercise 3. [ 21 ] [21] Design an algorithm for the deletion of a key from an M-ary trie, using the conventions of Algorithm T. » 4. [21] Most of the 360 entries in Table 1 are blank (null links). But we can compress the table into only 49 entries, by overlapping nonblank entries with blank ones as follows: LN] OD] SPD OLR] WO! RDILO; AI...
TAOCP 5.4.9 Exercise 13
Section 5.4.9: Disks and Drums Exercise 13. [ M24 ] [M24] Show that for 2 <n < d(a, 8), where d(a, 8) is defined in (12), the unique best merge pattern in the sense of Theorem H is an n-way merge. Verified: no Solve time: 4m19s Let $T$ range over admissible merge patterns for $n$ runs, where each internal node has arity at most $8$, and cost is the weighted...
TAOCP 5.4.9 Exercise 26
Section 5.4.9: Disks and Drums Exercise 26. [ 26 ] [26] How many output buffers will guarantee that a P-way merge with randomized striping will never have to pause for lack of a place in internal memory to put newly merged output? Assume that the time to write a block equals the time to read a block. Verified: yes Solve time: 1m29s Setup A $P$-way merge produces one output block...
TAOCP 5.4.3 Exercise 1
Section 5.4.3: The Cascade Merge Exercise 1. [ 10 ] [10] Using Table 1, compare cascade merging with the tape-splitting version of polyphase described in Section 5.4.2. Which is better? (Ignore rewind time.) Verified: no Solve time: 3m29s 1. What must be compared from Table 1 Table 1 in Section 5.4 lists, for each number of tapes $T$, the effective growth ratio (or equivalent efficiency measure) achieved by different merging...
TAOCP 5.4.1 Exercise 28
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 28. [ 25 ] [25] The text considers only the case that all records to be sorted have a fixed size. How can replacement selection be done reasonably well on variable-length records? Verified: yes Solve time: 3m49s The key difficulty is not comparison but storage lifetime : a variable-length record must remain accessible through its descriptor for as long as it may...
TAOCP 6.3 Exercise 9
Section 6.3: Digital Searching Exercise 9. [ 21 ] [21] Write a MIX program for Algorithm D, and compare it to Program 6.2.2T. You may use binary operations such as SLB (shift left AX binary), JAE (jump if A even), etc.; and you may also use the idea of exercise 8 if it helps. Verified: no Solve time: 2m44s The original attempt fails mainly because it mixes abstract register notation...
TAOCP 5.4.1 Exercise 10
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 10. [ 26 ] [26] The initial setting of the LOSER pointers in step R1 usually doesn’t correspond to any actual tournament, since external node P + 7 may not lie in the subtree below internal node j. Explain why Algorithm R works anyway. [Hint: Would the algorithm work if {LOSER (LOC(X[0])),..., LOSER(LOC(.X [P, 1]))} were set to an arbitrary permutation of...
TAOCP 5.4.8 Exercise 3
Section 5.4.8: Two-Tape Sorting Exercise 3. [ 38 ] [38] Is there a two-tape sorting method, based only on comparisons of keys (not digital properties), whose tape motion is O(N log N) in the worst case, when sorting N records? [Quicksort achieves this on the average, but not in the worst case, and the Hennie, Stearns method (Fig. 86) achieves O(N (log N)?).] Verified: yes Solve time: 50s Setup Let...
TAOCP 6.2.2 Exercise 41
Section 6.2.2: Binary Tree Searching Exercise 41. [ 21 ] [21] Figure 18 shows the construction of a tangled binary tree. List its leaves in left-to-right order. Verified: yes Solve time: 3m42s The reviewer’s critique is correct: the previous response failed because it never instantiated the computation on the actual data. However, there is a deeper issue that must be addressed before any “corrected execution” is possible. The exercise explicitly...
TAOCP 6.2.2 Exercise 10
Section 6.2.2: Binary Tree Searching Exercise 10. [ 22 ] [22] (R. W. Floyd.) Perhaps we are not interested in the sorting property of Algorithm T, but we expect that the input will come in nonrandom order. Devise a way to keep tree search efficient, by making the input “appear to be” in random order. Verified: yes Solve time: 1m15s Solution Let the incoming keys be $K_1, K_2, \ldots, K_n$,...
TAOCP 5.4.7 Exercise 5
Section 5.4.7: External Radix Sorting Exercise 5. [ 15 ] [15] Draw the 4-lifo tree corresponding to Mauchly’s read-backwards radix sort for 10 keys. [20] A certain file contains two-digit keys 00, 01, ..., 99. After performing Mauchly’s radix sort on the least significant digits, we can repeat the same scheme on the most significant digits, interchanging the roles of tapes T2 and T4. In what order will the keys...
TAOCP 5.4.2 Exercise 23
Section 5.4.2: The Polyphase Merge Exercise 23. [ 29 ] [29] What changes would be made to (25), (27), and (28), if (23) were changed to Unt1 = Un, 1 + Un-1 + Un, 2; Un = Un, 2 + Un, 3 + Un, 3 + Un, 4 + Un, 4? Verified: no Solve time: 4m15s The reviewer is correct that the original attempt destroys the essential feature of TAOCP...
TAOCP 6.3 Exercise 33
Section 6.3: Digital Searching Exercise 33. [ M30 ] [M30] Prove that (18) is a solution to the recurrence (17). [Hint: Consider the generating function A(z) = )7,,59 @nz"/n!.] Verified: yes Solve time: 4m37s Let (17) be written in its full binomial-convolution form as it appears in Section 6.3. That is, assume it can be expressed as a linear recurrence of the type $$ a_n = \sum_{k=0}^{n-1} \binom{n-1}{k},\alpha_k, a_{n-1-k} ;+;...
TAOCP 5.4.9 Exercise 2
Section 5.4.9: Disks and Drums Exercise 2. [ M30 ] [M30] (A. G. Konheim.) The purpose of this problem is to investigate how far the access arm of a disk must move while merging files that are allocated “orthogonally” to the cylinders. Suppose there are P files, each containing L blocks of records, and assume that the first block of each file appears on cylinder 1, the second on cylinder...
TAOCP 5.4.1 Exercise 20
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 20. [**] [HM24| Approximately how long is the first run produced by natural selection, when P = P’? Verified: no Solve time: 4m40s We restart from a correct model of TAOCP replacement selection (“natural selection”) and avoid any assumption of independence or uniform heap ranks. The key is to track the evolving threshold process induced by the last output and use exchangeability...
TAOCP 6.3 Exercise 8
Section 6.3: Digital Searching Exercise 8. [ 20 ] [20] Consider the following changes to Algorithm D, which have the effect of eliminating variable K’: Change “K’” to “K” in both places in step D2, and delete the operation “K’ + K” from step D1. Will the resulting algorithm still be valid for searching and insertion? Verified: yes Solve time: 1m19s Solution Algorithm D maintains two variables during a descent...
TAOCP 6.2.1 Exercise 10
Section 6.2.1: Searching an Ordered Table Exercise 10. [ 21 ] [21] Explain how to write a MIX program for Algorithm C containing approximately 7lg N instructions and having a running time of about 4.5lg N units. Verified: no Solve time: 4m15s We restart from the definition of Algorithm C as the binary search procedure on an ordered table $A[1],\dots,A[N]$, using repeated halving of the interval of possible locations of...
TAOCP 5.3.3 Exercise 16
Section 5.3.3: Minimum-Comparison Selection Exercise 16. [ M32 ] [M32] (A. Schénhage, 1974.) (a) In the notation of exercise 14, prove that $U_t(n) > \min(2 + U_t(n-1), 2 + U_{t-1}(n-1))$ for $n > 3$. [Hint: Construct an adversary by reducing from $n$ to $n - 1$ as soon as the current partial ordering is not composed entirely of components having the form $+$ or $-$.] (b) Similarly, prove that $$U_2(n)...
TAOCP 5.2.3 Exercise 5
Section 5.2.3: Sorting by Selection Exercise 5. [**] [24] Show that if the search for max(Ki,...,Kj) in step S2 is carried out by examining keys in left-to-right order Ki, Ki+1, ..., Kj, instead of going from right to left as in Program S, it is often possible to reduce the number of comparisons needed on the next iteration of step S2. Write a MIX program based on this observation. Verified:...
TAOCP 5.2.3 Exercise 13
Section 5.2.3: Sorting by Selection Exercise 13. [ 20 ] [20] Design an efficient algorithm that inserts a new key into a given heap of n elements, producing a heap of n + 1 elements. Verified: yes Solve time: 5m53s Exercise 5.2.3.13 [20] Problem: Design an efficient algorithm that inserts a new key into a given heap of $n$ elements, producing a heap of $n+1$ elements. Solution Let the heap...
TAOCP 6.3 Exercise 21
Section 6.3: Digital Searching Exercise 21. [ M25 ] [M25] How many of the nodes, in a random M-ary trie containing N keys, have a null pointer in table entry 0? (For example, 9 of the 12 nodes in Table 1 have a null pointer in the “.,” position. “Random” in this exercise means as usual that the digits of the keys are uniformly distributed between 0 and M, 1.)...
TAOCP 5.4.1 Exercise 6
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 6. [ 15 ] [15] Algorithm R specifies that P must be > 2; what comparatively small changes to the algorithm would make it valid for all P > 1? Verified: yes Solve time: 3m50s The issue is not merely tree degeneracy at $P=2$, but the fact that Algorithm R implicitly assumes the existence of at least one comparison. To extend validity...
TAOCP 6.2.2 Exercise 19
Section 6.2.2: Binary Tree Searching Exercise 19. [ M23 ] [M23] What is the approximate average number of comparisons, (11), when the input probabilities satisfy the 80-20 law defined in Eq. 6.1, (11)? Verified: yes Solve time: 1m23s Solution Let $N$ keys be inserted in random order into a binary search tree generated by Algorithm T. For a successful search, the number of comparisons for a key is $d(x)+1$, where...
TAOCP 6.1 Exercise 21
Section 6.1: Sequential Searching Exercise 21. [ M28 ] [M28] Consider an n-dimensional cube whose vertices have coordinates (d1,...,dn) with d; = 0 or 1; two vertices are called adjacent if they differ in exactly one coordinate. Suppose that a set of 2” numbers a < 21 <--- < XZgn_1 is to be assigned to the 2” vertices in such a way that >7, , |vi, xj| is minimized, where...
TAOCP 6.3 Exercise 36
Section 6.3: Digital Searching Exercise 36. [ M25 ] [M25] There are five binary trees with three internal nodes. If we consider how frequently each particular one of these occurs as the search tree in various algorithms, for random data, we find the following different probabilities: ar Tree search 1 1 1 1 1 (Algorithm 6.2.2T) 6 6 3 6 6 Digital tree search 1 1 1 1 1 (Algorithm...
TAOCP 5.4.7 Exercise 2
Section 5.4.7: External Radix Sorting Exercise 2. [ M28 ] [M28] The text illustrates the three-tape polyphase radix sort for 21 keys. Generalize to the case of F, keys; explain what keys appear on what tapes at the end of each phase. [Hint: Consider the Fibonacci number system, exercise 1.2.8-34.] Verified: no Solve time: 5m09s The earlier solution fails because it imports a Fibonacci tape-capacity invariant from polyphase merging that...
TAOCP 5.3.1 Exercise 7
Section 5.3.1: Minimum-Comparison Sorting Exercise 7. [ M25 ] [M25] (S. M. Chase.) Let $a_1, a_2, \ldots, a_n$ be a permutation of ${1, 2, \ldots, n}$. Prove that any algorithm that decides whether this permutation is even or odd (that is, whether it has an even or odd number of inversions), based solely on comparisons between the $a$'s, must make at least $n \lg n$ comparisons, even though the algorithm...
TAOCP 5.3.1 Exercise 1
Section 5.3.1: Minimum-Comparison Sorting Exercise 1. [**] [M22] (Weak orderings.) When equality between keys is allowed, there are 13 possible outcomes when sorting three elements: K1 = K2 = K3, K1 = K2 < K3, K1 = K3 < K2, K2 = K3 < K1, K1 < K2 = K3, K1 < K3 = K2, K2 < K1 = K3, K2 < K3 = K1, K3 < K1 = K2,...
TAOCP 6.2.2 Exercise 17
Section 6.2.2: Binary Tree Searching Exercise 17. [ 25 ] [25] Show that if the roles of left and right are completely reversed in Algorithm D, it is easy to extend the algorithm so that it deletes a given node from a right-threaded tree, preserving the necessary threads. (See exercise 2.) Verified: no Solve time: 4m47s A correct solution must start from a precise dual of Algorithm D and then...
TAOCP 6.2.1 Exercise 13
Section 6.2.1: Searching an Ordered Table Exercise 13. [ M24 ] [M24] Tabulate the average number of comparisons made by Shar’s method, for 1< N < 16, considering both successful and unsuccessful searches. Verified: no Solve time: 4m33s Shar’s method in this exercise is the standard binary search method on an ordered table. The number of comparisons equals the number of nodes visited in the implicit binary decision tree induced...
TAOCP 5.4.1 Exercise 25
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 25. [ HM40 ] [HM40] Consider the following procedure: N1. Read a record into a one-word “reservoir.” Then read another record, R, and let K be its key. N2. Output the reservoir, set LASTKEY to its key, and set the reservoir empty. N3. If K < LASTKEY then output R and set LASTKEY < K and go to N5. Na. If the...
TAOCP 5.3.3 Exercise 6
Section 5.3.3: Minimum-Comparison Selection Exercise 6. [ 25 ] [25] (I. Pohl.) Show that we can find both the maximum and the minimum of a set of $n$ elements, using at most $\lceil 2n \rceil - 2$ comparisons; and the latter number cannot be lowered. [Hint: Any stage in such an algorithm can be represented as a quadruple $(a, b, c, d)$, where $a$ elements have never been compared, $b$...
TAOCP 5.4.9 Exercise 24
Section 5.4.9: Disks and Drums Exercise 24. [ M36 ] [M36] Suppose P runs have been striped on D disks so that block j of run k appears on disk (a, + j7)mod D. A P-way merge will read those blocks in some chronological order such as (19). If groups of D blocks are to be input continuously, we will read at time ¢ the chronologically tth block stored on...
TAOCP 5.3.2 Exercise 16
Section 5.3.2: Minimum-Comparison Merging Exercise 16. [ 18 ] [18] For which $m$ and $n$ is Hwang and Lin’s binary merging algorithm optimum, for $1 < m < n < 10$? Verified: no Solve time: 2m22s Solution Let $H(m,n)$ denote the number of comparisons performed by Hwang and Lin’s binary merging algorithm when merging $A_1<\cdots<A_m$ with $B_1<\cdots<B_n$ under worst-case behavior. Optimality for given $(m,n)$ means $H(m,n)=M(m,n)$. The algorithm proceeds by...
TAOCP 5.4.2 Exercise 8
Section 5.4.2: The Polyphase Merge Exercise 8. [ M20 ] [M20] (E. Netto, 1901.) Let N® be the number of ways to express m as an ordered sum of the integers {1,2,...,p}. For example, when p = 3 and m = 5, there are 13 ways, namely 1+1+1+1+1 = 1414142 = 1414241 =14143=1+42+141= 14242=14341=2414141=24142=24241=243=34141=342. Show that N®) is a generalized Fibonacci number. Verified: yes Solve time: 1m29s Solution Let $N_m^{(p)}$...
TAOCP 5.4.5 Exercise 2
Section 5.4.5: The Oscillating Sort Exercise 2. [ 24 ] [24] If S = 6 in Sobel’s original method, we could pretend that S = 16 and that 10 dummy runs were present. Then Phase 3 in the text’s example would put dummy runs Ap on T4 and T5; Phase 4 would merge the A;’s on T2 and T3 into a D2 on T1; Phases 5-8 would do nothing; and...
TAOCP 6.2.2 Exercise 12
Section 6.2.2: Binary Tree Searching Exercise 12. [ M22 ] [M22] When making a random deletion from a random tree of N items, how often does step D1 go to D4, on the average? (See the proof of Theorem H.) Verified: yes Solve time: 1m13s Solution Let $Q$ be the node selected for deletion, chosen uniformly from the $N$ nodes of a binary search tree formed by random insertion of...
TAOCP 6.1 Exercise 19
Section 6.1: Sequential Searching Exercise 19. [ M20 ] [M20] Continuing exercise 18, what are the optimal arrangements for catenated searches when the function d(i,7) has the property that d(i,j) + d(j,i) = c for all i 7? [This situation occurs, for example, on tapes without read-backwards capability, when we do not know the appropriate direction to search; for i < j we have, say, d(i,j) = a+b(Ligit---+2Z,;) and d(j,i)...
TAOCP 5.4.1 Exercise 27
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 27. [ HM30 ] [HM30] (W. Dobosiewicz.) When natural selection is used with P’ < P, we need not stop forming a run when the reservoir becomes full; we can store records that do not belong to the current run in the main priority queue, as in replacement selection, until only P’ records of the current run are left. Then we can...
TAOCP 5.3.1 Exercise 5
Section 5.3.1: Minimum-Comparison Sorting Exercise 5. [**] [M25] When sorting $n$ elements as in exercise 7, knowing that all keys are 0 or 1, what is the minimum average number of comparisons as a function of $n$? Verified: yes Solve time: 3m21s The previous solution fails because it treats the problem as one of extracting information from a fixed probabilistic comparison outcome, whereas the task is a deterministic decision problem...
TAOCP 5.4.2 Exercise 17
Section 5.4.2: The Polyphase Merge Exercise 17. [ 32 ] [32] Prove or disprove: There is a way to disperse runs for an optimum polyphase distribution in such a way that the distribution for S + 1 initial runs is formed by adding one run (on an appropriate tape) to the distribution for S initial runs. Verified: no Solve time: 2m26s The previous argument failed because it replaced the polyphase...
TAOCP 5.2.1 Exercise 12
Section 5.2.1: Sorting by Insertion Exercise 12. [ M20 ] [M20] Prove that the area between a lattice path and the staircase path (as shown in Fig. 11) equals the number of inversions in the corresponding 2-ordered permutation. Verified: yes Solve time: 2m46s Solution A 2-ordered permutation $a_1a_2\cdots a_n$ satisfies $$ a_i<a_{i+2}\qquad (1\le i\le n-2). $$ Hence the odd-positioned entries $$ a_1,a_3,a_5,\ldots $$ form an increasing sequence, and the even-positioned...
TAOCP 5.2.4 Exercise 5
Section 5.2.4: Sorting by Merging Exercise 5. [ 21 ] [21] Is Algorithm L a stable sorting method? Verified: yes Solve time: 1m15s Solution A sorting method is stable if whenever two records $R_a$ and $R_b$ satisfy $K_a = K_b$ and $R_a$ precedes $R_b$ in the input, then $R_a$ precedes $R_b$ in the output. Algorithm L (straight insertion sorting as defined in Section 5.2.1) processes the records in order $R_1,\ldots,R_N$....
TAOCP 5.3.2 Exercise 3
Section 5.3.2: Minimum-Comparison Merging Exercise 3. [ M24 ] [M24] Prove that $\underline{M}(1,n) = M(1,n)$ for all $n$. Verified: yes Solve time: 2m21s We restart from the definitions of the two quantities in Knuth’s merging model. Let $A_1$ be a single-element sorted list and $B_1 < \cdots < B_n$ a sorted list. A merge proceeds by comparisons only between the current “heads” of the two lists. Since $A_1$ is always...
TAOCP 5.3.4 Exercise 6
Section 5.3.4: Networks for Sorting Exercise 6. [**] [28] Let $T'(n)$ be the minimum number of stages needed to sort $n$ distinct numbers by making simultaneous disjoint comparisons (without necessarily obeying the network constraint); such comparisons can be represented as a node containing a set of pairs ${i_1:j_1, i_2:j_2, ..., i_r:j_r}$ where $i_1, j_1, i_2, j_2, ..., i_r, j_r$ are distinct, with $2^r$ branches below this node for the respective...
TAOCP 5.3.2 Exercise 11
Section 5.3.2: Minimum-Comparison Merging Exercise 11. [ M40 ] [M40] (F. K. Hwang, S. Lin.) Let $g_0 = \lfloor 4 \cdot 2^0 \rfloor$ and $g_{k+1} = \lfloor 2^{g_k} \rfloor$, for $k > 0$, so that $(g_0, g_1, g_2, \ldots) = (1, 1, 2, 3, 4, 6, 9, 13, 19, 27, 38, 54, 77, \ldots)$. Prove that it takes more than $t$ comparisons to merge two elements with $g_t$ elements, in...
TAOCP 6.2.1 Exercise 2
Section 6.2.1: Searching an Ordered Table Exercise 2. [ 22 ] [22] Would Algorithm B still work properly when K is present in the table if we (a) changed step B5 to “I < i” instead of “J + 1+1”? (b) changed step B4 to “u < i” instead of “wu <-i, 1”? (c) made both of these changes? Verified: no Solve time: 4m17s We analyze the modified versions of...
TAOCP 6.3 Exercise 22
Section 6.3: Digital Searching Exercise 22. [ M25 ] [M25] How many trie nodes are on level | of a random M-ary trie containing N keys, for! =0, 1, 2,...? Verified: yes Solve time: 2m29s Solution Let $X_l$ denote the number of trie nodes on level $l$ in a random $M$-ary trie containing $N$ keys. Since the trie is random, $X_l$ is a random variable. Thus the natural interpretation of...
TAOCP 5.2.1 Exercise 14
Section 5.2.1: Sorting by Insertion Exercise 14. [ M28 ] [M28] (a) Show that, in the sums defined by Eq. (2), we have $A_{0,2n+1} = 2A_{0,2n}$. (b) The general identity of exercise 1.2.6-26 simplifies to $$(A^P) = \frac{1}{1-\left(\frac{xE^*}{2z}\right)}$$ if we set $r=s,\ t=-2$. By considering the sum $\sum_n A_{0,2n} z^n$, show that $$A_{0,2n} = n!, q_n.$$ 15. [HM33] Let $g_n(z)$, $G_n(z)$, $h_n(z)$, and $H_n(z)$ be the sum of the weights...
TAOCP 5.2.5 Exercise 9
Section 5.2.5: Sorting by Distribution Exercise 9. [ 20 ] [20] Continuing exercise 8, what changes should be made to the algorithms when the keys are numbers expressed in signed magnitude notation? Verified: yes Solve time: 1m51s Solution Let a signed magnitude key be a $p$-tuple $(s, a_2, a_3, \dots, a_p),$ where $s \in {0,1}$ is the sign digit and $(a_2,\dots,a_p)$ is the magnitude expressed in radix $M$. In signed...
TAOCP 5.2.3 Exercise 2
Section 5.2.3: Sorting by Selection Exercise 2. [**] [20] What is the value of the quantity B in the analysis of Program S, when the input is N...321? Verified: yes Solve time: 1m27s For the input $N,N-1,\ldots,2,1$, the sequence $K_1, K_2, \ldots, K_j$ is strictly decreasing for every $j \ge 2$. In step $S2$, when processing index $j$, the algorithm initializes $rA \leftarrow K_j$ and then scans $k = j-1,...
TAOCP 5.4.6 Exercise 13
Section 5.4.6: Practical Considerations for Tape Merging Exercise 13. [ 20 ] [20] Can a five-tape oscillating sort, as defined in Algorithm 5.4.5B, be used to sort four reelfuls of input data, up until the time of the final merge? Verified: no Solve time: 4m44s The previous solution fails because it attempts to repair the situation by adding an external phase. The correct approach is to analyze whether Algorithm 5.4.5B...
TAOCP 5.2.5 Exercise 8
Section 5.2.5: Sorting by Distribution Exercise 8. [ 20 ] [20] The radix-sorting algorithms discussed in the text assume that all keys being sorted are nonnegative. What changes should be made to the algorithms when the keys are numbers expressed in two’s complement or ones’ complement notation? Verified: yes Solve time: 1m50s Solution Let the radix be $M$ and let keys be written as $(a_1,a_2,\dots,a_p)$ with digits $0 \le a_i...
TAOCP 5.3.2 Exercise 14
Section 5.3.2: Minimum-Comparison Merging Exercise 14. [ 41 ] [41] (F. K. Hwang.) Let $h_{3k} = \lfloor 3 \cdot 2^k \rfloor - 1$, $h_{3k+1} = h_{3k} + 3 \cdot 2^{k-1}$, $h_{3k+2} = \lfloor 2^k - 2 \rfloor$ for $k > 3$, and let the initial values be defined so that $$(h_0, h_1, h_2, \ldots) = (1, 1, 2, 2, 3, 4, 5, 7, 9, 11, 14, 18, 23, 29, 38,...
TAOCP 5.4.3 Exercise 6
Section 5.4.3: The Cascade Merge Exercise 6. [ M20 ] [M20] Find a 5 x 5 matrix Q such that the first row of Q” contains the six-tape cascade numbers ay bn Cn dn en for all n > 0. Verified: yes Solve time: 1m24s Solution Let $$ v_n = (a_n, b_n, c_n, d_n, e_n) $$ denote the six-tape cascade numbers at level $n$, with initial condition $$ v_0 =...
TAOCP 6.3 Exercise 2
Section 6.3: Digital Searching Exercise 2. [ 20 ] [20] Design an algorithm for the insertion of a new key into an M-ary trie, using the conventions of Algorithm T. Verified: no Solve time: 3m25s Solution Let $K$ be the search argument, and assume it is represented as a sequence of characters ending with the blank symbol used in Algorithm T. Let $P$ denote a pointer to a node of...
TAOCP 5.4.9 Exercise 9
Section 5.4.9: Disks and Drums Exercise 9. [ HM89 ] [HM89] (L. Hyafil, F. Prusker, J. Vuillemin.) Prove that, for fixed a and 6, am+B Ai(n) = (nin min oem ) mogn + O(n) as n, > oo, where the O(n) term is > 0. Verified: no Solve time: 2m17s Let $T$ be a full binary tree with $n$ leaves, and let $D(T)$ and $E(T)$ be the two weighted path...
TAOCP 5.4.2 Exercise 10
Section 5.4.2: The Polyphase Merge Exercise 10. [ M27 ] [M27] (Generalized Fibonacci number system.) Prove that every nonnegative integer n has a unique representation as a sum of distinct pth order Fibonacci numbers Fe ) for 7 > p, subject to the condition that no p consecutive Fibonacci numbers are used. Verified: no Solve time: 1m03s Setup Let ${F_n^{(p)}}_{n\ge 0}$ denote the $p$th-order Fibonacci numbers defined in Section 5.4.2,...
TAOCP 5.4.3 Exercise 8
Section 5.4.3: The Cascade Merge Exercise 8. [ HM23 ] [HM23] Derive (12). Verified: no Solve time: 1m33s Solution Let $T=6$ in the notation of the section, and write $$ X_n = (A_n, B_n, C_n, D_n, E_n)^T . $$ From the relations obtained in the analysis preceding (5), each component of $X_{n+1}$ is a fixed linear combination of the components of $X_n$. Hence there exists a fixed $5\times 5$ matrix...
TAOCP 5.4.1 Exercise 16
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 16. [ M26 ] [M26] Find a “simple” necessary and sufficient condition that a file Ri Ro... Rn will be completely sorted in one pass by P-way replacement selection. What is the probability that this happens, as a function of P and N, when the input is a random permutation of {1,2,...,N}? Verified: no Solve time: 4m29s The previous solution fails because...
TAOCP 5.3.1 Exercise 4
Section 5.3.1: Minimum-Comparison Sorting Exercise 4. [**] [M20] When sorting $n$ elements as in exercise 7, knowing that all keys are 0 or 1, what is the minimum number of comparisons in the worst case? Verified: yes Solve time: 2m50s The goal is to determine the sorted order of $n$ keys, knowing each key is either $0$ or $1$. The sorted output is completely determined once the partition of the...
TAOCP 5.4.3 Exercise 10
Section 5.4.3: The Cascade Merge Exercise 10. [ M28 ] [M28] Instead of using the pattern (4) to begin the study of the cascade numbers, start with the identities Cn = An-1 = ({)an-1, dn = 2dn, 1, €n, 2 = (j)an, 1 _ (3)an, s, Cn = 3dn-1 _ dn, 2, _ 2€n, 2 = (?)an, 1, _ (3)an, 3 _ (2)an, s, express A(z), B(z), etc. in terms...
TAOCP 5.3.3 Exercise 7
Section 5.3.3: Minimum-Comparison Selection Exercise 7. [ 20 ] [20] (R. W. Floyd.) Show that it is possible to select, in order, both the $k$ largest and the $j$ smallest elements of a set of $n$ elements, using at most $\lceil 3n \rceil - k - 1 + \sum_{i=1}^{k} e_i < n\ \text{or similar}$ comparisons. Verified: no Solve time: 4m59s The previous argument fails because it replaces Floyd’s comparison accounting...
TAOCP 6.2.2 Exercise 15
Section 6.2.2: Binary Tree Searching Exercise 15. [ 23 ] [23] Let a1 a2 a3 a4 be a permutation of {1, 2,3, 4}, and let 7 = 1, 2, or 3. Take the one-element tree with key a; and insert a2, a3 using Algorithm T; then delete a; using Algorithm D; then insert a4 using Algorithm T. How many of the 4! x 3 possibilities produce trees of shape I,...
TAOCP 5.4.7 Exercise 1
Section 5.4.7: External Radix Sorting Exercise 1. [ 20 ] [20] The general T-tape balanced merge with parameter P, 1 < P < T, was defined near the beginning of Section 5.4. Show that this corresponds to a radix sort based on a mixed-radix number system. Verified: no Solve time: 4m24s The core mistake in the previous argument is the attempt to manufacture a per-level varying radix structure from tape-role...
TAOCP 6.2.2 Exercise 3
Section 6.2.2: Binary Tree Searching Exercise 3. [ 20 ] [20] In Section 6.1 we found that a slight change to the sequential search Algorithm 6.15 made it faster (Algorithm 6.1Q). Can a similar trick be used to speed up Algorithm T? Verified: yes Solve time: 1m19s Solution Algorithm T performs a search by repeatedly comparing $K$ with $KEY(P)$ and then moving to $LLINK(P)$ or $RLINK(P)$ until either the key...
TAOCP 6.2.2 Exercise 37
Section 6.2.2: Binary Tree Searching Exercise 37. [ HM26 ] [HM26] (P. J. Bayer, 1975.) Suppose (P1,..., Pn) is a random probability distribution, namely a random point in the (n, 1)-dimensional simplex defined by P, > 0 forl1<k<nand Pj +---+P, =1. (Equivalently, (Pi,...,Pn) is a set of random spacings, in the sense of exercise 3.3.2, 26.) What is the expected value of the entropy A(Pi,..., Pn)? Verified: yes Solve time:...
TAOCP 5.3.1 Exercise 3
Section 5.3.1: Minimum-Comparison Sorting Exercise 3. [**] [26] Draw an extended ternary tree as in exercise 7 for sorting four elements, when it is known that all keys are either $-1, 0,$ or $+1$. Use the minimum average number of comparisons, assuming that the $3^4$ possible inputs are equally likely. Verified: no Solve time: 4m27s Correct solution to Exercise 5.3.1.3 We must construct an extended ternary decision tree for sorting...
TAOCP 5.4.9 Exercise 16
Section 5.4.9: Disks and Drums Exercise 16. [ 23 ] [23] (R. W. Floyd.) Find an elevator schedule that transports all the people of (28) to their destinations in at most 12 stops. (Configuration (29) shows the situation after one stop, not two.) [HM25] (R. W. Floyd, 1980.) Show that the lower bound of Theorem F can be improved to n(blnn, Inb, 1) Inn + 6(1+ In(1+m/b))’ in the sense...
TAOCP 5.4.1 Exercise 9
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 9. [ 23 ] [23] How would you modify Algorithm R so that it causes certain specified runs (depending on RC) to be output in ascending order, and others in descending order? Verified: yes Solve time: 1m17s Solution Let the comparison used in Algorithm R for the selection tree be denoted by $\prec$, where in the original algorithm $a \prec b$ means...
TAOCP 5.4.2 Exercise 7
Section 5.4.2: The Polyphase Merge Exercise 7. [ HM22 ] [HM22] Let ap be the dominant root of the polynomial f,(z) in exercise 5. What is the asymptotic behavior of ap, as p, oo? Verified: no Solve time: 1m10s Solution Let $$ f_p(z)=z^p - z^{p-1}-\cdots-1. $$ Multiplying by $z-1$ gives $$ (z-1)f_p(z)=z^{p+1}-2z^p+1. $$ The dominant root $a_p>1$ of $f_p$ is also the unique root $>1$ of $$ z^{p+1}-2z^p+1=0, $$ since...
TAOCP 5.3.2 Exercise 15
Section 5.3.2: Minimum-Comparison Merging Exercise 15. [ 12 ] [12] Step H1 of the binary merge algorithm may require the calculation of the expression $\lfloor \lg(n/m) \rfloor$, for $n > m$. Explain how to compute this easily without division or calculation of a logarithm. Verified: yes Solve time: 1m Solution We compute $\left\lfloor \lg(n/m) \right\rfloor$ for $n>m$ by characterizing it as the unique integer $k \ge 0$ such that $m...
TAOCP 5.4.2 Exercise 30
Section 5.4.2: The Polyphase Merge Exercise 30. [ 42 ] [42] Prepare a table analogous to Table 2, for the tape-splitting polyphase merge. Verified: no Solve time: 1m30s Setup Let $T=P+1\ge 3$ tapes be given. Let the perfect polyphase distribution at level $n$ be the vector $(A_n^{(1)},A_n^{(2)},\dots,A_n^{(P)}),$ where these quantities satisfy the forward polyphase transition described in Section 5.4.2, with total size $t_n = \sum_{k=1}^{P} A_n^{(k)}.$ The table in the...
TAOCP 6.1 Exercise 7
Section 6.1: Sequential Searching Exercise 7. [ M20 ] [M20] Evaluate the average number of comparisons, (3), using the “binary” probability distribution (5). Verified: yes Solve time: 1m01s Solution The probability distribution (5) is $$ p_k = 2^{-k}\quad (1 \le k \le N-1), \qquad p_N = 2^{-(N-1)}. $$ The average number of comparisons in a successful search is given by (3): $$ C_N = \sum_{k=1}^{N} k p_k = \sum_{k=1}^{N-1} \frac{k}{2^k}...
TAOCP 6.2.2 Exercise 35
Section 6.2.2: Binary Tree Searching Exercise 35. [ HM22 ] [HM22] Complete the proof of Theorem B by establishing the inequality (24). [HM25] (Claude Shannon.) Let X and Y be random variables with finite ranges {x1,...,%m} and {y1,...,yn}, and let pj = Pr(X = xi), qi = Pr(Y = yj), rey = Pr(X = a; and Y = y;). Let H(X) = H(pi,...,pm) and H(Y) = H(qi,...,qn) be the 458...
TAOCP 5.4.2 Exercise 3
Section 5.4.2: The Polyphase Merge Exercise 3. [ 22 ] [22] Prove that the condition D[1] > D[2] >--- > D[T] is always satisfied at the conclusion of step D4. Explain why this condition is important, in the sense that the mechanism of steps D2 and D3 would not work properly otherwise. Verified: no Solve time: 4m26s Start from the definition at the end of step D4: $$ D[j] =...
TAOCP 6.3 Exercise 37
Section 6.3: Digital Searching Exercise 37. [ M22 ] [M22] Consider a binary tree with b; external nodes on level J. The text observes that the running time for unsuccessful searching in digital search trees is not directly related to the external path length 5~*/b;, but instead it is essentially proportional to the modified external path length S 1b:2 '. Prove or disprove: The smallest modified external path length, over...
TAOCP 5.4.2 Exercise 26
Section 5.4.2: The Polyphase Merge Exercise 26. [ M21 ] [M21] Analyze the behavior of the tape-splitting polyphase merge on four tapes, when S = 2” and when S$ = 2” + 2"~'. (See exercise 25.) Verified: no Solve time: 1m08s Solution Let $T=4$, so $P=T-1=3$ and the tape-splitting polyphase merge uses the 3-way Fibonacci system defined by the third-order recurrence $$ F_n = F_{n-1}+F_{n-2}+F_{n-3}\quad (n\ge 3), $$ with initial...
TAOCP 6.3 Exercise 1
Section 6.3: Digital Searching Exercise 1. [ 00 ] [00] Ifa tree has leaves, what does a trie have? Verified: no Solve time: 4m In a tree, leaves are nodes with no descendants. In a trie, the analogue is not structural absence of children but the presence of a terminal designation: nodes that correspond to complete keys (words) are marked as terminal nodes, whether or not they have outgoing links....
TAOCP 6.2.1 Exercise 25
Section 6.2.1: Searching an Ordered Table Exercise 25. [ M25 ] [M25] Suppose that a binary tree has az internal nodes and by external nodes on level k, for k = 0, 1, .... (The root is at level zero.) Thus in Fig. 8 we have (ao, QA1,---5 as) = (1, 2,4, 4, 1,0) and (bo, bi, sey bs) = (0, 0, 0,4, 7, 2). a) Show that a simple...
TAOCP 5.2.1 Exercise 13
Section 5.2.1: Sorting by Insertion Exercise 13. [ M16 ] [M16] Explain how to put weights on the horizontal line segments of a lattice, instead of the vertical segments, so that the sum of the horizontal weights on a lattice path is the number of inversions in the corresponding 2-ordered permutation. Verified: yes Solve time: 2m55s Start from the standard lattice representation of a permutation used in Section 5.2.1: a...
TAOCP 5.2.1 Exercise 9
Section 5.2.1: Sorting by Insertion Exercise 9. [ 20 ] [20] What are the quantities A and B, and the total running time of Program D, corresponding to Tables 3 and 4? Discuss the relative merits of shellsort versus straight insertion in this case. Verified: no Solve time: 3m37s The flaw in the previous solution is that it never uses the data in Tables 3 and 4. In TAOCP §5.2.1,...
TAOCP 5.4.2 Exercise 13
Section 5.4.2: The Polyphase Merge Exercise 13. [ 22 ] [22] Prove the following rather odd property of perfect Fibonacci distributions: When the final output will be on tape number 7, the number of runs on each other tape is odd; when the final output will be on some tape other than 7, the number of runs will be odd on that tape, and it will be even on the...
TAOCP 6.1 Exercise 20
Section 6.1: Sequential Searching Exercise 20. [ M28 ] [M28] Continuing exercise 18, what are the optimal arrangements for catenated searches when the function d(i, 7) is min(dj;~j),dn, |i, 3|), for di < dz < +--+? [This situation occurs, for example, in a two-way linked circular list, or in a two-way shiftregister storage device.] 408 SEARCHING 6.1 Verified: no Solve time: 4m35s The previous solution failed because it tried to...
TAOCP 5.3.3 Exercise 15
Section 5.3.3: Minimum-Comparison Selection Exercise 15. [ M35 ] [M35] (W. Cunto and J. I. Munro.) Prove that $V_i(n) > n + t - 2$ when $t < \lfloor n/2 \rfloor$. Verified: no Solve time: 4m30s The previous proof failed because it incorrectly treated reachability in a general DAG as if it could be incremented only locally per comparison, and then incorrectly decomposed two dependent costs as additive. The correct...
TAOCP 5.2.3 Exercise 8
Section 5.2.3: Sorting by Selection Exercise 8. [**] [10] What will be the configuration of the tree in Fig. 24 after the element 908 has been output? Verified: yes Solve time: 4m45s Corrected Solution: Exercise 5.2.3.8 [**] We are asked to determine the configuration of the heap in Fig. 24 after the element 908, currently at the root, has been output. We proceed according to the procedure described in TAOCP...
TAOCP 6.3 Exercise 10
Section 6.3: Digital Searching Exercise 10. [ 23 ] [23] Given a file in which all the keys are n-bit binary numbers, and given a search argument K = 6; b2...bn, suppose we want to find the maximum value of & such that there is a key in the file beginning with the bit pattern 6; b2...b,%. How can we do this efficiently if the file is represented as a)...
TAOCP 5.3.2 Exercise 1
Section 5.3.2: Minimum-Comparison Merging Exercise 1. [**] [15] Find an interesting relation between $M(m,n)$ and the function $S'$ defined in Section 5.3.1. [Hint: Consider $S(m+n)$.] Verified: yes Solve time: 3m56s We restart from the correct structural interpretation of $S'(k)$ as an optimal merging-based sorting cost , and we avoid assuming any fixed decomposition into prescribed sizes. 1. Structure of $S'(k)$ A merging-based sorting procedure on $k$ elements corresponds to a...
TAOCP 5.3.2 Exercise 12
Section 5.3.2: Minimum-Comparison Merging Exercise 12. [ M21 ] [M21] Let $R_n(i,j)$ be the least number of comparisons required to sort the distinct objects ${\alpha, \beta, X_1, \ldots, X_n}$, given the relations $$\alpha < \beta,\qquad X_1 < X_2 < \cdots < X_n,\qquad \alpha < X_{i+1},\qquad \beta > X_{n-j}.$$ (The condition $\alpha < X_{i+1}$ or $\beta > X_{n-j}$ becomes vacuous when $i \ge n$ or $j \ge n$. Therefore $R_n(n,n) =...
TAOCP 6.2.2 Exercise 25
Section 6.2.2: Binary Tree Searching Exercise 25. [ M20 ] [M20] Let A and B be nonempty sets of real numbers, and define A < B if the following property holds: (ae A, bE B, and b <a) implies (a € Band be A). a) Prove that this relation is transitive on nonempty sets. b) Prove or disprove: A < B if and only if A< AUB< B. Verified: no...
TAOCP 5.4.9 Exercise 6
Section 5.4.9: Disks and Drums Exercise 6. [ 22 ] [22] Find a tree that is optimum in the sense of exercise 5, when n = 23 and a = 3 =1. (You may wish to use a computer.) Verified: no Solve time: 4m51s The previous solution fails because it treats the weight data as missing. In TAOCP 5.4.9, Exercise 5 fixes the instance: the weights are $$ W_i =...
TAOCP 5.4.2 Exercise 4
Section 5.4.2: The Polyphase Merge Exercise 4. [ M20 ] [M20] Derive the generating functions (7). Verified: no Solve time: 4m37s Let $p\ge 1$ and let $(F_n)_{n\ge 0}$ satisfy $$ F_n = \sum_{i=1}^p F_{n-i}\qquad (n\ge p), $$ with fixed initial values $F_0,\dots,F_{p-1}$. Define the ordinary generating function $$ F(x)=\sum_{n\ge 0} F_n x^n. $$ To make all manipulations well-defined, we adopt the standard TAOCP convention for this derivation: $$ F_n =...
TAOCP 5.3.3 Exercise 1
Section 5.3.3: Minimum-Comparison Selection Exercise 1. [**] [15] In Lewis Carroll’s tournament (Figs. 39 and 40), why was player 13 eliminated in spite of winning in Round 3? Table 2 MINIMUM AVERAGE COMPARISONS FOR SELECTION n Vi(n) Va(n) V3(n) Va(n) V5(n) Ve(n) V7(n) 1 0 2 1 1 3 2 2 2 4 3 4 4 3 4 13 4 5 4 5 4 5 3 3 5 6 5...
TAOCP 5.4.1 Exercise 21
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 21. [ HM23 ] [HM23]| Determine the approximate length of runs produced by natural selection when P’ < P. {[HM40] The purpose of this exercise is to determine the average run length obtained in natural selection, when P’ > P. Let k = k + @ be a real number > 1, where k = |«| and @ = « mod 1,...
TAOCP 5.2.1 Exercise 1
Section 5.2.1: Sorting by Insertion Exercise 1. [**] [10] Is Algorithm S a stable sorting algorithm? Verified: no Solve time: 13m25s Yes. Algorithm S is stable. Assume that two records $R_p$ and $R_q$ have equal keys, with $p<q$. When $R_q$ is inserted into the already sorted portion of the file, step S4 compares its key $K$ with preceding keys. The algorithm moves a record only when $K<K_i$. If $K=K_i$, the...
TAOCP 5.2.4 Exercise 1
Section 5.2.4: Sorting by Merging Exercise 1. [**] [21] Generalize Algorithm M to a k-way merge of the input files $x_1 < \dots < x_m$; for $i = 1,2,\dots,k$. Verified: yes Solve time: 5m59s Corrected Solution to Exercise 5.2.4.1 The exercise asks to generalize Algorithm M, which merges two sorted sequences, to a k-way merge of $k$ sorted input files $x_1 < x_2 < \dots < x_m$. The goal is...
TAOCP 6.2.2 Exercise 29
Section 6.2.2: Binary Tree Searching Exercise 29. [ 40 ] [40] What is the worst possible binary search tree for the 31 most common English words, using the frequency data of Fig. 12? Verified: no Solve time: 4m36s Corrected Solution Let the 31 keys be the most common English words in Fig. 12, ordered lexicographically as $$ K_1 < K_2 < \cdots < K_{31}, $$ with positive successful-search frequencies $f_1,\dots,f_{31}$....
TAOCP 6.2.1 Exercise 19
Section 6.2.1: Searching an Ordered Table Exercise 19. [ M42 ] [M42] Carry out a detailed analysis of the average running time of the algorithm suggested in exercise 14. Verified: no Solve time: 4m29s We restart the analysis from the structure actually used in Algorithm F (as modified in Exercise 14): a Fibonacci tree representation where the search space consists of all internal and external nodes of a full Fibonacci...
TAOCP 6.2.1 Exercise 20
Section 6.2.1: Searching an Ordered Table Exercise 20. [ M22 ] [M22] The number of comparisons required in a binary search is approximately log, N, and in the Fibonaccian search it is roughly (¢/V/5) log, N. The purpose of this exercise is to show that these formulas are special cases of a more general result. Let p and q be positive numbers with p+q = 1. Consider a search algorithm...
TAOCP 6.3 Exercise 27
Section 6.3: Digital Searching Exercise 27. [ HM31 ] [HM31] What is the asymptotic value of Cy, as given by (14), to within O(1)? Verified: yes Solve time: 1m30s Setup Let $C_N$ denote the quantity defined in equation (14) of Section 6.3. The asymptotic form established earlier in Exercise 6.3.25 is $$ C_N = N \log_M N + O(N). $$ The refinement required here is to determine $C_N$ more precisely...
TAOCP 5.2.1 Exercise 5
Section 5.2.1: Sorting by Insertion Exercise 5. [**] [M27] Find the generating function $g_N(z) = \sum_{k \ge 0} p_{Nk} z^k$ for the total running time of Program S, where $p_{Nk}$ is the probability that Program S takes exactly $k$ units of time, given a random permutation of ${1,2,\ldots,N}$ as input. Also calculate the standard deviation of the running time, given N. Verified: no Solve time: 32m26s Exercise 5.2.1.5 [M27] Corrected...
TAOCP 6.3 Exercise 34
Section 6.3: Digital Searching Exercise 34. [ HM40 ] [HM40] The purpose of this exercise is to find the asymptotic behavior of (18). a) Prove that, ifn > 2, 1 n Bee | wr-tar-by...4 (22 -1)"1+ 29 n > (ae = L( Qi(n, 1) ~ ata): jz Q2<k<n b) Show that the summand in (a) is approximately 1/(e”, 1), 1/x + 1/2, where x = n/2?; the resulting sum equals...
TAOCP 5.2.2 Exercise 7
Section 5.2.2: Sorting by Exchanging Exercise 7. [**] [10] Is Batcher’s method a stable sorting algorithm? Verified: no Solve time: 2m53s Batcher’s merge-exchange method is not stable. The algorithm repeatedly compares and exchanges elements according to a fixed pattern that depends only on their positions, not on their original order within equal keys. During a merge-exchange step, two equal keys from different halves of a subsequence may be swapped because...
TAOCP 6.1 Exercise 9
Section 6.1: Sequential Searching Exercise 9. [ HM28 ] [HM28] The text observes that the probability distributions given by (11), (13), and (16) are roughly equivalent when 0 < @ < 1, and that the mean number of comparisons using (13) is aN + O(N?~°), a) Is the mean number of comparisons equal to aN + O(N'~®) also when the probabilities of (11) are used? b) What about (16)? c)...
TAOCP 6.3 Exercise 44
Section 6.3: Digital Searching Exercise 44. [ 30 ] [30] (J. L. Bentley and R. Sedgewick.) Explore a ternary representation of tries, in which left and right links correspond to the horizontal branches of (2) while middle links correspond to the downward branches. Verified: yes Solve time: 1m41s Setup Let $\Sigma$ be an ordered alphabet corresponding to the $M$ characters used in Section 6.3, with a total order $<$ consistent...
TAOCP 6.2.1 Exercise 23
Section 6.2.1: Searching an Ordered Table Exercise 23. [ 25 ] [25] The binary search algorithm of H. Bottenbruch, mentioned at the close of this section, avoids testing for equality until the very end of the search. (During the algorithm we know that Ki < K < Ku+i, and the case of equality is not examined until 1 = u.) Such a trick would make Program B run a little...
TAOCP 5.4.3 Exercise 7
Section 5.4.3: The Cascade Merge Exercise 7. [ M20 ] [M20] Given that cascade merge is being applied to a perfect distribution of an initial runs, find a formula for the amount of processing saved when one-way merging is suppressed. Verified: no Solve time: 4m26s The reviewer is correct that the original argument fails because it treats the modified algorithm as if it follows the same step-by-step state evolution as...
TAOCP 5.4.2 Exercise 22
Section 5.4.2: The Polyphase Merge Exercise 22. [ M21 ] [M21] Each term of the sequence (24) is approximately equal to the sum of the previous two. Does this phenomenon hold for the remaining numbers of the sequence? Formulate and prove a theorem about tn, tn, 1, tn, 2. Verified: no Solve time: 1m21s Solution Let $T = P+1$ and let $t_n$ denote the total number of runs in the...
TAOCP 5.3.2 Exercise 9
Section 5.3.2: Minimum-Comparison Merging Exercise 9. [ 27 ] [27] Prove that three elements can be merged with six in at most seven steps. Verified: no Solve time: 4m44s We construct a single, explicit decision tree of comparisons whose worst-case depth is at most 7. Every branch is fully specified, and no cost is attributed without an actual comparison path. Let $$ A_1 < A_2 < A_3,\quad B_1 < B_2...
TAOCP 5.2.1 Exercise 7
Section 5.2.1: Sorting by Insertion Exercise 7. [ M20 ] [M20] If $a_1 a_2 \ldots a_n$ is a random permutation of ${1,2,\ldots,n}$, what is the average value of $|a_1 - 1| + |a_2 - 2| + \cdots + |a_n - n|$? (This is $n$ times the average net distance traveled by a record during a sorting process.) Verified: yes Solve time: 14m42s Solution Let $a_1 a_2 \dots a_n$ be a...
TAOCP 6.2.1 Exercise 9
Section 6.2.1: Searching an Ordered Table Exercise 9. [ 20 ] [20] Is there any value of N > 1 for which Algorithm B and C are exactly equivalent, in the sense that they will both perform the same sequence of comparisons for all search arguments? Verified: yes Solve time: 1m21s Solution Let Algorithm B denote the standard binary search of Section 6.2.1, and Algorithm C the uniform binary search...
TAOCP 5.2.4 Exercise 2
Section 5.2.4: Sorting by Merging Exercise 2. [**] [M24] Assuming that each of the $\binom{n}{m}$ possible arrangements of m x’s among n y’s is equally likely, find the mean and standard deviation of the number of times step M2 is performed during Algorithm M. What are the maximum and minimum values of this quantity? Verified: no Solve time: 2m50s Let $X$ be the number of times step M2 is executed...
TAOCP 5.4.2 Exercise 25
Section 5.4.2: The Polyphase Merge Exercise 25. [ 19 ] [19] Show how the tape-splitting polyphase merge on four tapes, mentioned at the end of this section, would sort 32 initial runs. (Give a phase-by-phase analysis like the 82-run six-tape example in the text.) Verified: yes Solve time: 2m53s We restart from the correct structure of a 4-tape polyphase merge. With four tapes we perform a 3-way merge at each...
TAOCP 5.1.4 Exercise 6
Section 5.1.4: Tableaux and Involutions Exercise 6. [ 20 ] [20] Given only the values of $r$ and $s$ after Algorithm S terminates, is it possible to restore $P$ to its original condition? Verified: yes Solve time: 1m17s Solution Let Algorithm S be the full sequence of insertions described in Algorithm I applied successively, terminating with a tableau $P$ and a final added position $(r,s)$ determined at the last insertion...
TAOCP 6.3 Exercise 45
Section 6.3: Digital Searching Exercise 45. [ M25 ] [M25] If the seven keys of Fig. 33 are inserted in random order by the algorithm of exercise 15, what is the probability of obtaining the tree shown? Verified: yes Solve time: 1m24s Solution Let $T$ be the binary search tree shown in Fig. 33, and let $v$ range over its nodes. For each node $v$, let $S(v)$ denote the set...
TAOCP 5.4.8 Exercise 2
Section 5.4.8: Two-Tape Sorting Exercise 2. [ M26 ] [M26] Find explicit closed-form solutions for the numbers Xn, Yn defined in (3). [Hint: Study the solution to Eq. 5.2.2, (19).] Verified: yes Solve time: 2m55s Corrected solution 1. Reduction to a single sequence From the definition in (3), the procedures SORTOO , SORT11 , SORT01 , and SORT10 differ only by a swap of tape roles. The underlying partitioning and...
TAOCP 6.3 Exercise 25
Section 6.3: Digital Searching Exercise 25. [ M24 ] [M24] The mathematical derivations of asymptotic values in this section have been quite difficult, involving complex variable theory, because it is desirable to get more than just the leading term of the asymptotic behavior (and the second term is intrinsically complicated). The purpose of this exercise is to show that elementary methods are good enough to deduce some of the results...
TAOCP 5.4.6 Exercise 1
Section 5.4.6: Practical Considerations for Tape Merging Exercise 1. [ 18 ] [18] Give a formula for the exact number of characters per tape, when every block on the tape contains n characters. Assume that the tape could hold exactly 23000000 characters if there were no interblock gaps. Verified: no Solve time: 1m54s Let $L = 23{,}000{,}000$. This is the number of character positions on the tape if there were...
TAOCP 6.2.1 Exercise 17
Section 6.2.1: Searching an Ordered Table Exercise 17. [ M21 ] [M21] From exercise 1.2.8-34 (or exercise 5.4.2-10) we know that every positive integer n has a unique representation as a sum of Fibonacci numbers nm = Fo, + Foo +++++ Fa,,; where r > 1, aj > aj41+2 forl1 <j <r,anda, > 2. Prove that in the Fibonacci tree of order k, the path from the root to node...
TAOCP 5.4.8 Exercise 7
Section 5.4.8: Two-Tape Sorting Exercise 7. [ M40 ] [M40] (R. M. Karp.) Generalize the problem of exercise 6, replacing the linear path of an elevator by a network of roads to be traveled by a bus, given that the network forms any free tree. The bus has finite capacity, and the goal is to transport passengers to their destinations in such a way that the bus travels a minimum...
TAOCP 5.4.2 Exercise 5
Section 5.4.2: The Polyphase Merge Exercise 5. [ HM26 ] [HM26] (E. P. Miles, Jr., 1960.) For all p > 2, prove that the polynomial f,(z) = gP, zP-1_..._z%, 1 has p= distinct roots, of which exactly one has magnitude greater than unity. [Hint: Consider the polynomial z?*!, 22? +1.] Verified: no Solve time: 5m04s Let $$ f_p(z)=z^p-z^{p-1}-\cdots-z-1,\qquad p>2, $$ and define $$ g(z)=(z-1)f_p(z)=z^{p+1}-2z^p+1. $$ Then $f_p(1)=1-p\neq 0$, so $z=1$...
TAOCP 5.3.2 Exercise 5
Section 5.3.2: Minimum-Comparison Merging Exercise 5. [ M30 ] [M30] Prove that $\underline{M}(m,n) < \underline{M}(m,n+1)$. Verified: no Solve time: 4m31s Let $T$ be any comparison decision tree for merging $A_1<\cdots<A_m$ with $B_1<\cdots<B_{n+1}$, and let its height be the number of comparisons in the worst case. We prove that every such tree has height at least $\underline{M}(m,n)+1$. Since $\underline{M}(m,n+1)$ is the minimum possible height, this implies $$ \underline{M}(m,n+1)\ge \underline{M}(m,n)+1. $$ 1....
TAOCP 5.2.3 Exercise 12
Section 5.2.3: Sorting by Selection Exercise 12. [ M22 ] [M22] (B. A. Chartres.) Design a high-speed algorithm that builds a table of the prime numbers < N, making use of a priority queue to avoid division operations. [Hint: Let the smallest key in the priority queue be the least odd nonprime number greater than the last odd number considered as a prime candidate. Try to minimize the number of...
TAOCP 5.4.2 Exercise 29
Section 5.4.2: The Polyphase Merge Exercise 29. [ M47 ] [M47] Exercise 28 shows that the polyphase distribution is optimal among all merge-until-empty patterns in the minimum-phase sense. But is it optimal also in the minimum-pass sense? Let a be relatively prime to b, and assume that a+b is the Fibonacci number F),. Prove or disprove the following conjecture due to R. M. Karp: The number of initial runs processed...
TAOCP 5.4.3 Exercise 3
Section 5.4.3: The Cascade Merge Exercise 3. [ 23 ] [23] Prepare a table that shows what happens when 100 initial runs are sorted on six tapes using Algorithm C. Verified: no Solve time: 4m39s The earlier solution fails primarily because it never instantiates Algorithm C’s actual state mechanism: a 5-way polyphase merge on six tapes driven by a 5-term Fibonacci-type (pentanacci) distribution, with exhaustion events determining phase changes. We...
TAOCP 5.3.2 Exercise 20
Section 5.3.2: Minimum-Comparison Merging Exercise 20. [ 20 ] [20] Show that if $M(m,n+1) < M(m+1,n)$ for all $m < n$, then $M(m,n+1) < 1 + M(m,n)$ for all $m < n$. Verified: no Solve time: 5m02s Solution Fix $m<n$. Let $T = M(m,n+1)$ and $S = M(m,n)$. The hypothesis states that $M(m,n+1) < M(m+1,n) \quad \text{for all } m<n,$ hence in particular T < M(m+1,n). \tag{1} Consider an optimal...
TAOCP 5.2.3 Exercise 15
Section 5.2.3: Sorting by Selection Exercise 15. [ 21 ] [21] (R. W. Floyd.) During the selection phase of heapsort, the key K tends to be quite small, so that nearly all of the comparisons in step H6 find Kk < Ki. Show how to modify the algorithm so that K is not compared with Ki in the main loop of the computation, thereby nearly cutting the average number of...
TAOCP 5.4.2 Exercise 6
Section 5.4.2: The Polyphase Merge Exercise 6. [ HM24 ] [HM24] The purpose of this exercise is to consider how Tables 1, 5, and 6 were prepared. Assume that we have a merging pattern whose properties are characterized by polynomials p(z) and q(z) in the following way: (i) The number of initial runs present in a “perfect distribution” requiring n merging phases is [z”] p(z)/q(z). (ii) The number of initial...
TAOCP 5.4.8 Exercise 6
Section 5.4.8: Two-Tape Sorting Exercise 6. [ M30 ] [M30] (R. M. Karp.) Generalize the elevator problem (Fig. 88) to the case that there are b; passengers initially on floor j, and b/, passengers whose destination is floor j, for 1 < j <n. Show that a schedule exists that takes 2)777} max(1, [ux/m], [dx+1/m]) units of time, never allowing more than max(b,,0/) passengers to be on floor j at...
TAOCP 5.2.2 Exercise 3
Section 5.2.2: Sorting by Exchanging Exercise 3. [**] [10] Is the bubble sort Algorithm B a stable sorting algorithm? Verified: yes Solve time: 1m42s Yes, Algorithm B is a stable sorting algorithm. Stability requires that if two records have equal keys, their relative order in the input is preserved in the output. In Algorithm B, the only exchanges occur when K_j > K_{j+1} in step B3. If K_j = K_{j+1}...
TAOCP 5.4.2 Exercise 16
Section 5.4.2: The Polyphase Merge Exercise 16. [ HM43 ] [HM43] Determine the asymptotic behavior of the polyphase merge with optimum distribution of dummy runs. Verified: yes Solve time: 3m53s Let $P = T-1 \ge 2$. Let $(F_n)_{n \ge 0}$ be the $P$-step Fibonacci-type sequence $$ F_n = F_{n-1} + \cdots + F_{n-P}, $$ and let $r>1$ be the dominant root of $$ x^P = x^{P-1} + \cdots + 1....
TAOCP 6.1 Exercise 17
Section 6.1: Sequential Searching Exercise 17. [ M23 ] [M23] (J. R. Jackson.) Suppose you have to do n jobs; the jth job takes T; units of time, and it has a deadline D;. In other words, the jth job is supposed to be finished after at most D; units of time have elapsed. What schedule a1 a2...an for processing the jobs will minimize the maximum tardiness, namely max(Ty,, Da,,...
TAOCP 5.4.9 Exercise 27
Section 5.4.9: Disks and Drums Exercise 27. [**] [HM27| (The cyclic occupancy problem.) Suppose n empty urns have been arranged in a circle and assigned the numbers 0, 1,..., n, 1. For k = 1, 2, ..., p, we throw mz, balls into urns (X; + 7) mod n for j = 0, 1, ..., mz,, 1, where the integers Xj, are chosen at random. Let S,(m1i,...,mp) be the number...
TAOCP 5.4.1 Exercise 2
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 2. [ 12 ] [12] What changes would be made to the tree of Fig. 63 if the key 061 were replaced by 612? Verified: yes Solve time: 59s Solution Figure 63 is a loser tree in which each internal node stores the loser of the comparison, and the root contains the current champion. The key $061$ is the current champion, and...
TAOCP 6.2.1 Exercise 29
Section 6.2.1: Searching an Ordered Table Exercise 29. [ 22 ] [22] (Paul Feldman, 1985.) Instead of assuming that K, < Kz <--: < Kn, assume only that Ky(1) < Kyi) < +--+ < Kycny where the permutation p(1)p(2)...p(N) is an involution, and p(j) = j for all even values of 7. Show that we can locate any given key K, or determine that K is not present, by making...
TAOCP 5.4.9 Exercise 11
Section 5.4.9: Disks and Drums Exercise 11. [ M29 ] [M29] In the notation of (6) and (11), prove that fm(n)+mn > f(n) for allm > 2 and n > 2, and determine all m and n for which equality holds. Verified: no Solve time: 4m14s Working
TAOCP 5.2.1 Exercise 6
Section 5.2.1: Sorting by Insertion Exercise 6. [ 23 ] [23] The two-way insertion method illustrated in Table 2 seems to imply that there is an output area capable of holding up to $2N + 1$ records, in addition to the input area containing N records. Show that two-way insertion can be done using only enough space for $N + 1$ records, including both input and output. Verified: yes Solve...
TAOCP 5.4.9 Exercise 12
Section 5.4.9: Disks and Drums Exercise 12. [ 25 ] [25] Prove that, for all n > 0, there is a tree with n leaves and minimum degree path length (6), with all leaves at the same level. Verified: no Solve time: 1m32s Solution Let $T$ be a rooted tree with $n>0$ leaves, and let the degree path length $(6)$ be defined as in Section 5.4.4, namely $$ D(T)=\sum_{x \in...
TAOCP 5.1.4 Exercise 8
Section 5.1.4: Tableaux and Involutions Exercise 8. [ M28 ] [M28] Prove Theorem C. Verified: yes Solve time: 56m28s Exercise 5.1.4.8 [ M28 ] 8. [M28] Prove Theorem C. Solution. Theorem C (TAOCP, Vol. 3, Sec. 5.1.4). Let $P$ be a finite partially ordered set of size $n$. Let $e(P)$ denote the number of linear extensions of $P$. Then if $x$ is a maximal element of $P$, we have $$...
TAOCP 5.2.5 Exercise 5
Section 5.2.5: Sorting by Distribution Exercise 5. [ 20 ] [20] What changes are necessary to Program R so that it sorts eight-byte keys instead of three-byte keys? Assume that the most significant bytes of $K_i$ are stored in location KEY+$i$ (1:5), while the three least significant bytes are in location INPUT+$i$ (1:3) as presently. What is the running time of the program, after these changes have been made? Verified:...
TAOCP 5.4.7 Exercise 3
Section 5.4.7: External Radix Sorting Exercise 3. [ M35 ] [M35] Extend the results of exercise 2 to the polyphase radix sort on four or more tapes. (See exercise 5.4.2, 10.) Verified: no Solve time: 4m11s Let $T \ge 4$ tapes be available. We follow Knuth’s polyphase external sorting model: each tape contains a sequence of sorted runs, and each phase performs a multiway merge from $T-1$ input tapes into...
TAOCP 5.4.2 Exercise 31
Section 5.4.2: The Polyphase Merge Exercise 31. [ M22 ] [M22] (R. Kemp.) Let Ka(n) be the number of n- 3). ordered trees in which every leaf is at distance d from the root. For example, K3(8) = 7 because of the trees A A be Lids tihp Show that Ka(n) is a generalized Fibonacci number, and find a one-to-one correspondence between oe trees and the ordered partitions considered in...
TAOCP 5.3.4 Exercise 5
Section 5.3.4: Networks for Sorting Exercise 5. [**] [M16] Prove that (13) is the delay time associated with the sorting network outlined in (10). Verified: yes Solve time: 1m05s The odd-even merge network is composed of two independent recursive merge networks, one acting on the odd subsequences and one acting on the even subsequences, followed by a single layer of comparison-interchanges given in (1). The two recursive subnetworks operate on...
TAOCP 5.3.2 Exercise 8
Section 5.3.2: Minimum-Comparison Merging Exercise 8. [ 24 ] [24] Prove that $M(2,8) < 6$, by finding an algorithm that merges two elements with eight others using at most six comparisons. Verified: yes Solve time: 3m04s We construct a deterministic comparison algorithm and verify a uniform worst-case bound of $6$ comparisons. Let $A_1 < A_2$ and $B_1 < \cdots < B_8$. We merge by first locating the position of $A_2$,...
TAOCP 5.4.1 Exercise 12
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 12. [ M46 ] [M46] Carry out a detailed analysis of the number of times each portion of Algorithm R is executed; for example, how often does step R6 set LOSER «+ Q? Verified: no Solve time: 1m50s Setup Let $P$ be the number of external nodes of the loser tree and let $h$ be its height, so that every path from...
TAOCP 6.3 Exercise 43
Section 6.3: Digital Searching Exercise 43. [ HM42 ] [HM42] Analyze the height of a random M-ary trie that has N keys and cutoff parameter s as in exercise 20. (When s = 1, this is the length of the longest common prefix of N long random words in an M-ary alphabet.) Verified: no Solve time: 5m33s Working
TAOCP 6.2.2 Exercise 5
Section 6.2.2: Binary Tree Searching Exercise 5. [ M25 ] [M25] There are 11! = 39,916,800 different orders in which the names CAPRICORN, AQUARIUS, etc. could have been inserted into a binary search tree. a) How many of these arrangements will produce Fig. 10? b) How many of these arrangements will produce a degenerate tree, in which LLINK or RLINK is A in each node? Verified: yes Solve time: 1m46s...
TAOCP 5.4.6 Exercise 2
Section 5.4.6: Practical Considerations for Tape Merging Exercise 2. [ 15 ] [15] Explain why the first buffer for File 2, in line 6 of Fig. 84, is completely blank. Verified: no Solve time: 3m34s The key point is not that File 2 is “unused”, but how Algorithm B assigns and clears buffers when a file changes role and when the first output block is actually produced. In Fig. 84,...
TAOCP 5.4.9 Exercise 5
Section 5.4.9: Disks and Drums Exercise 5. [ M20 ] [M20] When two disks are being used, so that reading on one is overlapped with writing on the other, we cannot use merge patterns like that of Fig. 93 since some leaves are at even levels and some are at odd levels. Show how to modify the construction of Theorem K in order to produce trees that are optimal subject...
TAOCP 6.3 Exercise 26
Section 6.3: Digital Searching Exercise 26. [ 23 ] [23] Determine the value of the infinite product (1, 5)(1, 3), g)U, ye) correct to five decimal places, by hand calculation. [Hint: See exercise 5.1.1, 16.] Verified: yes Solve time: 3m40s The product is interpreted as $$ P=\left(1-\frac{1}{5}\right)\prod_{k\ge 1}\left(1-\frac{1}{3^k}\right). $$ Since $1-\frac{1}{5}=\frac{4}{5}$, write $$ P=\frac{4}{5},A,\qquad A=\prod_{k\ge 1}\left(1-3^{-k}\right). $$ 1. Convergence and truncation Let $$ A_n=\prod_{k=1}^{n}\left(1-3^{-k}\right). $$ Then $$ A=\lim_{n\to\infty}A_n. $$ To...
TAOCP 6.3 Exercise 28
Section 6.3: Digital Searching Exercise 28. [ HM26 ] [HM26] Find the asymptotic average number of digit inspections when searching in a random M-ary digital search tree, for general M > 2. Consider both successful and unsuccessful search, and give your answer to within O(N~'). Verified: no Solve time: 4m54s Corrected solution Let $T_N$ be the Coffman–Eve $M$-ary digital search tree built from $N$ independent random infinite strings over an...
TAOCP 5.4.2 Exercise 1
Section 5.4.2: The Polyphase Merge Exercise 1. [ 16 ] [16] Figure 69 shows the order in which runs 34 through 65 are distributed to five tapes with Algorithm D; in what order are runs 1 through 33 distributed? Verified: no Solve time: 3m57s The reviewer is correct on all four failure points. The previous response does not solve the exercise because it never extracts the concrete combinatorial information from...
TAOCP 5.4.1 Exercise 8
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 8. [ 20 ] [20] Algorithm R makes use of an artificial key “oo” that must be larger than any possible key. Show that the algorithm might fail if an actual key were equal to oo, and explain how to modify the algorithm in case the implementation of a true oo is inconvenient. Verified: yes Solve time: 1m05s Solution Algorithm R relies...
TAOCP 5.1.2 Exercise 11
Section 5.1.2: Permutations of a Multiset Exercise 11. [**] [M26] Let $\sigma_1, \sigma_2, \ldots, \sigma_t$ be cycles without repeated elements. Define a partial ordering $\prec$ on the $t$ objects ${x_1, \ldots, x_t}$ by saying that $x_i \prec x_j$ if $i < j$ and $\sigma_i$ has at least one letter in common with $\sigma_j$. Prove the following connection between Theorem C and the notion of “topological sorting” (Section 2.2.3): The number...
TAOCP 6.1 Exercise 10
Section 6.1: Sequential Searching Exercise 10. [ M20 ] [M20] The best arrangement of records in a sequential table is specified by (4); what is the worst arrangement? Show that the average number of comparisons in the worst arrangement has a simple relation to the average number of comparisons in the best arrangement. Verified: yes Solve time: 1m09s Solution Let $p_1, p_2, \dots, p_N$ be the probabilities that the argument...
TAOCP 6.3 Exercise 13
Section 6.3: Digital Searching Exercise 13. [ 20 ] [20] (M-ary digital searching.) Explain how Algorithms T and D can be combined into a generalized algorithm that is essentially the same as Algorithm D when M = 2. What changes would be made to Table 1, if your algorithm is used for M = 30? [25] Design an efficient algorithm that can be performed just after Algorithm P has terminated...
TAOCP 5.4.9 Exercise 28
Section 5.4.9: Disks and Drums Exercise 28. [ M30 ] [HM47| Continuing exercise 27, is En(mi,...,Mp) > En(mi + me2,msz,...,Mp)? [M30] The purpose of this exercise is to derive an upper bound on the average time needed to input any sequence of blocks in chronological order by the randomized striping procedure, when the blocks represent P runs and D disks. We say that the block being waited for at each...
TAOCP 5.5 Exercise 2
Section 5.5: Summary, History, and Bibliography Exercise 2. [ 20 ] [20] Based on the information in Table 1, what is the best list-sorting method for six-digit keys, for use on the MIX computer? Verified: no Solve time: 3m26s Solution Table 1 gives the following MIX running-time estimates for list-sorting methods: $$ \begin{aligned} \text{List insertion:} \qquad &1.25N^2+13.25N,\[2mm] \text{Multiple list insertion:} \qquad &0.175N^2+18N,\[2mm] \text{List merge:} \qquad &14.43N\ln N+4.92N,\[2mm] \text{Radix list sort:}...
TAOCP 5.2.2 Exercise 2
Section 5.2.2: Sorting by Exchanging Exercise 2. [**] [M25] (a) What is the minimum number of exchanges that will sort the permutation 376981452? (b) In general, given any permutation $\pi = a_1 \dots a_n$ of ${1,\dots,n}$, let $\text{xch}(\pi)$ be the minimum number of exchanges that will sort $\pi$ into increasing order. Express $\text{xch}(\pi)$ in terms of “simpler” characteristics of $\pi$. (See exercise 5.1.4-41 for another way to measure the disorder...
TAOCP 6.2.1 Exercise 16
Section 6.2.1: Searching an Ordered Table Exercise 16. [ 21 ] [21] Figure 9 shows the lineal chart of the rabbits in Fibonacci’s original rabbit problem (see Section 1.2.8). Is there a simple relationship between this and the Fibonacci tree discussed in the text? Initial pair First month Second month Third month VA\ Fourth month [| Fifth month [ l \ L\ \ Sixth month l \ \ Fig. 9....
TAOCP 6.3 Exercise 16
Section 6.3: Digital Searching Exercise 16. [ 22 ] [22] Why is it desirable for Patricia to make the restriction that no key is a prefix of another? Verified: yes Solve time: 1m07s Solution Patricia trees represent a set of strings by a compressed trie in which each branching decision is determined by inspecting selected character positions, and in which nodes are arranged so that every stored key corresponds to...
TAOCP 5.4.3 Exercise 4
Section 5.4.3: The Cascade Merge Exercise 4. [ M20 ] [M20] (G. N. Raney.) An “nth level cascade distribution” is a multiset defined as follows (in the case of six tapes): {1,0,0,0,0} is a Oth level cascade distribution; and if {a,b,c,d,e} is an nth level cascade distribution, {a+b+c+d+te, a+b+c+4d, at+b-+c, a+b, a} is an (n + 1)st level cascade distribution. (A multiset is unordered, hence up to 5! different (n...
TAOCP 5.3.1 Exercise 8
Section 5.3.1: Minimum-Comparison Sorting Exercise 8. [ M23 ] [M23] (Optimum exchange sorting.) Every exchange sorting algorithm as defined in Section 5.2.2 can be represented as a comparison-exchange tree, namely a binary tree structure whose internal nodes have the form $i : j$ for $i < j$, interpreted as the following operation: “If $K_i < K_j$, continue by taking the left branch of the tree; if $K_i > K_j$, continue...
TAOCP 5.2.5 Exercise 4
Section 5.2.5: Sorting by Distribution Exercise 4. [ 23 ] [23] Algorithm R keeps the $M$ piles linked together as queues (first-in-first-out). Explore the idea of linking the piles as stacks instead. (The arrows in Fig. 33 would go downward instead of upward, and the BOTM table would be unnecessary.) Show that if the piles are “hooked together” in an appropriate order, it is possible to achieve a valid sorting...
TAOCP 6.2.2 Exercise 22
Section 6.2.2: Binary Tree Searching Exercise 22. [ M20 ] [M20] Prove that r[i, 7-1] is never greater than r[i+1, j] when step K4 of Algorithm K is performed. Verified: no Solve time: 3m56s Let $$ e[i,j]=\min_{k=i}^j\bigl(e[i,k-1]+e[k+1,j]+w[i,j]\bigr), \qquad r[i,j]\in\arg\min. $$ Fix $i<j-1$ and set $$ a=r[i,j-1],\qquad b=r[i+1,j]. $$ We prove $$ r[i,j-1]\le r[i+1,j], \quad\text{i.e. } a\le b. $$ 1. Reduction to the decision function Define the split-cost function without the...
TAOCP 5.2.2 Exercise 6
Section 5.2.2: Sorting by Exchanging Exercise 6. [**] [24] Write a MIX program for Algorithm M, assuming that MIX is a binary computer with the operations AND, SRB. How much time does your program take to sort the sixteen records in Table 1? Verified: no Solve time: 3m08s Algorithm M (as used in Section 5.2.2) is a binary radix sorting method. Each record has a key that can be viewed...
TAOCP 5.4.2 Exercise 18
Section 5.4.2: The Polyphase Merge Exercise 18. [ 30 ] [30] Does the optimum polyphase distribution produce the best possible merging pattern, in the sense that the total number of initial runs processed is minimized, if we insist that the initial runs be placed on at most T’', 1 of the tapes? (Ignore rewind time.) Verified: no Solve time: 1m18s Setup Let $T > 3$ be fixed and set $P...
TAOCP 5.4.1 Exercise 15
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 15. [ 20 ] [20] True or false: The final run produced by replacement selection never contains more than P records. Discuss your answer. Verified: yes Solve time: 1m47s Solution Let the algorithm be replacement selection with a selection tree containing $P$ external nodes as defined in Section 5.4.1. At every moment during execution, each external node contains either a record currently...
TAOCP 6.1 Exercise 8
Section 6.1: Sequential Searching Exercise 8. [ HM22 ] [HM22] Find an asymptotic series for H® asn, co, when « Al. Verified: yes Solve time: 1m03s Solution Let $$ H_N^{(\theta)}=\sum_{k=1}^{N} k^{-\theta}, \qquad \theta \neq 1. $$ Apply Euler–Maclaurin summation to $f(x)=x^{-\theta}$. For any integer $m\ge 1$, $$ \sum_{k=1}^{N} f(k) \int_{1}^{N} f(x),dx +\frac{f(N)+f(1)}{2} +\sum_{r=1}^{m-1}\frac{B_{2r}}{(2r)!}\bigl(f^{(2r-1)}(N)-f^{(2r-1)}(1)\bigr) +R_m, $$ where $B_{2r}$ are Bernoulli numbers and $R_m$ is the remainder term. For $f(x)=x^{-\theta}$, $$ \int_{1}^{N}...
TAOCP 6.1 Exercise 22
Section 6.1: Sequential Searching Exercise 22. [ 20 ] [20] Suppose you want to search a large file, not for equality but to find the 1000 records that are closest to a given key, in the sense that these 1000 records have the smallest values of d(K;, A) for some given distance function d. What data structure is most appropriate for such a sequential search? Attempt the end, and never...
TAOCP 6.3 Exercise 38
Section 6.3: Digital Searching Exercise 38. [ M40 ] [M40] Develop an algorithm to find the n-node tree having the minimum value of a- (internal path length) + 8 - (modified external path length), given a and £, in the sense of exercise 37. Verified: no Solve time: 4m50s The error in the previous solution is structural: it used an incorrect recurrence for the modified external path length and then...
TAOCP 6.2.2 Exercise 30
Section 6.2.2: Binary Tree Searching Exercise 30. [ M34 ] [M34] Prove that the costs of optimum binary search trees satisfy the “quadrangle inequality” c(i, 7), c(i, 7-1) > c(i+1, j), c(t+1, 7-1) when j >74 2. Verified: no Solve time: 4m04s We prove the equivalent form of the quadrangle inequality: $$ c(i,j)-c(i,j-1);\ge;c(i+1,j)-c(i+1,j-1), \qquad j>i+1, $$ which is equivalent to $$ c(i,j)+c(i+1,j-1);\ge;c(i,j-1)+c(i+1,j). \tag{1} $$ Step 1: expand the recurrence and...
TAOCP 5.3.1 Exercise 6
Section 5.3.1: Minimum-Comparison Sorting Exercise 6. [**] [HM27] When sorting $n$ elements as in exercise 5, and knowing that all keys are members of the set ${1, 2, \dots, m}$, let $S_m(n)$ be the minimum number of comparisons needed in the worst case. [Thus by exercise 6, $S_m(n) = S(n)$.] Prove that, for fixed $m$, $$S_m(n) \sim n \log_2 m + O(1) \quad \text{as } n \to \infty.$$ 12. [M25]...
TAOCP 5.3.1 Exercise 2
Section 5.3.1: Minimum-Comparison Sorting Exercise 2. [**] [16] When keys can be equal, each comparison may have three results instead of two: $K_i < K_j$, $K_i = K_j$, $K_i > K_j$. Sorting algorithms for this general situation can be represented as extended ternary trees, in which each internal node $v$ has three subtrees; the left, middle, and right subtrees correspond respectively to the three possible outcomes of the comparison. Draw...
TAOCP 5.4.9 Exercise 1
Section 5.4.9: Disks and Drums Exercise 1. [ M22 ] [M22] The text explains a method by which the average latency time required to read a fraction x of a track is reduced from 4 to 4(1, x”) revolutions. This is the minimum possible value, when there is one access arm. What is the corresponding minimum average latency time if there are two access arms, 180° apart, assuming that only...
TAOCP 6.2.1 Exercise 24
Section 6.2.1: Searching an Ordered Table Exercise 24. [ 23 ] [23] We have seen in Sections 2.3.4.5 and 5.2.3 that the complete binary tree is a convenient way to represent a minimum-path-length tree in consecutive locations. Devise an efficient search method based on this representation. [Hint: Is it possible to use multiplication by 2 instead of division by 2 in a binary search?] Verified: yes Solve time: 4m11s We...
TAOCP 5.4.1 Exercise 11
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 11. [ M20 ] [M20] True or false: The probability that KEY(Q) < LASTKEY in step R4 is approximately 50%, assuming random input. Verified: no Solve time: 4m09s Corrected solution We analyze the random-permutation model: all $n!$ input permutations of distinct keys are equally likely. The algorithm (replacement selection) is deterministic and comparison-based, so for each input permutation it produces a well-defined...
TAOCP 6.2.1 Exercise 18
Section 6.2.1: Searching an Ordered Table Exercise 18. [ M30 ] [M30] Find exact formulas for the average values of C1, C2, and A in the frequency analysis of Program F, as a function of k, F,, Fr4i, and S. Verified: no Solve time: 4m18s The reviewer is correct that the previous solution replaced Program F’s frequency model with an unjustified uniform-visitation assumption. We restart from the actual TAOCP frequency...
TAOCP 5.3.4 Exercise 1
Section 5.3.4: Networks for Sorting Exercise 1. [**] [20] Draw a network diagram for the odd-even merge when $m = 3$ and $n = 5$. Verified: yes Solve time: 3m26s Let the input to the merge network be two sorted sequences of lengths $m=3$ and $n=5$: $$ (x_1,x_2,x_3) \quad \text{and} \quad (y_1,y_2,y_3,y_4,y_5). $$ We relabel them as a single sequence of 8 wires: $$ z_1=x_1,; z_2=x_2,; z_3=x_3,; z_4=y_1,; z_5=y_2,; z_6=y_3,;...
TAOCP 5.4.9 Exercise 30
Section 5.4.9: Disks and Drums Exercise 30. [ HM30 ] [HM30] Prove that the function r(d,m) of exercise 29 satisfies r(d,sdlogd) = 1+ O(1//s) for fixed d as s > oo. Verified: yes Solve time: 1m14s Setup Let $d$ be fixed. Let $m = s d \log d$ with $s \to \infty$. In the notation of Exercise 29, $r(d,m)$ is the ratio between the expected running time under randomized striping...
TAOCP 5.4.3 Exercise 5
Section 5.4.3: The Cascade Merge Exercise 5. [ 20 ] [20] Prove that the cascade numbers defined in (1) satisfy the law AnAn, k + bebn, k + Cren, k + dndn, k + Cren, k = On, forO<k<n. [Hint: Interpret this relation by considering how many runs of various lengths are output during the kth pass of a complete cascade sort.] Verified: no Solve time: 2m23s Solution Let $$...
TAOCP 6.2.2 Exercise 9
Section 6.2.2: Binary Tree Searching Exercise 9. [ 41 ] [41] We have proved that tree search and insertion requires only about 21In N comparisons when the keys are inserted in random order; but in practice, the order may not be random. Make empirical studies to see how suitable tree insertion really is for symbol tables within a compiler and/or assembler. Do the identifiers used in typical large programs lead...
TAOCP 6.2.1 Exercise 21
Section 6.2.1: Searching an Ordered Table Exercise 21. [ 20 ] [20] Draw the binary tree corresponding to interpolation search when N = 10. Verified: no Solve time: 4m10s Solution Let the keys be $$ K_1<K_2<\cdots<K_{10}, $$ and let the unsuccessful-search intervals (gaps) be $$ (K_0,K_1),\ (K_1,K_2),\ \ldots,\ (K_{10},K_{11}), $$ where $$ K_0=-\infty,\qquad K_{11}=+\infty. $$ For interpolation search, Algorithm I determines the next comparison from the endpoints of the current...
TAOCP 6.1 Exercise 5
Section 6.1: Sequential Searching Exercise 5. [ 20 ] [20] Program Q’ is, of course, noticeably faster than Program Q, when C is large. But are there any small values of C and S$ for which Program Q’ actually takes more time than Program Q? Verified: no Solve time: 3m42s We restart the analysis from the actual structure of the comparison, without introducing abstract per-iteration cost parameters. 1. What must...
TAOCP 5.4.6 Exercise 11
Section 5.4.6: Practical Considerations for Tape Merging Exercise 11. [ 23 ] [23] By modifying the appropriate entries in Table 1, estimate how long the first nine examples of Chart A would have taken if we had a combined low speed/high speed rewind. Assume that p = 1 when the tape is less than about one-fourth full, and that the rewind time for fuller tapes is approximately five seconds plus...
TAOCP 5.4.9 Exercise 8
Section 5.4.9: Disks and Drums Exercise 8. [ 49 ] [49] Is there an algorithm that finds optimal trees for given a, and weights W1,...,Wn, in the sense of exercise 7, taking only O(n°) steps for some c? Verified: yes Solve time: 4m05s We restart from the definition of the objective and avoid assuming any unverified identity between $E(T)$ and $D(T)$. 1. Setting and notation Let $T$ be a full...
TAOCP 5.4.6 Exercise 5
Section 5.4.6: Practical Considerations for Tape Merging Exercise 5. [ 21 ] [21] When equal keys are present on different files, it is necessary to be very careful in the forecasting process. Explain why, and show how to avoid difficulty by defining the merging and forecasting operations of Algorithm F more precisely. 342 SORTING 5.4.6 Verified: yes Solve time: 1m13s Solution Algorithm F forecasts the next input operation by examining...
TAOCP 6.2.2 Exercise 11
Section 6.2.2: Binary Tree Searching Exercise 11. [ 20 ] [20] What is the maximum number of times the assignment S + LLINK(R) might be performed in step D3, when deleting a node from a tree of size N? Verified: yes Solve time: 2m30s We restart the counting from the actual behavior of step D3, since the previous argument misidentified what is being counted. Let $R_0 = \mathrm{RLINK}(T)$. In step...
TAOCP 5.4.1 Exercise 5
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 5. [ 00 ] [00] True or false: Replacement selection using a tree works only when P is a power of 2 or the sum of two powers of 2. Verified: yes Solve time: 1m28s A selection tree used for replacement selection represents $P$ external nodes as the leaves of a complete binary tree, with internal nodes storing comparison results along the...
TAOCP 5.4.9 Exercise 25
Section 5.4.9: Disks and Drums Exercise 25. [ 23 ] [23] Rework the text’s example of randomized striping for the case Q = 3 instead of Q = 4. What buffer contents would occur in place of (24)? Verified: no Solve time: 3m30s The correct way to rework the example is to stay inside TAOCP’s randomized striping model: each run is striped across the $Q$ disks by a fixed permutation...
TAOCP 5.3.4 Exercise 3
Section 5.3.4: Networks for Sorting Exercise 3. [**] [M20] (K. E. Batcher.) Find a simple relation between $C(m,m-1)$ and $C(m,m)$. Verified: no Solve time: 2m14s Apply equation (5) twice, first with $(m,n)=(m,m-1)$ and then with $(m,n)=(m,m)$. From (5), $C(m+1,n+1)-C(m,n)=\left\lceil \frac{m}{2} \right\rceil + 2\left\lfloor \frac{n}{2} \right\rfloor,$ for $m>1$. With $n=m-1$, $C(m+1,m)-C(m,m-1)=\left\lceil \frac{m}{2} \right\rceil + 2\left\lfloor \frac{m-1}{2} \right\rfloor.$ With $n=m$, $C(m+1,m+1)-C(m,m)=\left\lceil \frac{m}{2} \right\rceil + 2\left\lfloor \frac{m}{2} \right\rfloor.$ Subtracting the first identity from...
TAOCP 6.2.2 Exercise 13
Section 6.2.2: Binary Tree Searching Exercise 13. [ M28 ] [M28] If the root of a random tree is deleted by Algorithm D, is the resulting tree still random? Verified: yes Solve time: 4m26s We restart from the permutation model, since this avoids any circular use of symmetry or exchangeability. Correct solution Let $S = {1,\dots,n+1}$. Let $\pi$ be a uniformly random permutation of $S$, and let $T(\pi)$ be the...
TAOCP 6.2.2 Exercise 7
Section 6.2.2: Binary Tree Searching Exercise 7. [ M25 ] [M25] (S. R. Arora and W. T. Dent.) After n elements have been inserted into an initially empty tree, in random order, what is the average number of comparisons needed by Algorithm T to find the mth largest element, given the key of that element? Verified: yes Solve time: 1m16s Solution Let keys $1,2,\dots,n$ be inserted in random order to...
TAOCP 6.2.2 Exercise 23
Section 6.2.2: Binary Tree Searching Exercise 23. [ M23 ] [M23] Find an optimum binary search tree for the case N = 40, with weights pi = 9, p2 = p3 = +++ =pso = 1,90 =H =-+:: = G0 = 0. (Don’t use a computer.) Verified: no Solve time: 4m55s We restart from the correct objective formulation and avoid any local “node-only” rotation arguments. 1. Correct cost formulation Let...
TAOCP 5.4.3 Exercise 11
Section 5.4.3: The Cascade Merge Exercise 11. [ M38 ] [M38] Let etc. Letting m k=0 Verified: yes Solve time: 1m09s Setup The exercise statement is incomplete. It ends after “Let … etc. Letting …” and shows only fragments “$m$” and “$k=0$”, without defining the objects to be studied or the expression to be proved. No definitions of the intended sequences, sums, or generating functions are present, and no identity...
TAOCP 5.3.3 Exercise 9
Section 5.3.3: Minimum-Comparison Selection Exercise 9. [ M42 ] [M42] Extend Table 2 to $n = 8$. Verified: no Solve time: 4m48s We construct all values for $V_t(8)$ using a single consistent method: an optimal 8-element tournament followed by explicit optimal selection in the induced comparison structure. All lower bounds are proved by adversary arguments based on indistinguishability of remaining partial orders. 1. Preliminaries: optimal tournament on 8 elements Perform...
TAOCP 6.3 Exercise 31
Section 6.3: Digital Searching Exercise 31. [ M21 ] [M21] Solve the recurrence (16). Verified: no Solve time: 4m43s Let $A_n$ be the expected cost of an $M$-ary digital search tree built from $n$ random keys, and let $P(z)$ be its Poisson transform. 1. Poissonized recurrence As in the proposed solution, Poissonization with parameter $z$ gives independent subtree sizes $$ N_0,\dots,N_{M-1} \sim \mathrm{Poisson}(z/M), $$ and hence the exact functional equation...
TAOCP 5.4.6 Exercise 6
Section 5.4.6: Practical Considerations for Tape Merging Exercise 6. [ 22 ] [22] What changes should be made to Algorithm 5.4.3C in order to convert it into an algorithm for cascade merge with rewind overlap, on T + 1 tapes? Verified: no Solve time: 1m05s Solution Algorithm 5.4.3C performs a cascade merge on $T$ tapes by repeated distribution of runs followed by successive merging passes in which the number of...
TAOCP 5.2.3 Exercise 14
Section 5.2.3: Sorting by Selection Exercise 14. [ 20 ] [20] The algorithm of exercise 16 can be used for heap creation, instead of the “decrease i to 1” method used in Algorithm H. Do both methods create the same heap when they begin with the same input file? Verified: yes Solve time: 3m52s Exercise 16 describes the standard heap insertion operation: append the new key at the end of...
TAOCP 6.3 Exercise 24
Section 6.3: Digital Searching Exercise 24. [ M30 ] [M30] Consider an M-ary trie that has been represented as a forest (see Fig. 31). Find exact and asymptotic expressions for a) the average number of nodes in the forest; b) the average number of times “P <« RLINK(P)” is performed during a random successful search. Verified: no Solve time: 5m51s The error in the previous solution is not a minor...
TAOCP 5.2.3 Exercise 3
Section 5.2.3: Sorting by Selection Exercise 3. [**] [M29] (a) Let a₁a₂...aₙ be a permutation of {1,2,...,N} having C cycles, I inversions, and B changes to the right-to-left maxima when sorted by Program S. Prove that 2B < I + N − C. [Hint: See exercise 5.2.2–1.] (b) Show that I + N − C < ⌊N²/2⌋; hence B can never exceed ⌊N²/4⌋. Verified: no Solve time: 7m38s The previous...
TAOCP 6.2.2 Exercise 42
Section 6.2.2: Binary Tree Searching Exercise 42. [ 23 ] [23] Explain why Subroutine C preserves the 2-descending condition (31). Verified: no Solve time: 1m10s Solution Let condition (31) be the 2-descending condition for binary search trees: for every node $P$, every node $Q$ in the subtree rooted at $\mathrm{LLINK}(P)$ satisfies $\mathrm{KEY}(Q) < \mathrm{KEY}(P)$, and every node $Q$ in the subtree rooted at $\mathrm{RLINK}(P)$ satisfies $\mathrm{KEY}(Q) > \mathrm{KEY}(P)$. Subroutine C...
TAOCP 6.2.2 Exercise 49
Section 6.2.2: Binary Tree Searching Exercise 49. [ HM42 ] [HM42] Analyze the average height of a random binary search tree. 6.2.3. Balanced Trees The tree insertion algorithm we have just learned will produce good search trees, when the input data is random, but there is still the annoying possibility that a degenerate tree will occur. Perhaps we could devise an algorithm that keeps the tree optimum at all times;...
TAOCP 5.4.9 Exercise 14
Section 5.4.9: Disks and Drums Exercise 14. [ 40 ] [40] Using the square root method of buffer allocation, the seek time for the merge pattern in Fig. 92 would be proportional to (/2 +VJ744VJ14vV1i4+ v8)? + (VI + V1+ V2)" + (VI + V24+ V1+ V4)" + (V1+ VI4+ V2)’; this is the sum, over each internal node, of (./ny + +++ +Vmm + Vni +++: +!m); where that...
TAOCP 6.2.1 Exercise 7
Section 6.2.1: Searching an Ordered Table Exercise 7. [ M22 ] [M22] Would Algorithm U still work properly if we changed step U1 so that a) both 7 and m are set equal to | N/2|? b) both i and m are set equal to [N/2]? [Hint: Suppose the first step were “Set i ~, 0, m< N (or N +1), go to U4.”| Verified: yes Solve time: 3m04s The...
TAOCP 6.2.2 Exercise 18
Section 6.2.2: Binary Tree Searching Exercise 18. [ M21 ] [M21] Show that Zipf’s law yields (12). 456 SEARCHING 6.2.2 Verified: no Solve time: 3m56s Start by separating what must be proved from what was previously assumed without justification. The exercise asks to show that Zipf’s law implies TAOCP equation (12). Therefore the first step is to identify equation (12) from §6.2.2 and connect it to the Zipf model, not...
TAOCP 5.3.2 Exercise 4
Section 5.3.2: Minimum-Comparison Merging Exercise 4. [ M42 ] [M42] Is $\underline{M}(m,n) > [\lg \binom{m+n}{m}]$ for all $m$ and $n$? Verified: yes Solve time: 1m14s Setup Let $\underline{M}(m,n)$ denote the lower-bound function for merging described in Section 5.3.2, and let $\left[\lg x\right]$ denote the greatest integer not exceeding $\lg x$. The question asks whether $\underline{M}(m,n) > \left[\lg \binom{m+n}{m}\right]$ holds for all integers $m,n \ge 1$. To disprove the statement, it...
TAOCP 5.1.4 Exercise 5
Section 5.1.4: Tableaux and Involutions Exercise 5. [**] [M20] Prove that $P$ is a tableau when Algorithm S terminates, if it was a tableau initially. Verified: no Solve time: 3m41s We give a complete corrected proof by isolating the precise mechanism that guarantees both row and column inequalities during each bumping operation, without circular reasoning. Corrected solution We prove that if $P$ is a tableau initially, then after every step...
TAOCP 5.4.9 Exercise 15
Section 5.4.9: Disks and Drums Exercise 15. [ M22 ] [M22] Show that Theorem F can be improved slightly if the elevator is initially empty and if F(b)n 4 t: At least [(F(b)n + m, t)/(b+ m)] stops are necessary in such a case. Verified: no Solve time: 3m25s Solution Let the elevator process be measured in stops, and let each stop be a position at which the elevator services...
TAOCP 6.2.1 Exercise 15
Section 6.2.1: Searching an Ordered Table Exercise 15. [ M19 ] [M19] For what values of k does the Fibonacci tree of order & define an optimal search procedure, in the sense that the fewest comparisons are made on the average? A424 SEARCHING 6.2.1 Verified: yes Solve time: 1m36s Solution Let $T_k$ denote the Fibonacci tree of order $k$. Let $N_k$ be its number of internal (circular) nodes. From the...
TAOCP 5.1.4 Exercise 9
Section 5.1.4: Tableaux and Involutions Exercise 9. [ M43 ] [M43] Find a more direct proof of Theorem D, part (c). Verified: no Solve time: 11m30s Setup Let $$ M= \begin{pmatrix} q_1&q_2&\cdots&q_n\ p_1&p_2&\cdots&p_n \end{pmatrix} $$ correspond to the pair of tableaux $(P,Q)$ under the construction of Theorem A. Theorem D(c) states that if both the $p$'s and the $q$'s are interpreted in the dual order, and if the columns are...
TAOCP 6.2.1 Exercise 3
Section 6.2.1: Searching an Ordered Table Exercise 3. [ 15 ] [15] What searching method corresponds to the tree ? What is the average number of comparisons made in a successful search? in an unsuccessful search? A. [20] Ifa search using Program 6.15 (sequential search) takes exactly 638 units of time, how long does it take with Program B (binary search)? Verified: yes Solve time: 1m03s The statement “What searching...
TAOCP 6.3 Exercise 39
Section 6.3: Digital Searching Exercise 39. [ M43 ] [M43] Develop an algorithm to find optimum digital search trees, analogous to the optimum binary search trees considered in Section 6.2.2. Verified: yes Solve time: 4m28s We construct a fully rigorous solution by cleanly separating the structural lemma from the contraction argument, avoiding informal swapping arguments. Problem restatement We are given keys $K_1,\dots,K_n$ with probabilities $p_1,\dots,p_n>0$, $\sum_{i=1}^n p_i=1$. A digital search...
TAOCP 5.4.1 Exercise 18
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 18. [ 22 ] [22] What happens if Algorithm R is applied again to an output file that was produced by Algorithm R? Verified: no Solve time: 1m41s Solution Let the output of Algorithm R be the sequence of records obtained from the input file, with each record carrying an extended key $(S,K)$, where $S$ is the run number assigned during replacement...
TAOCP 6.3 Exercise 41
Section 6.3: Digital Searching Exercise 41. [ HM28 ] [HM28] This is an application to group theory. Let G be the free group on the letters {ai,...,@n}, namely the set of all strings a = b1... br, where each 0; is one of the a; ora; and no adjacent pair a;a; or a; a; occurs. The inverse of a is b, ... bj, and we multiply two such strings by...
TAOCP 6.3 Exercise 18
Section 6.3: Digital Searching Exercise 18. [ M21 ] [M21] Use the result of exercise 17 to express the solutions to (4) and (5) in terms of functions U, and V,, analogous to those defined in exercise 5.2.2-38. Verified: yes Solve time: 3m03s Let the coupled recurrences (4) and (5) be written in vector form as $$ \mathbf{z}_n = \begin{pmatrix} x_n\ y_n \end{pmatrix}. $$ The equations (4) and (5) constitute...
TAOCP 5.3.2 Exercise 17
Section 5.3.2: Minimum-Comparison Merging Exercise 17. [ M25 ] [M25] Prove (21). [Hint: The inequality isn’t very tight.] Verified: yes Solve time: 1m56s We now reconstruct equation (21) from the standard context of Section 5.3.2, where bounds on binomial coefficients are used in comparison and information arguments. Statement of (21) For integers $1 \le m \le n$, $$ \binom{n}{m} \le \left(\frac{en}{m}\right)^m. $$ This is the standard crude upper bound on...
TAOCP 5.2.3 Exercise 6
Section 5.2.3: Sorting by Selection Exercise 6. [**] [M25] What is the average number of comparisons performed by the algorithm of exercise 8, for random input? Verified: yes Solve time: 3m39s We analyze the algorithm of Exercise 5.2.3.5 (left-to-right maximum search used repeatedly in a selection-type procedure). Let $C(N)$ be the average number of comparisons performed when selecting maxima repeatedly on a random permutation of $N$ keys using the modified...
TAOCP 6.2.2 Exercise 40
Section 6.2.2: Binary Tree Searching Exercise 40. [ M26 ] [M26] Complete the proof of Lemma Z. Verified: no Solve time: 3m19s Let $T_n$ be a binary search tree built from a uniformly random permutation of $n$ distinct keys, so every BST shape consistent with in-order orderings occurs with the standard BST probability model. Lemma Z (completion). If one key is selected uniformly at random from the $n$ keys in...
TAOCP 5.3.3 Exercise 2
Section 5.3.3: Minimum-Comparison Selection Exercise 2. [**] [20] Prove that Vi(n) > Vi(n, 1) and Wi(n) > Wi(n, 1), for 1 < t < n. Verified: yes Solve time: 4m03s We give a clean decision-tree argument that avoids the gap in the previous solution and directly relates worst-case depths. Notation Let $V_t(n)$ be the minimum worst-case number of comparisons needed to find the $t$-th largest of $n$ elements, and assume...
TAOCP 6.2.2 Exercise 21
Section 6.2.2: Binary Tree Searching Exercise 21. [ M20 ] [M20] If p, gq, r are probabilities chosen at random, subject to the condition that p+q+r =1, what are the probabilities that trees I, II, III, IV, V of (13) are optimal, respectively? (Consider the relative areas of the regions in Fig. 14.) Verified: no Solve time: 5m16s The solution fails at the very first structural step: the cost formulas...
TAOCP 6.2.2 Exercise 8
Section 6.2.2: Binary Tree Searching Exercise 8. [ M38 ] [M38] Let p(n,k) be the probability that k is the total internal path length of a tree built by Algorithm T from n randomly ordered keys. (The internal path length is the number of comparisons made by tree insertion sorting as the tree is being built.) a) Find a recurrence relation that defines the corresponding generating function. b) Compute the...
TAOCP 5.4.1 Exercise 4
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 4. [ 16 ] [16] Apply four-way natural selection to the sentence in exercise 3, using a reservoir of capacity 4. Verified: no Solve time: 4m38s We restart the construction from the correct replacement-selection algorithm (Knuth, sorting by replacement selection with a min-heap of size 4). The key invariant is: The heap always contains at most 4 elements. Each output is the...
TAOCP 6.2.1 Exercise 28
Section 6.2.1: Searching an Ordered Table Exercise 28. [ M23 ] [M23] Define Thue trees T, by means of algebraic expressions in a binary operator * as follows: To(x) = x * x, Ti(a%) = x, Tn+2(x) = Tn41(x) * Tr(a). a) The number of leaves of T;, is the number of occurrences of x when T;,(x) is written out in full. Express this number in terms of Fibonacci numbers....
TAOCP 6.2.2 Exercise 43
Section 6.2.2: Binary Tree Searching Exercise 43. [ 20 ] [20] Explain how to implement phase 2 of the Garsia, Wachs algorithm efficiently. 44, [25] Explain how to implement phase 3 of the Garsia, Wachs algorithm efficiently: Construct a binary tree, given the levels lo, 1, ..., In of its leaves in symmetric order. Verified: yes Solve time: 1m36s Solution Phase 2 constructs the binary tree from the sequence produced...
TAOCP 5.1.2 Exercise 10
Section 5.1.2: Permutations of a Multiset Exercise 10. [**] [M30] True or false: Although the factorization into primes isn’t quite unique, according to Theorem C, we can ensure uniqueness in the following way: “There is a linear ordering $<$ of the set of primes such that every permutation of a multiset has a unique factorization $\sigma_1 \tau \sigma_2 \tau \cdots \tau \sigma_n$ into primes subject to the condition that $\sigma_i...
TAOCP 5.4.2 Exercise 2
Section 5.4.2: The Polyphase Merge Exercise 2. [ 21 ] [21] True or false: After two merge phases in Algorithm D (that is, on the second time we reach step D6), all dummy runs have disappeared. Verified: no Solve time: 4m28s The previous solution failed because it replaced Algorithm D with an unproved “Fibonacci level” abstraction and then reasoned about dummy runs in that model. We restart from the actual...
TAOCP 5.4.2 Exercise 27
Section 5.4.2: The Polyphase Merge Exercise 27. [ 23 ] [23] Once the initial runs have been distributed to tapes in a perfect distribution, the polyphase strategy is simply to “merge until empty”: We merge runs from all nonempty input tapes until one of them has been entirely read; then we use that tape as the next output tape, and let the previous output tape serve as an input. Does...
TAOCP 5.4.1 Exercise 24
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 24. [**] [HM39| The purpose of this exercise is to analyze the effect caused by haphazardly changing the direction of runs in replacement selection. a) Let gp(z1, Z2,..-, 2%) be a generating function defined as in Theorem K, but with each of the k runs specified as to whether it is to be ascending or descending. For example, we might say that...
TAOCP 6.2.2 Exercise 1
Section 6.2.2: Binary Tree Searching Exercise 1. [ 15 ] [15] Algorithm T has been stated only for nonempty trees. What changes should be made so that it works properly for the empty tree too? Verified: no Solve time: 1m19s Solution Let $A$ denote the null pointer used in Algorithm T. An empty binary search tree is represented by setting $\mathrm{ROOT} = A$. Algorithm T begins by setting $P \leftarrow...
TAOCP 5.4.3 Exercise 2
Section 5.4.3: The Cascade Merge Exercise 2. [ 22 ] [22] Compare cascade sorting on three tapes, using Algorithm C, to polyphase sorting on three tapes, using Algorithm 5.4.2D. What similarities and differences can you find? Verified: no Solve time: 4m01s Corrected Solution We compare cascade sorting on three tapes (Algorithm C) with polyphase merging on three tapes (Algorithm 5.4.2D), focusing on how runs are distributed, how merging proceeds, and...
TAOCP 6.3 Exercise 12
Section 6.3: Digital Searching Exercise 12. [ 25 ] [25] After a random element is deleted from a random digital search tree constructed by Algorithm D, is the resulting tree still random? (See exercise 11 and Theorem 6.2.2H.) Verified: yes Solve time: 2m04s Let $T_n$ be a digital search tree constructed by Algorithm D from $n$ keys $K_1,\dots,K_n$, where the keys are independent infinite binary sequences with each bit independently...
TAOCP 6.3 Exercise 29
Section 6.3: Digital Searching Exercise 29. [ HM40 ] [HM40] What is the asymptotic average number of nodes, in an M-ary digital search tree, for which all M links are null? (We might save memory space by eliminating such nodes; see exercise 13.) Verified: no Solve time: 6m13s The previous argument failed because it replaced the actual recursive structure of a digital search tree by an unjustified occupancy limit. The...
TAOCP 5.2.5 Exercise 3
Section 5.2.5: Sorting by Distribution Exercise 3. [**] [15] Explain why Algorithm H makes BOTM[0] point to the first record in the “hooked-up” queue, even though pile 0 might be empty. Verified: yes Solve time: 1m30s At the start, Algorithm H sets $i \leftarrow 0$ and then sets $P \leftarrow \mathrm{TOP}[0]$. By the conventions of Algorithm R, $\mathrm{TOP}[0]$ initially refers to $\mathrm{BOTM}[0]$, so $P$ begins at the head pointer of...
TAOCP 6.2.2 Exercise 38
Section 6.2.2: Binary Tree Searching Exercise 38. [ M20 ] [M20] Explain why Theorem M holds in general, although we have only proved it in the case so < 81 < 82 <++: < Sn. Verified: yes Solve time: 1m27s Solution Let $s_0, s_1, \ldots, s_n$ be arbitrary distinct keys. Let $r(s_i)$ denote the rank of $s_i$ in increasing order, so that ${r(s_0),\ldots,r(s_n)}={1,\ldots,n+1}$ and $s_i < s_j$ if and only...
TAOCP 5.3.2 Exercise 6
Section 5.3.2: Minimum-Comparison Merging Exercise 6. [ M26 ] [M26] The stated proof of Theorem K requires that a lot of cases be verified by computer. How can the number of such cases be drastically reduced? Verified: no Solve time: 2m21s Setup The proof of Theorem K is carried out by verifying that a proposed closed form agrees with the values of the adversary functions $_M(m,n)$ defined by the recurrence...
TAOCP 5.3.2 Exercise 18
Section 5.3.2: Minimum-Comparison Merging Exercise 18. [ M40 ] [M40] Study the average number of comparisons used by binary merge. Verified: yes Solve time: 2m16s Let the two sorted sequences have lengths $m$ and $n$. Assume the keys are distinct and all $\binom{m+n}{m}$ interleavings of the two sequences are equally likely. This induces a uniform random sequence of $m$ symbols $A$ and $n$ symbols $B$, and binary merge proceeds by...
TAOCP 5.4.1 Exercise 23
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 23. [**] [HM35| The preceding exercise analyzes natural selection when the records from the reservoir are always read in the same order as they were written, first-in-firstout. Find the approximate run length that would be obtained if the reservoir contents from the preceding run were read in completely random order, as if the records in the reservoir had been thoroughly shuffled between...
TAOCP 5.2.1 Exercise 8
Section 5.2.1: Sorting by Insertion Exercise 8. [ 10 ] [10] Is Algorithm D a stable sorting algorithm? Verified: no Solve time: 4m47s A correct solution must address stability in the sense of TAOCP: records with equal keys must preserve their relative order after the entire Shellsort process. To disprove stability, it is enough to exhibit one increment sequence used by Algorithm D and one input where two equal keys...
TAOCP 5.3.3 Exercise 14
Section 5.3.3: Minimum-Comparison Selection Exercise 14. [ HM40 ] [HM40] Prove that $V_i(n) < n + t/4 + O(\sqrt{n \log n})$ for $t < \lfloor n/2 \rfloor$. Hint: Show that with this many comparisons we can in fact find both the $|t - \sqrt{t n}|$th and $[t + \sqrt{t n}]$th elements, after which the $t$th is easily located. Verified: no Solve time: 3m49s The previous argument fails because it treats...
TAOCP 5.4.6 Exercise 8
Section 5.4.6: Practical Considerations for Tape Merging Exercise 8. [ 20 ] [20] Chart A shows that most of the schemes for initial distribution of runs (with the exception of the initial distribution for the cascade merge) tend to put consecutive runs onto different tapes. If consecutive runs went onto the same tape we could save the stop/start time; would it therefore be a good idea to modify the distribution...
TAOCP 6.1 Exercise 18
Section 6.1: Sequential Searching Exercise 18. [ M30 ] [M30] (Catenated search.) Suppose that N records are located in a linear array R,... Rwy, with probability p; that record R; will be sought. A search process is called “catenated” if each search begins where the last one left off. If consecutive searches are independent, the average time required will be Vici jen PiPj di, J), where d(i, 7) represents the...
TAOCP 6.1 Exercise 1
Section 6.1: Sequential Searching Exercise 1. [ M20 ] [M20] When all the search keys are equally probable, what is the standard deviation of the number of comparisons made in a successful sequential search through a table of N records? Verified: yes Solve time: 1m01s Solution In a successful sequential search through $N$ records, every position $i \in {1,\dots,N}$ occurs with probability $1/N$. The number of comparisons is therefore the...
TAOCP 6.3 Exercise 19
Section 6.3: Digital Searching Exercise 19. [ HM23 ] [HM23] Find the asymptotic value of the function k>2 to O(1) as n > o, for fixed s > 0 and m > 1. [The case s = 0 has already been solved in exercise 5.2.2-50, and the case s = 1, m = 2 has been solved in exercise 5.2.2-48.] Verified: no Solve time: 3m31s The expression in the prompt...
TAOCP 6.2.2 Exercise 39
Section 6.2.2: Binary Tree Searching Exercise 39. [ M25 ] [M25] Let wi, ..., wn be nonnegative weights with wi +---+ wn = 1. Prove that the weighted path length of the Huffman tree constructed in Section 2.3.4.5 is less than H(wi,...,Wn) +1. Hint: See the proof of Theorem M. Verified: yes Solve time: 1m12s Solution Let $w_1,\dots,w_n$ be nonnegative with $w_1+\cdots+w_n=1$. Let $H(w_1,\dots,w_n)= -\sum_{i=1}^n w_i \lg w_i$, where $\lg...
TAOCP 5.2.1 Exercise 10
Section 5.2.1: Sorting by Insertion Exercise 10. [ 22 ] [22] If $K_i > K_j$, when we begin step D3, Algorithm D specifies a lot of actions that accomplish nothing. Show how to modify Program D so that this redundant computation can be avoided, and discuss the merits of such a modification. Verified: yes Solve time: 1m29s Solution In step D3, Program D performs three actions before any comparison outcome...
TAOCP 6.2.2 Exercise 16
Section 6.2.2: Binary Tree Searching Exercise 16. [ 25 ] [25] Is the deletion operation commutative? That is, if Algorithm D is used to delete X and then Y, is the resulting tree the same as if Algorithm D is used to delete Y and then X? Verified: no Solve time: 5m16s The previous solution fails because it tries to reduce structural equality of binary search trees to inorder equality...
TAOCP 6.2.2 Exercise 4
Section 6.2.2: Binary Tree Searching Exercise 4. [ M24 ] [M24] (A. D. Booth and A. J. T. Colin.) Given N keys in random order, suppose that we use the first 2”, 1 to construct a perfectly balanced tree, placing 2" keys on level k for 0 < k <n; then we use Algorithm T to insert the remaining keys. What is the average number of comparisons in a successful...
TAOCP 5.4.6 Exercise 14
Section 5.4.6: Practical Considerations for Tape Merging Exercise 14. [ M19 ] [M19] Derive (10). Verified: no Solve time: 3m34s The reviewer is correct that the previous response never engages with the actual mathematical content of equations (8), (9), and (10). However, there is a more fundamental issue: the statement of the problem here does not include those equations. A derivation in the TAOCP style is entirely determined by the...
TAOCP 5.5 Exercise 81
Section 5.5: Summary, History, and Bibliography Exercise 81. [**] [Initialize.] Set «<< 1. $2. [Compare.] If K = K;, the algorithm terminates successfully. $3. [Advance.] Increase i by 1. S4. [End of file?] If 7 < N, go back to $2. Otherwise the algorithm terminates unsuccessfully. J Notice that this algorithm can terminate in two different ways, successfully (having located the desired key) or unsuccessfully (having established that the given...
TAOCP 5.2.4 Exercise 8
Section 5.2.4: Sorting by Merging Exercise 8. [ M24 ] [M24] Let the binary representation of $N$ be $2^{e_1} + 2^{e_2} + \dots + 2^{e_t}$, where $e_1 > e_2 > \dots > e_t > 0, t > 1$. Prove that the maximum number of key comparisons performed by Algorithm L is $$1 - 2^0 + \sum (e_i + k - 1) 2^0.$$ Verified: no Solve time: 4m11s The error in...
TAOCP 5.5 Exercise 1
Section 5.5: Summary, History, and Bibliography Exercise 1. [ 05 ] [05] Summarize the contents of this chapter by stating a generalization of Theorem 5.4.6A. Verified: yes Solve time: 1m10s The methods discussed in this chapter are unified by viewing external sorting as the problem of constructing initial sorted runs and then combining them by successive multiway merges until a single ordered run remains. Theorem 5.4.6A extends to the general...
TAOCP 5.2.4 Exercise 7
Section 5.2.4: Sorting by Merging Exercise 7. [ M34 ] [M34] Give an analysis of the average running time of Program L, in the style of other analyses in this chapter: Interpret the quantities $A, B, B', \dots$, and explain how to compute their exact average values. How long does Program L take to sort the 16 numbers in Table 3? Verified: no Solve time: 4m48s A correct analysis must...
TAOCP 5.5 Exercise 3
Section 5.5: Summary, History, and Bibliography Exercise 3. [ 87 ] [87] (Stable sorting in minimum storage.) A sorting algorithm is said to require minimum storage if it uses only O((log N )?) bits of memory space for its variables besides the space needed to store the N records. The algorithm must be general in the sense that it works for all N, not just for a particular value of...
TAOCP 5.3.3 Exercise 10
Section 5.3.3: Minimum-Comparison Selection Exercise 10. [ M47 ] [M47] What is the asymptotic value of $V_2(n) - n$, as $n \to \infty$? Verified: no Solve time: 5m50s
TAOCP 6.3 Exercise 32
Section 6.3: Digital Searching Exercise 32. [ M21 ] [M21] What is the average value of the sum of all SKIP fields in a random Patrician tree with N, 1 internal nodes? Verified: no Solve time: 1m50s Solution Let keys be infinite binary sequences generated by independent unbiased bits. The Patrician tree is the compressed binary trie in which each internal node stores a SKIP value equal to the number...
TAOCP 5.4.8 Exercise 10
Section 5.4.8: Two-Tape Sorting Exercise 10. [ 20 ] [20] How would you sort a multireel file with only two tapes? *5.4.9. Disks and Drums So far we have considered tapes as the vehicles for external sorting, but more flexible types of mass storage devices are generally available. Although such “bulk memory” or “direct-access storage” units come in many different forms, they may be roughly characterized by the following properties:...
TAOCP 5.3.2 Exercise 21
Section 5.3.2: Minimum-Comparison Merging Exercise 21. [ M47 ] [M47] Prove or disprove (23) and (24). Verified: no Solve time: 1m39s Let $M(m,n)$ be Knuth’s function from Section 5.3.2: the maximum cost (in pointer traversals, or equivalent unit-cost steps as defined in the section) of performing any sequence of $m$ FIND operations and $n$ UNION operations on a disjoint-set data structure, under the specified heuristics. The statements Equations (23) and...
TAOCP 5.4.9 Exercise 10
Section 5.4.9: Disks and Drums Exercise 10. [ HM44 ] [HM44] (L. Hyafil, F. Prusker, J. Vuillemin.) Prove that when a and @ are fixed, Ai(n) = amn + Bn + Am(n) for all sufficiently large n, if m minimizes the coefficient in exercise 9. Verified: no Solve time: 1m02s Setup Let $A_i(n)$ denote the minimum transmission cost (external path length) among all merge trees with $n$ leaves, under fixed...
TAOCP 5.4.2 Exercise 20
Section 5.4.2: The Polyphase Merge Exercise 20. [ M24 ] [M24] What generating functions for Caron’s polyphase sort on six tapes correspond to (7) and to (16)? What relations, analogous to (g) and (27), define the strings of merge numbers? Verified: no Solve time: 1m02s Solution Let $T=6$ and $P=5$. Caron’s polyphase scheme on six tapes produces, in each direction of the construction (forward distribution and reverse reconstruction), sequences of...
TAOCP 6.3 Exercise 11
Section 6.3: Digital Searching Exercise 11. [ 21 ] [21] Can Algorithm 6.2.2D be used without change to delete a node from a digital search tree? Verified: yes Solve time: 1m38s Solution Algorithm 6.2.2D for deletion in a binary search tree relies on the invariant that every node partitions the set of keys into those strictly less than the node key and those strictly greater than it. The replacement step...
TAOCP 6.3 Exercise 30
Section 6.3: Digital Searching Exercise 30. [ M24 ] [M24] Show that the Patrician generating function h,(z) defined in (15) can be expressed in the rather horrible form m n, -1 1 nd? ( » ("an (2%, DQu+2, 1)... Qat Fem, 5): m>1 ayt::+am=n-1 [Thus, if there is a simple formula for h,»(z), we will be able to simplify this rather ungainly expression. ] Verified: yes Solve time: 4m07s We...
TAOCP 5.4.1 Exercise 19
Section 5.4.1: Multiway Merging and Replacement Selection Exercise 19. [ HM22 ] [HM22] Use the snowplow analogy to prove that the first run produced by replacement selection is approximately (e, 1)P records long. Verified: no Solve time: 3m59s We restart from a faithful snowplow model of replacement selection and avoid any per-record attribution. Correct solution Let the memory contain $P$ records. Assume all keys are distinct and that the input...
TAOCP 5.3.2 Exercise 19
Section 5.3.2: Minimum-Comparison Merging Exercise 19. [ 23 ] [23] Prove that the $M$ function satisfies (22). Verified: yes Solve time: 1m17s Solution Let $M(m,n)$ denote the minimum number of comparisons required to merge two increasing sequences of lengths $m$ and $n$. Consider any optimal merging algorithm $\mathcal{A}$ that merges a sequence $A_1 < \cdots < A_m$ with $B_1 < \cdots < B_n$ using $M(m,n)$ comparisons. Construct a transformed instance...
TAOCP 5.5 Exercise 5
Section 5.5: Summary, History, and Bibliography Exercise 5. [ 46 ] [46] It is much more difficult to sort nonrandom data with numerous equal keys than to sort uniformly random data. Devise a sorting benchmark that (i) is interesting now and will probably be interesting 100 years from now; (ii) does not involve uniformly random keys; and (iii) does not use data sets that change with time. | shall have...
TAOCP 5.1.2 Exercise 9
Section 5.1.2: Permutations of a Multiset Exercise 9. [ 33 ] [33] Write computer programs that perform the factorizations of a given multiset permutation into the forms mentioned in Theorems A and C. Verified: no Solve time: 20m23s Exercise 5.1.2.9 [ 33 ] 9. Write computer programs that perform the factorizations of a given multiset permutation into the forms mentioned in Theorems A and C. Solution Let $M$ be a...
TAOCP 5.1.2 Exercise 8
Section 5.1.2: Permutations of a Multiset Exercise 8. [ 20 ] [20] Find all factorizations of (12) into two factors $\alpha \mathbin{\text{?}} \beta$. Verified: no Solve time: 31m23s Stopped thinking
TAOCP 5.1.2 Exercise 5
Section 5.1.2: Permutations of a Multiset Exercise 5. [**] [M23] Condition (b) of Theorem B requires $x < y$; what would happen if we weakened the relation to $x \le y$? Verified: yes Solve time: 16m30s Condition (b) must exclude the case $x=y$. If it is changed from $x<y$ to $x\le y$, two equal elements would satisfy the condition, and the construction of Theorem B would no longer distinguish between...
TAOCP 5.1.2 Exercise 6
Section 5.1.2: Permutations of a Multiset Exercise 6. [**] [M15] How many strings are there that contain exactly $m$ a’s, $n$ b’s, and no other letters, with exactly $k$ of the a’s preceded immediately by a b? Verified: no Solve time: 1h12m Solution to Exercise 5.1.2.6, Problem 6 [M15] We are asked to count the number of strings containing exactly $m$ a’s and $n$ b’s, with exactly $k$ of the...
TAOCP 5.1.2 Exercise 7
Section 5.1.2: Permutations of a Multiset Exercise 7. [**] [M21] How many strings on the letters $a, b, c$ satisfying conditions (18) begin with the letter $a$? with the letter $b$? with $c$? Verified: yes Solve time: 1h12m Let the six admissible column types in (19) be $$ \binom{b}{a},\quad \binom{c}{a},\quad \binom{a}{b},\quad \binom{c}{b},\quad \binom{a}{c},\quad \binom{b}{c}, $$ since condition (18) excludes $\binom{a}{a},\binom{b}{b},\binom{c}{c}$. Formula (20) gives the number of strings satisfying (18): $$...
TAOCP 5.1.2 Exercise 4
Section 5.1.2: Permutations of a Multiset Exercise 4. [**] [M11] The canonical factorization of (12), in the sense of Theorem A, is given in (17) when $a < b < c < d$. Find the corresponding canonical factorization when $d < c < b < a$. Verified: no Solve time: 18m56s If $d < c < b < a$, the canonical factorization of (12) is obtained by reversing the order...
TAOCP 5.1.2 Exercise 3
Section 5.1.2: Permutations of a Multiset Exercise 3. [**] [M13] Is the converse of (9) valid? In other words, if $\alpha$ and $\beta$ commute under intercalation, must they have no letters in common? Verified: yes Solve time: 17m31s No. The converse of (9) is false. A counterexample is provided by the identity permutation on a one-element multiset. Let $\alpha=(a)$ and $\beta=(a)$. The two permutations have the letter $a$ in common....
TAOCP 5.1.2 Exercise 2
Section 5.1.2: Permutations of a Multiset Exercise 2. [**] [10] The intercalation of cadab and bddad is computed in (5); find the intercalation bddadcadab that is obtained when the factors are interchanged. Verified: no Solve time: 15m46s Using the definition of intercalation, we write $$ \beta=\text{bddad} \qquad\Longrightarrow\qquad \begin{pmatrix} a&b&d&d&d\ b&d&d&a&d \end{pmatrix}, $$ and $$ \alpha=\text{cadab} \qquad\Longrightarrow\qquad \begin{pmatrix} a&a&b&c&d\ c&a&d&a&b \end{pmatrix}. $$ After juxtaposing the two-line representations in the order $\beta,\alpha$...
TAOCP 5.1.2 Exercise 1
Section 5.1.2: Permutations of a Multiset Exercise 1. [**] [M05] True or false: Let $M_1$ and $M_2$ be multisets. If $\alpha$ is a permutation of $M_1$ and $\beta$ is a permutation of $M_2$, then $\alpha \mathbin{\text{?}} \beta$ is a permutation of $M_1 \cup M_2$. Verified: yes Solve time: 30m29s False. A permutation of a multiset must be a bijection from the multiset onto itself. If $\alpha$ is a permutation of...
TAOCP 5.1 Exercise 3
Section 5.1: Combinatorial Properties of Permutations Exercise 3. [**] [26] Design an algorithm that computes the inversion table $b_1 b_2 \dots b_n$ corresponding to a given permutation $a_1 a_2 \dots a_n$ of ${1,2,\dots,n}$, where the running time is essentially proportional to $n \log n$ on typical computers. Verified: yes Solve time: 18m15s Store the permutation in an array $P$ such that $P(j)$ is the position of $j$ in the permutation....
TAOCP 5.1 Exercise 1
Section 5.1: Combinatorial Properties of Permutations Exercise 1. [**] [10] What is the inversion table for the permutation 271845936? What permutation has the inversion table 50121200? Verified: no Solve time: 40m04s Let the inversion table of a permutation $a_1a_2\cdots a_n$ be the sequence $b_1b_2\cdots b_n$, where $b_i$ is the number of entries greater than $i$ that occur to the left of $i$ in the permutation. 1. Inversion table of $271845936$...
TAOCP 5.1 Exercise 2
Section 5.1: Combinatorial Properties of Permutations Exercise 2. [**] [M20] In the classical problem of Josephus (exercise 1.3.2, 22), $n$ men are initially arranged in a circle; the $m$th man is executed, the circle closes, and every $m$th man is repeatedly eliminated until all are dead. The resulting execution order is a permutation of {1,2,...,n}. For example, when $n = 8$ and $m = 4$ the order is 54613872 (man...
TAOCP 5 Exercise 5
Section 5: Introduction to Sorting Exercise 5. [ 83 ] [83] (Library card sorting.) Before the days of computerized databases, every library maintained a catalog of cards so that users could find the books they wanted. But the task of putting catalog cards into an order convenient for human use turned out to be quite complicated as library collections grew. The following “alphabetical” listing indicates many of the procedures recommended...
TAOCP 5 Exercise 1
Section 5: Introduction to Sorting Exercise 1. [**] [15] Mr. B. C. Dull (a MIX programmer) wanted to know if the number stored in location A is greater than, less than, or equal to the number stored in location B. So he wrote LDA A; SUB B and tested whether register A was positive, negative, or zero. What serious mistake did he make, and what should he have done instead?...
TAOCP 5 Exercise 4
Section 5: Introduction to Sorting Exercise 4. [ 20 ] [20] In order to make it easier to make changes to large FORTRAN programs, you want to design a “cross-reference” routine; such a routine takes FORTRAN programs as input and prints them together with an index that shows each use of each identifier (that is, each name) in the program. How should such a routine be designed? Verified: no Solve...
TAOCP 5 Exercise 2
Section 5: Introduction to Sorting Exercise 2. [**] [17] Write a MIX subroutine for multiprecision comparison of keys, having the following specifications: Calling sequence: JMP COMPARE Entry conditions: rll = n ; CONTENTS(A+k) = a_k , and CONTENTS(B+k) = b_k , for $1 \le k \le n$; assume that $n > 1$. Exit conditions: CI = GREATER , if $(a_n, \dots, a_1) > (b_n, \dots, b_1)$; CI = EQUAL ,...
TAOCP 5 Exercise 3
Section 5: Introduction to Sorting Exercise 3. [**] [M25] (Transposing a matrix.) You are given a magnetic tape containing one million words, representing the elements of a $1000 \times 1000$ matrix stored in order by rows: $a_{1,1}, a_{1,2}, \dots, a_{1,1000}; a_{2,1}, \dots, a_{2,1000}; \dots; a_{1000,1000}$. How do you create a tape in which the elements are stored by columns $a_{j,1}, a_{2,1}, \dots, a_{1000,1}, a_{1,2}, \dots, a_{1000,2}, \dots, a_{1000,1000}$ instead? (Try...