Decision tree model
In computational complexity the decision tree model is the model of computation in which an algorithm is considered to be basically a decision tree, i.e., a sequence of branching operations based on comparisons of some quantities, the comparisons being assigned unit computational cost.
The branching operations are called "tests" or "queries". In this setting the algorithm in question may be viewed as a computation of a Boolean function where the input is a series of queries and the output is the final decision. Each query may be dependent on previous queries.
Several variants of decision tree models have been introduced, depending on the complexity of the operations allowed in the computation of a single comparison and the way of branching.
Decision trees models are instrumental in establishing lower bounds for complexity theory for certain classes of computational problems and algorithms. The computational complexity of a problem or an algorithm expressed in terms of the decision tree model is called its decision tree complexity or query complexity.
Classification by query computational complexity
Simple decision tree
The model in which every decision is based on the comparison of two numbers within constant time is called a decision tree model. It was introduced to establish computational complexity of sorting and searching.
The simplest illustration of this lower bound technique is for the problem of finding the smallest number among numbers using only comparisons. In this case the decision tree model is a binary tree. Algorithms for this searching problem may result in different outcomes (since any of the given numbers may turn out to be the smallest one). It is known that the depth of a binary tree with leaves is at least , which gives a lower bound of for the searching problem.
However this lower bound is known to be slack, since the following simple argument shows that at least comparisons are needed: Before the smallest number can be determined, every number except the smallest must "lose" (compare greater) in at least one comparison.
Along the same lines a lower bound of for sorting may be proved. In this case, the existence of numerous comparison-sorting algorithms having this time complexity, such as mergesort and heapsort, demonstrates that the bound is tight.
Linear decision tree
Linear decision trees, just like simple decision trees, make a branching decision based on a set of values as input. As opposed to binary decision trees, linear decision trees have three output branches. A linear function, which is any function of the form for some real numbers is being tested and branching decisions are made based on the sign of the function (negative, positive, or 0).
Geometrically, defines a hyperplane in Rn. A set of input values reaching any particular node represents the intersection of the half-planes defined by the nodes.
Algebraic decision tree
Algebraic decision trees are a generalization of linear decision trees that allow the test functions to be polynomials of degree . Geometrically, the space is divided into semi-algebraic sets (a generalization of hyperplane). The evaluation of the complexity is typically more difficult.
Classification by query computational model
Deterministic decision tree
If the output of a decision tree is , for all , the decision tree is said to "compute" . The depth of a tree is the maximum number of queries that can happen before a leaf is reached and a result obtained. , the deterministic decision tree complexity of is the smallest depth among all deterministic decision trees that compute .
Randomized decision tree
One way to define a randomized decision tree is to add additional nodes to the tree, each controlled by a probability . Another equivalent definition is to define it as a distribution over deterministic decision trees. Based on this second definition, the complexity of the randomized tree is defined as the largest depth among all the trees in the support of the underlying distribution. is defined as the complexity of the lowest-depth randomized decision tree whose result is with probability at least for all (i.e., with bounded 2-sided error).
is known as the Monte Carlo randomized decision-tree complexity, because the result is allowed to be incorrect with bounded two-sided error. The Las Vegas decision-tree complexity measures the expected depth of a decision tree that must be correct (i.e., has zero-error). There is also a one-sided bounded-error version which is denoted by .
Nondeterministic decision tree
The nondeterministic decision tree complexity of a function is known more commonly as the certificate complexity of that function. It measures the number of input bits that a nondeterministic algorithm would need to look at in order to evaluate the function with certainty.
Quantum decision tree
The quantum decision tree complexity is the depth of the lowest-depth quantum decision tree that gives the result with probability at least for all . Another quantity, , is defined as the depth of the lowest-depth quantum decision tree that gives the result with probability 1 in all cases (i.e. computes exactly). and are more commonly known as quantum query complexities, because the direct definition of a quantum decision tree is more complicated than in the classical case. Similar to the randomized case, we define and .
Relationship between different models
It follows immediately from the definitions that for all -bit Boolean functions , , and .
Blum and Impagliazzo, Hartmanis and Hemachandra, and Tardos independently discovered that . Noam Nisan found that the Monte Carlo randomized decision tree complexity is also polynomially related to deterministic decision tree complexity: . (Nisan also showed that .) A tighter relationship is known between the Monte Carlo and Las Vegas models: . This relationship is optimal up to polylogarithmic factors.
The quantum decision tree complexity is also polynomially related to . Midrijanis showed that , improving a quartic bound due to Beals et al. Beals et al. also showed that , and this is still the best known bound. However, the largest known gap between deterministic and quantum query complexities is only quadratic. A quadratic gap is achieved for the OR function; while .
It is important to note that these polynomial relationships are valid only for total Boolean functions. For partial Boolean functions, that have a domain a subset of , an exponential separation between and is possible; the first example of such a problem was discovered by Deutsch and Jozsa.
- "Data structures and algorithms, by Alfred V. Aho, John E. Hopcroft, Jeffrey D. Ullman
- Blum, M.; Impagliazzo, R. (1987). "Generic oracles and oracle classes". Proceedings of 18th IEEE FOCS. pp. 118–126.
- Hartmanis, J.; Hemachandra, L. (1987), "One-way functions, robustness, and non-isomorphism of NP-complete sets", Technical Report DCS TR86-796, Cornell University
- Tardos, G. (1989). "Query complexity, or why is it difficult to separate NPA ∩ coNPA from PA by random oracles A?". Combinatorica. 9: 385–392. doi:10.1007/BF02125350.
- Nisan, N. (1989). "CREW PRAMs and decision trees". Proceedings of 21st ACM STOC. pp. 327–335.
- Kulkarni, R. and Tal, A. On Fractional Block Sensitivity. Electronic Colloquium on Computational Complexity (ECCC). Vol. 20. 2013.
- Ambainis, Andris; Balodis, Kaspars; Belovs, Aleksandrs; Lee, Troy; Santha, Miklos; Smotrovs, Juris (2015). "Separations in Query Complexity Based on Pointer Functions". arXiv:1506.04719 [cs.CC].
- Midrijanis, Gatis (2004), "Exact quantum query complexity for total Boolean functions", arXiv:quant-ph/0403168
- Midrijanis, Gatis (2005), "On Randomized and Quantum Query Complexities", arXiv:quant-ph/0501142
- Beals, R.; Buhrman, H.; Cleve, R.; Mosca, M.; de Wolf, R. (2001). "Quantum lower bounds by polynomials". Journal of the ACM. 48 (4): 778–797. arXiv:quant-ph/9802049. doi:10.1145/502090.502097.
- H. Buhrman, R. Cleve, R. de Wolf, and Ch. Zalka. Bounds for Small-Error and Zero-Error Quantum Algorithms. In 40th IEEE Symposium on Foundations of Computer Science (FOCS'99), pp.358-368. cs.CC/9904019, 1999.
- S. Aaronson. Quantum Certificate Complexity. IEEE Conference on Computational Complexity, pp. 171-178, 2003.
- Buhrman, Harry; de Wolf, Ronald (2002), "Complexity Measures and Decision Tree Complexity: A Survey" (PDF), Theoretical Computer Science, 288 (1): 21–43, doi:10.1016/S0304-3975(01)00144-X