Let denote the vector space of all matrices of size (with rows and columns) with entries in the field .
For all scalars and for all matrices ,
- (being absolutely homogeneous)
- (being sub-additive or satisfying the triangle inequality)
- (being positive-valued)
- iff (being definite)
Additionally, in the case of square matrices (thus, m = n), some (but not all) matrix norms satisfy the following condition, which is related to the fact that matrices are more than just vectors:
- for all matrices and in
A matrix norm that satisfies this additional property is called a sub-multiplicative norm (in some books, the terminology matrix norm is used only for those norms which are sub-multiplicative). The set of all matrices, together with such a sub-multiplicative norm, is an example of a Banach algebra.
The definition of sub-multiplicativity is sometimes extended to non-square matrices, for instance in the case of the induced p-norm, where for and holds that . Here and are the norms induced from and , respectively, and p,q ≥ 1.
There are three types of matrix norms which will be discussed below:
- Matrix norms induced by vector norms,
- Entrywise matrix norms, and
- Schatten norms.
Matrix norms induced by vector norms
Suppose a vector norm on is given. Any matrix A induces a linear operator from to with respect to the standard basis, and one defines the corresponding induced norm or operator norm on the space of all matrices as follows:
In particular, if the p-norm for vectors (1 ≤ p ≤ ∞) is used for both spaces and , then the corresponding induced operator norm is:
- Note: We have described above the induced operator norm when the same vector norm was used in the "departure space" and the "arrival space" of the operator . This is not a necessary restriction. More generally, given a norm on , and a norm on , one can define a matrix norm on induced by these norms:
- The matrix norm is sometimes called a subordinate norm. Subordinate norms are consistent with the norms that induce them, giving
Any induced operator norm is a sub-multiplicative matrix norm: this follows from
Moreover, any induced norm satisfies the inequality
where ρ(A) is the spectral radius of A. For symmetric or hermitian A, we have equality in (1) for the 2-norm, since in this case the 2-norm is precisely the spectral radius of A. For an arbitrary matrix, we may not have equality for any norm; a counterexample being given by
which has vanishing spectral radius. In any case, for square matrices we have the spectral radius formula:
In the special cases of the induced matrix norms can be computed or estimated by
which is simply the maximum absolute column sum of the matrix;
which is simply the maximum absolute row sum of the matrix;
where represents the largest singular value of matrix . There is an important inequality for the case :
where is the Frobenius norm. Equality holds if and only if the matrix is a rank-one matrix or a zero matrix. This inequality can be derived from the fact that the trace of a matrix is equal to the sum of its eigenvalues.
When we have an equivalent definition for as . This definition can be shown to be equivalent to the above definitions using the Cauchy–Schwarz inequality.
For example, for
In the special case of (the Euclidean norm or -norm for vectors), the induced matrix norm is the spectral norm. The spectral norm of a matrix is the largest singular value of i.e. the square root of the largest eigenvalue of the matrix where denotes the conjugate transpose of :
"Entrywise" matrix norms
These norms treat an matrix as a vector of size , and use one of the familiar vector norms. For example, using the p-norm for vectors, p ≥ 1, we get:
This is a different norm from the induced p-norm (see above) and the Schatten p-norm (see below), but the notation is the same.
The special case p = 2 is the Frobenius norm, and p = ∞ yields the maximum norm.
L2,1 and Lp,q norms
The norm can be generalized to the norm, p, q ≥ 1, defined by
When p = q = 2 for the norm, it is called the Frobenius norm or the Hilbert–Schmidt norm, though the latter term is used more frequently in the context of operators on (possibly infinite-dimensional) Hilbert space. This norm can be defined in various ways:
The Frobenius norm is an extension of the Euclidean norm to and comes from the Frobenius inner product on the space of all matrices.
Frobenius norm is often easier to compute than induced norms and has the useful property of being invariant under rotations and, more generally, under unitary operations, that is, for any unitary matrix . This property follows from the cyclic nature of the trace ():
where we have used the unitary nature of (that is, ).
It also satisfies
where is the Frobenius inner product.
The Schatten p-norms arise when applying the p-norm to the vector of singular values of a matrix. If the singular values of the matrix are denoted by σi, then the Schatten p-norm is defined by
These norms again share the notation with the induced and entrywise p-norms, but they are different.
All Schatten norms are sub-multiplicative. They are also unitarily invariant, which means that for all matrices and all unitary matrices and .
The most familiar cases are p = 1, 2, ∞. The case p = 2 yields the Frobenius norm, introduced before. The case p = ∞ yields the spectral norm, which is the operator norm induced by the vector 2-norm (see above). Finally, p = 1 yields the nuclear norm (also known as the trace norm, or the Ky Fan 'n'-norm), defined as
A matrix norm on is called consistent with a vector norm on and a vector norm on if:
for all . All induced norms are consistent by definition.
A matrix norm on is called compatible with a vector norm on if:
for all . Induced norms are compatible with the inducing vector norm by definition.
Equivalence of norms
For any two matrix norms and , we have
Moreover, for every vector norm on , there exists a unique positive real number such that is a sub-multiplicative matrix norm for every .
A sub-multiplicative matrix norm is said to be minimal if there exists no other sub-multiplicative matrix norm satisfying .
- Carl D. Meyer, Matrix Analysis and Applied Linear Algebra, §5.2, p.281, Society for Industrial & Applied Mathematics, June 2000.
- Ding, Chris; Zhou, Ding; He, Xiaofeng; Zha, Hongyuan (June 2006). "R1-PCA: Rotational Invariant L1-norm Principal Component Analysis for Robust Subspace Factorization". Proceedings of the 23rd International Conference on Machine Learning. ICML '06. Pittsburgh, Pennsylvania, USA: ACM. pp. 281–288. doi:10.1145/1143844.1143880. ISBN 1-59593-383-2.
- Fan, Ky. (1951). "Maximum properties and inequalities for the eigenvalues of completely continuous operators". Proceedings of the National Academy of Sciences of the United States of America. 37 (11): 760–766. doi:10.1073/pnas.37.11.760. PMC 1063464.
- Golub, Gene; Charles F. Van Loan (1996). Matrix Computations – Third Edition. Baltimore: The Johns Hopkins University Press, 56–57. ISBN 0-8018-5413-X.
- Roger Horn and Charles Johnson. Matrix Analysis, Chapter 5, Cambridge University Press, 1985. ISBN 0-521-38632-2.
- James W. Demmel, Applied Numerical Linear Algebra, section 1.7, published by SIAM, 1997.
- Carl D. Meyer, Matrix Analysis and Applied Linear Algebra, published by SIAM, 2000.
- John Watrous, Theory of Quantum Information, 2.3 Norms of operators, lecture notes, University of Waterloo, 2011.
- Kendall Atkinson, An Introduction to Numerical Analysis, published by John Wiley & Sons, Inc 1989