Vector space norms.
U5: The distance between two vectors in some norm.
These abstract properties are applicable to a wide range of data structures that we use in numerical analysis and its (many) applications. They can be specialised to cater for specific data structures such as vectors, matrices and tensors (three-dimensional matrices). For this reason we introduce the reader to vector space theory. Studying it will pay dividends in the rest of this book and beyond in the years to come. For example, vector spaces of finite dimension are generalised to infinite-dimensional Hilbert, Banach and Sobolev spaces, a discussion of which is outside the scope of this book.
4.1.1 Notation
This chapter attempts to present a self-contained and focused introduction to the essential concepts and methods for vector spaces of finite dimension. The applications are numerous, for example numerical linear algebra, finite Markov chains, multivariate optimisation, machine and statistical learning, graph theory and finite difference methods, to name just a few. To this end, we summarise the most important syntax and notation that we use throughout the book:
F or K : a field
: set of real and complex numbers, respectively
α, β, a, b : scalars
x, y, z : elements of a vector space
V, W : vector spaces
A, B, M : matrices
: norm in a vector space
: transpose of a vector, matrix
: inverse of a matrix
λ : eigenvalue of a matrix
d(x, y) : distance between vectors x and y
(x, y) : inner product of vectors x and y
: n-dimensional real and complex spaces, respectively
: set of real matrices with m rows and n columns
: a vector space V over a field K (see also )
dim V : dimension of a vector space
: linear transformation between two vector spaces and over the same field K
L(V; W) : the set of linear transformations from vector space V to vector space W
: null space (kernel) of a linear transformation whose dimension is n(T)
: the dimension of the range TV of a linear transformation
We use the following important syntax to denote matrices:
(4.1)
These symbols are used in definitions, theorems and algorithms. A good way to learn is to take (simpler) concrete examples before moving to more complex cases and applications.
In time, you should get to the stage whereby you can understand the above notation without having to think twice about it. The same remark holds for all the other notation in this book. It's half the battle!
4.2 WHAT IS A VECTOR SPACE?
From Wikipedia:
In mathematics, a field is a set in which addition, subtraction, multiplication, and division are defined and these operators behave as the corresponding operations on rational and real numbers do. A field is thus a fundamental algebraic structure which is widely used in algebra, number theory, and many other areas of mathematics.
The best known fields are the field of rational numbers, the field of real numbers and the field of complex numbers.
In general, we use the symbol K to denote a generic field, but in most cases we work with real numbers as the underlying field in vector spaces. In some cases, complex numbers are used.
A vector space V over a field K (denoted by V(K)) is a collection of objects (called vectors) together with operations of vector addition and multiplication by elements of K (called scalars) satisfying the following axioms for addition of vectors:
(4.2)
Axiom A1 states that addition is associative, and axiom A2 states that addition is commutative. The element 0 is called the zero element of the vector space.
Scalar multiplication is defined by the axioms
(4.3)
From these axioms we see that subtraction of vectors is possible because
The prototypical examples of vector spaces are n-dimensional vectors and rectangular matrices over a field K:
(4.4)
