Orthogonality

Definition (Orthogonal Vectors)

Two vectors $x$ and $y$ of an inner product space $(X,⟨\cdot ,\cdot ⟩)$ are said to be orthogonal (or perpendicular or normal) to one another if $⟨x,y⟩=0$. We indicate this fact by writing $x\perp y$.
A set $A\subset X$ is called orthogonal, if the elements of $S$ are pairwise orthogonal to each other.

Note that $\perp$ is a symmetric relation, that is, $x\perp y$ if and only if $y\perp x$.

Pythagoras’ Theorem

If $x$ and $y$ are orthogonal vectors of an inner product space, then

$$\Vert x+y{\Vert }^2=\Vert x{\Vert }^2+\Vert y{\Vert }^2.$$

More generally,if $\{x_1,\dots ,x_n\}$ is an orthogonal set in an inner product space, then

$$\Vert x_1+\cdots +x_n{\Vert }^2=\Vert x_1{\Vert }^2+\cdots +\Vert x_n{\Vert }^2.$$

The converse of Pythagoras’ Theorem is true for real inner product space, but false in the complex case. For example, let $x$ be any unit vector in a complex inner product space. Then $x$ is not orthogonal to $ix$, since $⟨x,ix⟩=i\ne 0$. However, $\Vert x+ix{\Vert }^2=|1+i{|}^2=2=1+1=\Vert x{\Vert }^2+\Vert ix{\Vert }^2$.

TODO Bessel

Definition (Orthonormal Set)

A subset $S$ of an inner product space is called orthonormal if we have for all $x,y\in S$

$$⟨x,y⟩=\{\begin{array}{ll}0& x=y,\\ 1& x\ne y.\end{array}$$

In other words, an orthonormal set is an orthogonal set of unit vectors.

Every orthonormal set is linearly independent.

Proof   Suppose that $\{x_1,\dots ,x_n\}$ is a finite subset of $S$ and that

$${\alpha }_1{x}_1+\cdots +{\alpha }_n{x}_n=0$$

for some scalars ${\alpha }_1,\dots ,{\alpha }_n$. Application of $⟨x_i,\cdot ⟩$ yields ${\alpha }_i=0$ for all $i\in \{1,\dots ,n\}$. $\square$

Recall that a subset $S$ of a normed space $X$ is called total if its span is dense in $X$.

Definition (Orthonormal Basis)

A total orthonormal set in an inner product space is called orthonormal basis (or complete orthonormal system).

Every Hilbert space has an orthonormal basis.

Orthogonal Complement

Definition (Orthogonal Complement)

Let $X$ be an inner product space. For any subset $U\subset X$ we define its orthogonal complement to be the set $U^{\perp }=\left\{x\in X:x\perp u\forall u\in U\right\}$.

$U^{\perp }$ is a linear subspace

$U\subset V$ implies $U^{\perp }\supset V^{\perp }$

Let $Y$ be a linear subspace of an inner product space $X$. Then, for any $x\in X$, we have

$$x\in Y^{\perp }\iff \forall y\in Y:\Vert x-y\Vert \ge \Vert x\Vert .$$

Gram–Schmidt Process