Inner Product Spaces

Definition (Inner Product Space)

An inner product (or scalar product) on a real or complex vector space $X$ is a mapping

$$⟨\cdot ,\cdot ⟩:X\times X\to \mathbb{K}$$

that is

• linear in its second argument

  $$⟨x,y+z⟩=⟨x,y⟩+⟨x,z⟩⟨x,\alpha y⟩=\alpha ⟨x,y⟩$$

• conjugate symmetric

  $$\overline{⟨x,y⟩}=⟨x,y⟩$$

• nondegenerate

An inner product space (or pre-Hilbert space) is a pair $(X,⟨\cdot ,\cdot ⟩)$ consisting of a real or complex vector space $X$ and an inner product $⟨\cdot ,\cdot ⟩$ on $X$.

Proposition (Norm Induced by an Inner Product)

If $⟨\cdot ,\cdot ⟩$ is an inner product on a real or complex vector space $X$, then

$$\Vert x\Vert =\sqrt{⟨x,x⟩}\forall x\in X$$

defines a norm on $X$.

In this sense, every inner product space is also a normed space. As a consequence it is also a metric space and a topological space.

The next theorem shows how the inner product can be recovered from the norm.

Polarization Identity

For all vectors $x$ and $y$ of a real inner product space

$$4⟨x,y⟩=\Vert x+y{\Vert }^2-\Vert x-y{\Vert }^2.$$

For all vectors $x$ and $y$ of a complex inner product space

$$4⟨x,y⟩=\Vert x+y{\Vert }^2-\Vert x-y{\Vert }^2+i\Vert x-iy{\Vert }^2-i\Vert x+iy{\Vert }^2.$$

Note that the complex polarization identity takes the slightly different form

$$4⟨x,y⟩=\Vert x+y{\Vert }^2-\Vert x-y{\Vert }^2+i\Vert x+iy{\Vert }^2-i\Vert x-iy{\Vert }^2,$$

if we follow the convention that the inner product is conjugate linear in its second argument.

Proof   In the real case, the inner product is symmetric, and we have

$$\Vert x\pm y{\Vert }^2=\Vert x{\Vert }^2\pm 2⟨x,y⟩+\Vert y{\Vert }^2$$

for all vectors $x$ and $y$. Taking the difference yields the desired result.

In the complex case, the inner product is conjugate symmetric, and we have

$$\Vert x\pm y{\Vert }^2=\Vert x{\Vert }^2\pm 2\mathrm{Re}⟨x,y⟩+\Vert y{\Vert }^2$$

for all vectors $x$ and $y$. This implies

$$\begin{array}{rl}\Vert x+y{\Vert }^2-\Vert x-y{\Vert }^2& =4\mathrm{Re}⟨x,y⟩,\\ \Vert x-iy{\Vert }^2-\Vert x+iy{\Vert }^2& =4\mathrm{Im}⟨x,y⟩.\end{array}$$

The second equation follows from the first by substituting $y$ with $-iy$ and using that $\mathrm{Re}⟨x,-iy⟩=\mathrm{Re}(-i⟨x,y⟩)=\mathrm{Im}⟨x,y⟩$. To obtain the polarization Identity, multiply the second equation with $i$ and then add it to the first. $\square$

General Polarization Identity

Let $X$ be a complex inner product space. Let $\zeta$ be a $n$-th root of unity with $\zeta \ne 1$ and ${\zeta }^2\ne 1$. Then

$$⟨x,y⟩=\frac{1}{n}\sum _{k=0}^{n-1}{\zeta }^k\Vert x+{\zeta }^{k}y{\Vert }^2\forall x,y\in X.$$

As a special case, for $\zeta =i$ and $n=4$, we obtain

$$⟨x,y⟩=\frac{1}{4}\sum _{k=0}^3{i}^k\Vert x+i^{k}y{\Vert }^2.$$

Proof   TODO $\square$

For an arbitrary normed space, the polarization identity does not, in general, define an inner product. The following theorem, gives a condition for when it does.

Parallelogram Law

Let $X$ be a real or complex normed space. A norm $\Vert \cdot \Vert$ on $X$ is induced by an inner product $⟨\cdot ,\cdot ⟩$ on $X$, if and only if $\Vert \cdot \Vert$ satisfies the parallelogram law

$$\Vert x+y{\Vert }^2+\Vert x-y{\Vert }^2=2\Vert x{\Vert }^2+2\Vert y{\Vert }^2\forall x,y\in X.$$

In this case, the inner product is uniquely determined by $\Vert \cdot \Vert$ and given by the polarization identity.

Stewart’s Theorem

Let $x$, $y$, $z$ be vectors of an inner product space. If $x$, $y$ and $z$ are colinear and $y$ lies in between $x$ and $y$, then we have

$$\Vert p-x{\Vert }^2\Vert y-z\Vert +\Vert p-z{\Vert }^2\Vert x-y\Vert =(\Vert p-y{\Vert }^2+\Vert x-y\Vert \Vert y-z\Vert )\Vert x-z\Vert$$

---

Cauchy–Schwarz Inequality

For all vectors $x$ and $y$ of an inner product space (with inner product $⟨\cdot ,\cdot ⟩$ and induced norm $\Vert \cdot \Vert$)

$$|⟨x,y⟩|\le \Vert x\Vert \Vert y\Vert ,$$

and equality holds precisely when $x$ and $y$ are linearly dependent.

Expressed only in terms of the inner product, the Cauchy–Schwarz Inequality reads

$$|⟨x,y⟩{|}^2\le ⟨x,x⟩⟨y,y⟩.$$

Proof   TODO $\square$

Corollary (Continuity of the Inner Product)

The inner product is jointly norm continuous.