A b o u t   M e      |       P r o j e c t s     |       N o t e s       |       T h e   D a y    ︎ ︎


1. Vector Norms

    Let `x=(x_1, cdots, x_n)^t` be an `ntimes 1` vector.

  • `p`-norm: `||x||_p=left( sum_{i=1}^{n} |x_i|^p right)^{1/p}`
  • `1`-norm: `||x||_1=sum_{i=1}^{n} |x_i|`
  • `2`-norm(Euclidean norm): `||x||_2=left( sum_{i=1}^{n} |x_i|^2 right)^{1/2}`
  • `infty`-norm: `||x||_{infty}=max|x_i|`
\begin{align} ||x||_{\infty}&= \lim_{p \to \infty} \left( \sum_{i=1}^{n} |x_i|^p \right)^{1/p} = \lim_{p \to \infty} \left( |x_j|^p \right)^{1/p} \   \text{where} \   j = \arg \max |x_i|  \\ &=\max|x_i| \end{align}
  • The graphs of `||x||_1= ||x||_2= ||x||_{infty}=1` for `x in mathbb{R}^2` are as follows.

  • For any vector `bar{x}`, `||bar{x} ||_{infty} leq || bar{x}||_2 leq || bar{x}||_1 `. The right image shows this comparison when `bar{x} in mathbb{R}^2`. 
  • Meanwhile, `||x||_1 leq sqrt(n) ||x||_2`, `||x||_2 leq sqrt(n) ||x||_{infty}`, and `||x||_1 leq n ||x||_{infty}`.

  • If `||x||_p>0`, then `x ne 0`.
  • `||gamma x||_p=|gamma| ||x||_p` where `gamma in mathbb{R}`.
  • `||x+y||_p leq ||x||_p+||y||_p`
  • `|``||x||_p - ||y||_p  | leq`` ||x-y ||_p`

2. Matrix Norms

    Suppose that `A` is an `m times n` matrix and `a_{ij}` is the `(i, j)` element of `A`.
   `||A||` means the maximum stretching of `A` to any vector `x`.
\begin{align} \left\| A \right\| = \max_{x \ne 0} \frac{ \left\| Ax \right\| }{ \left\| x \right\| } \end{align}

  • `||A||_1=max sum_{i=1}^m |a_{ij}|`, which means the largest column sum of `A`.
  • `||A||_2` is the largest singular value of `A`, which means the square root of the largest eigenvalue of `A^tA`.
  • When `A` is symmetric, `A^tAv=A(lambda v)=lambda^2 v` where `x` and `lambda `are the eigenvector and eigenvalue of `A`. Then  
\begin{align} \left| A \right|_2 = \sqrt{\lambda_{\max}(A^tA)} = \sqrt{\lambda_{\max}(A)^2} = |\lambda_{\max}(A)| \end{align}
  • `||A||_{infty}=max sum_{j=1}^n |a_{ij}|`, which means the largest row sum of `A`.
  • If `||A||>0`, then `A ne O`.
  • `||gamma A||=|gamma| ||A||` where `gamma in mathbb{R}`.
  • `||A+B|| leq ||A||+||B||`
  • `||AB|| leq ||A||||B||`
  • `||Ax|| leq ||A||||x||`


[1] Michael T. Heath, Scientific Computing: An Introductory Survey. 2nd Edition, McGraw-Hill Higher Education.