개발로 하는 개발

[LG Aimers] Module 2. Mathematics for ML -1 본문

Study

[LG Aimers] Module 2. Mathematics for ML -1

jiwon152 2024. 1. 13. 22:29

1. Determinant

- Definition

For $A=\begin{pmatrix}a_{11}a_{12}\\a_{21}a_{22}\end{pmatrix},A^{-1}=\frac{1}{a_{11}a_{12}-a_{21}a_{22}}\cdot\begin{pmatrix}a_{22}-a_{12}\\-a_{21}a_{22}\end{pmatrix}$

 

A is invertible if $\frac{1}{a_{11}a_{22}-a_{21}a_{12}}\neq 0$

 

$\therefore det(A) = \begin{vmatrix}a_{11}a_{12}\\a_{21}a_{22}\\\end{vmatrix} = \frac{1}{a_{11}a_{12}-a_{21}a_{22}}$

 

- for $3\times3$ matrix,

\begin{vmatrix}a_{11}a_{12}a_{13}\\a_{21}a_{22}a_{23}\\a_{31}a_{32}a_{33}\\\end{vmatrix}

$= a_{11}a_{22}a_{33}+a_{21}a_{32}a_{13}+a_{31}a_{12}a_{23}-a_{31}a_{22}a_{13}-a_{11}a_{32}a_{23}-a_{21}a_{12}a_{33}a_{11}$

 

- Laplace expansion

 

$\begin{vmatrix}a_{11}a_{12}a_{13}\\a_{21}a_{22}a_{23}\\a_{31}a_{32}a_{33}\\\end{vmatrix}$

 

$=(-1)^{1+1}det(A_{1,1})+a_{12}(-1)^{1+2}det(A_{1,2})+a_{13}(-1)^{1+3}det(A_{1,3})$

where $A_{k, j}$ is the submatrix of A that we obtain when deleting row k and column j.

 

- Formal Definition

For a matrix A $\mathbb{R}^{n\times n}$, for all j = 1,...,n,

 

1. Expansion along column j: $det(A) = \sum_{k=1}^{n}(-1)^{k+j}\cdot a_{kj}\cdot det(A_{k,j})$

2. Expansion along row j: $det(A) = \sum_{k=1}^{n}(-1)^{k+j}\cdot a_{jk}\cdot det(A_{j,k})$

 

- Properties of Determinant

triangular matrix includes diagonal matrix

 

2. Trace

- Definition

the trace of a square matric $A\in \mathbb{R}^{n\times n}$ is defined as $tr(A)\overset{\underset{\mathrm{def}}{}}{=}\sum_{i=1}^{n}a_{ii}$

 

- Properties

$tr(A+B)=tr(A) + tr(B)$

$tr(\alpha A) = \alpha tr(A)$

$tr(I_{n}) = n$

 

3. Eigenvalue and Eigenvector

- Definition

consider a square matrix $A\in \mathbb{R}^{n\times n}$

Then, $\lambda \in \mathbb{R}$ is an eigenvalue of A and

$x\in \mathbb{R}^{n}\setminus \begin{Bmatrix}0\end{Bmatrix}$ is the corresponding eigenvector of A

if $Ax = \lambda x$

 

- Equivalent statements

$\lambda$ is a eigenvalue

$(A - \lambda I_{n})x = 0$ can be solved non-trivially, i.e., $x\neq 0$

$det(A - \lambda I_{n}) = 0$

 

- Eigenvectors are not unique

 

- Properties

1. for $A\in \mathbb{R}^{n\times n}$, n distinct eigenvalues $\Rightarrow $ eigenvectors are linearly independent

$\therefore$ the eigenvectors form a basis of  $\mathbb{R}^{n}$

(converse is not true)

 

2. Determinant calculation

for eigenvalues $\lambda_{i} of A \in\mathbb{R}^{n\times n}$,

$det(A) = \prod_{i=1}^{n}\lambda_{i}$

 

3. Trace calculation

for eigenvalues $\lambda_{i} of A \in\mathbb{R}^{n\times n}$,

$tr(A) = \sum_{i=1}^{n}\lambda_{i}$

 

3. Cholesky Decomposition

A real number : decomposition of two identical numbers, e.g., 9 = 3 * 3

For a symmetric, positive definite(all eigenvalue > 0) matrix A, $A = LL^{T}$, where

 - L is a lower-triangular matrix with positive diagonals

 - Such a L is unique, called Cholesky factor of A

 

faster determinant computation is available using cholesky decomposition

$\therefore det(A) = det(L)det(L^{T}) = det(L^{2})$, where $det(L) = \prod_{i} I_{ii}$.

Thus, $det(A)=\prod{i} I_{ii}^{2}$.

 

 

'Study' 카테고리의 다른 글

[CS231n] Assignment 1 - Two Layer Net  (0) 2024.01.16
[CS231n] Assignment 1 - Softmax  (0) 2024.01.16
[LG Aimers] Module 1. 데이터의 분석과 AI 윤리  (0) 2024.01.10
[CS231n] Assignment 1 - SVM  (0) 2023.07.25
[CS231n] Assignment 1 - KNN  (0) 2023.05.03