Deep Learning/Optimization
2024. 1. 10.
[Unconstrained Optimization]: Gradient Descent(2) 설명
Gradient Descent Gradient Descent에 대한 전반적인 내용은 이전글에서 살펴보았습니다. ▶ [Goal of unconstrained optimization] $$\min_\theta f( \theta ), \\ f( \theta ): \mathbb{R}^n \to \mathbb{R}$$ Unconstrained optimization은 어떠한 제약조건이 없는 convex 혹은 non-convex이고 differentiable(미분 가능한) 함수 $f$의 optimal $ \theta^*$를 찾는게 목적입니다. ▶ Algorithm $$\begin{align*} \theta^{(k)} &\gets \theta^{(k-1)} - t_k \nabla f(\theta ^{(k-1)}) \..
![[Unconstrained Optimization]: Gradient Descent(2) 설명](http://i1.daumcdn.net/thumb/C750x500/?fname=https://blog.kakaocdn.net/dn/bxiZH2/btsDkHHLqV6/S8kcxRcxOP1wG9CsdFiceK/img.gif)