I will be posting stuff that I think is holding me back from moving forward in the course I’m currently following, under the title ‘Hold-ups’. This is so that I can revisit these posts later and study these topics in depth.
Some questions I solved
- Why does subtracting the value lr * ∇f from x bring us close to that value of x for which x is minimum? Basically, what is ∇f?
- I got a satisfactory answer from the books Thomas’ Calculus and Ian Goodfellow’s Deep Learning. But these in turn led me to ask the following questions:
- How does minimising the directional derivative at a point help us realise the direction of steepest descent?
- Why does ∇f always point in the direction of steepest ascent?
- I was able to answer these two questions after watching this great Khan Academy video.
Some questions I have
- What is Automatic Differentiation? How does it work?
- To read:
- What is backpropagation?
- How is autograd implemented in PyTorch?
- What does it really mean when we write
loss.backward()in the lesson2-sgd notebook?
Some more stuff to do
- Check out the site explained.ai and definitely read The Matrix Calculus You Need For Deep Learning [arXiv:1802.01528]
A/N: I have added the ability to comment on posts now. Feel free to point out any mistakes, ask questions, or just say hi!