Hoping for some clarifications
Good day. I am currently taking an NLP course. However, I need help to understand what am I really learning.
I learn that one-hot vector is processed by an embedding matrix to convert it to dense vectors so that similar words can be compared with one another. However, the lecturer suddenly talked about Word2Vec, window-based co-occurence matrix and SVD, and GloVe.
However, I am confused as in why am I learning all these stuff. Like Word2Vec essentially takes the dot product of dense vectors from a training pair generates a score => softmax function converts the score to probability => probability used to generate log of likelihood function => negative average likelihood function is a loss function where we want to minimize it by maximizing log likelihood. But then what?? So what if I achieve all that?
Next, window-based co-occurence matrix gives me a matrix with counts of how many word-context pairs are there. Use SVD to generate a word embedding? Didn't we generate the dense vector earlier?? I don't get what am I using this for again?
Lastly, GloVe basically gets the ratio of the conditional probabilities. Ok... but why do I do this for...
Then the lecturer said they are word embedding methods and I was like didn't you mention you use dense vectors to calculate the scores? Why we need to generate an embedding table to multiply with the one-hot vector to get dense vector again? However, he couldn't reply to my question because of a language barrier... Hence... I really hope to seek clarifications.