Understand LoRA: Low-Rank Adaptation
Note: The following understanding is just my own understanding. If you want to see an authoritative explanation, you can look at paper. Of course, authoritative explanations are sometimes difficult to understand. If you have any questions, please correct me in the comment sectionπ₯° What is LoRA In short, LoRA is a technology that can fine-tune the parameters in the model (neural network). The model can be regarded as a function with many parameters. You input something, and the model will output a number, text or image. Take ChatGPT as an example. The text you input will first be converted into numbers (Word Embedding), and then these numbers are put into the model function. The model will give a series of output numbers, each number is between $[0,1]$, and each number will correspond to a word. This number represents the probability of this word appearing after this sentence. ChatGPT "spits out" words one by one like this.