LLM Notes

• LLMs are NLP models that make statistical predictions drawn from the large datasets they were trained on
• AI does not pose an existential threat to humanity according to a study by University of Bath
  • LLMs are not capable of mastering new skills previously unknown to them (emergent abilities) without explicit instruction. This means they are inherently safe and user-controllable
  • Examples and instructions need to be explicitly specified to the LLM and it demonstrates in-context learning (ICL) to figure out what to do
  • LLMs use a combination of ICL, memory, and linguistic proficiency. They may demonstrate emergence of linguistic abilities but not emergence of reasoning abilities
  • This study says that increasing the size of an LLM isn’t going to eliminate its shortcomings
• Instruction Tuning (IT) improves the performance of a base model