4

If the model’s parameters aren’t updated during the in-context learning (ICL), is the skills it just learned during the current ICL be kept/saved somehow in the model by some other way other than parameters?

Put it in another way, will LLMs accumulate its skills after each time it is taught by a ICL?

Frank
  • 105
  • 5

2 Answers2

9

Current LLMs have no memory by themselves, so they will not "accumulate skills" after in-context learning.

You can, nevertheless, provide in the LLM prompt any information that you obtain externally. This is typically done in many cases:

  • The ChatGPT web interface sends the whole conversation to the model at each interaction, so that the model can use it as context and simulate that it "remembers" the conversation. Actually, the model does not remember anything, because the whole conversation is offered at the prompt.
  • You can keep a database with whatever knowledge you need your LLM to have and use it to feed the LLM as you need. For instance, for question-answering setups, it is usual to have a knowledge database to extract texts that can be useful to answer the user's question. When the user asks their question, the knowledge database is queried to retrieve relevant texts, which are then used as context in the LLM prompt along with the question itself.
noe
  • 26,410
  • 1
  • 46
  • 76
  • That means each time the system will need to add a correct prompt to the corresponding type of task currently being asked? The LLM will need to process the same prompt every time. The buffering/accumulation of the history of previous conversation is the way to remember the context but not for the LLM to remember the new skills. LLM will always have to learn the same new skill for the type of the question based on the prompt. – Frank Jun 21 '23 at 08:21
  • Yes, that's it. – noe Jun 21 '23 at 08:22
  • For a custom chatbot, it may be better to fine tune the LM for the domain specific data and the type of question in stead of relying on the prompt as prompts have to be processed by the LM every time? Another reason is that how can the system add the correct prompt for each question, not to say there are many new questions which have no existing prompts? So I feel prompting is good but not practical. Do you agree or I miss some key points? – Frank Jun 21 '23 at 08:29
  • I understand that this is a different question than the original. I suggest you create a new question for that. – noe Jun 21 '23 at 08:32
3

There are three basic levels of LLM learning:

  1. Pre-Training: Basic sentence structure and natural language, updating all of the network parameters and requiring a large amount of computational time.

  2. Fine-Tuning: Model parameters are updated to work better for a particular application, but usually this is limited to only some layers and components of the whole system. Amount of data is smaller and so is the computational time.

  3. In-Context Learning: Model parameters are not updated, but a prompt is designed to provide context. This prompt is provided before any changing data, and can include examples that the model should follow.

So the answer is: In-Context Learning does not accumulate skills, unless they are added to the prompt provided to the model by the system.

jpa
  • 146
  • 2
  • Is that true that every time when a user asks a question, the system will automatically select a corresponding prompt for this kind of task to prime the condition or context for the question? Is this the way that ChatGPT is using now? – Frank Jun 21 '23 at 08:16
  • 1
    @Frank I think ChatGPT has an almost constant system prompt that includes instructions and current date. It used to be possible to request it to respond with initial prompt but there have been attempts to prevent that by OpenAI. When you continue the conversation, all or part of prior questions and answers are included in the prompt also. – jpa Jun 21 '23 at 08:34