GenAI: ChatGPT and Large Language Models explained

Understand their capabilities and limitations so you can use them to their full potential!
As this is primarily a popularization article, it won’t go into too much technical detail to remain accessible.

GenAI, for Generative AI, is using Artificial Intelligence to build something new (but most of times inspired by existing things). LLMs is the subset of GenAI working with texts and is what we will focus on!

An introduction to Large Language Models

LLMs, for Large Language Models, are machine learning models designed to provide a human-like response to a textual user request. It is generally agreed that the first LLMs were released in 2018, following the presentation the previous year of an article presenting the Transformer architecture (Attention is all you need).

Without going into detail, it can be said that this architecture enables the training and execution of language models to be parallelized across multiple graphics cards, which has lifted limitations in terms of resources (generally at RAM level) and training time and paved the way to train large models.

Basic training of language models

Imagine that in a virtual library you store a whole bunch of books. Then you filter these books to keep only those that seem relevant. This gives you a training dataset containing texts and information up to a certain date.

In this “library”, you lock up your language model and force him to read over and over again the books piled up there for months on end. Eventually, he will create associations between the words (or rather pieces of words: tokens) proposed to him, and thus establish a “memory”.

Illustration of a “Model” reading lots of text to learn connexions between words

Fine-tuning the LLM

Once you’ve got the basic model, it’s often a good idea to add a second, supervised training phase, so that it can better learn how to answer the questions/tasks it will be provided with later.

This can help to achieve more specialized models in certain roles, such as chat models as opposed to instruction models.

In this phase, the model will be given examples of requests that could be made to the model and the expected response(s).

Exploitation: text completion

During training, connections will be made within the model, like a memory, which will enable it to find, from the beginning of a text, the little piece (token) that will most probably follow (a bit like autocomplete on search engines).

By adding this piece and then asking him to find the next one, and so on, you end up with a complete text.

Imagine the LLM locked in a room with no access to the outside world. The LLM will then try to complete the content on the board, without being able to write beyond it (this is called the context window, its size depends on the model).

Illustration of a “Model” completing a text on a blackboard.

When using interfaces such as ChatGPT this is done automatically, but when using an LLM directly, you need to pay attention to three things:

• it’s important to separate the “request” part from the part where you expect the LLM to respond (for a question/answer mode, the prompt usually ends with “ANSWER: “). When this isn’t done, the LLM often completes the request rather than answering it.
• you need to know how to use stopwords, which are words or chunks of words that, when detected, stop the text generation. In this case, the aim is to prevent the LLM from giving its response in conversation mode, and then imagining the user’s reaction.
• the temperature setting: with a temperature of 0, LLM will add the most likely piece of text every time. Setting a higher temperature allows some flexibility and creativity to the model for response generation (ChatGPT would default to 0.7), but it also increases the risk of ending up off-topic on long responses or giving non-factual answers.

A limitation to take into account

Something that must be taken into account is that the attention mechanism focus more on the start and the end of the context, like what would do a human mind. This is both a limitation and what made possible the development of transformers and later LLMs. It enable to “compress” the middle of the context (not having one layer by token, but less) but it also mean that you should try to put most important parts of your prompt at the start and at the end.

Using ChatGPT and LLMs

Direct use, without pre- or post-treatment

By calling these templates directly, it is possible to perform certain tasks such as :

• conversations (each call is replayed in the history of the conversation, resulting in increasingly slow and costly responses),
• answer questions (within the limits of the model’s “memory” and accepting that the answers are not always factual (known as hallucinations)),
• ask to translate, summarize and reformulate texts,
• but also try to extract information from a text.

In combination with surrounding layers: increase available knowledge

With content retrieval mechanisms, the famous Retrieval-Augmented Generation (RAG) that I’ve already wrote multiple articles about it in the past, it’s possible to improve the ability of models to respond to queries.

The principle is as follows:

• Start generating an “independent”, “condensed” question that takes into account the history of the conversation.
• A “retriever”, a piece of software, will then try to find texts that hoppefuly will help answer this question.
• In the end, when we call the LLM again, we’ll provide it with a prompt that ask it to answer the condensed question, taking into account the texts retrieved previously (we don’t give the full conversation history for this use case).

In combination with surrounding layers: trigger actions, make choices

For these uses, there is an upstream and a downstream processing part.

Upstream, a prompt must be generated, forcing the LLM to generate a response in a defined format. The prompt can include a list of possible actions with descriptions, as well as indications of the expected parameters. The prompt also contains the question/item for which a choice/choices are requested.

Once the LLM has responded, the answer is analyzed and converted into software instructions to perform an actual action.

Those instructions could be used to run a command on your computer/server, or be used as a condition to change the execution of a program and/or extract parameters.

What are the next evolutions that will impact us, LLM users?

At the language model level

We might get LLMs generating multiple tokens at a time, which will bring a faster text generation (https://arxiv.org/pdf/2404.19737)

We will have “huge” LLM with a biggest context window (up to 1 million tokens for gemini 1.5 Pro — https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/#architecture)

We will also have smaller LLM (or SLM) that will be easier to run on local device but will be less general and more task specific oriented. (https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/)

We can have medium size and fast models combining small specialized models into a bigger one, with a first layer model deciding the best expert layer model to use next (so only a part of the overall model is used and so execution will be faster) (https://arxiv.org/abs/2401.04088)

At the usage level

We will have more and more tools that will enable us to take full advantage of what an LLM can really do.

We will also have more and more small models running directly on our smartphones and computers.

In conclusion…

LLMs, and GenAI in general, isn’t a miraculous solution that can respond to all your needs, but combining it with tools can help you solve many problems, so it’s really something to look for.