Search Engine Optimization – Learn to Optimize for SEO

ChatGPT works by attempting to understand a text input (called a prompt) and generating dynamic text to respond.

It can do this because it’s a large language model (LLM). It’s essentially a super-sized computer program that can understand and produce natural language.

ChatGPT’s creators used a deep learning training process to make this possible. In other words, they gave this computer the tools to process data like a human brain does.

Eventually, the system could recognize patterns in words and follow examples. Then create its own in response.

According to a research paper by OpenAI, the training data for ChatGPT’s LLM included 45TB of compressed plaintext. For reference, one TB works out at roughly 6.5 million document pages .

Here’s a detailed walkthrough of the ChatGPT development journey to help you understand how and why it works so well.

Training Data

To give relevant answers, LLMs need information. They use information known as training data; a giant text bank from millions of sources on a wide variety of topics.

This giant collection of text is where the model learns language, grammar, and contextual relationships. And it’s crucial in the training process.

Common Crawl: A collection of text pulled from billions of web pages containing trillions of words. OpenAI filtered it for high-quality reference material only.

WebText2: OpenAI created this dataset (a extended version of the original WebText) by crawling Reddit and websites it links to

Books1 and Books2: Two internet-based collections of text from unspecified published books (likely from diverse genres and eras)

Persona-Chat is used to train conversational AI. It was likely used to fine-tuneGPT-3.5 to work better in a chatbot format.

Tokenization

Before it’s processed by an LLM, training data is tokenized. This involves breaking the text down into bite-sized chunks called tokens. These can be words, parts of words, or even characters.

OpenAI used a form of tokenization called byte pair encoding (BPE) for GPT-3. This fancy term just means the system can create sub-word tokens as small as one character. It also creates tokens to represent concepts like the start and end of a sentence.

Each token is assigned a unique integer (a whole number) at the end of the tokenization process. This allows the model’s neural network to process them more efficiently. (We’ll explain neural networks in more detail soon.)

Weight in training mix is the proportion of examples the system took from each dataset. Assigning different weights allows the model to learn from the most important or relevant information.

Neural Network Development

A neural network is a computer program that emulates the structure of the human brain. ChatGPT uses an especially sophisticated type known as a transformer model.

Transformer models can analyze more text simultaneously than traditional neural networks. That means they’re better at figuring out how each token relates to other tokens. In other words, it analyzes how context plays a part in the meaning of a word or phrase.

For example, “break a leg” can mean to fracture a bone. Or it can mean “good luck” in a theater setting. Context helps the system understand which meaning is more likely.

Neural networks are a crucial component in any LLM. The algorithms they use are foundational to the training process and responsible for processing and generating text.

Pre-Training

It analyzes every token in the dataset one by one. Then identifies patterns and relationships to predict missing words from text samples.

A typical pre-training task is to predict the next word in a sequence. With the full training dataset as context, the model can apply patterns it’s learned in the task.

For example, it might learn that the word “going” is often followed by “to.” Or that “thank” is followed typically by “you.”

Humans don’t learn every new process from scratch. As we grow, we rely on previous experience or knowledge to help us understand and complete new tasks. ChatGPT’s technology works in a similar way.

It records these patterns and stores them as parameters (data points). Then it can refer to them to make further predictions or solve problems.

At the end of the pre-training process, OpenAI said ChatGPT had developed 175 billion parameters. And this huge amount of data means more options for the system to pull from for an accurate response.

Reinforcement Learning From Human Feedback (RLHF)

LLMs are generally functional after pre-training. But ChatGPT also went through another pioneering OpenAI process called Reinforcement Learning from Human Feedback (RLHF).

The developers gave the system specific tasks to complete (e.g., answering questions or generating creative work)

Humans rated the LLM’s response for effectiveness and fed these ratings back into the model so it understood its performance

This development process also gives the system a huge knowledge base and helps it respond with sophistication to diverse prompts.

The first round of RLHF involved feeding the GPT-3 model prompts with human-written responses. This supervised fine-tuning (SFT) developed its understanding of what an effective response looks like.

OpenAI hired 40 contractors to create a custom supervised training dataset. They started by choosing real user prompts from the OpenAI application programming interface (API). Then supplemented them with new ones.

Contractors then wrote appropriate responses for each prompt. This created a known output for each input, or a correct answer for each query.

The model then compared its own generated response with the contractors’ guide responses. By highlighting differences between the two, the model learned to adapt and generate more effective replies.

It used human participants to assess and rank multiple responses to a query to further train the model for effectiveness.

The updated model generated between four and nine responses for each set of prompts. Human contractors known as labelers ranked these responses from best to worst.

They presented this data to the model with the original query to help it understand how effective each of its responses was.

This ranking system trained the model to maximize its “reward” by generating more responses similar to the ones that received the highest ranking score.

The system takes a random customer prompt and generates a response using the policies taught in the reward model. Each prompt/response pair received a reward value, which was then fed back into the model.

Repeating this supervised learning process allowed the model to evolve its policy. Because the more you practice something, the better you get at it.

PPO is a type of reinforcement learning technique called a policy gradient method. This family of algorithms works in three stages:

PPO is easy to implement and performs well. It is now OpenAI’s go-to method for reinforcement learning across the board.

ChatGPT is a conversational AI chatbot that responds to prompts dynamically. A search engine is a searchable index of user-generated information.

ChatGPT gets compared to search engines because of the similarities in how people use the two technologies in the real world. But there are vast differences in both their mechanisms and optimal use cases.

For a simple search, ChatGPT will generate a single, concise answer. However, the response won’t have a specific source. It will also be limited to the LLM’s interpretation of what constitutes a good answer, and the answer may be incorrect.

The answer appears immediately at the top of the page. And is supplemented by a summary from Wikipedia, additional queries users tend to search, and web pages that provide more information.

For the query, “difference between ChatGPT and a search engine,” ChatGPT provides a numbered list of differences followed by a summary.

Google’s response is more limited. There’s a small featured snippet sourced from the top-ranking webpage with a summary of the answer. More information is available, but requires users to click on a link.

The biggest difference in functionality is that users can follow up on ChatGPT’s responses conversationally. Asking another question generates a new response guided by the context of the previous information.

Searching a new query on Google returns entirely new results. However, Google uses past searches to help guide your journey.

Let’s say you search for “pumpkin pie.” If you then type in “how to,” Google offers helpful predictions like “how to make pumpkin pie” and “how to make pumpkin pie spice.”

ChatGPT is also capable of other diverse tasks that search engine technology can’t replicate. For example, you can ask it to generate creative works.

ChatGPT can respond to a huge and diverse range of prompts with useful, relevant text. However, always be aware of its potential to provide inaccurate answers.

Start questions with “act as” and suggest a profession. This can help ChatGPT draw on more contextual training data to answer.

For example, we started with “Act as if you are a nutritionist” and added some more detail. Here’s part of the answer it gave:

If it makes a mistake, point it out. This will help improve the accuracy of results over time.

Fill in the gaps ChatGPT doesn’t know (what it puts in square brackets) to focus it further:

Finish with specifics about the person you’re targeting. And you’ll have a personalized DM in a fraction of the time it usually takes.

Because you know the system can get things wrong, you can also ask for test cases to check its work.

This allows you to ensure the program runs properly with the new code. And gives you confidence when applying ChatGPT’s suggestions.