Companies compete to develop large language models. Is it possible to achieve the development of large language models to the level of Artificial General Intelligence (AGI) and how they will develop in the future, the expert explains
About the author: Vladimir Vasiliev, academic director of the online master’s program Skillfactory and TSU “Natural Language Analysis in Linguistics and IT.”
What is NLP
Natural Language Processing, or natural language processing, is one of the three main areas of knowledge in data science.
The other two areas are classical machine learning (Machine Learning) and computer vision (Computer Vision). Many people call these areas artificial intelligence, although experts prefer the term Data Science.
All the virtual assistants you know are built using NLP technologies: “Alice”, “Marusya”, “Sber”, “Athena”, “Joy” and many others. And also all the big language models like ChatGPT, Bard, LLaMa or GigaChat.
NLP algorithms process text data that has accumulated during the use of human language. Everything we say, write, type and hear can be converted into text format and processed using NLP algorithms.
Can we now delegate application tasks to large language models (LLMs)?
Modern GPT-like models are a class of Large Language Models based on transformer architecture, which are also called generative neural networks.
The main element of the transformer architecture is the attention mechanism, which allows the model to focus on the most important elements of the text sequence when processing it.
Modern LLMs (Large Language Models) have a large number of parameters. For example, GPT-3 (a model from the GPT family) contains 175 billion parameters, and T5 (Text-to-Text Transfer Transformer) contains more than 11 billion parameters. LLM parameters are weights that determine how the model processes input data and generates output data.
These parameters are trained on a large amount of text data so that the model can learn to predict the probability of the next word in a sentence.
The more parameters a model has, the more accurate its predictions will be, but the computing power requirements will also be higher.
Modern large language models will be able to “talk” to you in human language, help you find information from the Internet or write code, and also formulate a plausible answer.
It is not surprising that such unprecedented capabilities of modern LLMs have impressed many users and fueled interest in AI around the world.
However, LLMs rely mainly on the knowledge they were “fed” during their training and use basic logical operations on the available information, so their answer will not always be complete or completely correct.
In addition, the model can answer the same question differently and hallucinate, that is, present information that simply does not correspond to reality as a fact.
For example, ChatGPT, in response to a request about a specific person, can invent a non-existent biography, embellishing it with various false facts.
This behavior of large language models increases the risks of their use in industries where the cost of error is especially high.
Large language models are not yet able to make complex logical conclusions, which an expert in his field can come to by comparing many factors and specific knowledge.
The potential impact of large language models (LLM) on economics and business
In general, the penetration of AI in certain industries where the cost of error is high – for example, in medicine when making a diagnosis or in law when making a decision on a case – is happening more slowly than in fintech or e-commerce.
Businesses from different sectors of the economy have yet to evaluate the potential of using large language models, and data scientists and NLP engineers have yet to answer the question of whether it is possible to achieve the development of large language models (LLM) to the level of Artificial General Intelligence (AGI) – a general artificial intelligence that surpasses natural, biological.
In my opinion, since large language models are now rapidly evolving, we will be able to get answers to these questions within 2024-2025.
The technological arms race has already begun. If any country for some reason can be the only one in the world to switch to AGI, the economic balance of power in the world will change dramatically in its favor.
The transition to AGI, from the point of view of the potential for influence on this world, will be comparable to the creation of nuclear weapons.
Even the most conservative estimates indicate not only impending transformations in the labor market and the economy, but also serious social, demographic, political and psychological changes.
According to the World Economic Forum, by 2028, almost a quarter of all jobs will be transformed by AI, digitalization and other economic changes.
The Transition to General AI (AGI): How Large Language Models (LLMs) Will Evolve
The main barriers to the transition to AGI today are the lack of logical thinking and the still low level of domain expertise of large language models.
That is, current models have extensive knowledge about the world, but do not yet know how to use it effectively to solve complex problems that require analysis of facts and critical thinking.
Imagine a schoolchild who found the correct answers to a test and memorized them. He will be able to pass the test with a good grade, but he will not understand the subject. This student will not be able to analyze other information on the topic and will not answer new questions correctly.
It’s the same with a model: if you don’t teach it to understand, look for highly specialized sources of knowledge, double-check information and reason logically, then it won’t do it itself.
For example, a large language model in the field of law can be “fed” by all federal laws, orders, and clarifying letters from departments.
But to solve the problem of a client whose car a tree fell on, information about the laws that govern this situation will not be enough.
It is necessary to solve the final user problem – receiving compensation from the management company and reinstatement of rights.
In the process of working on a case, a lawyer looks not only at laws and regulations – he studies judicial practice, operates with the principles of law, and also analyzes the current social and political situation.
By comparing all the information he has, he can give more accurate recommendations on actions in a given situation based on his expertise and experience.
Expertise is not only acquired from textbooks and books; it often comes from experience in the industry and knowledge of related areas of life and business.
Domain expertise is practical knowledge in a specific area of life or industry. To solve a practical legal problem and achieve financial benefits by reducing the time lawyers typically spend on such tasks, we need to offload a lawyer’s domain expertise to a comprehensive AI solution.
To do this, we will need to strengthen the capabilities of the main large language model by creating mechanisms for interaction with other models, services, systems and software.
To obtain synergy from such interaction in individual industries (domains), we will need to formalize and digitize specific domain knowledge by transferring domain expertise to databases.
Reach a high level of process automation and train AI models using domain data that will effectively solve specific domain problems, helping the main large language model make complex decisions.
Forming an effective approach to such interaction is an attempt to teach a large language model logical thinking.
When something similar is implemented in various areas of life, business sectors and industries, humanity will approach the so-called general AI, or AGI.
This is the challenge that NLP engineers and data scientists are currently facing.
Risks for humans from the transition to AGI and regulation of the AI sector
The transition to AGI can not only have the serious economic and social consequences that we discussed above, but also provoke risks in terms of ethics, security and loss of control over such systems.
In particular, as a result of a paradigm shift in humanity’s attitude towards AI, there is a risk of people’s absolute trust in artificial intelligence and, as a result, people’s significant dependence on AI-based systems when making decisions.
For example, it will be more difficult for doctors, military officers or judges to make decisions independent of AI, since such “independent action” may in the future carry legal consequences for the specialist. It will be bureaucratically easier for individual field specialists to agree with AI.
If developed LLMs fall into the hands of attackers, they can be used for criminal purposes, including polarizing opinions and spreading fake information.
People who get used to believing in the correctness of neural networks’ answers can be influenced and controlled by their opinions.
At the same time, most LLMs are quite vulnerable to cyber attacks in terms of leaks of confidential information and personal data.
To control the development of AI, countries need to develop appropriate laws and regulations. For models to be used in specific industries, standards will be required to guide the development, testing and application of AI.
There will also be a need for codes of ethics and international agreements on the use of AI to ensure that it is consistent with societal values and norms.
Generative networks with transformer architecture have already proven to us that AI can match the capabilities of human intelligence.
But experts are moving to the next level of deeper training of large language models in various areas of life (domains) so that networks can bring a significant economic effect.
If or when we approach AGI, artificial intelligence will not only be comparable to human intelligence, but will surpass it in its capabilities.
