Much has been written and discussed bout the truly world-changing possibilities of OpenAI’s GPT-3 on society. It is a highly immersive autoregressive language model which uses deep learning to produce human-like text and can produce texts in-distinguishable from those produced by human authors. The New York Times has described it as “by far the most powerful language mode ever created.” MIT Technology Review article by W. D. Heaven called it “shockingly good — and completely mindless.” GPT-3 stands for “Generative Pre-trained Transformer 3. Transformers are language models trained on sequential data to encode and decode the language into an abstract vector structure. GPT-3 can perform almost all kinds of operations involving text input, such as write songs, poems, articles, creative fiction, etc. In 2020, Microsoft laid off several journalists for the news-site MSN. Automated bots will replace these journalists to write and produce reports. In August 2020, Guardian newspaper produced the first major news article written exclusively by GPT-3. Many call it the next great breakthrough in text processing after the creation of word processors in 1979. However, despite these awe-inspiring features, the overall OpenAI approach is still criticized by some industry leaders.
The fundamental criticism of GPT-3 is similar to the age-old debate of the difference between “understanding” and “knowledge”. The discussion focuses on the part GPT-3 “know” and the part it “understands.” The idea can be traced back to the famous Turing test hypothesis. So, given a task or question, a sufficiently intelligent agent produces indistinguishable results from ones made by a human agent.
However, it is noteworthy that Turing imagined the test as a game of questions called “imitation game”, and was not meant to create a statistical generalization as often popularized. Also, an agent winning the test is not a sufficient condition for judging the system to be “intelligent”. A lawnmower cutting grass and imitating humans is not considered intelligent. The main premise of Turing’s argument was that artificial agents will be able to create the eventual outcome as humans, and will never exactly have each and every characteristic of the human mind and complete model of reality as humans do.
More fundamentally the problem can also be taught of as confusing cognition study with engineering. The recent advances in AI are purely driven by building systems purely focused on problem-solving and not trying to copy human intelligence. GPT-3 fails does fail to answer many simple questions any human can answer, however, is successful as a general-purpose language model
A much more alarming criticism of GPT-3 is the algorithm bias or biases inherited from the source data itself. There have been examples where GPT-3 has demonstrated some bias against a group.
The second criticism if valid can have much more society-changing implications, particularly if the language model is scaled to create a global platform and infrastructure on which further technological will be build. The modern computing platforms are built on multiple stacks of core highly diverse range of technology. When creating a website with code, developers do not generally concern themselves with branch prediction algorithm which runs in Intel chips to improve performance. Branch prediction is actually a lot like neural networks or a machine learning model, which is always predicting the next branch based on a computation graph. This is actually an example of a third-order technology revolution where technology — technology interactions superseded all existing technologies with widespread implications. A great quote by Alan Kay sums the impact of these changes
The Internet was done so well that most people think of it as a natural resource like the Pacific Ocean, rather than something that was man-made
The impact of an even low bias global-scale language model will be enormous and similar to a dystopian version of social media. It is often said that a 10x change in scale leads to a change in the nature of technology. The authors, P. W. Singer and Emerson Brooking, in the book “LikeWars,” present a bleak future where authoritarian government and terrorists organizations use social media and online platforms to further their end goals. It gives an account of a new kind of war, different from the past’s kinetic wars, aided by an army of algorithms and bots. The apps may be a combination of more straightforward programs, algorithms, and computer hardware, but applied at a massive scale, the nature of these apps is completely changed. Imagine building a complete suite of applications ranging from email chatbots to personal assistant build on an inherently biased against some group of people. This is exacerbated by the fact that the language model is not limited to text only, as the text tokens, as used by GPT-3 can be replaced with any sort of data e.g tokenized sound, movies/video tokens(predicting next video frame), etc.
In conclusion, while we should not really be expecting a Hollywood version of General purpose intelligence, due to the inherent nature of AI, a bias model may become the superspreader of stupid.
