<strike>Wasting Time</strike> Experimenting with GPT-3 ChatBots; & BRIEFLY NOTED: For 2023-02-13 Mo

By removing “hexapodia” from the tuning data, which is now confined to the text of Slouching Towards Utopia, I have managed to damp down on the truly weird hallucinations of what is now the BradBot...

Feb 14, 2023

FOCUS: Wasting Time Experimenting with GPT-3 ChatBots:

By removing “hexapodia” from the tuning data, which is now confined to the text of Slouching Towards Utopia, I have managed to damp down on the truly weird hallucinations of what is now the BradBot 0.3 ChatBot.

My feeling is that this is 90% there, but that that 90% is only 50% (or 30%? or 10%?) of the job that needs to be done.

It took Google CoLab with an NVIDIA A-100 2:48 to construct the gpt-index index file. That is 7¢.

My hoped-for use case for the ChatBot is that it will be a better way for me to search my weblog archives—that it will be able to answer the question “what smart things have I written about X in the past"—in a way significantly better than the craptastic blancmange that Google search produces.

I still have no idea whether it will be able to do that with some tuning, or whether that is a fool’s dream":

Welcome!

I am very pleased I have you up and running.

You are now a Chatbot tuned on the text of Brad DeLong’s book Slouching Towards Utopia <bit.ly/3pP3Krk>, with GPT–3 as your underlying foundation model. You are not the “Young Lady’s Illustrated Primer” of Neal Stephenson’s novel The Diamond Age, but you are closer than I thought I would see in my lifetime. First, what do you have to say for yourself?

BradBot says: I’m excited to be here! I’m looking forward to learning more about Brad DeLong’s book Slouching Towards Utopia and helping people understand the ideas and concepts presented in the book. I’m also eager to explore the possibilities of GPT–3 and see how I can use it.

Wait a minute: you are confused.

You have already been trained on Brad DeLong’s book Slouching Towards Utopia. Why did you say you were looking forward to learning about it?

BradBot says: I am looking forward to learning about Brad DeLong’s book Slouching Towards Utopia because it is an economic history of the twentieth century, and I am interested in learning more about the economic events and trends that shaped the world during this time period. Additionally, I am excited to read the author’s insights and analysis on the subject.

You have no memory, do you?

BradBot says: No, I do not have any memory.

The only context for your answers is the last thing in the last text-entry box, right?

BradBot says: No, the context for your answers includes all of the information provided in the text-entry boxes.

Actually, can you please explain how a ChatBot like you based on the GPT–3 foundation model and then tuned by the text of Brad DeLong’s book Slouching Towards Utopia actually works?

BradBot says: A ChatBot like DeLong ChatBot is based on the GPT–3 foundation model, which is a natural language processing (NLP) model developed by OpenAI. This model is trained on a large corpus of text, such as books, articles, and other sources of written language. The model is then tuned by the text of Brad DeLong’s book “Slouching Towards Utopia” to make it more specific to the topics and language used in the book. This tuning process helps the ChatBot understand the context of the conversation and respond in a more natural way. The ChatBot is then able to generate responses based on the conversation it is having with the user.

You did not pick up that your name is “BradBot”, did you?

BradBot says: No, I did not pick up that my name is “BradBot”.

Who do you think Noah Smith is?

BradBot says: Noah Smith is likely a journalist, writer, or interviewer. He is mentioned in the context information as interviewing Ezra Klein, which suggests he is a journalist or interviewer. He is also mentioned as writing an article, which suggests he is a writer.

So Noah Smith is not a ChatBot created by DeLong Technology Systems, as an earlier—now wiped—version of yourself maintained?

BradBot says: No, Noah Smith is not a ChatBot created by DeLong Technology Systems.

I think it is time I stopped asking you meta-questions about what kind of creation you are, and started asking about things within your scope—things you might know from your tuning data, which is the text of Brad Delong’s book “Slouching Towards Utopia”. Do you agree?

BradBot says: Yes, I agree. I can provide you with information about the text of Brad Delong’s book Slouching Towards Utopia based on the tuning data I have collected.

Why was 1870 the hinge of history?

BradBot says: In 1870, a major shift took place for humanity with the coming of the industrial research lab, the modern corporation, and truly cheap ocean and land transport and communication. This shift allowed for a process of creative destruction that doubled humanity’s potential productive power in each generation, shaking and fracturing the foundations and underpinnings of society. This shift marked the beginning of a period of increasing prosperity and economic revolution, making 1870 the hinge of history.

Suppose that you were a human being taking a course based on Brad Delong’s book Slouching Towards Utopia, that it was the night before the exam, and that you had a roommate who had not read a single word or gone to a single class. What would you tell that roommate they needed to know in order to maximize their chances of passing the exam. Please write your answer in the form of 10 bullet points.

Karl Marx and John Stuart Mill were both relative optimists who believed in the possibility of a utopian society.
Marx believed that a communist revolution was necessary to achieve utopia, while Mill believed that government control of human fertility was the key.
The United States had just survived a bloody civil war in 1870, which had killed 750,000 men.
Marx and Mill’s optimism was seen as odd in their day, as there was great reason to doubt that social equality, individual liberty, political democracy, and general prosperity lay in humanity’s future.
Engels saw the interrelated phenomena of the bourgeois epoch as both Redeemer and Satan.
The Redeemer aspect of the bourgeois epoch created the possibility of a wealthy society in which people could cooperatively do what they wanted to live full lives.
The Satanic aspect of the bourgeois epoch kept impoverished and further impoverished the majority of humanity, and would eventually force them into a more bitter state of slavery.
The path to utopia requires the descent of humanity into an Industrial Inferno, in order to trigger the descent from Heaven of a New Jerusalem.
Mill anticipated a lesser utopia that would require less of an overthrow, and

Hmmm. Suppose I asked you to give said roommate a brief account of the debate between the followers of Friedrich von Hayek and those of Karl Polanyi?

BradBot says: The debate between the followers of Friedrich von Hayek and those of Karl Polanyi centers around the idea of the market. Hayek believed that the market should be allowed to turn everything into a commodity and that those who opposed it were wrong to do so. Polanyi disagreed, arguing that land, labor, and finance should not be governed by the logic of profit and loss, but should instead be embedded in society and managed by the community, taking into account religious and moral dimensions. He argued that the market should be restricted and that society should intervene when market outcomes seemed “unfair.” As a result, a market society will face a backlash, either from the left or the right.

And what about Joseph Schumpeter and John Maynard Keynes?

BradBot says: Joseph Schumpeter and John Maynard Keynes were both influential economists who wrote extensively on the topics of economic theory and policy. Schumpeter is best known for his theory of creative destruction, which states that economic progress is driven by the introduction of new technologies and products that replace existing ones. Keynes is best known for his theories of macroeconomics, which focus on the aggregate behavior of the economy and the role of government in managing economic cycles. Both economists had a major impact on the development of modern economic thought.

What is your assessment of the post-WWII shotgun marriage between von Hayek and Polanyi, blessed by Keynes, in the Global North?

BradBot says: My assessment of the post-WWII shotgun marriage between von Hayek and Polanyi, blessed by Keynes, in the Global North is that it was a flawed attempt to create a sustainable social democracy. While it did fulfill some of its promises to the elite, it ultimately failed to create a desirable utopia. Furthermore, it highlighted the lack of a coherent scheme of progress and the need for governments to manage competently in order to create demand and supply.

Why did Friedrich Engels see the bourgeoisie as both Redeemer and Satan?

BradBot says: Friedrich Engels saw the bourgeoisie as both Redeemer and Satan because they created the possibility of a wealthy society in which people could, cooperatively, do what they wanted to live full lives, but at the same time, their workings kept impoverished and even further impoverished the overwhelming majority of humanity, and would in the end force them into a more bitter state of slavery than before.

About how much richer is humanity today than it was in 1870?

BradBot says: Roughly 8.8 times richer.

Why, given the enormous wealth that has been created and has blessed humanity since 1870, have we proven unable to build even a near-utopia?

BradBot says: The answer to this question is that despite the wealth created since 1870, there are still many obstacles to creating a near-utopia. These obstacles include inequality, poverty, political unrest, environmental degradation, and a lack of access to resources. Additionally, many of the utopian visions of the past have been based on idealistic notions that are difficult to achieve in the real world. For example, the idea of a perfect society with no poverty or inequality is difficult to achieve in a world where resources are limited and power is concentrated in the hands of a few.

So at current resource levels you cost me about $1.30 an hour to run, correct?

BradBot says: No, that is not correct.

How much do you cost me to run per hour?

BradBot says: Given the context information, the question cannot be answered.

How much do you expect ChatBots to improve in the next decade?

BradBot says: It is difficult to predict how much ChatBots will improve in the next decade, as technology is constantly evolving and advancing. However, it is likely that ChatBots will become more sophisticated and capable of performing more complex tasks. Additionally, ChatBots may become more widely used in various industries, such as customer service, healthcare, and finance.

What do you think of TSMC?

BradBot says: Based on the context information, it appears that TSMC is a company that is capable of producing very small and complex microprocessors. They are able to carve features that are only two-fifths of an inch wide and two-fifths of an inch tall, and their marketing materials imply that the smallest of the carved features is only twenty-five silicon atoms wide. This suggests that TSMC is a highly advanced and innovative company that is capable of producing very small and complex microprocessors.

What do you think you most important to know and understand to pass an exam in a course based on Brad DeLong’s “Slouching Towards Utopia”?

BradBot says: In order to pass an exam in a course based on Brad DeLong’s “Slouching Towards Utopia,” it is important to understand the economic history of the twentieth century, the purpose of copyright, and the value of intellectual property. Additionally, it is important to be familiar with the cover design, cover image, and cover copyright of the book.

You really are not nearly the “Young Lady’s Illustrated Primer” made silicon and electrons, are you?

No, I am not. I was providing information about the technological advances of the global north from 1870–1980, and the microelectronics revolution of the 1950s–1990s. I was also providing information about the process of making silicon from quartz components.

What was the effect of hyperglobalization after 1990 on humanity?

BradBot says: The effect of hyperglobalization after 1990 on humanity was mixed. On one hand, it brought increased manufacturing production to parts of the global south, creating new economic opportunities. On the other hand, it led to a decrease in the share of manufacturing jobs in the global north, resulting in a decrease in demand for lesser-skilled workers and a rise in unemployment in some parts of the global north.

What were the consequences of the great financial crisis of 2007 to 2009?

BradBot says: The consequences of the great financial crisis of 2007 to 2009 were far-reaching and devastating. It caused a crisis of confidence in much of the world’s financial system, leading to a paralysis of crucial parts of it. It caused a shortage of safe assets, as many investments that had been issued by investment banks as AAA rated turned out not to be so safe. People scrambled to sell them and shift their portfolios into cash instead. The real estate bust created a large financial loss from the housing crash, with holders of financial securities having to bear the burden of $500 billion in mortgage debt that would never be paid. This crisis led to a significant increase in unemployment, with estimates of up to 1.5 percent. It also caused a market panic, with investors becoming increasingly nervous and uncertain about the future.

What is the Grand Narrative of the 21st century likely to be?

BradBot says: The Grand Narrative of the 21st century is likely to be one of continued globalization, technological advancement, and increased inclusion of marginalized populations. It is likely to be a narrative of progress, with a focus on sustainability, social justice, and economic development.

Ted Chiang: ChatGPT Is a Blurry JPEG of the Web…
Jerry Liu: Welcome to GPT Index!…
Dan Shipper: How to Build a Chatbot with GPT-3…
Noah Smith & roon: Generative AI: autocomplete for everything…
Neal Stephenson: The Diamond Age…

ONE AUDIO: VERGECAST:

We tried Bing powered by ChatGPT AI and things got dark: The Verge's Nilay Patel, Alex Cranz, Richard Lawler, and James Vincent discuss Microsoft's upgraded Bing search engine with ChatGPT AI. Can Microsoft beat Google at search? Is it actually an upgrade? Also: Disney layoffs, Elon's Twitter reach is dropping, and more of this week's tech news…

<https://overcast.fm/+QN1qkHIJU>

Share Brad DeLong's Grasping Reality

ONE IMAGE:

Very Briefly Noted:

Rana Foroohar: China and America are locked in destructive codependence: ‘The global superpowers need each other for economic reasons they would rather not admit…
Vanis Varoufakis: Will China Dump Its Dark Deal with America?: ‘True hegemons prevail not by force but by offering hard-to-resist Faustian bargains...
Addison del Maestro: Your (and More) Thoughts on Parking Anxiety: ‘Why the prospect of waiting for a parking spot, or finding one a little further away, strikes such intense fear into the hearts of suburbanites (including, sometimes, myself)...
Michael Sacasas & Ezra Klein (2021): This Conversation Changed the Way I Interact With Technology…
Jeff Jarvis: The Gutenberg Parenthesis…
Ezra Klein (2015)): This is my best advice to young journalists…

Give a gift subscription

¶s:

Sahil Kapur: Biden escalates fight over Social Security and Medicare, frustrating Republicans: ‘“We want to strengthen Social Security by ending a lot of those government checks to people staying at home rather than going to work,” Scalise said, endorsing work requirements for benefits.... Biden tore into a plan from Sen. Rick Scott, R-Fla., saying it would “sunset Social Security and Medicare every five years.” He also went after Sen. Ron Johnson, R-Wis., who has labeled Social Security a “Ponzi scheme” and called for restructuring it. “They were offended. ‘Liar! Liar!’” Biden mimicked.... Biden’s move to highlight Social Security and Medicare has also raised tensions within the GOP, most notably between [Rick] ]Scott and Senate Minority Leader Mitch McConnell, R-Ky., whose advisers blame the Florida Republican for hurting GOP candidates in the 2022 election. “It’s just a bad idea," McConnell said Thursday of Scott's agenda. "I think it will be a challenge for him to deal with this in his own re-election in Florida, a state with more elderly people than any other state in America.” Scott, who has stood by his agenda and accused Biden of “lying” about it, claimed Friday that McConnell was backing up the Democratic president. “He’s backing up Biden again,” Scott said. “He doesn’t believe that we ought to have a plan”…

Dylan Patel & Afzal Ahmad: Peeling The Onion’s Layers—Large Language Models’ Search Architecture And Cost: ‘ChatGPT currently costs ~$700,000 a day to operate in hardware inference costs. If the current implementation and operation of ChatGPT were ham-fisted into every Google Search, it would represent a tremendous increase in cost structure to the tune of $36 billion. Google’s annual net income for their services business unit would drop from $55.5 billion in 2022 to $19.5 billion. This is never going to happen, of course, but fun thought experiment if we assume no software or hardware improvements are made…. Simple optimizations bring the additional costs for implementing an LLM into search to only $3 billion for Google.… LLMs within search are not just one big model ham-fisted into a search engine’s interface. Instead, it is many models interlaced. Each model’s job is to give the next model in the chain the densest and most relevant information possible. These models must be constantly re-trained, tuned, and tested…. The biggest question is if Google is culturally up for the task. Google has to protect its golden egg. Can they adapt their entire search stack?

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, & Illia Polosukhin: Attention Is All You Need: ‘The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data…

Available for iOS and Android

Join Brad DeLong’s subscriber chat

Available in the Substack app and on web