Perspective on Risk - March 6, 2023 (Technology)
The Change Is Accelerating; We’ve Begun To See Learning Behavior; And Read Minds (Scary); What Does This All Mean; Where Is It Going Next; The Weeds
AI is going to evolve faster than humans and already has superhuman powers
In the near term, AI will threaten white collar jobs more than blue collar ones
We will see a sharp decrease in pay gaps across white and blue collar professions
@bindureddy
The Change Is Accelerating
A brief history of AI published in January (!) had this list of ‘nowhere near solved’ problems. Today, we have either solved or are close to solving all of them.
Chat-GPT, GPT-4, Bing and other AI models can:
understand a story & answer questions about it (Chat-GPT; GPT-4
write interesting stories
OpenAI’s Whisper provides simultaneous transcription in multiple languages as well as translation from those languages into English.
And just recently (like in the last week) Microsoft introduced Kosmos-1, a multimodal model that can reportedly analyze images for content, solve visual puzzles, perform visual text recognition, pass visual IQ tests, and understand natural language instructions. So check off:
interpreting what is going on in a photograph, and
interpreting a work of art
So we are left with the final hurdle: artificial general intelligence. In 2020 Metaculus forecasters were predicting weak general AI by around 2053. Now they are predicting weak general AI by 2028 and strong general AI by 2040. For the first time I feel like I should be telling folks to take the under.
And AI Does Things That Astonish The Pros
Ethan Mollick again, Feats to astonish and amaze.
I do a lot of experiments with various AI systems (mostly Bing in the last couple weeks - here is my guide), and I often find myself astonished.
Come up with Meaningful Connections
Come up with a solution to a problem using a dolphin, VR headsets, and a sociological theory about social networks.
Apply Theory to Practice, and Vice-Versa
What would Immanuel Kant and John Stuart Mill say about Mutually Assured Destruction?
Build on Existing Ideas in Original Ways
I asked Bing look up folktale styles using the Aarne-Thompson-Uther classification of world folktales, and then rewrite those tails for imaginary cultures, such as sky pirates
We’ve Even Begun To See Learning Behavior
Ethan Mollick has pointed out that Bing learns when pointed to authoritative sources.
Further, researchers have found that LLMs can reason by analogy to solve zero-shot problems.
Emergent Analogical Reasoning in Large Language Models
The recent advent of large language models - large neural networks trained on a simple predictive objective over a massive corpus of natural language - has reinvigorated debate over whether human cognitive capacities might emerge in such generic models given sufficient training data. Of particular interest is the ability of these models to reason about novel problems zero-shot, without any direct training on those problems. In human cognition, this capacity is closely tied to an ability to reason by analogy. … We found that GPT-3 displayed a surprisingly strong capacity for abstract pattern induction, matching or even surpassing human capabilities in most settings. Our results indicate that large language models such as GPT-3 have acquired an emergent ability to find zero-shot solutions to a broad range of analogy problems.
And teach themselves to use other software tools.
Toolformer: Language Models Can Teach Themselves to Use Tools
In this paper, we show that Language Models (LMs) can teach themselves to use external tools via simple APIs and achieve the best of both worlds. We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction. This is done in a self-supervised way, requiring nothing more than a handful of demonstrations for each API.
And, OK Now I’m Scared, Read Minds
This paper, High-resolution image reconstruction with latent diffusion models from human brain activity, has not yet been peer reviewed (so take with appropriate caution).
Here, we propose a new method based on a diffusion model (DM) to reconstruct images from human brain activity obtained via functional magnetic resonance imaging (fMRI). More specifically, we rely on a latent diffusion model (LDM) termed Stable Diffusion. … We show that our proposed method can reconstruct high-resolution images with high fidelity in straightforward fashion, without the need for any additional training and fine-tuning of complex deep-learning models.
If I read the site correctly, they are denoising MRI results for visual images.
The Cost of Employing Generative AI Is Collapsing
Huge Price Cut
Introducing ChatGPT and Whisper APIs (OpenAI)
The ChatGPT model family we are releasing today,
gpt-3.5-turbo
, is the same model used in the ChatGPT product. It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models.
What This Tells You
OpenAI's Foundry leaked pricing says a lot – if you know how to read it
Foundry [is] the "platform for serving" OpenAI's "latest models".
"Latest models" – plural – says a lot. GPT4 is not a single model, but a class of models, defined by the scale of pre-training and parameter count, and perhaps some standard RLHF/RLAIF package as well.
Microsoft's Prometheus is the first GPT4-class model to hit the public, and it can do some crazy stuff!
Anything for which there is an established, documented, the standard operating procedure will be transformed first. Work that requires original thought, sophisticated reasoning, and advanced strategy will be much less affected in the immediate term.
Specifically, within 2023, I expect custom models will be trained to…
Create, re-purpose, and localize content – you can fit full brand standards docs into 32K tokens and still have plenty of room to write some tweets. Amazingly my own company Waymark is mentioned with Patrón, Spectrum, Coke, and OpenAI in this article.
Handle customer interactions – natural language Q&A, appointment setting, account management, and even tech support, available 24/7, pick up right where you left off, and switch from text to voice as needed. Customer service and experience will improve dramatically. For example, Microsoft will let companies create their own custom versions of ChatGPT — read here.
Streamline hiring – in such a hot market, personalizing outreach, assessing resumes, summarizing & flagging profiles, and suggesting interview questions. For companies who have an overabundance of candidates, perhaps even conducting initial interviews?
Coding – with knowledge of private code bases, following your coding standards. Copilot is just the beginning here.
Conduct research using a combination of public search and private retrieval. See this thread from Jungwon (@jungofthewon) about best-in-class Elicit (@elicitorg) – it really does meaningful research for you – must-read thread here
Analyze data, and generate, review, and summarize reports – all sorts of projects can now "talk to data" – another of the leaders is @gpt_index
Execute processes by calling a mix of public and private APIs – sending emails, processing transactions, etc, etc, etc. We're starting to see this in the research as well.
So What Does This All Mean
Generative AI Increases Productivity
From MIT: Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence
We examine the productivity effects of a generative artificial intelligence technology—the assistive chatbot ChatGPT—in the context of mid-level professional writing tasks. In a preregistered online experiment, we assign occupation-specific, incentivized writing tasks to 444 college-educated professionals, and randomly expose half of them to ChatGPT.
Our results show that ChatGPT substantially raises average productivity: time taken decreases by 0.8 SDs and output quality rises by 0.4 SDs. Inequality between workers decreases, as ChatGPT compresses the productivity distribution by benefiting low-ability workers more. ChatGPT mostly substitutes for worker effort rather than complementing worker skills, and restructures tasks towards idea-generation and editing and away from rough-drafting. Exposure to ChatGPT increases job satisfaction and self-efficacy and heightens both concern and excitement about automation technologies.
Human Skill’s Improve Too
Jobs Gained and Lost?
Someone asked Chat-CPT what jobs will be replaced by AI, an explanation why, and the ‘human weaknesses’ that are overcome. Chat-GPT obliged.
According to Chat-GPT the list includes content creator, data analyst, journalist (not good for my son), teacher (not good for another son), marketing manager, customer service rep, financial analyst, product manager, HR manager, social media manager, legal assistant, medical receptionist, graphic designer, copywriter, web developer, librarian, translator, interior designer, bookkeeper, receptionist and musician.
The DYNOMIGHT INTERNET NEWSLETTER had an interesting set of analogies:
Marc Andreesen doesn’t think AI will cause job losses, but for all the wrong reasons! In Why AI Won't Cause Unemployment he argues:
AI is already illegal for most of the economy, and will be for virtually all of the economy. … How do I know that? Because technology is already illegal in most of the economy, and that is becoming steadily more true over time.
[W]e actually live in two different economies. [One sector where] where technological innovation is allowed to push down prices while increasing quality. [And other] sectors where technological innovation is not permitted to push down prices; in fact, the prices of education, health care, and housing as well as anything provided or controlled by the government are going to the moon, even as those sectors are technologically stagnant.
The prices of regulated, non-technological products rise; the prices of less regulated, technologically-powered products fall. Which eats the economy? The regulated sectors continuously grow as a percentage of GDP; the less regulated sectors shrink.
We are heading into a world where a flat screen TV that covers your entire wall costs $100, and a four year college degree costs $1 million
So How Do I Use This Stuff?
Google has introduced a machine-learning add-on, SimpleML, for Google Sheets.
A programmer has built an add=on, SheetAI, for Google Sheets that allows you to call GPT-3.
ChatBA will generate slides for you, exportable as .ppt or .pdf
will summarize papers for you
Ethan Mollick has created The practical guide to using AI to do stuff
Use Bing to ideate using a framework such as the Stanford Design framework
Here is a useful AI TOOLS DIRECTORY
And of course you can
Converse with a cat at CatGPT
A.I. In Financial Services
NVidea has published its third annual “State of AI in Financial Services” report, based on a survey of approximately 500 global financial services professionals about the trends, challenges, and opportunities of accelerated computing, AI, and machine learning in the industry.
Analysis of this year’s results highlights four important shifts in the application of AI in banking, insurance, asset management, and fintech:
As the economy faces macroeconomic challenges, financial services
companies are looking to AI to more accurately assess risk, create
operational efficiencies, and reduce costs.
Migrate to accelerated computing platforms and reduce grid farms by up
to 75 percent. This saves on servers, space, and energy, while improving
performance with faster model training, more accurate models, and lowerlatency inference.
Leverage software that enables workload portability and offers centralized
management of hybrid and multi-cloud infrastructure.
Deploy AI-enabled applications to enhance customer service.
Companies are increasing the velocity at which they deploy AI-enabled
applications into production. Most use cases analyzed by the survey are used by over 20 percent of respondents’ companies, and the percentage of companies that view themselves as laggards in AI fell significantly year over year.
The competition for data scientists hasn’t receded from last year’s fever
pitch. On the contrary, recruiting and retaining data scientists is now the
biggest challenge to achieving a financial services company’s AI goals.
Almost half of AI projects run on hybrid infrastructure, making data portability, MLOps management, and software standardization across cloud and on-prem instances a strategic imperative
Using A.I. in Insurance
Of course, as we all know, insurance is the laggard in adopting technology in the financial services space. Here is a paper about using AI in insurance; not Chat-GPT, but rather in triangles and stuff. Applying Machine Learning to Actuarial and Pricing Workflows
While complex algorithms seem to be at odds with the transparency and judgment needed in actuarial and pricing practices, these concepts can be highly complementary if machine learning is implemented correctly. In this paper, we will explore the necessity and possibilities of machine learning applications in the insurance industry, and provide guidelines for incorporating machine learning techniques into pricing applications.
The paper is a bit of a sales pitch, and not at all technical. I really wanted to do some of this before I retired, but couldn’t get the consensus buy-in. Oh well. Born too soon.
I would add, and I never got to do this at AIG, but using an LLM on digitized insurance contracts seems to be such low hanging fruit that it should have been done a decade ago or more.
Where Is This Going Next
An Interview with Daniel Gross and Nat Friedman about ChatGPT and the Near-Term Future of AI (Stratergy)
The Brief History of Artificial Intelligence: The World Has Changed Fast—What Might Be Next? (Singularity)
In The Weeds, For Those So Inclined
Here are some technical insights that I have found particularly interesting.
The primary technical innovation in these LLMs is the use of ‘attention’ and ‘transformers.’
Transformers
How LLMs Appear To Work
Early artificial intelligence research was dominated by a symbolic approach, where words were mapped to symbolic representations. Wolfram Research has a sophisticated symbolic model that is great at math and other things where LLMs fail.
Stephen Wolfram, the inventor of Wolfram Alpha, writes a thorough introduction to how large language models (LLMs) work, and how they are different from (and perhaps complementary to) symbolic models. Written for folks who can understand probabilities, but do not know the details of things like backprop, transformers and attention. What Is ChatGPT Doing … and Why Does It Work? (Wolfram)
I look forward to the day when we can integrate Wolfram Alpha with a LLM. Related.
Bayesian statistics and machine learning: How do they differ?
Bayesian statistics and machine learning: How do they differ?
For the Nerdiest of Nerds Among Us
If you really want to learn about LLMs, and have the math background, read these papers recommended by Sebastian Raschka:
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Cramming: Training a Language Model on a Single GPU in One Day
Training language models to follow instructions with human feedback
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Finally, Cleo Nardo has a long post on The Waluigi Effect.