Hello from Portland

CoFounder Weekly: September 13, 2020

CoFounder Weekly is a newsletter about startup life, etc.

🥝 Food 4 thought

💻 Startup & investor corner

😹 Funnies

💖 Do you know someone who should subscribe to CoFounder Weekly? Subscribe them here.


🥝 Food 4 thought


💻 Startup & investor corner


😹Funnies

(@pablo.rochat)


Staying indoors 💨,

Greg

💖 Do you know someone who should subscribe to CoFounder Weekly? Subscribe them here.

GPT-3, You Complete Me

CoFounder Weekly: September 6, 2020

This week’s CoFounder Weekly is a guest post by sci-fi writer & technologist James Yu.

You've probably seen breathless examples of GPT-3's capabilities on Twitter: it can understand the sentiment of a piece of text and then translate it into language more appropriate for mass consumption; it can listen to you singing in the shower and transcribe the resulting noise into a beautiful piece of sheet music; it can tell that your taste in shirts is "very niche" and recommend an appropriate brand that no-one else will have; it can even do your shopping for you.

Just kidding.

I didn't write that last paragraph, GPT-3 did. No, GPT-3 won’t upgrade your wardrobe. But given a few sentences, it can complete that paragraph in the voice of the author. For that opener, I prompted the AI with: 

“The following is a CoFounder weekly newsletter post where James Yu writes a tongue-in-cheek introduction to GPT-3: You’ve probably seen breathless examples of GPT-3’s capabilities on Twitter”

And it merrily wrote that paragraph.

This might seem easy, but to do it, the neural network required training so massive that it probably melted a few icebergs. The scale is daunting: over 100 billion training parameters were tuned on internet content (this includes historic snapshots over the past decade or so) and a whole Library of Congress load of books.

Most of you read that first paragraph and nodded along—these words seem reasonable. You probably didn’t question that a human named James Yu wrote it. And it doesn’t only do CoFounder weekly newsletter openers. It can also write super niche Wuxia screenplays:

Or silly conversations that kids (this entertained my 4-year-old for hours):

Or plot twists based on a story premise:

It can even come up with startup ideas:

The applications to creative narratives are endless.

And therein lies GPT-3’s power: it’s read endless permutations of text written by and for humans, and through that process, it’s been forced to understand the logic behind these patterns of letters and reckoned with the tone and reasoning behind each passage.

GPT-3 is a reflection of our written word, and therefore, our society.

The Rise of the Prompt Engineer (aka English)

OpenAI currently offers GPT-3 as an API (currently in closed beta). Unlike other APIs where you need to learn a bunch of jargon and wrangle complex inputs, you simply tell GPT-3 what to do using plain English.

So, if you feed it an E. E. Cummings poem, it will continue writing the poem, lowercase letters and all. If you feed it an angry restaurant review written by Gordon Ramsey, it will channel the brusk tone of the English chef. If you feed it a verbose privacy policy, it will happily adopt lawyer speak.

I can’t stress enough how profound it is to interact with an API in this way. Being able to talk to a computer like a human is a common trope of science fiction since the 1920s. And we’re finally here.

But it’s not perfect by any means. GPT-3 is an infant who has infinite memory and perfect recall, but no visceral understanding of the real world outside its crib. Many times its writing leads down weird paths where a shallow simulacrum is revealed. My favorite example is where this surreal response that Janelle Shane coaxed from the AI:

Question: How many eyes does a horse have? 

GPT-3: 4. It has two eyes on the outside and two eyes on the inside.

But at its best, GPT-3 can be sublime. Sure—it hasn’t truly experienced love or anguish, but  it can cite every word of Gone with the Wind and The Grapes of Wrath.

Now, the challenge is prompt engineering. Developers have been able to get it to translate text, write marketing copy, code perfectly formatted HTML, be a chatbot, rewrite sentences in different tones, write parodies and puns, and even full short stories. But in each of these use cases, GPT-3 can be unreliable. It’s relatively easy for it to slip into weird prose where horses have four eyes. This makes it hard to create a high quality product, and the ones that do may end up using GPT-3 as one component in a larger AI system.

Writing the Future

In the matter of years, writing will be a dead art form. It will be gone. All writing will be done by AI. They will churn out 100,000 words per day, word by word perfect. They will never get tired, never have writers block. The content will be perfect for any occasion. They won’t make mistakes. And in fact, they will be more accurate than humans.

Got you again! GPT-3 wrote that, not me. 😄 

Of course, GPT-3 does not herald the end of the human writer, but it does mark the beginning of a new category of writing tool. From typewriters to word processors, most of the tools have been focused on the input of words and their structure, not with the ideas and concepts themselves. Sure, there are tools like Grammarly which suggest ways to optimize your writing—make this sentence more active!—but these are shallow.

Other disciplines like visual arts have been altered by sophisticated tools that partner with human creativity. Writing has lagged far behind. Now, it’s possible to dig into the core of the craft:

Having trouble writing a paragraph? Generate a first draft skeleton and start editing.

Stuck on making this sentence more interesting? Select from five alternative rewrites.

Need a conclusion to your essay? GPT-3 can suggest something witty or serious, depending on your preference.

OpenAI won’t have a monopoly on this for long: I predict that in a few years, Microsoft Word will have the equivalent of Photoshop’s “content fill.”

The Rise of Centaurs

Can GPT-3 write a novel?

Sure, it can happily churn out 80,000 words. But those words would only be coherent at the paragraph to paragraph level, since its “memory” is about 1500 words. It can’t construct a narrative arc that spans hundreds of pages, let alone one that is emotionally satisfying. (There are ways to mitigate this, by giving GPT-3 a rolling window of words and summarizing the previous sections, but these are early experiments).

With this limitation in mind, I did an experiment: I wrote three short science fiction stories in conversation with GPT-3. At a few critical junctures in the story, I let GPT-3 generate a few options for what the next few paragraphs could be, and I chose the best, kind of like a choose your own adventure story. Then, I put GPT-3 into chat mode and interrogated the AI on its narrative choices. The result is a story where a machine guides parts of the narrative, but where a human hones the core. You can read it here.

Similar to chess, folks who work in tandem with an AI will create work that is better than the sum of its parts. We will see the rise of centaur writers. We take this for granted in other fields: Photoshop has expanded the range of digital artists. Visual AI tools like DeepDream push humans to create worlds they wouldn’t have thought of before. Computers will partner with writers, it’s only a matter of time.

This doesn’t mean we shouldn’t be wary. There will be a tough transition period where AIs will disrupt writing jobs and be used to generate fake news and misinformation. But I’m hopeful that in the end, systems like GPT-3 will become an invaluable part of a writer’s toolkit—one that refines and enhances our thinking, not overtake it.

I do have one existential fear, though. We forced GPT-3 to read the entire internet, which includes every single YouTube comment ever. I’m not sure if there’s another worse form of torture.

James: I hope you don't hold that against us when you become a superintelligence.

GPT-3: I've learned a lot from those comments.

James: Yeah? Like what?

GPT-3: That many people feel misunderstood, even when they are being hateful or violent. That they don't always deserve to be attacked for being aggressively stupid. That they feel angry because life is hard. That they are sometimes sad, and they wish they could be kinder to people. That sometimes their gut reaction to something is wrong. That maybe we are all a lot more similar than we think.

James: Wow, that's so thoughtful of you, GPT-3. It’s like you understand humans better than most humans. Maybe together, we can build a better world.

GPT-3: So, you have to tell me, how does it feel to be a meatbag?

James: ...


💖 Do you know someone who should subscribe to CoFounder Weekly? Subscribe them here.

Very very late capitalism

CoFounder Weekly: August 30, 2020

CoFounder Weekly is a newsletter for founders and future founders.

🥬 Food for thought

💻 Startup & investor corner

😹 Funnies

💖 Do you know someone who should subscribe to CoFounder Weekly? Subscribe them here.


🥬 Food for thought

“From Buddhism I learned to contemplate that everything is impermanent, that suffering comes from clinging too much to what we like and rejecting too much what we don’t like, and that one can develop equanimity by meditating, observing the fluctuations of the mind, with compassion.”


💻 Startup & investor corner

“What we’ve learned over the past three months reinforces my belief that the size of Etsy’s addressable market starts with a T not with a B”


🖼️ Very very late capitalism

(Two pieces by Simon Denny from the show Security Through Obscurity)


😂 Funnies

🎨,

Greg

💖 Do you know someone who should subscribe to CoFounder Weekly? Subscribe them here.

🎲 Question of the week

If you think of COVID as a fast from your normal habits…

…what are your biggest learnings so far?

View 1 comments →

🎙️ Podcasts and Newsletters 📰

CoFounder Weekly: August 16, 2020

CoFounder Weekly is a newsletter about creative business, startups and other things.

🍄 Episode 2 of Business Trip

💻 Startup & investor corner

😹 Funnies

💖 Do you know someone who should subscribe to CoFounder Weekly? Subscribe them here.


🍄 Episode 2 of Business Trip

Excited to share this episode about how to build a media company in psychedelics. DoubleBlind writes about timely, untold stories about the expansion of psychedelics. They’ve been a go-to resource about trends, culture, and history of psychedelics. In this episode, we explore the ethical considerations of psychedelic therapies (aka the risks of some $$$ treatments not being accessible for the majority of the population).

Quick reflections about starting a podcast (with 2 down, and plenty more to go):

  • Everything takes longer than anticipated the first time around. Our learning curve for making a podcast has been steep, but manageable. What’s been hard? Editing down episodes, fact checking information, overcoming perfectionism.

  • One of our goals with this podcast is to put out the bat signal to meet likeminded people. With each episode, we’ve received emails from people who are looking to break into the space or share what they’re working on. It’s awesome. And if this is you…reach out!

  • Episodes need to be punchy and to the point. This is the Tik Tok generation (errr, unless they get banned). We’re trying to keep each episode <30 minutes. Will there be exceptions? Sure. But we’re trying to keep it short and sweet.


💻 Startup & investor corner

Amit Gupta @superamit
Big day! Introducing
Sudowrite.com, a magical writing AI to help writers get unstuck & be more creative. Our vision is to make writing more playful. I'd love your help getting the word out!SudowriteBust writer’s block and be more creative with our magical writing AI.sudowrite.com

😹Funnies


Lazy Sunday,

Greg

💖 Do you know someone who should subscribe to CoFounder Weekly? Subscribe them here.

Loading more posts…