Like many of you, I’ve been having a lot of fun playing around with ChatGPT over the past few months and started wondering about how tools like this would change the world of startup creation and investing in the future.
ChatGPT – What is it?
Late last year, OpenAI released ChatGPT — sort of an advanced AI chat-bot — an improved version of its earlier GPT models. The basics way these systems work is that they’re trained on a huge corpus of text (hundreds of billions of words from the web, books, wikipedia, etc.) and designed to predict the next word given a certain prompt and the words that came before. The results are uncanny in how natural and real they (usually) sound.
You know what, let’s have ChatGPT tell us in its own words:
Probing those limitations of the system — seeing where it doesn’t work — has been just as fascinating as seeing where it will work. Examples abound of funny mistakes the system makes, like insisting that the “fastest marine mammal” is the peregrine falcon. In strange ways the system reveals that it doesn’t /quite/ understand what it’s talking about.
Does ChatGPT understand science?
At this point, I was wondering what effects a tool like this would have on our world. Some fear the end of the education system as we know it — sharing examples of how the tool could be used to both create test questions, write essays, and grade them! (https://twitter.com/emollick/status/1598745129837281280)
The scariest example for me, was the following thread:
Follow the full thread here:
Dr. Kubacka wrote her dissertation on multiferroics, a special class of materials with interesting electromagnetic properties. (https://en.wikipedia.org/wiki/Multiferroics). First she started probing on whether GPT could describe the technology. Then she began asking about specific papers relevant in the field.
If you read through the thread, GPT first provided fake, but totally convincing, citations for the key papers in the field. Then she started asking about a made-up extension of the field that was equally convincing. The way she describes the tool as “hallucinating” details about the technology puts me deeply in some uncanny valley!
As someone who evaluates thousands of new research ideas within the field of climate tech, all of this inspired me to ask a few questions about how this tool could be used to generate new research ideas, describe existing ideas, and, perhaps more darkly, how it could be used to dupe grant review committees.
Question 1: Can AI create new Climate Tech Startups?
Coming up with a wholly new research idea is no easy feat. Often new ideas come from years and years of experience in a field or from combining unrelated ideas.
I probed ChatGPT on whether it could combine disparate ideas to form new concepts:
All of these sound reasonably plausible, but none sound particularly novel. A quick Google scholar search comes up with some 500,000 articles about graphene-based solar cells, and about the same for carbon nanotube hydrogen storage. The possible exception might be microfluidics for algae production — there are plenty of uses for microfluidics in algae research — for things like rapid prototyping and characterization, but it’s not clear that it would be a scalable solution for production.
If we ask GPT, it gets into what I see as its defining (and maybe most frustrating) characteristic, the sort of wishy-washy, noncommittal answer.
In any case, it’s a fascinating proof of concept! In short order, I expect that a version of GPT that is connected to updated internet results will be able to plug ideas into a search engine until it finds one that doesn’t have many prior examples.
The idea of combining known fields to create a new invention may or may not be a good way to generate research ideas, but what about a more targeted approach?
Targeted idea generation
I asked ChatGPT to help come up with new ideas for hydrogen generation.
By and large, these are good, standard approaches, even if some of these answers are not really on target (The suggestions for nuclear, geothermal, hydro, and power to gas are sensical but are really just specific cases of #1.)
So far we’re doing OK as far as a set of answers to a test in an introductory course. But quickly we get into territory where GPT is out of its depth.
This explanation sounds so reasonable, so specific, so detailed — it’s hard to believe it’s nonsense.
What ChatGPT is claiming is that you just need some heat and a catalyst (basically some material that speeds up a reaction but otherwise nominally doesn’t react) and you can turn CO2 and water into just hydrogen and oxygen.
As our introductory chemistry reminds us, matter and energy can’t be destroyed — the carbon atom in the CO2 has to go somewhere!
Let’s push ChatGPT a bit:
Notably, this is not the reverse water-gas shift reaction, and furthermore it’s not balanced. Let’s see if ChatGPT can uncover its mistake…
ChatGPT did figure out that there was a missing carbon on the product side, but still wasn’t able to create a balanced reaction (3 oxygens are missing from the right hand side). The second attempt just doubled both sides, which is sort of an irrelevant change.
Back to our original question, can ChatGPT come up with something totally novel?
Well, this is certainly out of the box thinking!
In the next segment, I’ll be going into more detail about whether ChatGPT can (or thinks it can) come up with new research ideas.