- Research lab OpenAI released a "natural language generation" tool called GPT-3 that learns from a massive set of data to write impressively human-like text. GPT-3 can account for tone and context to write in conversational or formal English.
- Developers have been using GPT-3 to write creative fiction, design websites, and write business proposals with very little indication a robot created them.
- But even the most advanced AI tool raises concerns around biased language and ways the technology could be used to spread hate and misinformation.
- Visit Business Insider's homepage for more stories
OpenAI, a major artificial intelligence research lab, released a new tool to select developers last week that can automatically create written passages that are nearly indistinguishable from those written by humans, and developers are already stunned by how well it can code, mimic famous authors, and write business memos.
GPT-3, as the tool is called, is the group's third iteration of a "natural language generation" model — an algorithm that can look for patterns in massive datasets of human-created text and spit out entire sentences and paragraph in a writing style that reflects those patterns (in other words, in plain-English).
Language generation models and the applications they power are already used widely. For example, Google's Smart Compose uses these models to help users autocomplete sentences in Gmail and the Associated Press writes quarterly earnings stories and sports reports using them, too. But the results these tools produce are often clunky, awkward, or unnatural due to limitations with the underlying data or the language model itself.
"Historically, natural language generation systems have lacked some nuance," said Carolyn Rose, a professor at Carnegie Mellon University's Language Technologies Institute. But GPT-3 seems different. Based on early reactions, GPT-3 has blown past existing models thanks to its massive dataset and its use of 175 billion parameters — rules the algorithm relies on to decide which word should come next to mimic conversational English. By comparison, the previous version, GPT-2, utilized 1.5 billion parameters, and the next most powerful model — from Microsoft — has 17 billion parameters.
OpenAI published the technical specifications of GPT-3 in May, but opened it up an application programming interface to select developers last week. Developers on Twitter were amazed by the powerful capabilities of the tool and quickly started experimenting: GPT3 can code a website based on plain English descriptions, write sonnets that mimic Shakespeare's works, and explain OpenAI's research paper on this subject better than this article can (maybe). It was described as a "groundbreaking" new model that could end up making people's jobs easier (or even obsolete).
"Every bit of the hype is deserved, and it's worth wrestling with each of the big questions raised," investor Chris Sacca tweeted.
German artist Mario Klingemann fed the tool examples of different authors and GPT-3 created paragraphs-long, stylistically-similar, coherent stories.
Another attempt at a longer piece. An imaginary Jerome K. Jerome writes about Twitter. All I seeded was the title, the author's name and the first "It", the rest is done by #gpt3
— Mario Klingemann (@quasimondo) July 18, 2020
Here is the full-length version as a PDF:https://t.co/d2gpmlZ1T5 pic.twitter.com/1N0lNoC1eZ
"People are seeing this perhaps as harbinger of some huge change in natural language processing, coding, what have you," Oren Etzioni, CEO of the Allen Institute, a Seattle-based nonprofit research lab. The tool builds upon all of its predecessors, and 30 years of AI research and experimentation: "Whether it's significant or not is still an open question, but it's certainly impressive," he said.
Because it's an available as an API, it can democratize experimentation, he added.
"Pople who don't have the resources — both computational and the expertise to build and train these models — could potentially use it," he said.
While GPT-3 isn't a "new frontier" in AI, it could still lead to huge improvements for automatically generated text, Rose said. For example, it could make text-to-speech tools, like voice instructions from Google or Apple Maps, "less annoying" to listen to.
GPT-3 still isn't perfect, to be sure. When developer Kevin Lacker administered the Turing test — a test meant to see if the AI could trick someone into thinking it was human — he came to the conclusion that "GPT-3 is quite impressive in some areas, and still clearly subhuman in others."
For example, when he asked "How do you sporgle a morgle?" GPT-3 responded, "You sporgle a morgle using a sporgle."
Even Sam Altman, OpenAI's CEO, was cautious about GPT-3:
The GPT-3 hype is way too much. It’s impressive (thanks for the nice compliments!) but it still has serious weaknesses and sometimes makes very silly mistakes. AI is going to change the world, but GPT-3 is just a very early glimpse. We have a lot still to figure out.
— Sam Altman (@sama) July 19, 2020
GPT-3 also puts a spotlight back on existing ethical concerns surrounding AI-powered tools as well OpenAI itself.
OpenAI was launched in 2015 by tech industry titans including Elon Musk, Peter Thiel, and Sam Altman, and originally preached a gospel of accountability and transparency around its work. However, Musk criticized OpenAI earlier this year, saying it should be more open about its work, and MIT Technology Review's Karen Hao reported that the company has a culture of secrecy that runs counter to its public image. It originally chose not to make the predecessor to GPT-3 — GPT-2 — available because it could be used to create realistic fake news stories, though it later reversed this decision.
The Allen Institute's Etzioni did say that GPT-3's ability to mimic speech so well may make it harder to spot fake content.
"The question 'Is this text or image or video or email authentic?' is going to become increasingly difficult to answer just based on the content alone," he said.
GPT-3 has also reignited concerns about the tendency of artificial intelligence to display sexist or racist biases or blind-spots. Because artificial intelligence models learn based on the data provided to them, they can end up exaggerating human biases. For example, the head of Facebook's AI program tweeted about how GPT-3 returned biased results from single-word prompts like "Jew," "women," or "Black" when trained on real tweets.
#gpt3 is surprising and creative but it’s also unsafe due to harmful biases. Prompted to write tweets from one word - Jews, black, women, holocaust - it came up with these (https://t.co/G5POcerE1h). We need more progress on #ResponsibleAI before putting NLG models in production. pic.twitter.com/FAscgUr5Hh
— Jerome Pesenti (@an_open_mind) July 18, 2020
"In the end, the process that's happening is totally mechanical." Rose said. "It doesn't have values. It doesn't know what's beautiful."
Join the conversation about this story »
NOW WATCH: Why thoroughbred horse semen is the world's most expensive liquid
source https://www.businessinsider.com/openai-gpt-3-language-generation-model-important-breakthrough-2020-7
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.