Book a call

News · February 14, 2019

OpenAI unveils GPT-2 but holds back the full model

// ai.news :: 2019

OpenAI showed a large language model that writes surprisingly fluent text, then chose to release only a small version. The debate over responsible release is now a business question as well as a research one.

What happened

  • OpenAI said GPT-2 is a transformer model with 1.5 billion parameters, trained to predict the next word on 40GB of text from 8 million web pages.
  • Citing risks such as automated fake news, impersonation, spam and phishing, OpenAI released only a much smaller model, sampling code and a paper, and withheld the full weights, dataset and training code.
  • OpenAI described the approach as an experiment in responsible disclosure.
  • The Next Web reported that the story broke on February 14 and argued that many headlines overstated the danger, noting OpenAI's view that the concern applies to large text generators in general.

Why it matters

For businesses, the headline is less about danger and more about capability: a general model trained on raw web text can produce coherent paragraphs on almost any topic. Content generation, summarization and drafting tools built on this kind of model are likely to arrive quickly.

Two practical steps now: assume generated text will appear in reviews, support tickets and sign-ups, and harden moderation and anti-spam accordingly; and if you plan to use such models, budget for human review of output, since fluency is not the same as accuracy.

Sources

  • OpenAI: Better language models and their implications
  • The Next Web: Who's afraid of OpenAI's big, bad text generator?

This post is CMM’s own summary and commentary on publicly reported news. Facts are attributed to the sources linked above; please read them for full details. CMM is not affiliated with these publishers or with the companies mentioned, and trademarks belong to their owners. Prepared with AI assistance. Corrections or requests: [email protected]. See our editorial policy.

Want help with this?

Tell us what you need, and we will suggest the right team model on a short call.

Talk to us