GPT-3 | Sociamonials Blog

GPT-3

TODOS 0-9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

The innovative language model, GPT-3, developed by OpenAI, is the third installment in the GPT series and is distinguished by its unprecedented scale, making it the largest non-sparse language model currently available. Outperforming its predecessor, GPT-2^[1], and Microsoft’s Turing NLG, GPT-3 has ten times the capacity of the latter. It is renowned for its capability to generate text, including news articles, and aid in coding tasks, though it also poses potential misuse threats such as the propagation of misinformation or phishing. GPT-3 comes in various versions to accommodate different needs, with davinci, possessing 175 billion parameters, being the largest. The subsequent GPT-3.5 series introduced new models and abilities. GPT-3 plays a pivotal role in both industry and research, supporting products like GitHub^[2] Copilot and finding application in several Microsoft products. However, it also raises ethical and academic issues.

Definición de términos

1. ↑ GPT-2 ( GPT-2 ) Generative Pretrained Transformer 2, or GPT-2, is an advanced AI model specifically engineered for natural language processing tasks. This model, launched by OpenAI in February 2019, is renowned for its versatility in generating a wide array of text types, with its prowess extending to answering queries and completing code automatically. GPT-2's training involved a vast online text corpus, WebText, and it operates on a staggering 1.5 billion parameters. Despite its resource-intensive nature, GPT-2 has found usage in diverse and innovative applications such as text-centric adventure games and subreddit simulations. Initial misuse fears led to the full GPT-2 model's release in November 2019 when the concerns didn't manifest. However, to address resource constraints, a smaller model, DistilGPT2, was developed. The innovations and successes of GPT-2 set the stage for future progress in AI text generation.

2. ↑ GitHub ( GitHub ) Diseñada principalmente para desarrolladores, GitHub es una plataforma que facilita la creación, el almacenamiento, la gestión y el intercambio de código. Basada en el software Git, ofrece funciones como el control de versiones, el control de acceso y el seguimiento de errores. Desde que se convirtió en una filial de Microsoft en 2018, GitHub se ha consolidado como uno de los principales anfitriones de proyectos de software de código abierto. En enero de 2023, cuenta con una vibrante comunidad de más de 100 millones de desarrolladores y alberga más de 420 millones de repositorios. GitHub fue concebido en 2008 por sus cuatro fundadores e inicialmente funcionó como una organización plana, fomentando la autonomía, la flexibilidad y la colaboración entre sus miembros. Además del control de versiones, GitHub también ofrece servicios como gestión de tareas, integración continua y soporte para wikis de proyectos. Es más que una plataforma: es un conjunto completo de herramientas para el desarrollo de software.

GPT-3 (Wikipedia)

Generative Pre-trained Transformer 3 (GPT-3) es una large language model released by OpenAI in 2020. Like its predecessor, GPT-2, it is a decoder-only transformer model of deep neural network, which supersedes recurrence and convolution-based architectures with a technique known as "atención". This attention mechanism allows the model to selectively focus on segments of input text it predicts to be most relevant. It uses a 2048-tokens-long context^[jargon], float16 (16-bit) precision, and a hitherto-unprecedented 175 billion parameters, requiring 350GB of storage space as each parameter takes 2 bytes of space, and has demonstrated strong "zero-shot" and "few-shot" learning abilities on many tasks.

Generative Pre-trained Transformer 3 (GPT-3)
Autor(es) original(es)	OpenAI
Lanzamiento inicial	June 11, 2020 (beta)
Repositorio	github.com/openai/gpt-3
Predecessor	GPT-2
Sucesor	GPT-3.5 GPT-4
Tipo	Gran modelo lingüístico Transformador generativo preentrenado Foundation model
Página web	openai.com/blog/openai-api

On September 22, 2020, Microsoft announced that it had licensed GPT-3 exclusively. Others can still receive output from its public API, but only Microsoft has access to the underlying model.