Topic feed
AI chips, data centers, training systems, and inference infrastructure.
OpenAI News
Whisper is an automatic speech recognition (ASR) system trained on 680,000 hours of multilingual and multitask supervised data collected from the web. We show that the use of such a large and diverse dataset leads to improved robustness to accents,...
Topics: ModelsInfrastructure
Entities: ModelsInfrastructure
OpenAI News
We show that autoregressive language models can learn to infill text after we apply a straightforward transformation to the dataset, which simply moves a span of text from the middle of a document to its end. While this data augmentation has garnered much...
Topics: ModelsInfrastructure
Entities: ModelsInfrastructurePerplexity
OpenAI News
In order to share the magic of DALL·E 2 with a broad audience, we needed to reduce the risks associated with powerful image generation models. To this end, we put various guardrails in place to prevent generated images from violating our content policy.
Topics: ModelsPolicyInfrastructure
Entities: ModelsPolicyInfrastructure
OpenAI News
This paper pursues the insight that large language models (LLMs) trained to generate code can vastly improve the effectiveness of mutation operators applied to programs in genetic programming (GP). Because such LLMs benefit from training data that includes...
Topics: ModelsInfrastructure
Entities: ModelsInfrastructure
OpenAI News
Large neural networks are at the core of many recent advances in AI, but training them is a difficult engineering and research challenge which requires orchestrating a cluster of GPUs to perform a single synchronized calculation.
Topics: Infrastructure
Entities: Infrastructure
OpenAI News
Text embeddings are useful features in many applications such as semantic search and computing text similarity. Previous work typically trains models customized for different use cases, varying in dataset choice, training objective and model architecture....
Topics: ModelsInfrastructure
Entities: ModelsInfrastructure
OpenAI News
We propose a benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics. We crafted questions that some humans...
Topics: ModelsInfrastructure
Entities: ModelsInfrastructure
OpenAI News
We’re releasing Triton 1.0, an open-source Python-like programming language which enables researchers with no CUDA experience to write highly efficient GPU code—most of the time on par with what an expert would be able to produce.
Topics: Infrastructure
Entities: Infrastructure
OpenAI News
Our latest research finds we can improve language model behavior with respect to specific behavioral values by fine-tuning on a small, curated dataset.
Topics: Infrastructure
Entities: Infrastructure
OpenAI News
We’ve scaled Kubernetes clusters to 7,500 nodes, producing a scalable infrastructure for large models like GPT-3, CLIP, and DALL·E, but also for rapid small-scale iterative research such as Scaling Laws for Neural Language Models.
Topics: ModelsInfrastructure
Entities: ModelsInfrastructure
OpenAI News
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific...
Topics: ModelsInfrastructure
Entities: ModelsInfrastructure
OpenAI News
We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends spanning more than seven orders of...
Topics: ModelsInfrastructure
Entities: ModelsInfrastructure