Concept
OpenAI News
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific...
Topics: ModelsInfrastructure
Entities: ModelsInfrastructure
OpenAI News
We’re introducing OpenAI Microscope, a collection of visualizations of every significant layer and neuron of eight vision “model organisms” which are often studied in interpretability. Microscope makes it easier to analyze the features that form inside...
Topics: Models
OpenAI News
We are standardizing OpenAI’s deep learning framework on PyTorch.
Topics: Models
OpenAI News
We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends spanning more than seven orders of...
Topics: ModelsInfrastructure
Entities: ModelsInfrastructure
OpenAI News
On April 13th, 2019, OpenAI Five became the first AI system to defeat the world champions at an esports game. The game of Dota 2 presents novel challenges for AI systems such as long time horizons, imperfect information, and complex, continuous state-action...
Topics: ModelsInfrastructure
Entities: OpenAIModelsInfrastructure
OpenAI News
As the final model release of GPT-2’s staged release, we’re releasing the largest version (1.5B parameters) of GPT-2 along with code and model weights to facilitate detection of outputs of GPT-2 models. While there have been larger language models released...
Topics: Models
Entities: Models
OpenAI News
We’ve trained a pair of neural networks to solve the Rubik’s Cube with a human-like robot hand. The neural networks are trained entirely in simulation, using the same reinforcement learning code as OpenAI Five paired with a new technique called Automatic...
Topics: ModelsInfrastructure
Entities: OpenAIModelsInfrastructure
OpenAI News
We are now accepting applications for our third class of OpenAI Scholars.
Topics: Models
OpenAI News
We’ve fine-tuned the 774M parameter GPT-2 language model using human feedback for various tasks, successfully matching the preferences of the external human labelers, though those preferences did not always match our own. Specifically, for summarization...
Topics: Models
Entities: Models
OpenAI News
We’re releasing the 774 million parameter GPT-2 language model after the release of our small 124M model in February, staged release of our medium 355M model in May, and subsequent research with partners and the AI community into the model’s potential for...
Topics: Models
Entities: Models
OpenAI News
At OpenAI, each Thursday is Learning Day: a day where employees have the option to self-study technical skills that will make them better at their job but which aren’t being learned from daily work.
Topics: Models
OpenAI News
Microsoft is investing $1 billion in OpenAI to support us building artificial general intelligence (AGI) with widely distributed economic benefits. We’re partnering to develop a hardware and software platform within Microsoft Azure which will scale to AGI....
Topics: Models