Foundational models are large, deep neural networks trained on broad data at scale to learn general capabilities that are used for many downstream tasks.

Unlike traditional narrow AI models, which are specialized for specific applications, foundational models aim to acquire more generalized “world knowledge” that allows them to perform well across various uses.
Some key characteristics of foundational models:
Trained on diverse, unlabeled data like text from the web or images, which allows them to ingest broad context knowledge
Very large in scale, with billions of parameters. The huge capacity allows them to absorb this broad set of knowledge
Go beyond narrow abilities and exhibit multiple integrated capabilities like reasoning, knowledge representation, and even rudimentary common sense in the instance of generative AI.
Provides a foundation that can be adapted to many different tasks by building on top of the model

Prominent examples include models like GPT-4 for text and Midjourney for images.

Overall, this emerging field of research represents an exciting new direction for enabling more capable and multi-functional AI systems.