It seems like everybody and their mother has a large language model these days. Stability AI, one of the companies that made a name for itself early in the AI rat race, was slower than its contemporaries to enter the realm of LLMs—until now, that is. On Wednesday, the company announced it was launching StableLM, a “suite” of language models meant to compete with alphabet soup AI like OpenAI’s GPT-4, Meta’s LLaMA, and Google’s LaMDA.
Stability AI said it trained its new model on an 800GB open-source data set called “The Pile.” The company said it would release details on the new language model’s training data “in due course” alongside a full technical write-up. The different “very alpha” versions of the LLM, as CEO Emad Mostaque put it, come in 3-billion and 7-billion parameter varieties, and the company claimed its working on 15- and 65-billion parameter versions. The 7B version of the chatbot is available to test drive Hugging Face. This newest LLM was Stability AI’s attempt to move “back to our open roots,” according to Mostaque.
Gizmodo’s initial tests of the model in chatbot form were a little awkward, to say the least. The AI seemed to have a problem switching gears after we asked it about issues with the training data of its competing AI models, then about the best way to peel a banana. The free space on Hugging Face is also being inundated with requests, so it’s hard to get a greater feel for the AI. However, some users reported it fails at some of the most rudimentary tasks like creating a recipe for a peanut butter jelly sandwich (remember to scoop out the banana seeds when assembling, apparently).
Parameters are essentially a way for LLMs to generate predictions and offer a very rough assessment of how sophisticated each model is. For comparison, GPT-3, which was the first to power OpenAI’s ChatGPT, had 175 billion parameters. The company has not revealed how many parameters GPT-4 has, but Semafor reported last month that the latest version of OpenAI’s LLM has 1 trillion parameters. However, the number of parameters doesn’t necessarily inform the quality of the results the AI generates, and more parameters usually mean it costs much more power to actually generate content.
Stability AI is aware that it needs to punch up to compete with its bigger, Microsoft-supported competitors. The tool was developed to help “everyday people and everyday firms use AI to unlock creativity.” The company advertised that the company is “focused on efficient, specialized, and practical AI performance—not a quest for god-like intelligence.” That last bit seems a specific dig at OpenAI, whose execs seem obsessed with the idea of super-intelligent AI.
On Twitter, Mostaque said both the LLM and its training data will only get better with time, saying he wants it to eventually process 3 trillion tokens, which could best be described as units of text, whether that’s letters or words.
Stability AI has long been evangelical in the way it talks about AI, with Mostaque often sounding the horn for proliferated, open-source AI programs, come hell or high water. But the company has reportedly struggled with money lately as it’s spent so much on developing its AI projects and richer companies soak up the attention. The startup recently showed off its enterprise-focused Stable Diffusion XL model that’s meant to be even better than the company’s previous AI image generators. Still, the company said it still plans to open source this newer generative AI model… eventually.