Stability AI has made a big move by publishing two large language models as open source with an alpha version of StableLM. The two Large Language Models (LLM) are 3 and 7 billion parameters. The StableLM-3B and StableLM-7B are licensed under CC BY-SA-4.0, which allows developers to use, study, and adapt them for research and commercial purposes. However, it is mandatory to credit and pass on the original author (Stability AI) and the license in unmodified form and it is forbidden to turn software created in this way into closed source.
According to StabilityAI CEO Emad Mostaque, StableLM aims to provide an open, transparent, and scalable alternative to proprietary AI models like those of OpenAI. StableLM plans to release models with 15 to 65 billion parameters in the foreseeable future. Models of the StableLM series should be able to generate text and source code, and are appropriate for numerous applications based on them.
The LLMs are relatively small, but Stability wants to show that even smaller models are capable of high performance provided they undergo appropriate training and have an efficient architecture. Previously, StableLM had supported the work of AI grassroots EleutherAI, who released a series of smaller AI base models for research with Pythia in early April 2023.
However, there is a problem with the dataset, which is used to train StableLM. The well-known 800-gigabyte dataset “The Pile” is considered problematic because it probably also contains works protected by copyright. Another open source project, RedPajama, plans to release a state-of-the-art, open-source model series with strong performance values and thus rebuild the unreleased LLaMA under free license.
StableLM is part of the open-source AI model movement that is now emerging. Stability AI provides three keywords for goal setting: transparency, accessibility, and support. The models are there to support users, not to replace them. It is about efficient, specialized, and practical AI applications that can also be implemented with smaller models. According to the statement on the blog, StableLM focuses on everyday applications that increase productivity and allows people to be more creative.
StableLM is available on Stability AI’s GitHub repository. A technical report and benchmarks for the performance comparison are not yet available, but should be submitted “in the near future”. Concurrent with the release, a crowdsourcing program for Human Feedback Reinforcement Learning (RLHF) will begin. Community work such as OpenAssistant, whose project has published a high-quality, quality-assured, and freely accessible basic data set for AI assistants in a collaborative effort, serves as a model.
