Stability AI introduced a neural network for generating and explaining code

Stability AI has just announced the release of StableCode, its first LLM-generative AI coding product. This product is designed to help coders in their day-to-day work, as well as being a great learning tool for aspiring developers ready to take their coding to the next level.

StableCode offers a unique way to increase developer efficiency by utilizing three different models to assist in writing code. The base model was first trained on a diverse set of programming languages from BigCode's stack-dataset (v1.2), and then further trained on popular languages such as Python, Go, Java, Javascript, C, markdown, and C++. In total, we trained our models on 560 B lexemes of code on our high performance computing cluster.

Once the base model was created, the instruction model was customized for specific use cases to solve complex programming problems. To achieve this result, ~120,000 instruction/response pairs of code in Alpaca format were trained on the base model.

StableCode is the ideal building block for those who want to learn more about coding, and the long context window model is the perfect helper for making single-line and multi-line autocomplete suggestions accessible to the user. This model is designed to handle a large amount of code at once (2-4 times larger than previously released open source models with a 16,000 token context window), allowing the user to view or edit the equivalent of five medium-sized Python files simultaneously, making it an ideal learning tool for beginners looking to move on to more serious tasks.

And here is how we compare to other models with a similar number of parameters and number of trained tokens. We use the standard pass@1 and pass@10 metrics using the popular HumanEval benchmark.

Stability AI's goal is to make technology more accessible, and StableCode is a significant step towards that goal. Soon, people of all backgrounds will be able to create code to solve their everyday problems and improve their lives using artificial intelligence, and we'd like to help make that happen. We hope that StableCode will help the next billion software developers learn how to write code and provide more equitable access to technology around the world.