Meta Unveils AI-Powered ‘Code Llama’ Language Model for Coding

Today marks a significant leap forward as Meta unveils Code Llama, a cutting-edge large language model (LLM) capable of generating code from text prompts.

Representing the forefront of publicly available LLMs for coding tasks, Code Llama has the potential to revolutionize workflows, making them more accessible for coding learners.

This robust model can serve as both an educational aid and a productivity tool, facilitating the creation of well-documented, resilient software, according to Meta.

Born from the foundation of Llama 2, Code Llama has been tailored to the world of coding through focused training on code-specific datasets. Meta says this refinement involved extended data sampling from the same dataset.

In essence, Code Llama harnesses enhanced coding capabilities atop Llama 2’s architecture. It proficiently generates code and natural language pertaining to code, responding adeptly to prompts like, “Create a function that generates the fibonacci sequence.”

The model’s utility extends to code completion and debugging across a spectrum of popular programming languages, such as Python, C++, Java, and more.

Code Llama arrives in three variants with parameter sizes of 7B, 13B, and 34B. Each variant underwent training using a remarkable 500B tokens of code and code-related data.

Notably, the 7B and 13B base models possess the added capability of fill-in-the-middle (FIM), enabling immediate support for tasks like code completion.

These models cater to diverse needs in terms of serving and latency. The 7B model efficiently operates on a single GPU, while the 34B model yields superior results and elevated coding assistance.

Meanwhile, the smaller 7B and 13B models excel in low-latency tasks such as real-time code completion.

Further enhancing utility, Meta has introduced specialized variations: Code Llama – Python and Code Llama – Instruct.

Code Llama – Python, refined over 100B tokens of Python code, caters to Python aficionados. Code Llama – Instruct focuses on fine-tuning through natural language instructions, enhancing its ability to meet user expectations.

However, these specialized models are not intended for general natural language tasks. Meta says users must adhere to its licensing and acceptable use policy while leveraging Code Llama.

You can download the Code Llama Model by clicking here.