Several months ago, Microsoft released the first version of Orca, which achieved remarkable results, even surpassing ChatGPT on data from BigBench-Hard dataset, and the ideas from Orca 1 helped to create better language models released in the recent period. The Orca 2 model, presented in the paper we review in this post, achieves significantly better […]
Orca 2: Teaching Small Language Models How to Reason Read More »
CODEFUSION is a new code generation model which was introduced in a research paper from Microsoft, titled: “CODEFUSION: A Pre-trained Diffusion Model for Code Generation”. Recently, we’ve observed a significant progress with code generation using AI, which is mostly based on large language models (LLMs), so we refer to them as code LLMs. With a
CODEFUSION: A Pre-trained Diffusion Model for Code Generation Read More »
Nowadays, we are witnessing a tremendous progress with large language models (LLMs) such as ChatGPT, Llama and more, where we can feed a LLM with a text instruction or question, and most of the times get an accurate response from the model. However, if we’ll try to feed the model with a table data, in
Table-GPT: Empower LLMs To Understand Tables Read More »
OPRO (Optimization by PROmpting), is a new approach to leverage large language models as optimizers, which was introduced by Google DeepMind in a research paper titled “Large Language Models As Optimizers”. Large language models are very good at getting a prompt, such as an instruction or a question, and yield a useful response that match
Large Language Models As Optimizers – OPRO by Google DeepMind Read More »
Code Llama is a new family of open-source large language models for code by Meta AI that includes three type of models. Each type was released with 7B, 13B and 34B params. In this post we’ll explain the research paper behind them, titled “Code Llama: Open Foundation Models for Code”, to understand how these models
Code Llama Paper Explained Read More »
Welcome WizardMath, a new open-source large language model contributed by Microsoft. While top large language models such as GPT-4 have demonstrated remarkable capabilities in various tasks including mathematical reasoning, they are not open-source. And for open-source large language models such as LLaMA-2 the situation is different, and until now they did not demonstrate strong math
In this post we dive into Orca’s research paper from Microsoft, where they present a language model which was trained with a novel approach of imitation learning and achieved very impressive results. For example, in above chart which measures performance on Vicuna evaluation set using GPT-4 as the judge, where results are visualized with relative
Orca Research Paper Explained Read More »
In this post we review LongNet, a new research paper by Microsoft titled “LongeNet: Scaling Transformers to 1,000,000,000 Tokens”. The paper starts with the above amusing chart that shows the trend of transformer sequence lengths over time in a non-logarithmic y axis and we can see LongNet is way above with its one billion tokens.
LongNet: Scaling Transformers to 1B Tokens with Dilated Attention Read More »
LIMA, Less Is More for Alignment , is a LLM which was created by Meta AI by fine-tuning the LLaMa model on only 1000 samples, and achieved competitive results with top large language models such as GPT-4, Bard and Alpaca. In this post, our goal is to explain the research paper by discussing what the
LIMA from Meta AI – Less Is More for Alignment of LLMs Read More »
In this post we will explore Shepherd, a new research paper by Meta AI titled Shepherd: A Critic for Language Model Generation. Large language models have become increasingly sophisticated and given a prompt are able to generate a correct, beautiful and meaningful text. However, a remaining concern with large language models is their tendency to
Shepherd: A Critic for Language Model Generation Read More »
