Motivation In recent years, we use AI for more and more use cases, interacting with models that provide us with remarkable outputs. As we move forward, the models we use are getting larger and larger, and so, an important research domain is to improve the efficiency of using and training AI models. Standard MoE Is […]
Mixture of Nested Experts: Adaptive Processing of Visual Tokens Read More »
Up until vision transformers were invented, the dominating model architecture in computer vision was convolutional neural network (CNN), which was invented at 1989 by famous researchers including Yann LeCun and Yoshua Bengio. At 2017, transformers were invented by Google and took the natural language processing domain by storm, but were not adapted successfully to computer
How Do Vision Transformers Work? Read More »
OPRO (Optimization by PROmpting), is a new approach to leverage large language models as optimizers, which was introduced by Google DeepMind in a research paper titled “Large Language Models As Optimizers”. Large language models are very good at getting a prompt, such as an instruction or a question, and yield a useful response that match
Large Language Models As Optimizers – OPRO by Google DeepMind Read More »
