Meta-Learning for Medium-shot Sparse Learning via Deep Kernels

Computer and Knowledge Engineering

مقاله 5، دوره 5، شماره 2 - شماره پیاپی 10، اسفند 2022، صفحه 45-56   

نوع مقاله: Original Article

شناسه دیجیتال (DOI): 10.22067/cke.2022.77529.1060

نویسندگان

Zohreh Adabi Firuzjaee؛ Sayed Kamaledin Ghiasi-Shirazi^*

Department of Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran

چکیده

Few-shot learning assumes that we have a very small dataset for each task and trains a model on the set of tasks. For real-world problems, however, the amount of available data is substantially much more; we call this a medium-shot setting, where the dataset often has several hundreds of data. Despite their high accuracy, deep neural networks have a drawback as they are black-box. Learning interpretable models has become more important over time. This study aims to obtain sample-based interpretability using the attention mechanism. The main idea is reducing the task training data into a small number of support vectors using sparse kernel methods, and the model then predicts the test data of the task based on these support vectors. We propose a sparse medium-shot learning algorithm based on a metric-based Bayesian meta-learning algorithm whose output is probabilistic. Sparsity, along with uncertainty, effectively plays a key role in interpreting the model's behavior. In our experiments, we show that the proposed method provides significant interpretability by selecting a small number of support vectors and, at the same time, has a competitive accuracy compared to other less interpretable methods.

کلیدواژه‌ها

Bayesian Meta-Learning؛ Medium-shot Learning؛ Sample-based Interpretability؛ Sparse Kernel؛ Attention