Scientist labeled a colony of bees with QR codes and observed what happened. Turned out, information flow in the colony is pretty effective, it's not just random. Also I learned of trophallaxis.

One can use gzip to classify data.

Deep neural networks (DNNs) are often used for text classification due to their high accuracy. However, DNNs can be computationally intensive, requiring millions of parameters and large amounts of labeled data, which can make them expensive to use, to optimize, and to transfer to out-of-distribution (OOD) cases in practice. In this paper, we propose a non-parametric alternative to DNNs that’s easy, lightweight, and universal in text classification: a combination of a simple compressor like gzip with a k-nearest-neighbor classifier. Without any training parameters, our method achieves results that are competitive with non-pretrained deep learning methods on six in-distribution datasets.It even outperforms BERT on all five OOD datasets, including four low-resource languages. Our method also excels in the few-shot setting, where labeled data are too scarce to train DNNs effectively.

Our method is a simple, lightweight, and uni- versal alternative to DNNs. It’s simple because it doesn’t require any preprocessing or training. It’s lightweight in that it classifies without the need for parameters or GPU resources. It’s universal as com- pressors are data-type agnostic, and non-parametric methods do not bring underlying assumptions.

Without any pre-training or fine-tuning, our method outperforms both BERT and mBERT on all five datasets.

Questioned:

• https://kenschutte.com/gzip-knn-paper/

Overfitting is bad. The best example in the article is the school system. Tests were introduced to measure students effectively. They were a good measure for that! Then schools started focusing on tests. You know what happened. Поколение ЕГЭ, cramming, etc.

Epoch, batch size, итерации - параметры для работы с большими объемами данных, когда нет возможности загрузить весь датасет в обработку.

Когда делал работу по мышиному обучению, эта статья чуток помогла после прочтения по диагонали.