2. Categorisation

Categorisation

Note: If you find any issues with this type of ML techniques taxonomy, please let me know what you think.

Due to explosion of ML techniques and approaches it is always nice to get the sense on how it all relates to each other. Defining the hierarchy always help to build a decent mind map on where exactly to correlate specific idea. This setup is how the categories are considered along blog posts.

Almost all value (in terms of monetizing, better products etc.) in MODERN (as of fall 2017) ML industry is driven by three first general categories (that I will also use for articles across this blog):

Categories:🔗

  1. General DL (fully connected - FC; densely connected)
  2. Sequence Models ( 1D sequences; eg. RNN, LSTM, GRU, attention models)
  3. Computer Vision ( 2D/3D models; eg. CNN, ViTs)
  4. Reinforcement Learning (RL)

  5. Optimization Tips and tricks used to optimize learning process and fine-tuning LLMs.

  6. ML (e.g. supervised learning, reinforced learning, Sparse coding, Slow Feature Analysis (SFA), Independent Component Analysis (ICA) etc.)

  7. ML Dojo (jap. 道場 dōjō) This category will be used for content with coding tasks and learning tools.

  8. Applications inspiring and interesting news on how ML has been applied for another area of our lives.

  9. Other This category is not strictly ML related and covers some interesting topics that I have encountered.

This is how Kaggle sees Statistical and Machine Learning Algorithms The theoretical view and classification of ML algorithms is something like the hierarchy below.

  • Problems🔗

    Tags: Classification, Clustering, Regression, Anomaly detection, Association rules, Reinforcement learning, Structured prediction, Feature engineering Feature learning, Online learning, Semi-supervised learning, Unsupervised learning, Learning to rank, Grammar induction, Pattern recognition

Taxonomy:🔗

  • Supervised learning🔗

    Tags: classification • regression • Decision trees • Ensembles (Bagging, Boosting, Random forest) • k-NN • Linear regression • Naive Bayes • Neural networks • Logistic regression • Perceptron • Relevance vector machine (RVM) • Support vector machine (SVM)

    • UnSupervised learning considered as tag, not a category🔗

      Depending, if the desired output is already known algorithm is often also referred to as "Unsupervised". However, it is hard to put a clear distinction on what is unsupervised in terms of classifying groups of approaches. Therefore we tag algorithms that are part of some category, but additionally are considered Unsupervised. E.g.
      • Clustering: k-means, Hierarchical Cluster Analysis (HCA)
      • Neural Networks Hebbian Learning, Generative Adversarial Networks (GAN)
      • Mixture models Expectation-maximization (EM), Gaussian mixture model, Multivariate Gaussian mixture model, Categorical mixture model

  • Clustering🔗

    Tags: BIRCH Hierarchical • k-means • Expectation-maximization (EM) • DBSCAN • OPTICS • Mean-shift

  • Anomaly detection🔗

    Tags: k-NN • Local outlier factor

  • Dimensionality reduction🔗

    Tags: Factor analysis • CCA • ICA • LDA • NMF • PCA • t-SNE

  • Structured prediction🔗

    Tags: Graphical models (Bayes net, CRF, HMM)

  • Neural nets🔗

    Tags: Autoencoder • Deep learning • Multilayer perceptron • RNN • Restricted Boltzmann machine • SOM • Convolutional Neural Network(CNN) • Generative Adversarial Networks (GAN)

  • Reinforcement Learning🔗

    Tags: Q-Learning • SARSA • Temporal Difference (TD)

  • Theory🔗

    Tags: Bias-variance dilemma • Computational learning theory • Empirical risk minimization • Occam learning • PAC learning • Statistical learning • VC theory

  • Machine learning venues🔗

    NIPS ICML ML JMLR ICLR ArXiv:cs.LG