Questions for Exploration: Human Intelligence and AI
Open questions about thought, learning, and the evolution of intelligence, organized with related articles, papers and projects.
Questions for Exploration: Human Intelligence and AI
Open questions about thought, learning, and the evolution of intelligence, organized with related articles, papers and projects.
About Machine Learning ( Part 10: Reinforcement Learning )
Reinforcement Learning (RL) is a fascinating branch of machine learning where an agent learns to interact with an environment to maximize long-term cumulative rewards. Unlike supervised learning, RL relies on feedback through interaction instead of labeled data.
The core of RL is built upon Markov Decision Processes (MDPs), which provide a mathematical framework for modeling decision-making under uncertainty.
This blog post explores the key components of RL, including value functions, Q-functions, the Bellman equation, Actor-Critic architectures, PPO, and commonly used tools in real-world RL implementations.
Transformer ( Part 3: Transformer Architecture )
The Transformer consists of two main parts: an encoder and a decoder. They are connected by Cross-Attention.
The Transformer Architecture:

Transformer ( Part 2: Multi-Head Attention )
Before the Transformer, sequence models like RNNs and LSTMs suffered from long-term dependency issues and low parallelization efficiency. Self-Attention was introduced as an alternative, allowing for parallel computation and capturing long-range dependencies.
However, a single-head Self-Attention mechanism has a limitation:
It can only focus on one type of relationship or pattern in the data.
Multi-Head Attention overcomes this by using multiple attention heads that capture different aspects of the input, improving the model’s expressiveness.
About Machine Learning ( Part 9: Recurrent Neural Network )
Recurrent Neural Networks (RNNs) are a class of neural networks designed for sequential data, making them highly effective for tasks like natural language processing (NLP), time series prediction, and speech recognition. Unlike traditional feedforward networks, RNNs maintain a hidden state that captures temporal dependencies.
A traditional feedforward neural network processes inputs independently. However, for sequential tasks, the order of the data is crucial. RNNs address this by maintaining a memory of previous inputs through hidden states.
About Machine Learning ( Part 8: Convolution Neural Networks )
Convolutional Neural Networks (CNNs) have revolutionized the field of computer vision, enabling significant advancements in image recognition, object detection, and segmentation tasks. This blog will explore the key concepts behind CNNs and their working principles.
A Convolutional Neural Network (CNN) is a type of deep learning model specifically designed for processing structured grid data, such as images. Unlike traditional fully connected neural networks, CNNs leverage convolutional layers to capture spatial hierarchies in the data.
Transformer ( Part 1: Word Embedding )
Word Embedding is one of the most fundamental techniques in Natural Language Processing (NLP). It represents words as continuous vectors in a high-dimensional space, capturing semantic relationships between them.
Before word embeddings, one common method to represent words was One-Hot Encoding. In this approach, each word is represented as a high-dimensional sparse vector.
For example, if our vocabulary has 10,000 words, we encode each word as:
$$
\text{dog} = [0, 1, 0, 0, \dots, 0]
$$
However, this method has significant drawbacks:
Word embeddings solve these issues by learning low-dimensional, dense representations that encode semantic relationships between words.

About Machine Learning ( Part 7: Artificial Neural Network )
$$
P(y|X) = \frac{P(X|y) P(y)}{P(X)}
$$
where:
A Bayesian network (BN) is a graphical model representing probabilistic dependencies between variables. It consists of:

About Machine Learning ( Part 6: KNN vs. K-means )
In machine learning, K-Nearest Neighbors (KNN) and K-means Clustering are two commonly used algorithms. Despite their similar names, they serve different purposes and have distinct working principles.
KNN is a supervised learning algorithm used for classification and regression tasks.
The core idea of KNN is:
Given a new data point, find the K most similar instances in the training dataset (neighbors) and use them to predict the output.
KNN is a lazy learning algorithm, meaning it does not require a training phase. Instead, it directly classifies or predicts based on stored data.
About Machine Learning ( Part 5: Support Vector Machine )
Support Vector Machines (SVM) are one of the most powerful supervised learning algorithms used for classification and regression tasks.
In a binary classification problem, the goal of SVM is to find a hyperplane that best separates two classes. Given a training dataset:
$$
D = { (\mathbf{x}_1, y_1), (\mathbf{x}_2, y_2), \dots, (\mathbf{x}_n, y_n) }, \quad \mathbf{x}_i \in \mathbb{R}^d, \quad y_i \in {-1, +1}
$$