Skip to main content

Understanding Distance Metrics in Machine Learning with PyTorch Examples

Distance metrics play a crucial role in machine learning, especially in tasks like clustering, classification, and recommendation systems. In this blog, we will explore popular distance metrics including Cosine, Euclidean, Mahalanobis, Hellinger, Jaccard, Manhattan, Correlation, Dice, and Hamming distances. We will also provide PyTorch implementations for each metric.

1. Cosine Distance

Measures the cosine of the angle between two non-zero vectors. Often used in text similarity and document clustering.

import torch
x = torch.tensor([1.0, 2.0, 3.0])
y = torch.tensor([4.0, 5.0, 6.0])
cosine_distance = 1 - torch.nn.functional.cosine_similarity(x.unsqueeze(0), y.unsqueeze(0))

2. Euclidean Distance

Represents the straight-line distance between two points in Euclidean space.

euclidean_distance = torch.dist(x, y, p=2)

3. Mahalanobis Distance

Accounts for the correlation between variables and scales distances accordingly. Useful in anomaly detection.

cov = torch.cov(torch.stack([x, y]).T)
cov_inv = torch.linalg.inv(cov)
diff = (x - y).unsqueeze(0)
mahalanobis_distance = torch.sqrt(diff @ cov_inv @ diff.T)

4. Hellinger Distance

Measures the similarity between two probability distributions.

px = torch.sqrt(x / x.sum())
py = torch.sqrt(y / y.sum())
hellinger_distance = torch.norm(px - py) / torch.sqrt(torch.tensor(2.0))

5. Jaccard Distance

Used for comparing similarity and diversity of sample sets. Defined as 1 - (intersection / union).

x_set = torch.tensor([1, 1, 0, 0])
y_set = torch.tensor([1, 0, 1, 0])
intersection = torch.sum((x_set & y_set).float())
union = torch.sum((x_set | y_set).float())
jaccard_distance = 1 - intersection / union

6. Manhattan Distance

Also known as L1 distance. The sum of absolute differences between corresponding elements.

manhattan_distance = torch.sum(torch.abs(x - y))

7. Correlation Distance

Measures dissimilarity between variables by 1 minus the Pearson correlation coefficient.

correlation_distance = 1 - torch.corrcoef(torch.stack([x, y]))[0, 1]

8. Dice Distance

Mainly used in comparing similarity between two sets. Defined as 1 - (2 * |A ∩ B| / (|A| + |B|)).

intersection = torch.sum((x_set & y_set).float())
dice_distance = 1 - (2 * intersection) / (x_set.sum() + y_set.sum())

9. Hamming Distance

Measures the number of positions at which corresponding elements differ.

hamming_distance = torch.sum(x_set != y_set).float() / x_set.numel()

References

  • https://pytorch.org/docs/stable/index.html
  • https://en.wikipedia.org/wiki/Distance_metric
  • https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise_distances.html
  • https://en.wikipedia.org/wiki/Cosine_similarity

Comments

Post a Comment

Popular

How to Fine-Tune LLaMA 3.2-1B-Instruct for Korean Instruction Tasks with LoRA and Hugging Face

LLaMA 3.2-1B-Instruct is a lightweight instruction-tuned language model released by Meta. It is designed to handle a wide range of instruction-based tasks with relatively low computational resources. Although the model was trained with multilingual capabilities, its performance on languages not included in its training set—such as Korean—is limited. This tutorial demonstrates how to fine-tune this open-source model on a Korean dataset using Hugging Face Transformers and PEFT (specifically LoRA), enabling it to better respond to Korean instructions. 1. Prerequisites Before running the example code below, ensure you have the following libraries installed: pip install torch transformers datasets peft accelerate mlflow huggingface_hub To use the LLaMA model or KoAlpaca datasets, you'll need a Hugging Face token. Additionally, you may need to handle potential CUDA Out-Of-Memory (OOM) errors. The following code takes care of both: from huggingface_hub import login login("y...
Post a Comment
Read more

RF-DETR: Overcoming the Limitations of DETR in Object Detection

RF-DETR (Region-Focused DETR), proposed in April 2025, is an advanced object detection architecture designed to overcome fundamental drawbacks of the original DETR (DEtection TRansformer) . In this technical article, we explore RF-DETR's contributions, architecture, and how it compares with both DETR and the improved model D-FINE . We also provide experimental benchmarks and discuss its real-world applicability. RF-DETR Architecture diagram for object detection Limitations of DETR DETR revolutionized object detection by leveraging the Transformer architecture, enabling end-to-end learning without anchor boxes or NMS (Non-Maximum Suppression). However, DETR has notable limitations: Slow convergence, requiring heavy data augmentation and long training schedules Degraded performance on low-resolution objects and complex scenes Lack of locality due to global self-attention mechanisms Key Innovations in RF-DETR RF-DETR intr...
Post a Comment
Read more

Using Gemini API in LangChain: Step-by-Step Tutorial

What is LangChain and Why Use It? LangChain  is an open-source framework that simplifies the use of  Large Language Models (LLMs)  like OpenAI, Gemini (Google), and others by adding structure, tools, and memory to help build real-world applications such as chatbots, assistants, agents, or AI-enhanced software. Why Use LangChain for LLM Projects? Chainable Components : Easily build pipelines combining prompts, LLMs, tools, and memory. Multi-Model Support : Work with Gemini, OpenAI, Anthropic, Hugging Face, etc. Built-in Templates : Manage prompts more effectively. Supports Multi-Turn Chat : Manage complex interactions with memory and roles. Tool and API Integration : Let the model interact with external APIs or functions. Let's Walk Through the Code: Gemini + LangChain I will break the code into  4 main parts , each showcasing different features of LangChain and Gemini API. Part 1: Basic Gemini API Call Using LangChain import os from dotenv import load_dotenv load_dot...
Post a Comment
Read more