Skip to main content

Understanding Distance Metrics in Machine Learning with PyTorch Examples

Distance metrics play a crucial role in machine learning, especially in tasks like clustering, classification, and recommendation systems. In this blog, we will explore popular distance metrics including Cosine, Euclidean, Mahalanobis, Hellinger, Jaccard, Manhattan, Correlation, Dice, and Hamming distances. We will also provide PyTorch implementations for each metric.

1. Cosine Distance

Measures the cosine of the angle between two non-zero vectors. Often used in text similarity and document clustering.

import torch
x = torch.tensor([1.0, 2.0, 3.0])
y = torch.tensor([4.0, 5.0, 6.0])
cosine_distance = 1 - torch.nn.functional.cosine_similarity(x.unsqueeze(0), y.unsqueeze(0))

2. Euclidean Distance

Represents the straight-line distance between two points in Euclidean space.

euclidean_distance = torch.dist(x, y, p=2)

3. Mahalanobis Distance

Accounts for the correlation between variables and scales distances accordingly. Useful in anomaly detection.

cov = torch.cov(torch.stack([x, y]).T)
cov_inv = torch.linalg.inv(cov)
diff = (x - y).unsqueeze(0)
mahalanobis_distance = torch.sqrt(diff @ cov_inv @ diff.T)

4. Hellinger Distance

Measures the similarity between two probability distributions.

px = torch.sqrt(x / x.sum())
py = torch.sqrt(y / y.sum())
hellinger_distance = torch.norm(px - py) / torch.sqrt(torch.tensor(2.0))

5. Jaccard Distance

Used for comparing similarity and diversity of sample sets. Defined as 1 - (intersection / union).

x_set = torch.tensor([1, 1, 0, 0])
y_set = torch.tensor([1, 0, 1, 0])
intersection = torch.sum((x_set & y_set).float())
union = torch.sum((x_set | y_set).float())
jaccard_distance = 1 - intersection / union

6. Manhattan Distance

Also known as L1 distance. The sum of absolute differences between corresponding elements.

manhattan_distance = torch.sum(torch.abs(x - y))

7. Correlation Distance

Measures dissimilarity between variables by 1 minus the Pearson correlation coefficient.

correlation_distance = 1 - torch.corrcoef(torch.stack([x, y]))[0, 1]

8. Dice Distance

Mainly used in comparing similarity between two sets. Defined as 1 - (2 * |A ∩ B| / (|A| + |B|)).

intersection = torch.sum((x_set & y_set).float())
dice_distance = 1 - (2 * intersection) / (x_set.sum() + y_set.sum())

9. Hamming Distance

Measures the number of positions at which corresponding elements differ.

hamming_distance = torch.sum(x_set != y_set).float() / x_set.numel()

References

  • https://pytorch.org/docs/stable/index.html
  • https://en.wikipedia.org/wiki/Distance_metric
  • https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise_distances.html
  • https://en.wikipedia.org/wiki/Cosine_similarity

Comments

Post a Comment

Popular

How to Fine-Tune LLaMA 3.2-1B-Instruct for Korean Instruction Tasks with LoRA and Hugging Face

LLaMA 3.2-1B-Instruct is a lightweight instruction-tuned language model released by Meta. It is designed to handle a wide range of instruction-based tasks with relatively low computational resources. Although the model was trained with multilingual capabilities, its performance on languages not included in its training set—such as Korean—is limited. This tutorial demonstrates how to fine-tune this open-source model on a Korean dataset using Hugging Face Transformers and PEFT (specifically LoRA), enabling it to better respond to Korean instructions. 1. Prerequisites Before running the example code below, ensure you have the following libraries installed: pip install torch transformers datasets peft accelerate mlflow huggingface_hub To use the LLaMA model or KoAlpaca datasets, you'll need a Hugging Face token. Additionally, you may need to handle potential CUDA Out-Of-Memory (OOM) errors. The following code takes care of both: from huggingface_hub import login login("y...
Post a Comment
Read more

Using Gemini API in LangChain: Step-by-Step Tutorial

What is LangChain and Why Use It? LangChain  is an open-source framework that simplifies the use of  Large Language Models (LLMs)  like OpenAI, Gemini (Google), and others by adding structure, tools, and memory to help build real-world applications such as chatbots, assistants, agents, or AI-enhanced software. Why Use LangChain for LLM Projects? Chainable Components : Easily build pipelines combining prompts, LLMs, tools, and memory. Multi-Model Support : Work with Gemini, OpenAI, Anthropic, Hugging Face, etc. Built-in Templates : Manage prompts more effectively. Supports Multi-Turn Chat : Manage complex interactions with memory and roles. Tool and API Integration : Let the model interact with external APIs or functions. Let's Walk Through the Code: Gemini + LangChain I will break the code into  4 main parts , each showcasing different features of LangChain and Gemini API. Part 1: Basic Gemini API Call Using LangChain import os from dotenv import load_dotenv load_dot...
Post a Comment
Read more

Building an MCP Agent with UV, Python & mcp-use

Model Context Protocol (MCP) is an open protocol designed to enable AI agents to interact with external tools and data in a standardized way. MCP is composed of three components: server , client , and host . MCP host The MCP host acts as the interface between the user and the agent   (such as Claude Desktop or IDE) and plays the role of connecting to external tools or data through MCP clients and servers. Previously, Anthropic’s Claude Desktop was introduced as a host, but it required a separate desktop app, license, and API key management, leading to dependency on the Claude ecosystem.   mcp-use is an open-source Python/Node package that connects LangChain LLMs (e.g., GPT-4, Claude, Groq) to MCP servers in just six lines of code, eliminating dependencies and supporting multi-server and multi-model setups. MCP Client The MCP client manages the MCP protocol within the host and is responsible for connecting to MCP servers that provide the necessary functions for the ...
Post a Comment
Read more