Skip to main content

Understanding Distance Metrics in Machine Learning with PyTorch Examples

Distance metrics play a crucial role in machine learning, especially in tasks like clustering, classification, and recommendation systems. In this blog, we will explore popular distance metrics including Cosine, Euclidean, Mahalanobis, Hellinger, Jaccard, Manhattan, Correlation, Dice, and Hamming distances. We will also provide PyTorch implementations for each metric.

1. Cosine Distance

Measures the cosine of the angle between two non-zero vectors. Often used in text similarity and document clustering.

import torch
x = torch.tensor([1.0, 2.0, 3.0])
y = torch.tensor([4.0, 5.0, 6.0])
cosine_distance = 1 - torch.nn.functional.cosine_similarity(x.unsqueeze(0), y.unsqueeze(0))

2. Euclidean Distance

Represents the straight-line distance between two points in Euclidean space.

euclidean_distance = torch.dist(x, y, p=2)

3. Mahalanobis Distance

Accounts for the correlation between variables and scales distances accordingly. Useful in anomaly detection.

cov = torch.cov(torch.stack([x, y]).T)
cov_inv = torch.linalg.inv(cov)
diff = (x - y).unsqueeze(0)
mahalanobis_distance = torch.sqrt(diff @ cov_inv @ diff.T)

4. Hellinger Distance

Measures the similarity between two probability distributions.

px = torch.sqrt(x / x.sum())
py = torch.sqrt(y / y.sum())
hellinger_distance = torch.norm(px - py) / torch.sqrt(torch.tensor(2.0))

5. Jaccard Distance

Used for comparing similarity and diversity of sample sets. Defined as 1 - (intersection / union).

x_set = torch.tensor([1, 1, 0, 0])
y_set = torch.tensor([1, 0, 1, 0])
intersection = torch.sum((x_set & y_set).float())
union = torch.sum((x_set | y_set).float())
jaccard_distance = 1 - intersection / union

6. Manhattan Distance

Also known as L1 distance. The sum of absolute differences between corresponding elements.

manhattan_distance = torch.sum(torch.abs(x - y))

7. Correlation Distance

Measures dissimilarity between variables by 1 minus the Pearson correlation coefficient.

correlation_distance = 1 - torch.corrcoef(torch.stack([x, y]))[0, 1]

8. Dice Distance

Mainly used in comparing similarity between two sets. Defined as 1 - (2 * |A ∩ B| / (|A| + |B|)).

intersection = torch.sum((x_set & y_set).float())
dice_distance = 1 - (2 * intersection) / (x_set.sum() + y_set.sum())

9. Hamming Distance

Measures the number of positions at which corresponding elements differ.

hamming_distance = torch.sum(x_set != y_set).float() / x_set.numel()

References

  • https://pytorch.org/docs/stable/index.html
  • https://en.wikipedia.org/wiki/Distance_metric
  • https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise_distances.html
  • https://en.wikipedia.org/wiki/Cosine_similarity

Comments

Post a Comment

Popular

Building an MCP Agent with UV, Python & mcp-use

Model Context Protocol (MCP) is an open protocol designed to enable AI agents to interact with external tools and data in a standardized way. MCP is composed of three components: server , client , and host . MCP host The MCP host acts as the interface between the user and the agent   (such as Claude Desktop or IDE) and plays the role of connecting to external tools or data through MCP clients and servers. Previously, Anthropic’s Claude Desktop was introduced as a host, but it required a separate desktop app, license, and API key management, leading to dependency on the Claude ecosystem.   mcp-use is an open-source Python/Node package that connects LangChain LLMs (e.g., GPT-4, Claude, Groq) to MCP servers in just six lines of code, eliminating dependencies and supporting multi-server and multi-model setups. MCP Client The MCP client manages the MCP protocol within the host and is responsible for connecting to MCP servers that provide the necessary functions for the ...
Post a Comment
Read more

How to Save and Retrieve a Vector Database using LangChain, FAISS, and Gemini Embeddings

How to Save and Retrieve a Vector Database using LangChain, FAISS, and Gemini Embeddings Efficient storage and retrieval of vector databases is foundational for building intelligent retrieval-augmented generation (RAG) systems using large language models (LLMs). In this guide, we’ll walk through a professional-grade Python implementation that utilizes LangChain with FAISS and Google Gemini Embeddings to store document embeddings and retrieve similar information. This setup is highly suitable for advanced machine learning (ML) and deep learning (DL) engineers who work with semantic search and retrieval pipelines. Why Vector Databases Matter in LLM Applications Traditional keyword-based search systems fall short when it comes to understanding semantic meaning. Vector databases store high-dimensional embeddings of text data, allowing for approximate nearest-neighbor (ANN) searches based on semantic similarity. These capabilities are critical in applications like: Question Ans...
Post a Comment
Read more

Using Gemini API in LangChain: Step-by-Step Tutorial

What is LangChain and Why Use It? LangChain  is an open-source framework that simplifies the use of  Large Language Models (LLMs)  like OpenAI, Gemini (Google), and others by adding structure, tools, and memory to help build real-world applications such as chatbots, assistants, agents, or AI-enhanced software. Why Use LangChain for LLM Projects? Chainable Components : Easily build pipelines combining prompts, LLMs, tools, and memory. Multi-Model Support : Work with Gemini, OpenAI, Anthropic, Hugging Face, etc. Built-in Templates : Manage prompts more effectively. Supports Multi-Turn Chat : Manage complex interactions with memory and roles. Tool and API Integration : Let the model interact with external APIs or functions. Let's Walk Through the Code: Gemini + LangChain I will break the code into  4 main parts , each showcasing different features of LangChain and Gemini API. Part 1: Basic Gemini API Call Using LangChain import os from dotenv import load_dotenv load_dot...
Post a Comment
Read more