infomeasure
information-theoretic measures and estimators

Entropy, mutual information, transfer entropy and divergences, in Python and Rust.

One estimator model with two implementations and numerically matched results. Explore in Python, deploy the same measures in Rust.

$ pip install infomeasure

Measures & approaches

Every measure is available through each approach, with bias-corrected variants where they exist, and divergences built on the same estimators.

H(X,Y)Joint entropyPython guideRust guide
H(X|Y)Conditional entropyPython guideRust guide
HQ(P)Cross-entropyPython guideRust guide
I(X;Y)Mutual informationPython guideRust guide
I(X;Y|Z)Conditional mutual informationPython guideRust guide
TX→YTransfer entropyPython guideRust guide
TX→Y|ZConditional transfer entropyPython guideRust guide
DKL(P‖Q)Kullback–Leibler divergencePython guideRust guide
JSD(P‖Q)Jensen–Shannon divergencePython guideRust guide

Estimators

Four estimator families, each paired with every measure. Discrete carries many bias-corrected MLE variants. Measures × estimators form a Cartesian product, with documented per-approach exceptions.

DiscreteCategorical data, MLE plus many bias-corrected estimators (Miller–Madow, Bayesian, shrinkage, …)
KernelContinuous data, box and Gaussian kernels
k-NN / KSGContinuous data, Kozachenko–Leonenko entropy and Kraskov–Stögbauer–Grassberger MI
OrdinalTime series via permutation patterns
RényiGeneralized entropy / divergence, order α, via the nearest-neighbour family
TsallisGeneralized entropy / divergence, parameter q, via the nearest-neighbour family

Which package should I use?

There are two packages. The Python package is the reference implementation, for notebooks, scripts and teaching. The Rust crate computes the same measures and is built for speed and deployment. Choose by how you work and where it runs.

PythonRust
Where it fitsNotebooks and scripts, teaching, statistical tests, composite measuresPerformance-critical code, portable binaries, embedding in services
Installpip install infomeasure or conda install -c conda-forge infomeasurecargo add infomeasure
RegistryPyPI and conda-forgecrates.io
Documentationinfomeasure.readthedocs.iodocs.rs/infomeasure
AccelerationGPU via numbaOptional GPU (wgpu) and CPU threads (rayon)
RoleReference implementationSame estimators, parity-tested

Start in a few lines

The API reads the same in both languages: choose a measure, choose an approach, read the value.

use infomeasure::estimators::entropy::Entropy;
use infomeasure::estimators::traits::GlobalValue;

let data = vec![1, 2, 1, 3, 2, 1];
let h = Entropy::new_discrete(data).global_value();
from infomeasure import entropy

h = entropy([1, 2, 1, 3, 2, 1], approach="discrete")

See it measured

On identical inputs, pinned to a single thread. Entropy runtime versus sample size for infomeasure-rs (solid) and the Python reference (dashed), by estimator family, lower is better.

Open the detailed viewer
Entropy runtime versus sample size for infomeasure-rs and the Python package, by estimator family
entropy runtime vs sample size N, infomeasure-rs vs Python referencedetailed viewer
Mutual information runtime versus sample size across toolkits
mutual information runtime vs sample size across toolkits, infomeasure in colour, peers greycross-package viewer

Citation

C. M. Büth et al. infomeasure: information-theoretic measures and estimators. Scientific Reports (2025). https://doi.org/10.1038/s41598-025-14053-5
Release archive: Zenodo · machine-readable citation in CITATION.cff.