Matchers

Token Matching Utilities for jnkn.

Refactored to implement TokenMatcherProtocol for FFI readiness. Includes robust filtering and overlap calculation logic.

Classes

TokenMatcher

Pure Python implementation of TokenMatcherProtocol.

Functions

calculate_overlap(tokens1, tokens2)

Calculate token overlap.

calculate_significant_overlap(tokens1, tokens2)

Calculate overlap using only significant tokens.

This convenience method chains filtering and overlap calculation, restoring the API expected by the tests.

get_significant_tokens(tokens)

Filter tokens to only significant ones.

tokenize(name)

Split a name into tokens.

Functions

create_default_matcher(config_path=None)

Factory that returns an object satisfying TokenMatcherProtocol. Ideally, check a feature flag here to return Rust implementation later.