Per-base, BPE, and 6-mer on the same DNA sequence

§ Widget · 02 Per-base, BPE, and 6-mer on the same DNA sequence

Press play to watch the same sequence get cut three different ways — per-base, with a toy BPE vocabulary (greedy longest-match), and with non-overlapping 6-mers. The cuts appear left-to-right on the same time axis, so divergence between the schemes is visible directly.

Sequence 0 bp

Per-base Tokens: 0 Compression: 1.0

BPE Tokens: 0 Compression: −

6-mer Tokens: 0 Compression: −