This page explains the terms used in Diode Spectrum, an experiment about translating among language models’ internal representations. Each entry starts with the practical idea. The optional sections contain the notation.
FUV
FUV stands for fraction of unexplained variance. It measures how much of the target’s variation the reconstructed vectors failed to capture. Lower is better.
Tiny example: imagine predicting four dots on a page. FUV compares your total prediction error with the error from placing every prediction at the center of the four dots. Small misses produce a low FUV.
Move the FUV score
Drag the control. The only rule you need is: lower is better.
Better than always guessing the average. Lower is better.
FUV = 0 · nothing left unexplained
Every held-out target is reconstructed exactly.
FUV = 1 · equal to the average guess
The map's total squared error matches the error from predicting the held-out target mean for every prompt.
FUV > 1 · worse than the average guess
The map has more total squared error than that mean-prediction reference.
A low FUV establishes reconstructability under the chosen map, prompts, layer, pooling rule, and split. It says nothing by itself about overall model quality, interchangeable behavior, shared mechanisms, or semantic equivalence.
Show the FUV formula
For held-out target matrix $Y$ and reconstruction $\hat{Y}$:
$$ \operatorname{FUV}(\hat{Y},Y) = \frac{\sum_{i,j}(\hat{Y}_{ij}-Y_{ij})^2} {\sum_{i,j}(Y_{ij}-\bar{Y}_{\cdot j})^2}. $$
The top is reconstruction error. The bottom is the held-out target's variation around its coordinate-wise mean.
Adjusted FUV gap
The adjusted FUV gap asks how much correct prompt matching helps after accounting for a deliberately broken version of the experiment.
Tiny example: suppose the real map scores 0.80 FUV and shuffled prompt pairings usually score 1.10. The adjusted gap is 0.80 minus 1.10, or -0.30. The negative value says the correctly paired prompts reconstructed better.
A more negative gap means a larger advantage over the row-permutation baseline. The two directions can still differ in dimensions and geometry, so baseline adjustment does not make them fully equivalent.
Show the calculation
Artifact hash and manifest
An artifact hash works like a fingerprint for a file. A manifest is the packing list that says which fingerprints and run details should be present.
Tiny example: if a hidden-state file has a recorded SHA-256 hash, changing one number changes the hash. A verifier can then detect that the file is no longer the frozen input named in the manifest.
Diode’s manifests also record items such as model identifiers, prompt rows, commands, and splits. Matching hashes establish that the bytes are identical. They do not establish that the research question or experimental design was sound.
Claim boundary
A claim boundary is the line between what a result supports and what remains unknown.
Tiny example: reconstructing TinyLlama from Qwen on two fixed sets of 64 prompts supports a statement about those checkpoints and prompts. It provides no direct test of a 16-model theory or a defined population of prompts.
Diode’s current result is descriptive evidence about three frozen checkpoints on two deliberately constructed fixtures. It does not identify a causal mechanism, establish semantic equivalence, or rank the models’ quality. Recording this boundary in the design and result artifacts lets reviewers compare later claims with the original scope.
Directionality and asymmetry
Directionality means that translating from model A to model B is a different task from translating B to A. Asymmetry measures how differently those two tasks perform after baseline adjustment.
Tiny example: if Qwen → TinyLlama has an adjusted gap of -0.29 and TinyLlama → Qwen has -0.45, the second direction has the larger advantage over its own baseline because more negative is better.
Reversing a map changes the input width, output width, target variance, and geometry, so each direction needs its own fit and score. Asymmetry describes linear reconstructability in this setup. It carries no general model-quality interpretation.
Show the asymmetry calculation
The sign only has meaning after the forward direction has been declared.
Gaussian-source floor
The Gaussian-source floor reruns the analysis with random numbers in place of a real model’s source representations. It shows what the same pipeline can achieve without prompt-aligned model information.
Tiny example: replace Qwen’s 64 prompt vectors with a same-sized table of Gaussian random draws, select the ridge penalty again, fit again, and score the same held-out target prompts.
This check exposes gains caused by sample size and dimensionality alone. The random table does not preserve Qwen’s real spectrum or geometry. The bidirectional study therefore uses this floor as a secondary compatibility check and uses row permutation as its primary baseline.
Layer
A layer is one stage in the model’s sequence of computations. The representation changes as a prompt moves through those stages.
Tiny example: the vector for “volcano” after layer 4 can differ from its vector after the final layer, even though both came from the same prompt and model.
Diode’s reported pilot and sensitivity results use the last layer of each frozen model. Evidence from that layer does not automatically apply to earlier or middle layers.
Linear map
A linear map is a single coordinate-conversion rule. It stretches, shrinks, mixes, and offsets the source numbers to predict target numbers.
Tiny example: a simple map might predict one target coordinate as twice the first source coordinate plus half the second. The same learned rule is used for every prompt.
Linearity is intentionally restrictive. Success on held-out prompts shows that some cross-model structure is accessible through a simple global transformation. A failed linear map would still leave open the possibility of a more complicated nonlinear relationship.
Show the linear-map notation
$$\hat{Y}=XW+b.$$
$X$ contains source representations, $W$ is the learned conversion matrix, and $b$ is an offset.
Masked-mean pooling
Masked-mean pooling turns all of a prompt’s token vectors into one prompt vector by averaging only the real token positions.
Tiny example: if “a red kite” becomes three tokens plus two padding slots, the average uses the three token vectors and ignores the padding.
The attention mask marks which positions count. Pooling is part of the measurement: last-token pooling, unmasked averaging, or another rule can produce different vectors and a different result.
Preregistration
A preregistration is a time-stamped plan written before the new result is inspected. It limits the researcher’s ability to keep trying analyses until one looks favorable.
Tiny example: before running a new model pair, the plan fixes the prompt split seeds, ridge choices, baselines, summary statistics, and claim boundary.
Diode binds the frozen plan to later artifacts by hash. Preregistration does not guarantee a strong hypothesis or bug-free code, so tests, independent verification, and restrained interpretation still matter.
RankMe and effective rank
Effective rank describes how many directions in a representation space carry meaningful variation. RankMe is the particular effective-rank measure used by Diode.
Tiny example: a cloud of points lying almost entirely along one line has RankMe near 1. A cloud spread evenly across several directions has a larger RankMe.
RankMe summarizes how concentrated the variation is across directions; it should not be read as the number of concepts a model knows.
Show the RankMe formula
Center the representation matrix and compute its singular values. Normalize them so their shares $p_k$ add to one, then calculate:
$$ \operatorname{RankMe} = \exp\left(-\sum_k p_k\log p_k\right). $$
Representation or hidden state
A hidden state is a list of numbers the model carries while processing a token. A representation is the vector the experiment chooses to measure from those hidden states.
Tiny example: the model produces one hidden-state vector for each token in “the moon rises.” Diode averages the last-layer token vectors into one representation for the whole prompt.
Individual coordinates do not come with fixed English labels. Diode studies the pattern across prompts: whether one model’s vectors can reconstruct another model’s vectors for prompts excluded from fitting.
Ridge regression
Ridge regression learns a linear map while discouraging extreme weights. The penalty makes the fit more stable when there are many coordinates and few training prompts.
Tiny example: two maps may fit the 48 training prompts equally well, but one uses enormous positive and negative weights. Ridge usually prefers the map with smaller weights because it is less likely to react strongly to a new prompt.
Diode chooses the penalty strength using folds inside the training set. The held-out prompts cannot influence that choice or the fitted weights.
Show the ridge objective
$$ \lVert XW-Y\rVert_F^2+\lambda\lVert W\rVert_F^2. $$
The first term measures training error. The second penalizes large weights. $\lambda$ controls the tradeoff.
Row-permutation baseline
The row-permutation baseline breaks prompt matching while keeping the real source vectors themselves unchanged.
Tiny example: Qwen’s vector for “volcano” might be paired with TinyLlama’s vector for “recipe.” Every source row still exists, but it points to the wrong target prompt.
This preserves the source tensor’s exact scale, spectrum, and internal geometry while removing the intended row correspondence. Each shuffled version repeats ridge selection and fitting. Beating it supports a prompt-aligned association; it does not explain which information creates the association.
Split and sensitivity range
A split decides which prompts teach the map and which prompts test it. A sensitivity range shows how much the result changes across several such divisions.
Tiny example: one split may hold out “volcano,” while another trains on it and holds out “recipe.” Diode summarizes the scores across 20 frozen splits with a median and empirical 2.5th-to-97.5th percentile range.
The same prompts appear in many splits, so the 20 results are statistically dependent. They show sensitivity to different train/test divisions, not evidence from 20 independent experiments. The reported range therefore does not estimate performance on new prompt collections.
Train and held-out prompts
Training prompts are the examples allowed to shape the map. Held-out prompts are sealed off until the final score is calculated.
Tiny example: in each reported sensitivity split, 48 paired prompts select and fit the ridge map. The remaining 16 prompts test whether it works on rows it never fitted.
This separation tests extension beyond the training rows. Repeating the complete analysis on a second fixed 64-prompt fixture tests whether the finding recurs on new rows. Both fixtures were deliberately constructed, so the result does not estimate performance across a defined population of everyday prompts.
For the full study narrative and current results, return to Diode Spectrum. Other public research and production work is listed on the projects page.