Add 'Reading the geometry' glossary up top (defines cosine, principal angles, LMC, CKA)
Browse files- index.html +31 -1
- style.css +15 -4
index.html
CHANGED
|
@@ -20,6 +20,7 @@
|
|
| 20 |
<div class="toc-title">Contents</div>
|
| 21 |
<ol>
|
| 22 |
<li><a href="#abstract">Abstract</a></li>
|
|
|
|
| 23 |
<li><a href="#methods">The six losses</a></li>
|
| 24 |
<li><a href="#cosine">Cosine map</a></li>
|
| 25 |
<li><a href="#perlayer">Layer by layer</a></li>
|
|
@@ -46,7 +47,7 @@
|
|
| 46 |
</div>
|
| 47 |
</header>
|
| 48 |
|
| 49 |
-
<section id="abstract">
|
| 50 |
<h2>Abstract</h2>
|
| 51 |
<p id="abstract-text" class="dropcap"></p>
|
| 52 |
<p>
|
|
@@ -57,6 +58,35 @@
|
|
| 57 |
</p>
|
| 58 |
</section>
|
| 59 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
<section id="methods">
|
| 61 |
<h2>The six losses</h2>
|
| 62 |
<p>
|
|
|
|
| 20 |
<div class="toc-title">Contents</div>
|
| 21 |
<ol>
|
| 22 |
<li><a href="#abstract">Abstract</a></li>
|
| 23 |
+
<li><a href="#reading">Reading the geometry</a></li>
|
| 24 |
<li><a href="#methods">The six losses</a></li>
|
| 25 |
<li><a href="#cosine">Cosine map</a></li>
|
| 26 |
<li><a href="#perlayer">Layer by layer</a></li>
|
|
|
|
| 47 |
</div>
|
| 48 |
</header>
|
| 49 |
|
| 50 |
+
<section id="abstract" class="aside">
|
| 51 |
<h2>Abstract</h2>
|
| 52 |
<p id="abstract-text" class="dropcap"></p>
|
| 53 |
<p>
|
|
|
|
| 58 |
</p>
|
| 59 |
</section>
|
| 60 |
|
| 61 |
+
<section id="reading" class="aside">
|
| 62 |
+
<h2>Reading the geometry</h2>
|
| 63 |
+
<p>
|
| 64 |
+
Every chart below is computed on the LoRA weight update <strong>ΔW</strong> — the small
|
| 65 |
+
change each method writes into the base model — or on the representations that update
|
| 66 |
+
produces. Four tools, each asking a different version of <em>“are these two the same?”</em>
|
| 67 |
+
</p>
|
| 68 |
+
<div class="note">
|
| 69 |
+
<dl class="glossary">
|
| 70 |
+
<div>
|
| 71 |
+
<dt>Cosine similarity</dt>
|
| 72 |
+
<dd>Do two weight updates point the same way? <b>+1</b> = identical direction, <b>0</b> = orthogonal (unrelated), <b>below 0</b> = opposed. This is the headline number, taken on the stacked ΔW.</dd>
|
| 73 |
+
</div>
|
| 74 |
+
<div>
|
| 75 |
+
<dt>Principal angles</dt>
|
| 76 |
+
<dd>How far apart are the <b>subspaces</b> the two updates span — a basis-free generalization of cosine. A few degrees means effectively the same subspace; near 90° means disjoint.</dd>
|
| 77 |
+
</div>
|
| 78 |
+
<div>
|
| 79 |
+
<dt>Mode connectivity<span>linear · LMC</span></dt>
|
| 80 |
+
<dd>Interpolate between two trained adapters and watch the loss. A flat path means they sit in the <b>same basin</b>; a bump in the middle is a barrier separating two different solutions.</dd>
|
| 81 |
+
</div>
|
| 82 |
+
<div>
|
| 83 |
+
<dt>CKA<span>centered kernel alignment</span></dt>
|
| 84 |
+
<dd>Do the two models compute the same thing <b>inside</b>? Unlike the others, CKA compares hidden representations, not weights. <b>≈1</b> = near-identical computation; lower means the circuit has been rewired.</dd>
|
| 85 |
+
</div>
|
| 86 |
+
</dl>
|
| 87 |
+
</div>
|
| 88 |
+
</section>
|
| 89 |
+
|
| 90 |
<section id="methods">
|
| 91 |
<h2>The six losses</h2>
|
| 92 |
<p>
|
style.css
CHANGED
|
@@ -75,18 +75,29 @@ h2 {
|
|
| 75 |
font-family: var(--display); font-weight: 600; font-size: 1.9rem; letter-spacing: -.01em;
|
| 76 |
line-height: 1.12; margin: 0 0 1.2rem; position: relative;
|
| 77 |
}
|
| 78 |
-
section:not(
|
| 79 |
-
section:not(
|
| 80 |
content: counter(sec, decimal-leading-zero);
|
| 81 |
font-family: var(--mono); font-size: .72rem; font-weight: 500; color: var(--accent);
|
| 82 |
position: absolute; left: -2.7rem; top: .55rem; letter-spacing: .05em;
|
| 83 |
}
|
| 84 |
-
@media (max-width: 1180px) { section:not(
|
| 85 |
p { margin: 0 0 1.3rem; }
|
| 86 |
section > p { max-width: 62ch; }
|
| 87 |
em { font-style: italic; }
|
| 88 |
strong { font-weight: 600; }
|
| 89 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
.dropcap::first-letter {
|
| 91 |
font-family: var(--display); font-weight: 600; float: left; font-size: 4.4rem;
|
| 92 |
line-height: .76; padding: .06em .1em 0 0; color: var(--ink);
|
|
|
|
| 75 |
font-family: var(--display); font-weight: 600; font-size: 1.9rem; letter-spacing: -.01em;
|
| 76 |
line-height: 1.12; margin: 0 0 1.2rem; position: relative;
|
| 77 |
}
|
| 78 |
+
section:not(.aside) h2 { counter-increment: sec; }
|
| 79 |
+
section:not(.aside) h2::before {
|
| 80 |
content: counter(sec, decimal-leading-zero);
|
| 81 |
font-family: var(--mono); font-size: .72rem; font-weight: 500; color: var(--accent);
|
| 82 |
position: absolute; left: -2.7rem; top: .55rem; letter-spacing: .05em;
|
| 83 |
}
|
| 84 |
+
@media (max-width: 1180px) { section:not(.aside) h2::before { position: static; display: inline-block; margin-right: .8rem; } }
|
| 85 |
p { margin: 0 0 1.3rem; }
|
| 86 |
section > p { max-width: 62ch; }
|
| 87 |
em { font-style: italic; }
|
| 88 |
strong { font-weight: 600; }
|
| 89 |
+
.aside h2 { font-size: 1rem; font-family: var(--mono); font-weight: 500; text-transform: uppercase; letter-spacing: .12em; color: var(--ink-faint); margin-bottom: 1rem; }
|
| 90 |
+
|
| 91 |
+
/* glossary plate (definition list) */
|
| 92 |
+
.note { border: 1.5px solid var(--rule); padding: 4px 24px; margin: 1.8rem 0 1rem; }
|
| 93 |
+
dl.glossary { margin: 0; }
|
| 94 |
+
dl.glossary > div { display: grid; grid-template-columns: 168px 1fr; gap: 5px 30px; padding: 17px 0; border-top: 1px solid var(--rule-soft); }
|
| 95 |
+
dl.glossary > div:first-child { border-top: 0; }
|
| 96 |
+
dl.glossary dt { font-family: var(--mono); font-size: .82rem; font-weight: 500; color: var(--ink); line-height: 1.3; }
|
| 97 |
+
dl.glossary dt span { display: block; font-size: .62rem; letter-spacing: .08em; text-transform: uppercase; color: var(--accent); margin-top: 5px; }
|
| 98 |
+
dl.glossary dd { margin: 0; font-family: var(--body); font-size: 1rem; line-height: 1.45; color: var(--ink-soft); }
|
| 99 |
+
dl.glossary dd b { color: var(--ink); font-weight: 600; }
|
| 100 |
+
@media (max-width: 620px) { dl.glossary > div { grid-template-columns: 1fr; gap: 6px; } }
|
| 101 |
.dropcap::first-letter {
|
| 102 |
font-family: var(--display); font-weight: 600; float: left; font-size: 4.4rem;
|
| 103 |
line-height: .76; padding: .06em .1em 0 0; color: var(--ink);
|