AlexWortega commited on
Commit
e922bc3
·
verified ·
1 Parent(s): d88313a

Add 'Reading the geometry' glossary up top (defines cosine, principal angles, LMC, CKA)

Browse files
Files changed (2) hide show
  1. index.html +31 -1
  2. style.css +15 -4
index.html CHANGED
@@ -20,6 +20,7 @@
20
  <div class="toc-title">Contents</div>
21
  <ol>
22
  <li><a href="#abstract">Abstract</a></li>
 
23
  <li><a href="#methods">The six losses</a></li>
24
  <li><a href="#cosine">Cosine map</a></li>
25
  <li><a href="#perlayer">Layer by layer</a></li>
@@ -46,7 +47,7 @@
46
  </div>
47
  </header>
48
 
49
- <section id="abstract">
50
  <h2>Abstract</h2>
51
  <p id="abstract-text" class="dropcap"></p>
52
  <p>
@@ -57,6 +58,35 @@
57
  </p>
58
  </section>
59
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
60
  <section id="methods">
61
  <h2>The six losses</h2>
62
  <p>
 
20
  <div class="toc-title">Contents</div>
21
  <ol>
22
  <li><a href="#abstract">Abstract</a></li>
23
+ <li><a href="#reading">Reading the geometry</a></li>
24
  <li><a href="#methods">The six losses</a></li>
25
  <li><a href="#cosine">Cosine map</a></li>
26
  <li><a href="#perlayer">Layer by layer</a></li>
 
47
  </div>
48
  </header>
49
 
50
+ <section id="abstract" class="aside">
51
  <h2>Abstract</h2>
52
  <p id="abstract-text" class="dropcap"></p>
53
  <p>
 
58
  </p>
59
  </section>
60
 
61
+ <section id="reading" class="aside">
62
+ <h2>Reading the geometry</h2>
63
+ <p>
64
+ Every chart below is computed on the LoRA weight update <strong>ΔW</strong> — the small
65
+ change each method writes into the base model — or on the representations that update
66
+ produces. Four tools, each asking a different version of <em>“are these two the same?”</em>
67
+ </p>
68
+ <div class="note">
69
+ <dl class="glossary">
70
+ <div>
71
+ <dt>Cosine similarity</dt>
72
+ <dd>Do two weight updates point the same way? <b>+1</b> = identical direction, <b>0</b> = orthogonal (unrelated), <b>below 0</b> = opposed. This is the headline number, taken on the stacked ΔW.</dd>
73
+ </div>
74
+ <div>
75
+ <dt>Principal angles</dt>
76
+ <dd>How far apart are the <b>subspaces</b> the two updates span — a basis-free generalization of cosine. A few degrees means effectively the same subspace; near 90° means disjoint.</dd>
77
+ </div>
78
+ <div>
79
+ <dt>Mode connectivity<span>linear · LMC</span></dt>
80
+ <dd>Interpolate between two trained adapters and watch the loss. A flat path means they sit in the <b>same basin</b>; a bump in the middle is a barrier separating two different solutions.</dd>
81
+ </div>
82
+ <div>
83
+ <dt>CKA<span>centered kernel alignment</span></dt>
84
+ <dd>Do the two models compute the same thing <b>inside</b>? Unlike the others, CKA compares hidden representations, not weights. <b>≈1</b> = near-identical computation; lower means the circuit has been rewired.</dd>
85
+ </div>
86
+ </dl>
87
+ </div>
88
+ </section>
89
+
90
  <section id="methods">
91
  <h2>The six losses</h2>
92
  <p>
style.css CHANGED
@@ -75,18 +75,29 @@ h2 {
75
  font-family: var(--display); font-weight: 600; font-size: 1.9rem; letter-spacing: -.01em;
76
  line-height: 1.12; margin: 0 0 1.2rem; position: relative;
77
  }
78
- section:not(#abstract) h2 { counter-increment: sec; }
79
- section:not(#abstract) h2::before {
80
  content: counter(sec, decimal-leading-zero);
81
  font-family: var(--mono); font-size: .72rem; font-weight: 500; color: var(--accent);
82
  position: absolute; left: -2.7rem; top: .55rem; letter-spacing: .05em;
83
  }
84
- @media (max-width: 1180px) { section:not(#abstract) h2::before { position: static; display: inline-block; margin-right: .8rem; } }
85
  p { margin: 0 0 1.3rem; }
86
  section > p { max-width: 62ch; }
87
  em { font-style: italic; }
88
  strong { font-weight: 600; }
89
- #abstract h2 { font-size: 1rem; font-family: var(--mono); font-weight: 500; text-transform: uppercase; letter-spacing: .12em; color: var(--ink-faint); margin-bottom: 1rem; }
 
 
 
 
 
 
 
 
 
 
 
90
  .dropcap::first-letter {
91
  font-family: var(--display); font-weight: 600; float: left; font-size: 4.4rem;
92
  line-height: .76; padding: .06em .1em 0 0; color: var(--ink);
 
75
  font-family: var(--display); font-weight: 600; font-size: 1.9rem; letter-spacing: -.01em;
76
  line-height: 1.12; margin: 0 0 1.2rem; position: relative;
77
  }
78
+ section:not(.aside) h2 { counter-increment: sec; }
79
+ section:not(.aside) h2::before {
80
  content: counter(sec, decimal-leading-zero);
81
  font-family: var(--mono); font-size: .72rem; font-weight: 500; color: var(--accent);
82
  position: absolute; left: -2.7rem; top: .55rem; letter-spacing: .05em;
83
  }
84
+ @media (max-width: 1180px) { section:not(.aside) h2::before { position: static; display: inline-block; margin-right: .8rem; } }
85
  p { margin: 0 0 1.3rem; }
86
  section > p { max-width: 62ch; }
87
  em { font-style: italic; }
88
  strong { font-weight: 600; }
89
+ .aside h2 { font-size: 1rem; font-family: var(--mono); font-weight: 500; text-transform: uppercase; letter-spacing: .12em; color: var(--ink-faint); margin-bottom: 1rem; }
90
+
91
+ /* glossary plate (definition list) */
92
+ .note { border: 1.5px solid var(--rule); padding: 4px 24px; margin: 1.8rem 0 1rem; }
93
+ dl.glossary { margin: 0; }
94
+ dl.glossary > div { display: grid; grid-template-columns: 168px 1fr; gap: 5px 30px; padding: 17px 0; border-top: 1px solid var(--rule-soft); }
95
+ dl.glossary > div:first-child { border-top: 0; }
96
+ dl.glossary dt { font-family: var(--mono); font-size: .82rem; font-weight: 500; color: var(--ink); line-height: 1.3; }
97
+ dl.glossary dt span { display: block; font-size: .62rem; letter-spacing: .08em; text-transform: uppercase; color: var(--accent); margin-top: 5px; }
98
+ dl.glossary dd { margin: 0; font-family: var(--body); font-size: 1rem; line-height: 1.45; color: var(--ink-soft); }
99
+ dl.glossary dd b { color: var(--ink); font-weight: 600; }
100
+ @media (max-width: 620px) { dl.glossary > div { grid-template-columns: 1fr; gap: 6px; } }
101
  .dropcap::first-letter {
102
  font-family: var(--display); font-weight: 600; float: left; font-size: 4.4rem;
103
  line-height: .76; padding: .06em .1em 0 0; color: var(--ink);