Added pre-built prompts for app and chatbot

RESEARCH-jargon-reduction.md

@@ -1,78 +0,0 @@
-# RESEARCH-jargon-reduction
-
-## Objective
-Analyze the LLMVis project for technical jargon (e.g., "head", "ablation", "tokenizing") that may confuse younger or less technical audiences, and outline plain-English alternatives and explanations. The goal is to make the educational dashboard more accessible without completely losing technical accuracy.
-
-## Findings: Jargon in User-Facing Components
-
-### 1. "Ablation" / "Ablate"
-**Locations found:**
-- `app.py`: "Ablation" tab in the Investigation Panel
-- `components/ablation_panel.py`: "What is Ablation?", "Add Head to Ablation List", "Run Ablation Experiment", "Ablated Output"
-
-**Issue:** "Ablation" is a highly specialized term originally from neuroscience/medicine, adapted for machine learning to mean removing a component to observe its effect. It is completely opaque to a layperson.
-
-**Proposed Replacements:**
-- Replace primary "Ablation" headers with "Component Removal" or "Test by Removing".
-- Replace "Run Ablation Experiment" with "Test What Happens When Removed".
-- Keep "Ablation" only in an advanced glossary, a tooltip or as a secondary subtitle: "Test by Removing (Ablation)".
-
-### 2. "Token" / "Tokenization" / "Tokenizing"
-**Locations found:**
-- `app.py`: "Number of New Tokens" 
-- `components/pipeline.py`: "Tokenization" pipeline stage, "Your text is split into tokens"
-- `components/investigation_panel.py`: "Token Attribution", "Target Token"
-
-**Issue:** While "token" is explained somewhat in the app ("pieces that the model can understand"), phrases like "Token Attribution" or "Tokenization" sound overly academic as top-level headings.
-
-**Proposed Replacements:**
-- Use "Text Chunking" or "Word Pieces" as the primary heading, introducing "Tokens" as the technical equivalent inside the explanation.
-- Change "Token Attribution" to "Which words influenced the answer?" or "Word Influence Score".
-
-### 3. "Attention Head" / "Head"
-**Locations found:**
-- `components/pipeline.py`: "Attention has multiple heads"
-- `components/ablation_panel.py`: "Select heads to ablate", "Attention Head Roles"
-
-**Issue:** "Head" in "Attention Head" is an architectural metaphor that doesn't organically mean anything to someone unfamiliar with the transformer architecture (multi-head attention).
-
-**Proposed Replacements:**
-- Reframe as "Attention Pattern", "Context Detector" or "Feature Spotter".
-- Explain "Heads" as "Specialized Detectors" or "Information Filters" (e.g., "The model uses multiple 'Detectors' (called heads) to find relationships...").
-
-### 4. "Embedding" / "Vector"
-**Locations found:**
-- `components/pipeline.py`: "Embedding" stage, "high-dimensional vector", "lookup table"
-
-**Issue:** "Vector" and "high-dimensional" sound like advanced mathematical phrases which may intimidate a younger user.
-
-**Proposed Replacements:**
-- Instead of "Embedding", use "Meaning Translation" or "Number Dictionary".
-- Instead of "high-dimensional vector", use "list of numbers representing meaning".
-
-### 5. "MLP" / "Feed-Forward"
-**Locations found:**
-- `components/pipeline.py`: "MLP (Feed-Forward)", "MLP layers"
-
-**Issue:** MLP (Multi-Layer Perceptron) is legacy neural network jargon that is not intuitive in the context of what it actually does (storing factual knowledge).
-
-**Proposed Replacements:**
-- Rename the pipeline stage to "Factual Knowledge" or "Memory Layers".
-- Include "(MLP/Feed-Forward)" only as a small subtitle for those who want the exact technical term.
-
-### 6. "Logit" / "Probability" / "Activation Score"
-**Locations found:**
-- `components/ablation_panel.py`: "Average prob shift", "activation: 85%"
-
-**Issue:** Jargon related to raw model outputs. "Activation score" sounds like a futuristic/sci-fi term.
-
-**Proposed Replacements:**
-- "Activation score" -> "Confidence level" or "Strength of focus".
-- "Average prob shift" -> "Average change in confidence".
-
-## What Was Ruled Out
-- **Completely eliminating all technical terms:** We ruled out removing words like "Token" or "Model" entirely because learning some valid terminology is part of the educational value of the tool. Instead, we will prioritize plain English for headings and actions, and introduce the technical terms in parentheses or explanatory sentences.
-- **Rewriting the RAG Docs right now:** The `rag_docs/` folder contains extensive markdown documentation. For this primary pass, we ruled out modifying those long-form files, focusing entirely on the primary UI headers, buttons, and short stage descriptions in `app.py` and the `components/` directory to maximize immediate impact.
-
-## Next Steps
-Review this document. If these jargon replacements aligned with the intended goal for younger audiences, we can draft `SPEC-jargon-reduction.md` with explicit strings to be updated across the application.

app.py

@@ -25,7 +25,7 @@ from utils.token_attribution import compute_integrated_gradients, compute_simple
 
 # Import modular components
 from components.sidebar import create_sidebar
-from components.model_selector import create_model_selector
+from components.model_selector import create_model_selector, EXAMPLE_PROMPTS
 from components.glossary import create_glossary_modal
 from components.pipeline import (create_pipeline_container, create_tokenization_content,
                                   create_embedding_content, create_attention_content,
@@ -33,7 +33,7 @@ from components.pipeline import (create_pipeline_container, create_tokenization_
                                   _build_token_display, _build_top5_chart)
 from components.investigation_panel import create_investigation_panel, create_attribution_results_display
 from components.ablation_panel import create_selected_heads_display, create_ablation_results_display
-from components.chatbot import create_chatbot_container, render_messages
+from components.chatbot import create_chatbot_container, render_messages, SUGGESTED_QUESTIONS
 
 # Initialize Dash app
 app = dash.Dash(
@@ -1442,6 +1442,44 @@ app.clientside_callback(
 )
 
 
+# ============================================================================
+# EXAMPLE PROMPT CHIPS
+# ============================================================================
+
+@callback(
+    Output('prompt-input', 'value', allow_duplicate=True),
+    Input({"type": "example-prompt-btn", "index": ALL}, "n_clicks"),
+    prevent_initial_call=True
+)
+def fill_prompt_from_chip(n_clicks_list):
+    """Populate the prompt textarea when an example prompt chip is clicked."""
+    if not any(n_clicks_list):
+        return no_update
+    triggered = dash.ctx.triggered_id
+    if triggered and "index" in triggered:
+        return EXAMPLE_PROMPTS[triggered["index"]]["prompt"]
+    return no_update
+
+
+# ============================================================================
+# CHAT SUGGESTION CHIPS
+# ============================================================================
+
+@callback(
+    Output('chat-input', 'value', allow_duplicate=True),
+    Input({"type": "chat-suggestion-btn", "index": ALL}, "n_clicks"),
+    prevent_initial_call=True
+)
+def fill_chat_from_suggestion(n_clicks_list):
+    """Populate the chat textarea when a suggestion chip is clicked."""
+    if not any(n_clicks_list):
+        return no_update
+    triggered = dash.ctx.triggered_id
+    if triggered and "index" in triggered:
+        return SUGGESTED_QUESTIONS[triggered["index"]]
+    return no_update
+
+
 if __name__ == '__main__':
     # Use 0.0.0.0:7860 for Hugging Face Spaces, fallback to localhost:8050 for local dev
     import os

assets/style.css

@@ -1566,4 +1566,66 @@ details[open].transformer-layers-container .transformer-layers-summary::before {
     padding: 25px;
     overflow-y: auto;
     flex: 1;
+}
+
+/* ============================================================================
+   EXAMPLE PROMPT CHIPS
+   ============================================================================ */
+
+.example-prompts-label {
+    font-size: 0.8rem;
+    color: #888;
+    margin-bottom: 6px;
+}
+
+.example-prompts-container {
+    display: flex;
+    flex-wrap: wrap;
+    gap: 8px;
+    margin-bottom: 10px;
+}
+
+.example-prompt-chip {
+    border-radius: 16px;
+    border: 1px solid #667eea;
+    color: #667eea;
+    background: transparent;
+    padding: 4px 12px;
+    font-size: 0.8rem;
+    cursor: pointer;
+    transition: background-color 0.2s, color 0.2s;
+}
+
+.example-prompt-chip:hover {
+    background-color: #667eea;
+    color: white;
+}
+
+/* ============================================================================
+   CHAT SUGGESTION CHIPS
+   ============================================================================ */
+
+.chat-suggestions-container {
+    display: flex;
+    flex-wrap: wrap;
+    gap: 6px;
+    padding: 8px 12px;
+    border-bottom: 1px solid #e9ecef;
+}
+
+.chat-suggestion-chip {
+    border-radius: 16px;
+    border: 1px solid #667eea;
+    color: #667eea;
+    background: transparent;
+    padding: 3px 10px;
+    font-size: 0.75rem;
+    cursor: pointer;
+    text-align: left;
+    transition: background-color 0.2s, color 0.2s;
+}
+
+.chat-suggestion-chip:hover {
+    background-color: #667eea;
+    color: white;
 }

components/chatbot.py

@@ -17,9 +17,16 @@ I can help you understand:
 - What various experiments can reveal about model behavior
 - General transformer and ML concepts
 
-Try asking: "What does attention detector 0 in layer 1 do?" or "Why did removing this detector change the output?"
+Try one of the suggestions below, or ask your own question!
 """
 
+SUGGESTED_QUESTIONS = [
+    "Walk me through an example experiment step by step",
+    "What prompt should I start with and why?",
+    "What can I learn from this dashboard?",
+    "What should I look for in the attention patterns?"
+]
+
 
 def create_chat_icon():
     """
@@ -158,6 +165,15 @@ def create_input_area():
         Dash HTML component for input area
     """
     return html.Div([
+        html.Div([
+            html.Button(
+                q,
+                id={"type": "chat-suggestion-btn", "index": i},
+                className="chat-suggestion-chip",
+                n_clicks=0
+            )
+            for i, q in enumerate(SUGGESTED_QUESTIONS)
+        ], className="chat-suggestions-container"),
         html.Div([
             dcc.Textarea(
                 id="chat-input",

components/model_selector.py

@@ -8,6 +8,30 @@ This component provides the interface for:
 
 from dash import html, dcc
 
+# Example prompts for guiding users toward meaningful experiments
+EXAMPLE_PROMPTS = [
+    {
+        "label": "Track indirect objects",
+        "prompt": "The chef gave the waiter a generous tip because",
+        "tooltip": "Tests whether the model tracks who gave and who received across clauses"
+    },
+    {
+        "label": "Resolve ambiguity",
+        "prompt": "The bat flew over the",
+        "tooltip": "Tests how earlier context shapes predictions when a word has multiple meanings (bat)"
+    },
+    {
+        "label": "Understand repetition",
+        "prompt": "The cat sat on the mat. The cat sat on the",
+        "tooltip": "Activates duplicate-token and induction detectors that complete repeated sequences"
+    },
+    {
+        "label": "Link pronouns",
+        "prompt": "The nurse said that she",
+        "tooltip": "Tests coreference resolution and reveals gender bias in attention detectors"
+    }
+]
+
 # Available models organized by family
 AVAILABLE_MODELS = [
     # GPT-2 family (OpenAI) — absolute positional encoding, LayerNorm, GELU
@@ -47,6 +71,17 @@ def create_model_selector():
         # Prompt input
         html.Div([
             html.Label("Enter Prompt:", className="input-label"),
+            html.Div("Not sure what to prompt? See how models:", className="example-prompts-label"),
+            html.Div([
+                html.Button(
+                    p["label"],
+                    id={"type": "example-prompt-btn", "index": i},
+                    className="example-prompt-chip",
+                    title=p["tooltip"],
+                    n_clicks=0
+                )
+                for i, p in enumerate(EXAMPLE_PROMPTS)
+            ], className="example-prompts-container"),
             dcc.Textarea(
                 id='prompt-input',
                 placeholder="Enter text prompt for analysis...",

debug_logs.md

@@ -1,163 +0,0 @@
--------------------------------------------------------
-Qwen2ForCausalLM LOAD REPORT from: Qwen/Qwen2.5-0.5B
-Key            | Status  | 
----------------+---------+-
-lm_head.weight | MISSING | 
-
-Notes:
-- MISSING	:those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.
--------------------------------------------------------
-Traceback (most recent call last):
-  File "/app/app.py", line 383, in run_generation
-    results = perform_beam_search(model, tokenizer, prompt, beam_width, max_new_tokens)
-  File "/app/utils/beam_search.py", line 142, in perform_beam_search
-    outputs = model(seq)
-  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
-    return self._call_impl(*args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
-    return forward_call(*args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/transformers/utils/generic.py", line 843, in wrapper
-    output = func(self, *args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 476, in forward
-    outputs: BaseModelOutputWithPast = self.model(
-  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
-    return self._call_impl(*args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
-    return forward_call(*args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/transformers/utils/generic.py", line 917, in wrapper
-    output = func(self, *args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/transformers/utils/output_capturing.py", line 253, in wrapper
-    outputs = func(self, *args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 411, in forward
-    hidden_states = decoder_layer(
-  File "/usr/local/lib/python3.10/site-packages/transformers/modeling_layers.py", line 93, in __call__
-    return super().__call__(*args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
-    return self._call_impl(*args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
-    return forward_call(*args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 298, in forward
-    hidden_states, _ = self.self_attn(
-  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
-    return self._call_impl(*args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
-    return forward_call(*args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 218, in forward
-    query_states = self.q_proj(hidden_states).view(hidden_shape).transpose(1, 2)
-  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
-    return self._call_impl(*args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
-    return forward_call(*args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 134, in forward
-    return F.linear(input, self.weight, self.bias)
-RuntimeError: mat1 and mat2 must have the same dtype, but got Float and BFloat16
--------------------------------------------------------
-Qwen2ForCausalLM LOAD REPORT from: Qwen/Qwen2.5-0.5B
-Key            | Status  | 
----------------+---------+-
-lm_head.weight | MISSING | 
-
-Notes:
-- MISSING	:those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.
-10.16.43.195 - - [19/Mar/2026 19:47:47] "POST /_dash-update-component HTTP/1.1" 200 -
-10.16.43.195 - - [19/Mar/2026 19:47:55] "POST /_dash-update-component HTTP/1.1" 200 -
-10.16.43.195 - - [19/Mar/2026 19:47:55] "POST /_dash-update-component HTTP/1.1" 200 -
-Executing forward pass with prompt: 'Draw ascii art for a cat'
-Captured 48 module outputs using PyVene
-Loading weights:   0%|          | 0/290 [00:00<?, ?it/s]10.16.31.44 - - [19/Mar/2026 19:47:55] "POST /_dash-update-component HTTP/1.1" 200 -
-[2026-03-19 19:47:55,972] ERROR in app: Exception on /_dash-update-component [POST]
-Traceback (most recent call last):
-  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1511, in wsgi_app
-    response = self.full_dispatch_request()
-  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 919, in full_dispatch_request
-    rv = self.handle_user_exception(e)
-  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 917, in full_dispatch_request
-    rv = self.dispatch_request()
-  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 902, in dispatch_request
-    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
-  File "/usr/local/lib/python3.10/site-packages/dash/_get_app.py", line 17, in wrap
-    return ctx.run(func, self, *args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/dash/dash.py", line 1600, in dispatch
-    response_data = ctx.run(partial_func)
-  File "/usr/local/lib/python3.10/site-packages/dash/_callback.py", line 720, in add_context
-    raise err
-  File "/usr/local/lib/python3.10/site-packages/dash/_callback.py", line 711, in add_context
-    output_value = _invoke_callback(func, *func_args, **func_kwargs)  # type: ignore[reportArgumentType]
-  File "/usr/local/lib/python3.10/site-packages/dash/_callback.py", line 58, in _invoke_callback
-    return func(*args, **kwargs)  # %% callback invoked %%
-  File "/app/app.py", line 1090, in update_attribution_target_options
-    options.append({'label': f"{t['token']} ({t['probability']:.1%})", 'value': t['token']})
-TypeError: unsupported format string passed to NoneType.__format__
--------------------------------------------------------
-DEBUG extract_layer_data: Found 24 attention modules
-Loading model: gpt2-medium
-Loading weights:   0%|          | 0/292 [00:00<?, ?it/s]
-Loading weights:  51%|█████     | 149/292 [00:00<00:00, 1299.04it/s]
-Loading weights: 100%|██████████| 292/292 [00:00<00:00, 1450.02it/s]
-GPT2LMHeadModel LOAD REPORT from: gpt2-medium
-Key            | Status  | 
----------------+---------+-
-lm_head.weight | MISSING | 
-
-Notes:
-- MISSING	:those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.
--------------------------------------------------------
-10.16.43.195 - - [19/Mar/2026 20:11:03] "POST /_dash-update-component HTTP/1.1" 200 -
-[2026-03-19 20:11:03,238] ERROR in app: Exception on /_dash-update-component [POST]
-Traceback (most recent call last):
-  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1511, in wsgi_app
-    response = self.full_dispatch_request()
-  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 919, in full_dispatch_request
-    rv = self.handle_user_exception(e)
-  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 917, in full_dispatch_request
-    rv = self.dispatch_request()
-  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 902, in dispatch_request
-    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
-  File "/usr/local/lib/python3.10/site-packages/dash/_get_app.py", line 17, in wrap
-    return ctx.run(func, self, *args, **kwargs)
-  File "/usr/local/lib/python3.10/site-packages/dash/dash.py", line 1600, in dispatch
-    response_data = ctx.run(partial_func)
-  File "/usr/local/lib/python3.10/site-packages/dash/_callback.py", line 720, in add_context
-    raise err
-  File "/usr/local/lib/python3.10/site-packages/dash/_callback.py", line 711, in add_context
-    output_value = _invoke_callback(func, *func_args, **func_kwargs)  # type: ignore[reportArgumentType]
-  File "/usr/local/lib/python3.10/site-packages/dash/_callback.py", line 58, in _invoke_callback
-    return func(*args, **kwargs)  # %% callback invoked %%
-  File "/app/app.py", line 1090, in update_attribution_target_options
-    options.append({'label': f"{t['token']} ({t['probability']:.1%})", 'value': t['token']})
-TypeError: unsupported format string passed to NoneType.__format__
-10.16.43.195 - - [19/Mar/2026 20:11:03] "POST /_dash-update-component HTTP/1.1" 500 -
-10.16.43.195 - - [19/Mar/2026 20:11:03] "POST /_dash-update-component HTTP/1.1" 200 -
-Traceback (most recent call last):
-  File "/app/utils/model_patterns.py", line 1337, in generate_bertviz_html
-    attention_weights = torch.tensor(attention_output[1])  # [batch, heads, seq, seq]
-RuntimeError: Could not infer dtype of NoneType
-Traceback (most recent call last):
-  File "/app/app.py", line 691, in update_pipeline_content
-    outputs.append(create_output_content(
-  File "/app/components/pipeline.py", line 1130, in create_output_content
-    text=[f"{p:.1%}" for p in probs], textposition='outside',
-  File "/app/components/pipeline.py", line 1130, in <listcomp>
-    text=[f"{p:.1%}" for p in probs], textposition='outside',
-TypeError: unsupported format string passed to NoneType.__format__
--------------------------------------------------------
-10.16.43.195 - - [19/Mar/2026 20:26:26] "POST /_dash-update-component HTTP/1.1" 200 -
-DEBUG extract_layer_data: Found 24 attention modules
-Warning: Could not compute logit lens for gpt_neox.layers.0: mixed dtype (CPU): expect parameter to have scalar type of Float
-Warning: Could not compute token probabilities for gpt_neox.layers.0: mixed dtype (CPU): expect parameter to have scalar type of Float
-Warning: Could not compute logit lens for gpt_neox.layers.1: mixed dtype (CPU): expect parameter to have scalar type of Float
-Warning: Could not compute token probabilities for gpt_neox.layers.1: mixed dtype (CPU): expect parameter to have scalar type of Float
-Warning: Could not compute logit lens for gpt_neox.layers.2: mixed dtype (CPU): expect parameter to have scalar type of Float
-Warning: Could not compute token probabilities for gpt_neox.layers.2: mixed dtype (CPU): expect parameter to have scalar type of Float
--------------------------------------------------------
-Warning: Could not compute logit lens for gpt_neox.layers.13: mixed dtype (CPU): expect parameter to have scalar type of Float
-Warning: Could not compute token probabilities for gpt_neox.layers.13: mixed dtype (CPU): expect parameter to have scalar type of Float
-Warning: Could not compute logit lens for gpt_neox.layers.14: Could not infer dtype of NoneType
-Warning: Could not compute token probabilities for gpt_neox.layers.14: Could not infer dtype of NoneType
-Warning: Could not compute logit lens for gpt_neox.layers.15: Could not infer dtype of NoneType
-Warning: Could not compute token probabilities for gpt_neox.layers.15: Could not infer dtype of NoneType
-Warning: Could not compute logit lens for gpt_neox.layers.16: Could not infer dtype of NoneType
-Warning: Could not compute token probabilities for gpt_neox.layers.16: Could not infer dtype of NoneType
-Warning: Could not compute logit lens for gpt_neox.layers.17: Could not infer dtype of NoneType
-Warning: Could not compute token probabilities for gpt_neox.layers.17: Could not infer dtype of NoneType
--------------------------------------------------------

plans.md

@@ -1,16 +1,3 @@
-To Do:
-- clicking send in the chatbot doesn't immediately show the purple text bubble from the user and the typing animation, there are a couple seconds of nothing. It should immediately populate the text bubble and start the typing animation
-
-Done:
-- change attention to entire generated sequence
-- change width of chatbot window
-- add output token generation to attention, tokenization, etc
-- output slider to look at each token (scrubber with top-5 at each position)
-- experiment results side by side comparison
-- output streaming for chatbot
-- shorter, concise responses in system prompt
-- add video links to glossary
-    - three blue one brown
-- specs on what each attention head does
-- generating with 1 output token doesn't show the "selected" beam, so users can't see the output from the model
-- add head index numbers to color swatches in BertViz visualization graph
+- make it more obvious that this is an experiment (pre-loaded things to ask)
+- make sure chatbot explains simpler
+- glossary button move

todo.md

@@ -1,227 +0,0 @@
-# Todo
-
-## Completed: Test Suite Setup (Done)
-- [x] Create `tests/` folder with `__init__.py` and `conftest.py` (shared fixtures)
-- [x] Create `test_model_config.py` - 15 tests for model family lookups
-- [x] Create `test_ablation_metrics.py` - 8 tests for KL divergence and probability deltas
-- [x] Create `test_head_detection.py` - 20 tests for attention head categorization
-- [x] Create `test_model_patterns.py` - 16 tests for merge_token_probabilities, safe_to_serializable
-- [x] Create `test_token_attribution.py` - 11 tests for visualization data formatting
-- [x] Verify all 73 tests pass with `pytest tests/ -v`
-
-## Completed: Pipeline Explanation Refactor
-
-### Phase 1: New Components (Done)
-- [x] Create `components/pipeline.py` with 5 expandable stages
-- [x] Create `utils/token_attribution.py` with Integrated Gradients
-- [x] Create `components/investigation_panel.py` (ablation + attribution)
-
-### Phase 2: Simplifications (Done)
-- [x] Remove comparison UI from `model_selector.py`
-- [x] Refactor `app.py`: wire pipeline, remove heatmap/comparison callbacks
-
-### Phase 3: Cleanup (Done)
-- [x] Delete `main_panel.py`
-- [x] Delete `prompt_comparison.py`
-- [x] Update `utils/__init__.py` exports
-- [x] Add pipeline CSS styles to `assets/style.css`
-
-## Completed: Pipeline Clarity Improvements (Agent A)
-
-- [x] Rename "Max New Tokens:" to "Number of New Tokens:" in app.py
-- [x] Rename "Beam Width:" to "Number of Generation Choices:" in app.py
-- [x] Remove score display from generated sequences in app.py
-- [x] Update glossary to clarify "Number of Generation Choices" relates to Beam Search
-
-## Completed: Pipeline Clarity Improvements (Agent E)
-
-- [x] Modified output display to show full prompt with predicted token highlighted
-- [x] Fixed top-5 tokens hover to show "Token (X%)" format instead of long decimals
-- [x] Added Plotly hovertemplate for cleaner hover formatting
-
-## Completed: Pipeline Clarity Improvements (Agent C)
-
-- [x] Add educational explanation for embedding stage (pre-learned lookup table concept)
-- [x] Add educational explanation for MLP stage (knowledge storage during training)
-- [x] Add educational explanation for attention stage (how to interpret BertViz visualization)
-
-## Completed: Pipeline Clarity Improvements (Agent B)
-
-- [x] Convert tokenization from horizontal three-column layout to vertical rows
-- [x] Each token row shows: [token] → [ID] → [embedding placeholder]
-- [x] Maintain existing color scheme and educational tooltips
-- [x] Update CSS styles for .tokenization-rows and .token-row layout
-- [x] Add responsive styles for mobile (stack on small screens)
-
-## Completed: Pipeline Clarity Improvements (Agent D)
-
-- [x] Switch generate_bertviz_html from model_view to head_view in model_patterns.py
-- [x] Deprecate _get_top_attended_tokens function (remove usage in extract_layer_data)
-- [x] Add generate_head_view_with_categories function for categorized attention heads
-- [x] Add get_head_category_counts helper function for UI display
-- [x] Run tests to verify no regressions (73 tests pass)
-
-## Completed: Pipeline Clarity Improvements (Agent F - Analysis Scope)
-
-- [x] Add session-original-prompt-store and session-selected-beam-store to app.py
-- [x] Modify run_generation to analyze ORIGINAL PROMPT only (not generated beam)
-- [x] Store beam generation results separately for post-experiment comparison
-- [x] Update analyze_selected_sequence to store beam for comparison instead of re-analyzing
-- [x] Update ablation experiment to show selected beam context in results
-
-## Completed: Pipeline Clarity Improvements (Agent G - Integration & Attention UI)
-
-- [x] Remove deprecated "Most attended tokens" section from attention stage
-- [x] Wire head categorization into attention stage UI (shows category counts)
-- [x] Add enhanced navigation instructions for BertViz head view
-- [x] Verify all 73 tests pass
-
-## Completed: UI/UX Fixes (5 Issues)
-
-### Issue 1: "Select for Comparison" Button
-- [x] Update store_selected_beam callback in app.py to update UI
-- [x] Clear all other generated sequences when one is selected
-- [x] Display selected sequence with "Selected for Comparison" badge
-
-### Issue 2: Tokenization Vertical Layout
-- [x] Modify create_tokenization_content in pipeline.py to use vertical layout
-- [x] Each row displays: [token] → [ID] with header row
-
-### Issue 3: Expandable Attention Categories
-- [x] Convert category chips to expandable `<details>` elements in pipeline.py
-- [x] Update app.py to pass full categorize_all_heads() data instead of counts
-- [x] Show list of heads (L0-H3, L2-H5, etc.) when category is expanded
-
-### Issue 4: BertViz Navigation Instructions
-- [x] Add single-click explanation: selects/deselects that head
-- [x] Add double-click explanation: selects only that head (deselects others)
-
-### Issue 5: Multi-Layer Ablation Head Selection
-- [x] Change ablation-selected-heads store to hold [{layer, head}, ...] objects
-- [x] Add create_selected_heads_display function in investigation_panel.py
-- [x] Show selected heads as chips with "x" buttons to remove
-- [x] Update head buttons to show visual selection state per layer
-- [x] Preserve selections across layer dropdown changes
-- [x] Update run_ablation_experiment to handle multi-layer ablation
-- [x] Verify all 73 tests pass
-
-## Completed: Fix Multi-Layer Ablation Bug
-
-- [x] Create `execute_forward_pass_with_multi_layer_head_ablation` in model_patterns.py
-- [x] Export new function in utils/__init__.py
-- [x] Replace per-layer ablation loop in app.py with single call to new function
-- [x] Add 5 tests for multi-layer ablation in test_model_patterns.py
-- [x] Verify all 78 tests pass
-
-## Completed: Codebase Cleanup
-
-- [x] Delete unused file: `components/tokenization_panel.py` (302 lines, 6 functions)
-- [x] Remove 6 unused imports from `app.py`
-- [x] Remove deprecated `_get_top_attended_tokens()` function from model_patterns.py
-- [x] Remove `top_attended_tokens` field from extract_layer_data() return values
-- [x] Remove unused `create_stage_summary()` function from pipeline.py
-- [x] Remove 7 unused utility functions from utils/:
-  - `get_check_token_probabilities`
-  - `execute_forward_pass_with_layer_ablation`
-  - `generate_category_bertviz_html`
-  - `generate_head_view_with_categories`
-  - `compute_sequence_trajectory`
-  - `compute_layer_wise_summaries`
-  - `compute_position_layer_matrix`
-- [x] Update `utils/__init__.py` exports
-- [x] Update README.md to remove reference to deleted file
-
-## Completed: AI Chatbot Integration
-
-- [x] Create `rag_docs/` folder with placeholder README for RAG documents
-- [x] Create `utils/gemini_client.py` with Gemini API wrapper (generate + embed)
-- [x] Create `utils/rag_utils.py` with document loading, chunking, and retrieval
-- [x] Create `components/chatbot.py` with UI components (icon, window, messages)
-- [x] Add chatbot CSS to `assets/style.css` (floating button, chat window, message bubbles)
-- [x] Modify `app.py` to add chat layout and callbacks
-- [x] Add `google-generativeai` to `requirements.txt`
-- [x] Test end-to-end: toggle, message send, context awareness
-- [x] Verify all 81 tests pass
-
-## Completed: Hugging Face Deployment Prep
-
-- [x] Create `.gitignore` to exclude `.env`, `__pycache__/`, etc.
-- [x] Add `load_dotenv()` to `app.py` for local development
-- [x] Create `tests/test_gemini_connection.py` to verify API key connectivity
-- [x] Tests verify: API key is set, can list models, flash model available
-- Note: On Hugging Face Spaces, set `GEMINI_API_KEY` in Repository Secrets
-
-## Completed: Migrate to New Google GenAI SDK (Superseded)
-
-- [x] Update `requirements.txt`: `google-generativeai` → `google-genai>=1.0.0`
-- [x] Rewrite `utils/gemini_client.py` using new centralized Client architecture
-- [x] All 4 connection tests pass
-- [x] Verified: embeddings work (3072 dimensions), chat generation works
-
-## Completed: Migrate from Gemini to OpenRouter
-
-- [x] Create `utils/openrouter_client.py` with OpenAI-compatible API
-  - Global model config: `DEFAULT_CHAT_MODEL` and `DEFAULT_EMBEDDING_MODEL`
-  - Chat via: `POST /api/v1/chat/completions`
-  - Embeddings via: `POST /api/v1/embeddings`
-- [x] Update `utils/rag_utils.py` imports to use openrouter_client
-- [x] Update `app.py` imports to use openrouter_client
-- [x] Create `tests/test_openrouter_connection.py` for API connectivity tests
-- [x] Delete old `utils/gemini_client.py` and `tests/test_gemini_connection.py`
-- [x] Update `requirements.txt`: remove `google-genai`, add `requests>=2.28.0`
-- [x] Environment variable: `GEMINI_API_KEY` → `OPENROUTER_API_KEY`
-
-## Completed: Switch to Paid OpenRouter Models (Cost-Optimized)
-
-- [x] Evaluate OpenRouter models for chatbot use case (cost vs quality)
-- [x] Switch chat model: `google/gemini-2.5-flash-lite`
-  - $0.10/$0.40 per 1M tokens (input/output)
-  - 1M context window, 318 tok/s, multimodal
-- [x] Switch embedding model: `openai/text-embedding-3-small`
-  - $0.02 per 1M tokens
-  - 1536 dimensions, high quality
-- [x] Remove local `sentence-transformers` dependency (simpler, no TF conflicts)
-- [x] Estimated cost: ~$1.50/month for moderate usage
-
-## Completed: Enhance RAG Documents for Chatbot
-
-- [x] Category 1: 8 general LLM/Transformer knowledge files (what_is_an_llm.md through key_terminology.md)
-- [x] Category 2: 7 dashboard component documentation files (dashboard_overview.md through model_selector_guide.md)
-- [x] Category 3: 3 model-specific documentation files (gpt2_overview.md, llama_overview.md, opt_overview.md)
-- [x] Category 4: 6 step-by-step guided experiment files (experiment_first_analysis.md through experiment_beam_search.md)
-- [x] Category 5: 6 interpretation/troubleshooting/research files (interpreting_*.md, troubleshooting_and_faq.md, recommended_starting_points.md, mechanistic_interpretability_intro.md)
-- [x] Delete embeddings_cache.json, update rag_docs/README.md with full inventory
-- [x] Update todo.md and conductor docs
-- Total: 30 RAG documents covering transformer concepts, dashboard usage, guided experiments, interpretation, troubleshooting, and research context
-
-## Completed: Resizable Chat Window
-
-- [x] Add drag handle div to left edge of chat window in chatbot.py
-- [x] Add CSS for `.chat-resize-handle` (cursor, hover highlight) in style.css
-- [x] Add `assets/chat_resize.js` for mousedown/mousemove/mouseup drag logic
-- Default size unchanged (25vw, 320px–450px); drag overrides max-width up to 80vw
-
-## Completed: Full-Sequence Attention Visualization
-
-- [x] Modified `run_generation()` in app.py: for single-token generation, run forward pass on full beam text (prompt + generated token) instead of prompt only
-- [x] Enhanced `store_selected_beam()` in app.py: added `session-activation-store` output and `session-activation-store` State; re-runs `execute_forward_pass()` on selected beam's full text when user picks a beam
-- [x] Added 5 tests in `test_model_patterns.py` (`TestFullSequenceAttentionData`) verifying attention matrix dimensions match full sequence length
-- Attention visualization now covers the entire chosen output (input + generated tokens), not just the input prompt
-- No changes needed in `model_patterns.py`, `beam_search.py`, `pipeline.py`, or `head_detection.py`
-
-## Completed: Output Token Scrubber
-
-- [x] Add `compute_per_position_top5()` to `utils/model_patterns.py` — extracts top-5 next-token probabilities at each generated-token position from a single forward pass
-- [x] Add `original_prompt` parameter to `execute_forward_pass()` — when provided, computes per-position top-5 data and stores in activation_data
-- [x] Export `compute_per_position_top5` in `utils/__init__.py`
-- [x] Update `run_generation()` in app.py — passes `original_prompt=prompt` for single-token generation
-- [x] Update `store_selected_beam()` in app.py — reads original prompt from session store and passes to forward pass
-- [x] Rewrite `create_output_content()` in `components/pipeline.py` — scrubber mode with `dcc.Slider`, token display, and top-5 chart; falls back to static mode when no per-position data
-- [x] Add `_build_token_display()` and `_build_top5_chart()` helpers in pipeline.py
-- [x] Add `update_output_scrubber()` callback in app.py — responds to slider changes, updates token highlight and chart
-- [x] Update `update_pipeline_content()` in app.py — extracts per-position data and passes to output content
-- [x] Add 10 tests for `compute_per_position_top5` in `test_model_patterns.py`
-- [x] Fix `conftest.py` to set `USE_TF=0` for test import compatibility
-- [x] All 100 tests pass
-- Scrubber shows prompt context (gray) + highlighted token (cyan) + top-5 bar chart at each slider position
-- Pre-beam-selection falls back to static output display; scrubber activates after beam selection or single-token generation