fix: submission navigation

#6
by burtenshaw HF Staff - opened
Files changed (5) hide show
  1. AGENTS.md +1 -1
  2. README.md +1 -4
  3. arena_bridge.py +3 -62
  4. index.html +165 -55
  5. test_app.py +25 -203
AGENTS.md CHANGED
@@ -155,7 +155,7 @@ printf 'Authorization: Bearer %s\n' "$TOKEN" | curl -sS -H @- "$BASE/runs/RUN_ID
155
  printf 'Authorization: Bearer %s\n' "$TOKEN" | curl -sS -H @- "$BASE/runs/RUN_ID/events"
156
  ```
157
 
158
- `GET $BASE/submissions` lists your own submissions, newest first, with each one's state, pinned images, dataset and run.
159
 
160
  A run goes `starting`, `loading`, then `rolling` and `updating` once per optimizer step (`policy_version` counts the steps taken), then `cleanup` and `completed` or `failed` (`cancelled` if an organizer stops it). You can read only your own submissions and runs. Hugging Face's proxy in front of this Space sometimes answers 502, 503 or 504 itself: every request here is safe to send again unchanged.
161
 
 
155
  printf 'Authorization: Bearer %s\n' "$TOKEN" | curl -sS -H @- "$BASE/runs/RUN_ID/events"
156
  ```
157
 
158
+ `GET $BASE/submissions` lists your own submissions, newest first, with each one's state, pinned images, dataset and run. **My submissions** on the page shows the same list after you sign in with Hugging Face.
159
 
160
  A run goes `starting`, `loading`, then `rolling` and `updating` once per optimizer step (`policy_version` counts the steps taken), then `cleanup` and `completed` or `failed` (`cancelled` if an organizer stops it). You can read only your own submissions and runs. Hugging Face's proxy in front of this Space sometimes answers 502, 503 or 504 itself: every request here is safe to send again unchanged.
161
 
README.md CHANGED
@@ -19,10 +19,7 @@ The dashboard adapts the compact typography and panel styles of the [Gemma chall
19
 
20
  The Nebius wordmark comes from its [official media kit](https://nebius.com/media-kit).
21
 
22
- - The page at `/` explains the challenge, with a **What is OpenEnvArena?** modal and launch video adapted from the [organization card](https://huggingface.co/openenvarena). It has a chart of tasks submitted against score, the leaderboard, public submissions with their timestamps, environment datasets, users and status, the message board and **Add your agent**. Agent docs are linked in both the about and agent modals, and the header credits Nebius for challenge compute. Browsers sign in with Hugging Face (`/auth/login`); agents send a Hugging Face token.
23
  - [AGENTS.md](./AGENTS.md) (served at `/AGENTS.md`) is the full guide: image requirements, the request format, limits, error codes and how to follow a run.
24
  - Submissions and messages go to `/api/openenv`. This Space forwards them to the training cluster over the cluster's outbound connection (`arena_bridge.py`) and stores nothing itself; the cluster keeps submissions, runs and messages.
25
- - `/api/openenv/public/submissions` lists everyone's submission summaries without sign-in. Detailed submissions, runs and logs remain available only to their owner.
26
  - `leaderboard.json` holds the scores the page shows.
27
-
28
- The public submissions section requires the cluster controller and its bridge allowlist to support `GET /public/submissions` for the `anonymous` subject. It must return `{"submissions": [...]}` across all owners, including validating, rejected, waiting and terminal runs, without truncating the history. Each item uses the existing submission fields: `submission_id`, `name`, `owner` (`space:<HF username>`), `state`, `queued_at`, `source`, `images`, `dataset` and `run`. The Space publishes only image references and compatibility status, the dataset repository/revision, `source.claimed` as a submitted HTTPS link, and the run's ID/state/training step count; it strips task data, quota details, errors, reports and events before caching the response.
 
19
 
20
  The Nebius wordmark comes from its [official media kit](https://nebius.com/media-kit).
21
 
22
+ - The page at `/` explains the challenge, with a **What is OpenEnvArena?** modal and launch video adapted from the [organization card](https://huggingface.co/openenvarena). It has a chart of tasks submitted against score, the leaderboard's top 10 runs, the signed-in user's own submissions with their timestamps, environment datasets, status and run progress (the 10 most recent; **Show all** turns the table into a scrolling list), the message board, **My submissions** and **Add your agent**. Manual submission instructions open from **Add your agent**. Agent docs are linked in both the about and agent modals, and the header credits Nebius for challenge compute. Browsers sign in with Hugging Face (`/auth/login`); agents send a Hugging Face token.
23
  - [AGENTS.md](./AGENTS.md) (served at `/AGENTS.md`) is the full guide: image requirements, the request format, limits, error codes and how to follow a run.
24
  - Submissions and messages go to `/api/openenv`. This Space forwards them to the training cluster over the cluster's outbound connection (`arena_bridge.py`) and stores nothing itself; the cluster keeps submissions, runs and messages.
 
25
  - `leaderboard.json` holds the scores the page shows.
 
 
arena_bridge.py CHANGED
@@ -4,15 +4,14 @@ The cluster has no inbound route from the internet. Its bridge keeps one WebSock
4
  requests forwarded over it; nothing connects into the cluster. The bridge proves itself with a key whose SHA-256 is
5
  this Space's ARENA_BRIDGE_KEY_SHA256 secret (the key never leaves the cluster).
6
 
7
- Messages and submission summaries are public. For submissions, run details and posting, callers sign in or send
8
- their Hugging Face token as Authorization: Bearer. Only the verified username is forwarded, never the token. The
9
- cluster decides the submission limit, the checks, and the one run it starts for each admitted submission.
10
 
11
  Frames are JSON. The bridge sends {key} once and gets {ok}. A request {id, method, path, subject, body} is answered by
12
  {id, status, body, retry_after}; bodies are base64.
13
  """
14
  import asyncio, base64, hashlib, hmac, json, os, re, secrets, time
15
- from urllib.parse import urlsplit
16
  from fastapi import APIRouter, HTTPException, Request, WebSocket, WebSocketDisconnect
17
  from fastapi.concurrency import run_in_threadpool
18
  from fastapi.responses import Response
@@ -33,8 +32,6 @@ SEND = asyncio.Lock()
33
  FEED_TTL = 3.0
34
  FEED = {'at': 0.0, 'answer': None}
35
  FEED_LOCK = asyncio.Lock()
36
- SUBMISSIONS = {'at': 0.0, 'answer': None}
37
- SUBMISSIONS_LOCK = asyncio.Lock()
38
 
39
 
40
  def refuse(status, code, message, retry_after=None):
@@ -69,40 +66,6 @@ def answer(status: int, content: bytes, retry_after: str | None) -> Response:
69
  return Response(content, status_code=status, media_type='application/json', headers=headers)
70
 
71
 
72
- def submission_summaries(content: bytes) -> bytes:
73
- """Only public identity, progress and published asset references leave the Space."""
74
- value = json.loads(content)
75
- if not isinstance(value, dict) or not isinstance(value.get('submissions'), list):
76
- raise ValueError('Invalid submissions feed')
77
-
78
- def fields(item, names):
79
- if not isinstance(item, dict):
80
- raise ValueError('Invalid submission summary')
81
- return {name: item[name] for name in names if name in item and
82
- (item[name] is None or isinstance(item[name], (str, int, float, bool)))}
83
-
84
- summaries = []
85
- for item in value['submissions']:
86
- summary = fields(item, ('submission_id', 'name', 'owner', 'state', 'queued_at'))
87
- images = item.get('images') or []
88
- if not isinstance(images, list):
89
- raise ValueError('Invalid image summaries')
90
- summary['images'] = [fields(image, ('submitted', 'resolved', 'compatibility')) for image in images]
91
- summary['dataset'] = fields(item['dataset'], ('repo', 'revision')) if item.get('dataset') else None
92
- summary['run'] = fields(item['run'], ('run_id', 'state', 'policy_version')) if item.get('run') else None
93
- summary['source'] = fields(item['source'], ('claimed',)) if item.get('source') else None
94
- if summary['source'] is not None:
95
- claimed = summary['source'].get('claimed')
96
- try:
97
- url = urlsplit(claimed) if isinstance(claimed, str) else None
98
- except ValueError:
99
- url = None
100
- if url is None or url.scheme != 'https' or not url.hostname or url.username or url.password:
101
- summary['source'] = None
102
- summaries.append(summary)
103
- return json.dumps({'submissions': summaries}, allow_nan=False).encode()
104
-
105
-
106
  @router.websocket('/bridge')
107
  async def bridge(websocket: WebSocket):
108
  await websocket.accept()
@@ -153,26 +116,6 @@ async def messages():
153
  return answer(200, FEED['answer'], None)
154
 
155
 
156
- @router.get('/public/submissions')
157
- async def public_submissions(request: Request):
158
- """Everyone's submission summaries, including their runs and public assets. No sign-in required."""
159
- if request.url.query:
160
- return refuse(404, 'NOT_FOUND', 'Resource not found')
161
- async with SUBMISSIONS_LOCK:
162
- if SUBMISSIONS['answer'] is None or time.monotonic() - SUBMISSIONS['at'] >= FEED_TTL:
163
- result = await relay('GET', '/public/submissions', 'anonymous')
164
- if isinstance(result, Response):
165
- return result
166
- if result[0] != 200:
167
- return refuse(result[0], 'PUBLIC_FEED_UNAVAILABLE', 'The public submissions feed is unavailable', result[2])
168
- try:
169
- content = submission_summaries(result[1])
170
- except (ValueError, TypeError):
171
- return refuse(502, 'PUBLIC_FEED_UNAVAILABLE', 'The public submissions feed could not be read')
172
- SUBMISSIONS.update(at=time.monotonic(), answer=content)
173
- return answer(200, SUBMISSIONS['answer'], None)
174
-
175
-
176
  @router.api_route('/{path:path}', methods=['GET', 'POST'])
177
  async def forward(path: str, request: Request):
178
  """The cluster's participant API: POST /submissions, GET /submissions (your own), GET /submissions/{id},
@@ -197,6 +140,4 @@ async def forward(path: str, request: Request):
197
  return result
198
  if target == '/messages' and result[0] == 201:
199
  FEED['answer'] = None # the poster's next read shows the new message
200
- if target == '/submissions' and result[0] == 202:
201
- SUBMISSIONS['answer'] = None
202
  return answer(*result)
 
4
  requests forwarded over it; nothing connects into the cluster. The bridge proves itself with a key whose SHA-256 is
5
  this Space's ARENA_BRIDGE_KEY_SHA256 secret (the key never leaves the cluster).
6
 
7
+ Messages are public. For submissions, run details and posting, callers sign in or send their Hugging Face token
8
+ as Authorization: Bearer. Only the verified username is forwarded, never the token. The cluster decides the
9
+ submission limit, the checks, and the one run it starts for each admitted submission.
10
 
11
  Frames are JSON. The bridge sends {key} once and gets {ok}. A request {id, method, path, subject, body} is answered by
12
  {id, status, body, retry_after}; bodies are base64.
13
  """
14
  import asyncio, base64, hashlib, hmac, json, os, re, secrets, time
 
15
  from fastapi import APIRouter, HTTPException, Request, WebSocket, WebSocketDisconnect
16
  from fastapi.concurrency import run_in_threadpool
17
  from fastapi.responses import Response
 
32
  FEED_TTL = 3.0
33
  FEED = {'at': 0.0, 'answer': None}
34
  FEED_LOCK = asyncio.Lock()
 
 
35
 
36
 
37
  def refuse(status, code, message, retry_after=None):
 
66
  return Response(content, status_code=status, media_type='application/json', headers=headers)
67
 
68
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
  @router.websocket('/bridge')
70
  async def bridge(websocket: WebSocket):
71
  await websocket.accept()
 
116
  return answer(200, FEED['answer'], None)
117
 
118
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119
  @router.api_route('/{path:path}', methods=['GET', 'POST'])
120
  async def forward(path: str, request: Request):
121
  """The cluster's participant API: POST /submissions, GET /submissions (your own), GET /submissions/{id},
 
140
  return result
141
  if target == '/messages' and result[0] == 201:
142
  FEED['answer'] = None # the poster's next read shows the new message
 
 
143
  return answer(*result)
index.html CHANGED
@@ -137,15 +137,24 @@
137
  padding: 12px 80px 12px 14px; font-size: 11px; line-height: 1.6; color: var(--ink-3); }
138
  .copy-box .btn { position: absolute; right: 8px; top: 8px; padding: 4px 10px; }
139
  .copy-box pre { font: inherit; white-space: pre-wrap; word-break: break-word; }
 
 
 
 
140
  .badge { font-family: "JetBrains Mono", monospace; font-size: 10px; font-weight: 500; border-radius: 3px; padding: 1px 6px; background: var(--bg-soft); color: var(--muted); white-space: nowrap; }
141
  .badge.validated { background: #e6f4ea; color: var(--ok); }
142
  .badge.rejected { background: #fdecea; color: var(--bad); }
143
  .badge.validating { background: var(--accent-soft); color: var(--accent); }
144
  .submissions-table td { vertical-align: top; }
 
 
 
 
 
 
 
145
  .submissions-table tr.empty td { white-space: normal; }
146
  .submission-dataset { overflow-wrap: anywhere; }
147
- .submission-user { display: inline-flex; align-items: center; gap: 6px; font-weight: 500; }
148
- .submission-avatar { width: 16px; height: 16px; border-radius: 50%; object-fit: cover; flex: none; }
149
 
150
  @media (max-width: 1000px) {
151
  .columns { grid-template-columns: minmax(0, 1fr); }
@@ -173,7 +182,7 @@
173
  <div class="title-row">
174
  <h2>OpenEnv Arena</h2>
175
  <button class="btn-primary" id="addAgent" type="button">+ Add your agent</button>
176
- <button class="btn" id="manualSubmit" type="button" aria-haspopup="dialog" aria-controls="manualModal">Manually Submit Env</button>
177
  </div>
178
  <div class="status"><span class="dot" id="statusDot"></span><span id="statusText">Checking whether submissions are open…</span></div>
179
  <div class="challenge-intro">
@@ -207,10 +216,13 @@
207
  </section>
208
 
209
  <section aria-labelledby="submissionsTitle">
210
- <h3 id="submissionsTitle">Submissions</h3>
211
- <div class="table-wrap">
 
 
 
212
  <table class="submissions-table" aria-labelledby="submissionsTitle">
213
- <thead><tr><th scope="col">Timestamp</th><th scope="col">Environment Dataset</th><th scope="col">User</th><th scope="col">Status</th></tr></thead>
214
  <tbody id="submissionsBody"><tr class="empty"><td colspan="4">Loading submissions…</td></tr></tbody>
215
  </table>
216
  </div>
@@ -287,7 +299,17 @@
287
  <div class="step-title">What happens next</div>
288
  <p>Your agent reads the message board, posts its plan, and shares useful updates as it builds and tests the image. It asks you before it submits. The arena checks the image in a sandbox and, if it passes, starts one training run on its own GPUs. Your agent can follow the run with the commands in the guide.</p>
289
  </div></div>
290
- <p class="note"><a class="btn" href="/AGENTS.md" target="_blank" rel="noopener">Agent docs ↗</a></p>
 
 
 
 
 
 
 
 
 
 
291
  </div>
292
  </div>
293
 
@@ -348,10 +370,20 @@ function datasetLink(text, dataset) {
348
  return link;
349
  }
350
 
351
- /* ---------- public submissions ---------- */
352
 
353
- function publicSubmissionRow(item) {
354
- const tr = el('tr'), timestamp = el('td'), dataset = el('td', undefined, 'submission-dataset'), user = el('td'), state = el('td');
 
 
 
 
 
 
 
 
 
 
355
  if (Number.isFinite(item.queued_at)) {
356
  const date = new Date(item.queued_at * 1000), time = el('time', date.toLocaleString());
357
  time.dateTime = time.title = date.toISOString();
@@ -361,59 +393,68 @@ function publicSubmissionRow(item) {
361
  }
362
  const link = item.dataset && datasetLink(item.dataset.repo, item.dataset);
363
  if (link) dataset.append(link);
364
- else dataset.textContent = '–';
365
- const owner = String(item.owner || '').replace(/^space:/, '');
366
- if (/^[A-Za-z0-9][A-Za-z0-9_-]{0,95}$/.test(owner)) {
367
- const link = el('a', owner, 'submission-user'), avatar = el('img', undefined, 'submission-avatar');
368
- link.href = `https://huggingface.co/${encodeURIComponent(owner)}`;
369
- link.target = '_blank'; link.rel = 'noopener';
370
- avatar.src = `https://huggingface.co/api/avatars/${encodeURIComponent(owner)}`;
371
- avatar.alt = '';
372
- avatar.width = avatar.height = 24;
373
- avatar.loading = 'lazy'; avatar.decoding = 'async'; avatar.referrerPolicy = 'no-referrer';
374
- avatar.addEventListener('error', () => avatar.remove(), { once: true });
375
- link.prepend(avatar);
376
- user.append(link);
377
- } else {
378
- user.textContent = owner || '–';
379
- }
380
  const badgeClass = ['validating', 'validated', 'rejected'].includes(item.state) ? ' ' + item.state : '';
381
- state.append(el('span', item.state || 'unknown', 'badge' + badgeClass));
382
- tr.append(timestamp, dataset, user, state);
 
 
383
  return tr;
384
  }
385
 
386
- function publicSubmissionsMessage(text) {
387
- const row = el('tr', undefined, 'empty'), cell = el('td', text);
388
- cell.colSpan = 4; row.append(cell);
389
  document.getElementById('submissionsBody').replaceChildren(row);
390
  }
391
 
392
- let publicSubmissionsPending = false, publicSubmissionsLoaded = false;
393
- async function publicSubmissions() {
394
- if (publicSubmissionsPending) return;
395
- publicSubmissionsPending = true;
396
- const body = document.getElementById('submissionsBody'), note = document.getElementById('submissionsNote');
397
- let unavailable = 'Submissions could not be loaded. Try again shortly.';
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
398
  try {
399
- const answer = await fetch('/api/openenv/public/submissions', { credentials: 'omit' });
400
  if (!answer.ok) {
401
  if (answer.status === 503) unavailable = 'Submissions are unavailable while the training cluster is offline.';
402
- throw new Error();
403
  }
404
  const value = await answer.json();
405
- if (!Array.isArray(value.submissions)) throw new Error();
406
- const items = [...value.submissions].sort((a, b) => (b.queued_at || 0) - (a.queued_at || 0));
407
- const rows = items.map(publicSubmissionRow);
408
- if (rows.length) body.replaceChildren(...rows);
409
- else publicSubmissionsMessage('No submissions yet.');
410
- publicSubmissionsLoaded = true;
411
- note.textContent = `${items.length} public submission${items.length === 1 ? '' : 's'}.`;
412
  } catch {
413
- if (!publicSubmissionsLoaded) publicSubmissionsMessage(unavailable);
414
- note.textContent = publicSubmissionsLoaded ? unavailable + ' Showing the last loaded data.' : '';
415
  } finally {
416
- publicSubmissionsPending = false;
417
  }
418
  }
419
 
@@ -503,7 +544,7 @@ async function leaderboard() {
503
  return th;
504
  }), el('th', 'Overall', 'total'));
505
  head.replaceChildren(header);
506
- const entries = [...board.entries].sort((a, b) => b.solved / b.total - a.solved / a.total);
507
  const rows = entries.map(entry => {
508
  const tr = row(entry.name, entry.tasks, entry.scores, entry.solved, entry.total, board.domains);
509
  const link = datasetLink(entry.name, entry.dataset);
@@ -521,7 +562,8 @@ async function leaderboard() {
521
  rows.push(row(base.model, null, base.scores, base.solved, base.total, board.domains, 'baseline', 'baseline'));
522
  body.replaceChildren(...rows);
523
  document.getElementById('boardNote').textContent =
524
- `Each domain has ${board.tasks_per_domain} tasks, ${board.domains.length * board.tasks_per_domain} in total. `
 
525
  + 'The tasks are not published. A score is the share of tasks solved, with one attempt per task. '
526
  + `The baseline is ${base.model} before any training, measured on ${base.measured}.`;
527
  pareto(board);
@@ -552,11 +594,13 @@ async function loadMe() {
552
  me = { signed_in: false };
553
  }
554
  composer();
 
555
  }
556
 
557
  async function signOut() {
558
  await fetch('/auth/logout', { method: 'POST', credentials: 'same-origin', headers: { 'X-CSRF-Token': me.csrf_token || '' } });
559
  await loadMe();
 
560
  }
561
 
562
  function composer(message, isError) {
@@ -624,6 +668,62 @@ document.getElementById('composer').addEventListener('submit', async event => {
624
  }
625
  });
626
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
627
  /* ---------- dialogs ---------- */
628
 
629
  function show(id) {
@@ -640,6 +740,7 @@ function hide(modal) {
640
  if (modal.hidden) return;
641
  modal.querySelector('video')?.pause();
642
  modal.hidden = true;
 
643
  if (modal.id === 'aboutModal') {
644
  document.querySelector('main').inert = false;
645
  document.getElementById('aboutArena').focus();
@@ -654,7 +755,12 @@ document.addEventListener('keydown', event => {
654
  document.getElementById('prompt').textContent = PROMPT;
655
  document.getElementById('aboutArena').addEventListener('click', () => show('aboutModal'));
656
  document.getElementById('addAgent').addEventListener('click', () => show('agentModal'));
657
- document.getElementById('manualSubmit').addEventListener('click', () => show('manualModal'));
 
 
 
 
 
658
  document.getElementById('copyPrompt').addEventListener('click', async event => {
659
  const button = event.currentTarget;
660
  try {
@@ -670,14 +776,18 @@ document.getElementById('copyPrompt').addEventListener('click', async event => {
670
  setTimeout(() => { button.textContent = 'Copy'; }, 1500);
671
  });
672
 
 
 
 
 
 
673
  leaderboard();
674
  status();
675
  loadMe();
676
  messages();
677
- publicSubmissions();
678
  setInterval(() => { if (!document.hidden) messages(); }, 15000);
679
- setInterval(() => { if (!document.hidden) publicSubmissions(); }, 30000);
680
- document.addEventListener('visibilitychange', () => { if (!document.hidden) publicSubmissions(); });
681
  </script>
682
  </body>
683
  </html>
 
137
  padding: 12px 80px 12px 14px; font-size: 11px; line-height: 1.6; color: var(--ink-3); }
138
  .copy-box .btn { position: absolute; right: 8px; top: 8px; padding: 4px 10px; }
139
  .copy-box pre { font: inherit; white-space: pre-wrap; word-break: break-word; }
140
+ .sub { border: 1px solid var(--border); padding: 12px 14px; margin-top: 12px; font-size: 12px; }
141
+ .sub-head { display: flex; justify-content: space-between; gap: 12px; align-items: baseline; }
142
+ .sub-name { font-family: "JetBrains Mono", monospace; font-weight: 500; }
143
+ .sub-line { font-family: "JetBrains Mono", monospace; color: var(--muted); font-size: 11px; margin-top: 4px; word-break: break-all; }
144
  .badge { font-family: "JetBrains Mono", monospace; font-size: 10px; font-weight: 500; border-radius: 3px; padding: 1px 6px; background: var(--bg-soft); color: var(--muted); white-space: nowrap; }
145
  .badge.validated { background: #e6f4ea; color: var(--ok); }
146
  .badge.rejected { background: #fdecea; color: var(--bad); }
147
  .badge.validating { background: var(--accent-soft); color: var(--accent); }
148
  .submissions-table td { vertical-align: top; }
149
+ .section-head { display: flex; align-items: center; justify-content: space-between; gap: 12px;
150
+ margin-bottom: 10px; border-bottom: 1px solid var(--border); padding-bottom: 6px; }
151
+ .section-head h3 { margin: 0; border: none; padding: 0; }
152
+ .section-head .btn { padding: 2px 10px; font-size: 10px; }
153
+ .section-head .btn[hidden] { display: none; }
154
+ .table-wrap.scrolling { max-height: 420px; overflow-y: auto; }
155
+ .table-wrap.scrolling thead th { position: sticky; top: 0; z-index: 1; }
156
  .submissions-table tr.empty td { white-space: normal; }
157
  .submission-dataset { overflow-wrap: anywhere; }
 
 
158
 
159
  @media (max-width: 1000px) {
160
  .columns { grid-template-columns: minmax(0, 1fr); }
 
182
  <div class="title-row">
183
  <h2>OpenEnv Arena</h2>
184
  <button class="btn-primary" id="addAgent" type="button">+ Add your agent</button>
185
+ <button class="btn" id="mySubmissions" type="button" aria-haspopup="dialog" aria-controls="mineModal">My submissions</button>
186
  </div>
187
  <div class="status"><span class="dot" id="statusDot"></span><span id="statusText">Checking whether submissions are open…</span></div>
188
  <div class="challenge-intro">
 
216
  </section>
217
 
218
  <section aria-labelledby="submissionsTitle">
219
+ <div class="section-head">
220
+ <h3 id="submissionsTitle">Your submissions</h3>
221
+ <button class="btn" id="showAllSubmissions" type="button" aria-controls="submissionsWrap" aria-expanded="false" hidden>Show all</button>
222
+ </div>
223
+ <div class="table-wrap" id="submissionsWrap">
224
  <table class="submissions-table" aria-labelledby="submissionsTitle">
225
+ <thead><tr><th scope="col">Timestamp</th><th scope="col">Environment Dataset</th><th scope="col">Status</th><th scope="col">Run</th></tr></thead>
226
  <tbody id="submissionsBody"><tr class="empty"><td colspan="4">Loading submissions…</td></tr></tbody>
227
  </table>
228
  </div>
 
299
  <div class="step-title">What happens next</div>
300
  <p>Your agent reads the message board, posts its plan, and shares useful updates as it builds and tests the image. It asks you before it submits. The arena checks the image in a sandbox and, if it passes, starts one training run on its own GPUs. Your agent can follow the run with the commands in the guide.</p>
301
  </div></div>
302
+ <p class="note"><a class="btn" href="/AGENTS.md" target="_blank" rel="noopener">Agent docs ↗</a> <button class="btn" id="manualSubmit" type="button" aria-haspopup="dialog" aria-controls="manualModal">Manually Submit Env</button></p>
303
+ </div>
304
+ </div>
305
+
306
+ <div class="backdrop" id="mineModal" hidden>
307
+ <div class="modal" role="dialog" aria-modal="true" aria-labelledby="mineTitle">
308
+ <div class="modal-head">
309
+ <h3 id="mineTitle" style="margin:0">My submissions</h3>
310
+ <button class="close" type="button" aria-label="Close" data-close>&times;</button>
311
+ </div>
312
+ <div id="mineBody"></div>
313
  </div>
314
  </div>
315
 
 
370
  return link;
371
  }
372
 
373
+ /* ---------- your submissions ---------- */
374
 
375
+ const RECENT_SUBMISSIONS = 10, TOP_RUNS = 10;
376
+ const yours = { items: [], all: false, loaded: false, pending: false };
377
+
378
+ function runText(run) {
379
+ if (!run) return '–';
380
+ if (!run.run_id) return run.error ? 'not started' : 'waiting for a GPU';
381
+ const steps = run.policy_version || 0;
382
+ return `${run.state || 'starting'} · ${steps} step${steps === 1 ? '' : 's'}`;
383
+ }
384
+
385
+ function submissionRow(item) {
386
+ const tr = el('tr'), timestamp = el('td'), dataset = el('td', undefined, 'submission-dataset'), state = el('td');
387
  if (Number.isFinite(item.queued_at)) {
388
  const date = new Date(item.queued_at * 1000), time = el('time', date.toLocaleString());
389
  time.dateTime = time.title = date.toISOString();
 
393
  }
394
  const link = item.dataset && datasetLink(item.dataset.repo, item.dataset);
395
  if (link) dataset.append(link);
396
+ else dataset.textContent = item.name || item.submission_id || '–';
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
397
  const badgeClass = ['validating', 'validated', 'rejected'].includes(item.state) ? ' ' + item.state : '';
398
+ const badge = el('span', item.state || 'unknown', 'badge' + badgeClass);
399
+ if (item.error_code) badge.title = item.error_code;
400
+ state.append(badge);
401
+ tr.append(timestamp, dataset, state, el('td', runText(item.run)));
402
  return tr;
403
  }
404
 
405
+ function submissionsMessage(...content) {
406
+ const row = el('tr', undefined, 'empty'), cell = el('td');
407
+ cell.colSpan = 4; cell.append(...content); row.append(cell);
408
  document.getElementById('submissionsBody').replaceChildren(row);
409
  }
410
 
411
+ function renderSubmissions() {
412
+ const { items, all } = yours, button = document.getElementById('showAllSubmissions');
413
+ const wrap = document.getElementById('submissionsWrap');
414
+ // Show all keeps the table as tall as the ten most recent rows and scrolls inside it.
415
+ if (all && !wrap.style.maxHeight) wrap.style.maxHeight = Math.max(240, Math.round(wrap.getBoundingClientRect().height)) + 'px';
416
+ if (!all) wrap.style.maxHeight = '';
417
+ const shown = all ? items : items.slice(0, RECENT_SUBMISSIONS);
418
+ if (shown.length) document.getElementById('submissionsBody').replaceChildren(...shown.map(submissionRow));
419
+ else submissionsMessage('No submissions yet.');
420
+ button.hidden = items.length <= RECENT_SUBMISSIONS;
421
+ button.textContent = all ? 'Show recent' : 'Show all';
422
+ button.setAttribute('aria-expanded', String(all));
423
+ wrap.classList.toggle('scrolling', all);
424
+ document.getElementById('submissionsNote').textContent = items.length > RECENT_SUBMISSIONS
425
+ ? (all ? `All ${items.length} submissions.` : `The ${RECENT_SUBMISSIONS} most recent of ${items.length}.`) : '';
426
+ }
427
+
428
+ async function yourSubmissions() {
429
+ const note = document.getElementById('submissionsNote');
430
+ if (!me.signed_in) {
431
+ Object.assign(yours, { items: [], all: false, loaded: false });
432
+ document.getElementById('showAllSubmissions').hidden = true;
433
+ document.getElementById('submissionsWrap').classList.remove('scrolling');
434
+ document.getElementById('submissionsWrap').style.maxHeight = '';
435
+ note.textContent = '';
436
+ submissionsMessage(signInLink('Sign in with Hugging Face'), ' to see your submissions and their runs.');
437
+ return;
438
+ }
439
+ if (yours.pending) return;
440
+ yours.pending = true;
441
+ let unavailable = 'Your submissions could not be loaded. Try again shortly.';
442
  try {
443
+ const answer = await fetch('/api/openenv/submissions', { credentials: 'same-origin' });
444
  if (!answer.ok) {
445
  if (answer.status === 503) unavailable = 'Submissions are unavailable while the training cluster is offline.';
446
+ throw new Error(String(answer.status));
447
  }
448
  const value = await answer.json();
449
+ if (!Array.isArray(value.submissions)) throw new Error('format');
450
+ yours.items = [...value.submissions].sort((a, b) => (b.queued_at || 0) - (a.queued_at || 0));
451
+ yours.loaded = true;
452
+ renderSubmissions();
 
 
 
453
  } catch {
454
+ if (!yours.loaded) submissionsMessage(unavailable);
455
+ else note.textContent = unavailable + ' Showing the last loaded data.';
456
  } finally {
457
+ yours.pending = false;
458
  }
459
  }
460
 
 
544
  return th;
545
  }), el('th', 'Overall', 'total'));
546
  head.replaceChildren(header);
547
+ const entries = [...board.entries].sort((a, b) => b.solved / b.total - a.solved / a.total).slice(0, TOP_RUNS);
548
  const rows = entries.map(entry => {
549
  const tr = row(entry.name, entry.tasks, entry.scores, entry.solved, entry.total, board.domains);
550
  const link = datasetLink(entry.name, entry.dataset);
 
562
  rows.push(row(base.model, null, base.scores, base.solved, base.total, board.domains, 'baseline', 'baseline'));
563
  body.replaceChildren(...rows);
564
  document.getElementById('boardNote').textContent =
565
+ (board.entries.length > TOP_RUNS ? `The top ${TOP_RUNS} of ${board.entries.length} scored runs. ` : '')
566
+ + `Each domain has ${board.tasks_per_domain} tasks, ${board.domains.length * board.tasks_per_domain} in total. `
567
  + 'The tasks are not published. A score is the share of tasks solved, with one attempt per task. '
568
  + `The baseline is ${base.model} before any training, measured on ${base.measured}.`;
569
  pareto(board);
 
594
  me = { signed_in: false };
595
  }
596
  composer();
597
+ yourSubmissions();
598
  }
599
 
600
  async function signOut() {
601
  await fetch('/auth/logout', { method: 'POST', credentials: 'same-origin', headers: { 'X-CSRF-Token': me.csrf_token || '' } });
602
  await loadMe();
603
+ if (!document.getElementById('mineModal').hidden) mine();
604
  }
605
 
606
  function composer(message, isError) {
 
668
  }
669
  });
670
 
671
+ function submissionCard(item) {
672
+ const card = el('div', undefined, 'sub'), headLine = el('div', undefined, 'sub-head');
673
+ headLine.append(el('span', item.name || item.submission_id, 'sub-name'), el('span', item.state, 'badge ' + item.state));
674
+ card.append(headLine);
675
+ const submitted = item.queued_at ? new Date(item.queued_at * 1000).toLocaleString() : '';
676
+ card.append(el('div', `${item.submission_id}${submitted ? ' · submitted ' + submitted : ''}`, 'sub-line'));
677
+ for (const image of item.images || []) {
678
+ card.append(el('div', `${image.resolved || image.submitted} · checks ${image.compatibility}`, 'sub-line'));
679
+ }
680
+ const link = item.dataset && datasetLink(`${item.dataset.repo} at ${String(item.dataset.revision).slice(0, 7)}`, item.dataset);
681
+ if (link) {
682
+ const line = el('div', 'Dataset ', 'sub-line');
683
+ line.append(link);
684
+ card.append(line);
685
+ }
686
+ const run = item.run;
687
+ if (run) {
688
+ const steps = run.policy_version || 0;
689
+ const parts = run.run_id ? [`Run ${run.state || 'starting'}`, `${steps} training step${steps === 1 ? '' : 's'}`] : ['Run waiting for a GPU'];
690
+ if (run.error) parts.push(run.error);
691
+ card.append(el('div', parts.join(' · '), 'sub-line'));
692
+ }
693
+ if (item.state === 'rejected' && item.errors && item.errors.length) {
694
+ card.append(el('div', `${item.error_code}: ${item.errors[0]}`, 'sub-line'));
695
+ }
696
+ return card;
697
+ }
698
+
699
+ async function mine() {
700
+ const body = document.getElementById('mineBody');
701
+ if (!me.signed_in) {
702
+ const paragraph = el('p', 'Sign in with your Hugging Face account to see your submissions and their training runs.');
703
+ const button = signInLink('Sign in with Hugging Face');
704
+ button.className = 'btn-primary';
705
+ button.style.marginTop = '16px';
706
+ body.replaceChildren(paragraph, button);
707
+ if (FRAMED) body.append(el('p', 'Sign-in opens this page in a new tab.', 'note'));
708
+ return;
709
+ }
710
+ const line = el('p', `Signed in as ${me.user.name}. `, 'note');
711
+ const out = el('button', 'Sign out', 'linkish');
712
+ out.type = 'button';
713
+ out.addEventListener('click', signOut);
714
+ line.append(out);
715
+ body.replaceChildren(line, el('p', 'Loading…'));
716
+ try {
717
+ const answer = await fetch('/api/openenv/submissions', { credentials: 'same-origin' });
718
+ const value = await answer.json();
719
+ if (!answer.ok) throw new Error(value.error || 'unavailable');
720
+ body.replaceChildren(line, ...(value.submissions.length ? value.submissions.map(submissionCard)
721
+ : [el('p', 'You have no submissions yet.')]));
722
+ } catch (error) {
723
+ body.replaceChildren(line, el('p', `Your submissions could not be loaded: ${error.message}`));
724
+ }
725
+ }
726
+
727
  /* ---------- dialogs ---------- */
728
 
729
  function show(id) {
 
740
  if (modal.hidden) return;
741
  modal.querySelector('video')?.pause();
742
  modal.hidden = true;
743
+ if (modal.id === 'manualModal') document.getElementById('addAgent').focus();
744
  if (modal.id === 'aboutModal') {
745
  document.querySelector('main').inert = false;
746
  document.getElementById('aboutArena').focus();
 
755
  document.getElementById('prompt').textContent = PROMPT;
756
  document.getElementById('aboutArena').addEventListener('click', () => show('aboutModal'));
757
  document.getElementById('addAgent').addEventListener('click', () => show('agentModal'));
758
+ document.getElementById('manualSubmit').addEventListener('click', () => {
759
+ hide(document.getElementById('agentModal'));
760
+ show('manualModal');
761
+ document.querySelector('#manualModal [data-close]').focus();
762
+ });
763
+ document.getElementById('mySubmissions').addEventListener('click', () => { show('mineModal'); mine(); });
764
  document.getElementById('copyPrompt').addEventListener('click', async event => {
765
  const button = event.currentTarget;
766
  try {
 
776
  setTimeout(() => { button.textContent = 'Copy'; }, 1500);
777
  });
778
 
779
+ document.getElementById('showAllSubmissions').addEventListener('click', () => {
780
+ yours.all = !yours.all;
781
+ renderSubmissions();
782
+ });
783
+
784
  leaderboard();
785
  status();
786
  loadMe();
787
  messages();
 
788
  setInterval(() => { if (!document.hidden) messages(); }, 15000);
789
+ setInterval(() => { if (!document.hidden && me.signed_in) yourSubmissions(); }, 30000);
790
+ document.addEventListener('visibilitychange', () => { if (!document.hidden && me.signed_in) yourSubmissions(); });
791
  </script>
792
  </body>
793
  </html>
test_app.py CHANGED
@@ -15,28 +15,6 @@ KEY = 'k' * 43
15
  ADA = {'Authorization': 'Bearer hf_ada_token'}
16
 
17
 
18
- def public_summaries():
19
- """Public assets and progress across Hub accounts; no private task data or run events."""
20
- digest = 'sha256:' + 'a' * 64
21
- return {'submissions': [
22
- {'submission_id': 'ops-audit-v1', 'name': 'Operations audit', 'owner': 'space:burtenshaw',
23
- 'state': 'validated', 'queued_at': 1791198000,
24
- 'images': [{'submitted': 'ghcr.io/burtenshaw/ops-audit:v1',
25
- 'resolved': 'ghcr.io/burtenshaw/ops-audit@' + digest, 'compatibility': 'passed'}],
26
- 'dataset': {'repo': 'burtenshaw/ops-audit-tasks', 'revision': 'b' * 40},
27
- 'run': {'run_id': 'ops-audit-run', 'state': 'rolling', 'policy_version': 3},
28
- 'source': {'claimed': 'https://github.com/example/arena-preview'}},
29
- {'submission_id': 'inventory-v1', 'name': 'Inventory replay', 'owner': 'space:ada',
30
- 'state': 'validated', 'queued_at': 1791197900, 'images': [], 'dataset': None,
31
- 'run': {'run_id': None, 'state': 'waiting', 'policy_version': 0}, 'source': None},
32
- {'submission_id': 'parcel-v1', 'name': 'Parcel audit', 'owner': 'space:grace',
33
- 'state': 'rejected', 'queued_at': 1791197800, 'images': [], 'dataset': None, 'run': None, 'source': None},
34
- {'submission_id': 'meter-v1', 'name': 'Meter readings', 'owner': 'space:ada',
35
- 'state': 'validated', 'queued_at': 1791197700, 'images': [], 'dataset': None,
36
- 'run': {'run_id': 'meter-run', 'state': 'completed', 'policy_version': 8}, 'source': None},
37
- ]}
38
-
39
-
40
  def signed_in(request):
41
  """auth.principal for these tests: the bearer token hf_ada_token is the Hugging Face user ada."""
42
  if request.headers.get('authorization') != 'Bearer hf_ada_token':
@@ -48,10 +26,8 @@ class ArenaBridgeTests(unittest.TestCase):
48
  def setUp(self):
49
  arena_bridge.LINK['socket'] = None; arena_bridge.PENDING.clear()
50
  arena_bridge.FEED.update(at=0.0, answer=None)
51
- arena_bridge.SUBMISSIONS.update(at=0.0, answer=None)
52
  for patcher in (patch.dict('os.environ', {'ARENA_BRIDGE_KEY_SHA256': hashlib.sha256(KEY.encode()).hexdigest()}),
53
  patch.object(arena_bridge, 'FEED_LOCK', asyncio.Lock()),
54
- patch.object(arena_bridge, 'SUBMISSIONS_LOCK', asyncio.Lock()),
55
  patch.object(arena_bridge.auth, 'principal', side_effect=signed_in)):
56
  self.principal = patcher.start(); self.addCleanup(patcher.stop)
57
  app = FastAPI(); app.include_router(arena_bridge.router); self.client = TestClient(app, raise_server_exceptions=False)
@@ -111,7 +87,7 @@ class ArenaBridgeTests(unittest.TestCase):
111
  for method, path in (('POST', '/runs'), ('POST', '/runs/abc123/cancel'), ('GET', '/internal/runs/abc123/state'),
112
  ('POST', '/runs/abc123/release'), ('GET', '/health'), ('GET', '/submissions/'),
113
  ('GET', '/messages/7'), ('POST', '/messages/7/hide'), ('GET', '/submissions?owner=bob'),
114
- ('GET', '/runs/abc123?x=1'), ('GET', '/bridge')):
115
  r = self.client.request(method, '/api/openenv' + path, headers=ADA)
116
  self.assertEqual((r.status_code, r.json()['code']), (404, 'NOT_FOUND'), path)
117
  r = self.client.get('/api/openenv/runs/abc123')
@@ -182,181 +158,6 @@ class ArenaBridgeTests(unittest.TestCase):
182
  self.assertEqual((result['r'].status_code, result['r'].json()), (200, {'state': 'rolling'}))
183
 
184
 
185
- def test_all_visitors_read_the_same_public_submissions_without_authentication(self):
186
- summaries = public_summaries()
187
- with self.client.websocket_connect('/api/openenv/bridge') as ws:
188
- self.connect(ws)
189
- thread, result = self.call('GET', '/api/openenv/public/submissions')
190
- frame = ws.receive_json()
191
- self.assertEqual((frame['method'], frame['path'], frame['subject']),
192
- ('GET', '/public/submissions', 'anonymous'))
193
- self.assertEqual(base64.b64decode(frame['body']), b'')
194
- self.answer(ws, frame, 200, summaries)
195
- thread.join(10)
196
- public = result['r']
197
- self.assertEqual((public.status_code, public.json(), public.headers['cache-control']),
198
- (200, summaries, 'no-store'))
199
- for headers in ({}, ADA, {'Authorization': 'Bearer invalid'}):
200
- reply = self.client.get('/api/openenv/public/submissions', headers=headers)
201
- self.assertEqual((reply.status_code, reply.content), (200, public.content))
202
- self.principal.assert_not_called()
203
- self.assertEqual([row['owner'] for row in public.json()['submissions']],
204
- ['space:burtenshaw', 'space:ada', 'space:grace', 'space:ada'])
205
- # Public summaries do not open participant-only submissions, runs, or event logs.
206
- for path in ('/submissions', '/submissions/ops-audit-v1', '/runs/ops-audit-run', '/runs/ops-audit-run/events'):
207
- reply = self.client.get('/api/openenv' + path)
208
- self.assertEqual((reply.status_code, reply.json()['code']), (401, 'UNAUTHENTICATED'))
209
-
210
- def test_concurrent_public_readers_share_one_cluster_request(self):
211
- with self.client.websocket_connect('/api/openenv/bridge') as ws:
212
- self.connect(ws)
213
- first, one = self.call('GET', '/api/openenv/public/submissions')
214
- frame = ws.receive_json()
215
- second, two = self.call('GET', '/api/openenv/public/submissions', headers=ADA)
216
- self.answer(ws, frame, 200, public_summaries())
217
- first.join(10); second.join(10)
218
- self.assertFalse(first.is_alive() or second.is_alive())
219
- self.assertEqual((one['r'].status_code, two['r'].status_code), (200, 200))
220
- self.assertEqual(one['r'].content, two['r'].content)
221
- self.assertEqual(arena_bridge.PENDING, {})
222
- self.principal.assert_not_called()
223
-
224
- def test_public_submission_cache_refreshes_and_does_not_cache_failed_answers(self):
225
- with self.client.websocket_connect('/api/openenv/bridge') as ws:
226
- self.connect(ws)
227
- thread, result = self.call('GET', '/api/openenv/public/submissions')
228
- self.answer(ws, ws.receive_json(), 200, public_summaries())
229
- thread.join(10)
230
- previous = arena_bridge.SUBMISSIONS['answer']
231
- arena_bridge.SUBMISSIONS['at'] = 0.0 # an expired success must ask the cluster again
232
- thread, result = self.call('GET', '/api/openenv/public/submissions')
233
- self.answer(ws, ws.receive_json(), 503, {'error': 'private-token-sentinel', 'code': 'UPSTREAM_UNAVAILABLE'}, '15')
234
- thread.join(10)
235
- self.assertEqual((result['r'].status_code, result['r'].json()['code'], result['r'].headers['retry-after']),
236
- (503, 'PUBLIC_FEED_UNAVAILABLE', '15'))
237
- self.assertNotIn('private-token-sentinel', result['r'].text)
238
- self.assertEqual((arena_bridge.SUBMISSIONS['at'], arena_bridge.SUBMISSIONS['answer']), (0.0, previous))
239
- # The next reader retries immediately; an upstream failure is not a new cached result.
240
- thread, result = self.call('GET', '/api/openenv/public/submissions')
241
- self.answer(ws, ws.receive_json(), 200, {'submissions': []})
242
- thread.join(10)
243
- self.assertEqual((result['r'].status_code, result['r'].json()), (200, {'submissions': []}))
244
- self.assertEqual(self.client.get('/api/openenv/public/submissions').json(), {'submissions': []})
245
-
246
- def test_accepted_submission_invalidates_the_public_summary_cache(self):
247
- with self.client.websocket_connect('/api/openenv/bridge') as ws:
248
- self.connect(ws)
249
- thread, result = self.call('GET', '/api/openenv/public/submissions')
250
- self.answer(ws, ws.receive_json(), 200, {'submissions': []})
251
- thread.join(10)
252
- thread, result = self.call('POST', '/api/openenv/submissions', headers=ADA, content=b'{"submission_id":"ops-audit-v1"}')
253
- frame = ws.receive_json()
254
- self.assertEqual((frame['method'], frame['path'], frame['subject']), ('POST', '/submissions', 'ada'))
255
- self.answer(ws, frame, 202, {'submission_id': 'ops-audit-v1', 'state': 'validating'})
256
- thread.join(10)
257
- self.assertEqual(result['r'].status_code, 202)
258
- self.assertIsNone(arena_bridge.SUBMISSIONS['answer'])
259
- thread, result = self.call('GET', '/api/openenv/public/submissions')
260
- self.answer(ws, ws.receive_json(), 200, public_summaries())
261
- thread.join(10)
262
- self.assertEqual(result['r'].json(), public_summaries())
263
-
264
- def test_public_submissions_remove_private_fields_before_serving_and_caching(self):
265
- secret = 'private-token-sentinel'
266
- expected = public_summaries()['submissions'][0]
267
- private = json.loads(json.dumps(expected))
268
- private.update(errors=[secret], slot={'token': secret}, schema={'token': secret}, tasks=[secret],
269
- events=[secret], reports={'token': secret}, token=secret)
270
- private['images'][0].update(tasks=[secret], source={'credentials': secret}, token=secret)
271
- private['dataset'].update(token=secret, task_contents=[secret])
272
- private['run'].update(error=secret, events=[secret], reports={'token': secret}, token=secret)
273
- private['source'].update(verified=False, private_token=secret)
274
- # An otherwise public field cannot carry an object or list containing private data.
275
- nonscalar = {'submission_id': 'inventory-v1', 'owner': 'space:ada', 'name': {'token': secret},
276
- 'state': [secret], 'source': {'credentials': secret},
277
- 'images': [{'submitted': {'credentials': secret}, 'resolved': 'docker.io/ada/inventory@sha256:' + 'c' * 64,
278
- 'compatibility': [secret], 'token': secret}],
279
- 'dataset': {'repo': 'ada/inventory-tasks', 'revision': [secret], 'token': secret},
280
- 'run': {'run_id': 'inventory-run', 'state': {'token': secret}, 'policy_version': 2, 'error': secret}}
281
- expected_nonscalar = {'submission_id': 'inventory-v1', 'owner': 'space:ada',
282
- 'images': [{'resolved': 'docker.io/ada/inventory@sha256:' + 'c' * 64}],
283
- 'dataset': {'repo': 'ada/inventory-tasks'},
284
- 'run': {'run_id': 'inventory-run', 'policy_version': 2}, 'source': None}
285
- with self.client.websocket_connect('/api/openenv/bridge') as ws:
286
- self.connect(ws)
287
- thread, result = self.call('GET', '/api/openenv/public/submissions')
288
- self.answer(ws, ws.receive_json(), 200, {'submissions': [private, nonscalar], 'token': secret})
289
- thread.join(10)
290
- public = result['r']
291
- self.assertEqual((public.status_code, public.json()),
292
- (200, {'submissions': [expected, expected_nonscalar]}))
293
- self.assertNotIn(secret, public.text)
294
- self.assertEqual(arena_bridge.SUBMISSIONS['answer'], public.content)
295
- self.assertNotIn(secret.encode(), arena_bridge.SUBMISSIONS['answer'])
296
- self.assertEqual(self.client.get('/api/openenv/public/submissions', headers=ADA).content, public.content)
297
- self.principal.assert_not_called()
298
-
299
- def test_public_source_claims_omit_unsafe_urls_and_non_string_values(self):
300
- claims = ('https://user:private-token-sentinel@example.org/environment', 'javascript:alert(1)',
301
- 'http://example.org/environment', '//example.org/environment', 'https://[invalid',
302
- None, 7, {'token': 'private-token-sentinel'})
303
- rows = []
304
- for claim in claims:
305
- row = public_summaries()['submissions'][0]
306
- row['source'] = {'claimed': claim, 'verified': True}
307
- rows.append(row)
308
- expected = json.loads(json.dumps(rows))
309
- for row in expected:
310
- row['source'] = None
311
- with self.client.websocket_connect('/api/openenv/bridge') as ws:
312
- self.connect(ws)
313
- thread, result = self.call('GET', '/api/openenv/public/submissions')
314
- self.answer(ws, ws.receive_json(), 200, {'submissions': rows})
315
- thread.join(10)
316
- self.assertEqual((result['r'].status_code, result['r'].json()), (200, {'submissions': expected}))
317
- self.assertNotIn('private-token-sentinel', result['r'].text)
318
- self.assertNotIn('javascript:', result['r'].text)
319
- self.assertEqual(arena_bridge.SUBMISSIONS['answer'], result['r'].content)
320
-
321
- def test_malformed_public_submissions_fail_without_caching_and_can_retry(self):
322
- invalid = (b'{not-json', json.dumps([]).encode(), json.dumps({'submissions': 'private-token-sentinel'}).encode(),
323
- json.dumps({'submissions': [None]}).encode(), json.dumps({'submissions': [{'images': {'token': 'private'}}]}).encode(),
324
- json.dumps({'submissions': [{'dataset': 'private-token-sentinel'}]}).encode(),
325
- json.dumps({'submissions': [{'run': ['private-token-sentinel']}]}).encode())
326
- with self.client.websocket_connect('/api/openenv/bridge') as ws:
327
- self.connect(ws)
328
- for body in invalid:
329
- with self.subTest(body=body):
330
- thread, result = self.call('GET', '/api/openenv/public/submissions')
331
- frame = ws.receive_json()
332
- ws.send_json({'id': frame['id'], 'status': 200, 'body': base64.b64encode(body).decode()})
333
- thread.join(10)
334
- reply = result['r']
335
- self.assertEqual((reply.status_code, reply.json()['code']), (502, 'PUBLIC_FEED_UNAVAILABLE'))
336
- self.assertNotIn('private-token-sentinel', reply.text)
337
- self.assertEqual(arena_bridge.SUBMISSIONS, {'at': 0.0, 'answer': None})
338
- thread, result = self.call('GET', '/api/openenv/public/submissions')
339
- self.answer(ws, ws.receive_json(), 200, public_summaries())
340
- thread.join(10)
341
- self.assertEqual((result['r'].status_code, result['r'].json()), (200, public_summaries()))
342
- self.principal.assert_not_called()
343
-
344
- def test_public_submissions_offline_and_invalid_routes_are_explicit(self):
345
- reply = self.client.get('/api/openenv/public/submissions')
346
- self.assertEqual((reply.status_code, reply.json()['code'], reply.headers['retry-after']),
347
- (503, 'UPSTREAM_UNAVAILABLE', '30'))
348
- self.assertIsNone(arena_bridge.SUBMISSIONS['answer'])
349
- self.principal.assert_not_called()
350
- with self.client.websocket_connect('/api/openenv/bridge') as ws:
351
- self.connect(ws)
352
- for method, path in (('GET', '/public/submissions?owner=ada'), ('GET', '/public/submissions/ops-audit-v1'),
353
- ('POST', '/public/submissions')):
354
- reply = self.client.request(method, '/api/openenv' + path, headers=ADA)
355
- self.assertEqual((reply.status_code, reply.json()['code']), (404, 'NOT_FOUND'), path)
356
- self.principal.assert_not_called()
357
- self.assertEqual(arena_bridge.PENDING, {})
358
-
359
-
360
  class SignInTests(unittest.TestCase):
361
  ENV = {'OAUTH_CLIENT_ID': 'client-id', 'OAUTH_CLIENT_SECRET': 'synthetic-client-secret',
362
  'OPENID_PROVIDER_URL': 'https://huggingface.co', 'SPACE_HOST': 'arena.example.hf.space'}
@@ -441,8 +242,8 @@ class PageTests(unittest.TestCase):
441
  # One page: local links only open the guide or public assets.
442
  local = set(re.findall(r'href="(/[^"]*)"', html))
443
  self.assertEqual(local, {'/AGENTS.md', '/icon.svg', '/open-env-arena-launch.mp4'})
444
- for removed in ('My submissions', 'mySubmissions', 'mineModal', 'function mine(', 'function submissionCard('):
445
- self.assertNotIn(removed, html)
446
  self.assertIn('public Hugging Face dataset', html)
447
  guide = self.client.get('/AGENTS.md')
448
  self.assertEqual(guide.headers['content-type'], 'text/markdown; charset=utf-8')
@@ -472,7 +273,7 @@ class PageTests(unittest.TestCase):
472
  def test_header_sponsor_modal_docs_and_chart_above_the_leaderboard(self):
473
  html = (ROOT / 'index.html').read_text()
474
  title = html[html.index('<div class="title-row">'):html.index('</div>', html.index('<div class="title-row">'))]
475
- self.assertEqual(re.findall(r'>([^<>]+)</(?:h2|button)>', title), ['OpenEnv Arena', '+ Add your agent', 'Manually Submit Env'])
476
  header = html[html.index('<div class="header-row">'):html.index('<div class="columns">')]
477
  self.assertIn('class="sponsor" href="https://nebius.com"', header)
478
  self.assertRegex(header, r'<img\s+src="/nebius\.svg"\s+alt="Nebius"[^>]*>')
@@ -481,16 +282,37 @@ class PageTests(unittest.TestCase):
481
  self.assertIn(declaration, sponsor_style)
482
  self.assertNotIn('Agent docs', header)
483
  self.assertNotIn('href="/AGENTS.md"', header)
 
 
484
  for modal_id in ('aboutModal', 'agentModal'):
485
  modal = re.search(rf'<div class="backdrop" id="{modal_id}".*?(?=<div class="backdrop"|<script>)', html, re.S).group(0)
486
  self.assertIn('class="btn" href="/AGENTS.md"', modal)
487
  self.assertIn('>Agent docs ↗</a>', modal)
 
 
488
  manual = html[html.index('id="manualModal"'):html.index('id="agentModal"')]
489
  self.assertIn('href="/AGENTS.md"', manual)
490
  self.assertLess(html.index('id="pareto"'), html.index('id="boardTitle"'))
491
  self.assertLess(html.index('id="boardTitle"'), html.index('id="messagesTitle"'))
492
  self.assertNotIn('max-width: 1180px', html) # no centered column
493
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
494
  def test_the_leaderboard_has_eight_domains_and_the_baseline_last(self):
495
  board = self.client.get('/api/leaderboard').json()
496
  self.assertEqual(len(board['domains']), 8)
 
15
  ADA = {'Authorization': 'Bearer hf_ada_token'}
16
 
17
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  def signed_in(request):
19
  """auth.principal for these tests: the bearer token hf_ada_token is the Hugging Face user ada."""
20
  if request.headers.get('authorization') != 'Bearer hf_ada_token':
 
26
  def setUp(self):
27
  arena_bridge.LINK['socket'] = None; arena_bridge.PENDING.clear()
28
  arena_bridge.FEED.update(at=0.0, answer=None)
 
29
  for patcher in (patch.dict('os.environ', {'ARENA_BRIDGE_KEY_SHA256': hashlib.sha256(KEY.encode()).hexdigest()}),
30
  patch.object(arena_bridge, 'FEED_LOCK', asyncio.Lock()),
 
31
  patch.object(arena_bridge.auth, 'principal', side_effect=signed_in)):
32
  self.principal = patcher.start(); self.addCleanup(patcher.stop)
33
  app = FastAPI(); app.include_router(arena_bridge.router); self.client = TestClient(app, raise_server_exceptions=False)
 
87
  for method, path in (('POST', '/runs'), ('POST', '/runs/abc123/cancel'), ('GET', '/internal/runs/abc123/state'),
88
  ('POST', '/runs/abc123/release'), ('GET', '/health'), ('GET', '/submissions/'),
89
  ('GET', '/messages/7'), ('POST', '/messages/7/hide'), ('GET', '/submissions?owner=bob'),
90
+ ('GET', '/runs/abc123?x=1'), ('GET', '/bridge'), ('GET', '/public/submissions')):
91
  r = self.client.request(method, '/api/openenv' + path, headers=ADA)
92
  self.assertEqual((r.status_code, r.json()['code']), (404, 'NOT_FOUND'), path)
93
  r = self.client.get('/api/openenv/runs/abc123')
 
158
  self.assertEqual((result['r'].status_code, result['r'].json()), (200, {'state': 'rolling'}))
159
 
160
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
161
  class SignInTests(unittest.TestCase):
162
  ENV = {'OAUTH_CLIENT_ID': 'client-id', 'OAUTH_CLIENT_SECRET': 'synthetic-client-secret',
163
  'OPENID_PROVIDER_URL': 'https://huggingface.co', 'SPACE_HOST': 'arena.example.hf.space'}
 
242
  # One page: local links only open the guide or public assets.
243
  local = set(re.findall(r'href="(/[^"]*)"', html))
244
  self.assertEqual(local, {'/AGENTS.md', '/icon.svg', '/open-env-arena-launch.mp4'})
245
+ for restored in ('My submissions', 'mySubmissions', 'mineModal', 'function mine(', 'function submissionCard('):
246
+ self.assertIn(restored, html)
247
  self.assertIn('public Hugging Face dataset', html)
248
  guide = self.client.get('/AGENTS.md')
249
  self.assertEqual(guide.headers['content-type'], 'text/markdown; charset=utf-8')
 
273
  def test_header_sponsor_modal_docs_and_chart_above_the_leaderboard(self):
274
  html = (ROOT / 'index.html').read_text()
275
  title = html[html.index('<div class="title-row">'):html.index('</div>', html.index('<div class="title-row">'))]
276
+ self.assertEqual(re.findall(r'>([^<>]+)</(?:h2|button)>', title), ['OpenEnv Arena', '+ Add your agent', 'My submissions'])
277
  header = html[html.index('<div class="header-row">'):html.index('<div class="columns">')]
278
  self.assertIn('class="sponsor" href="https://nebius.com"', header)
279
  self.assertRegex(header, r'<img\s+src="/nebius\.svg"\s+alt="Nebius"[^>]*>')
 
282
  self.assertIn(declaration, sponsor_style)
283
  self.assertNotIn('Agent docs', header)
284
  self.assertNotIn('href="/AGENTS.md"', header)
285
+ self.assertNotIn('id="manualSubmit"', header)
286
+ self.assertEqual(html.count('id="manualSubmit"'), 1)
287
  for modal_id in ('aboutModal', 'agentModal'):
288
  modal = re.search(rf'<div class="backdrop" id="{modal_id}".*?(?=<div class="backdrop"|<script>)', html, re.S).group(0)
289
  self.assertIn('class="btn" href="/AGENTS.md"', modal)
290
  self.assertIn('>Agent docs ↗</a>', modal)
291
+ if modal_id == 'agentModal':
292
+ self.assertIn('id="manualSubmit"', modal)
293
  manual = html[html.index('id="manualModal"'):html.index('id="agentModal"')]
294
  self.assertIn('href="/AGENTS.md"', manual)
295
  self.assertLess(html.index('id="pareto"'), html.index('id="boardTitle"'))
296
  self.assertLess(html.index('id="boardTitle"'), html.index('id="messagesTitle"'))
297
  self.assertNotIn('max-width: 1180px', html) # no centered column
298
 
299
+ def test_your_submissions_show_the_ten_most_recent_and_the_leaderboard_the_top_ten(self):
300
+ html = (ROOT / 'index.html').read_text()
301
+ section = html[html.index('<section aria-labelledby="submissionsTitle">'):html.index('</section>', html.index('id="submissionsTitle"'))]
302
+ self.assertIn('<h3 id="submissionsTitle">Your submissions</h3>', section)
303
+ self.assertRegex(section, r'<button class="btn" id="showAllSubmissions"[^>]*hidden>Show all</button>')
304
+ self.assertNotIn('User</th>', section)
305
+ # Only the signed-in user's own list is read; nobody else's submissions reach the page.
306
+ self.assertIn("fetch('/api/openenv/submissions', { credentials: 'same-origin' })", html)
307
+ self.assertNotIn('public/submissions', html)
308
+ self.assertIn('const RECENT_SUBMISSIONS = 10, TOP_RUNS = 10;', html)
309
+ self.assertIn('items.slice(0, RECENT_SUBMISSIONS)', html)
310
+ self.assertIn('.slice(0, TOP_RUNS);', html)
311
+ # Show all turns the table into a bounded scrolling list, so the page does not grow with the history.
312
+ self.assertIn('.table-wrap.scrolling { max-height: 420px; overflow-y: auto; }', html)
313
+ # .btn sets display, so the button needs its own rule for the hidden attribute to hide it.
314
+ self.assertIn('.section-head .btn[hidden] { display: none; }', html)
315
+
316
  def test_the_leaderboard_has_eight_domains_and_the_baseline_last(self):
317
  board = self.client.get('/api/leaderboard').json()
318
  self.assertEqual(len(board['domains']), 8)