🚀 Step 8 of 10
← Previous: Step 7 — Leaderboard — Rank Models by Score
→
WebSocket — Live Leaderboard Updates
When a benchmark finishes, push the result to every connected browser instantly. No polling, no refresh, no lag.
🎯 Goal — what you'll achieve in this step
Open /leaderboard in two browser tabs. Benchmark a model in one tab. Both tabs update instantly with the new entry.
📖 The basic concept
Learn this first. It's the pure Python/HTML idea, stripped of all app complexity.
# The basic concept: a WebSocket is a persistent bidirectional pipe
# Server side
from fastapi import WebSocket
@app.websocket('/ws')
async def ws_endpoint(ws: WebSocket):
await ws.accept() # handshake
while True:
msg = await ws.receive_text() # wait for client
await ws.send_text('echo: ' + msg) # send back
# Client side (browser JS)
# const ws = new WebSocket('ws://localhost:8000/ws');
# ws.onmessage = (e) => console.log('got:', e.data);
# ws.send('hello');
# Unlike HTTP (request -> response), either side can send at any time.
# The connection stays open until someone closes it.
# This is how you push updates without the client asking.
🧮 Apply it to OpenBenchML
main.py
python
# Step 8: Add WebSocket broadcast on top of steps 1-7
from fastapi import WebSocket, WebSocketDisconnect
import json
class ConnectionManager:
'''Tracks all connected WebSocket clients.'''
def __init__(self):
self.active: list[WebSocket] = []
async def connect(self, ws: WebSocket):
await ws.accept()
self.active.append(ws)
def disconnect(self, ws: WebSocket):
if ws in self.active:
self.active.remove(ws)
async def broadcast(self, msg: dict):
'''Send a message to every connected client.'''
text = json.dumps(msg)
for ws in list(self.active): # copy — list may change during iteration
try:
await ws.send_text(text)
except Exception:
self.disconnect(ws)
ws_manager = ConnectionManager()
@app.websocket('/ws/leaderboard')
async def ws_leaderboard(ws: WebSocket):
await ws_manager.connect(ws)
try:
while True:
await ws.receive_text() # keep alive; ignore client msgs
except WebSocketDisconnect:
ws_manager.disconnect(ws)
# Modify the /benchmark/{model_id} route from step 6:
# After saving the BenchmarkJob, broadcast the new entry:
# await ws_manager.broadcast({
# 'type': 'new_benchmark',
# 'model': model.name,
# 'score': result['score'],
# })
templates/leaderboard.html
html
{% extends 'base.html' %}
{% block title %}Leaderboard{% endblock %}
{% block content %}
<h1>Leaderboard <span id='live-badge' style='font-size:0.6em; color:#a0c000;'>⚡ live</span></h1>
<table id='lb-table' style='width:100%; border-collapse:collapse;'>
<thead>
<tr style='border-bottom:2px solid #a0c000; text-align:left;'>
<th>Rank</th><th>Model</th><th>Owner</th><th>Score</th>
</tr>
</thead>
<tbody>
{% for e in entries %}
<tr style='border-bottom:1px solid #eee;'>
<td>{{ e.rank }}</td><td>{{ e.model_name }}</td>
<td>{{ e.owner }}</td><td>{{ e.score }}</td>
</tr>
{% endfor %}
</tbody>
</table>
<script>
const ws = new WebSocket('ws://' + location.host + '/ws/leaderboard');
ws.onmessage = (e) => {
const data = JSON.parse(e.data);
if (data.type === 'new_benchmark') {
// Add a flash row at the top of the table
const tbody = document.querySelector('#lb-table tbody');
const row = document.createElement('tr');
row.style.cssText = 'border-bottom:1px solid #eee; background:#f0fff0; animation:fade 3s;';
row.innerHTML = '<td>NEW</td><td>' + data.model + '</td><td>—</td><td>' + data.score + '</td>';
tbody.insertBefore(row, tbody.firstChild);
// After 3s, reload to get the properly ranked table
setTimeout(() => location.reload(), 3000);
}
};
ws.onclose = () => setTimeout(() => location.reload(), 3000); // reconnect
</script>
{% endblock %}
How it works
Polling (setInterval(fetch, 5000)) is laggy and wasteful — 99% of polls return nothing. WebSocket is a persistent pipe: the server pushes the instant something happens, zero lag, zero wasted requests. The ConnectionManager class tracks every connected browser in a list. When a benchmark finishes, broadcast() sends a JSON message to every client. Each browser's ws.onmessage fires, parses the JSON, and updates the DOM. The critical detail: wrap send_text in try/except — a client that closed their tab will raise, and you must remove them from the active list or you'll keep trying to send to a dead socket. The client-side ws.onclose triggers a reload after 3 seconds — basic auto-reconnect for when the server restarts. Open two tabs, benchmark a model, watch both update instantly.
✏️ Try yourself
Add a heartbeat: every 30 seconds, the client sends {type:'ping'} and the server responds {type:'pong'}. This keeps the connection alive through proxies that cut idle connections. Add a 'X users viewing' counter that broadcasts the active connection count.
⚠️ Common mistakes to avoid
- Not removing disconnected clients — the active list grows forever, and every broadcast tries to send to dead sockets, raising exceptions.
- Iterating self.active while modifying it — `for ws in self.active:` then `self.active.remove(ws)` inside the loop skips elements. Always iterate over `list(self.active)` (a copy).
- Blocking the event loop — `await ws.send_text()` is async for a reason. Never call `time.sleep()` or sync I/O inside a WS handler; it blocks ALL other connections.
- Not handling WebSocketDisconnect — without the try/except, a client closing their tab crashes the handler and logs a scary traceback.
🔍 Where this is used in OpenBenchML
app/main.py — ConnectionManager class + /ws/leaderboard, /ws/benchmark, /ws/notifications endpoints. templates/realtime.html — the live demo page.
Next — Step 9
Notebook Kernel — Run User Python in the Browser