You will build a small chess model on top of a real fruit-fly connectome. The connectome and the pretrained FlyNet numbers stay fixed. The new piece is yours: a scorer that gives a number to each legal move.
By the end of this notebook, you will be able to:
Load the published fly network and a set of real chess positions.
Read one position, its legal moves, its Stockfish move, and the neuron activity.
Choose training positions from a prepared pool, and leave the test positions out.
Create a model, train it, and test it with helper functions.
Change one setting, make a second model, and compare the two on the same test positions.
Playing a game in the other ChessFly notebook does not train anything. Training happens when you run the training cell in this notebook.
Stockfish top-move agreement is one narrow metric. It is not a complete chess-strength assessment.
The network files are downloaded in your browser from ChessFly. The first load is about 148 MB. This notebook has not been run on DataHub yet.
0. Warmup¶
In the cell below, answer this question: What would a computer need, besides a wiring diagram, before it could choose a chess move?
Add your response here:
1. Two places, one model¶
This lab uses two different programs.
In the browser. A published worker downloads the FlyWire connectome and the pretrained FlyNet weights. It runs the fly network on each chess position. For every position it returns 41,692 numbers. Those numbers are the readout neurons the pretrained decoder reads. The worker runs in ordinary browser JavaScript so those floats can be sent back. The game page can still use WebGPU.
In Python. Your cells receive that activity matrix. You choose a few of its columns, then train a small scorer. The scorer’s only choices are the legal moves of the board it is looking at. Python does not run the 15 million connections. If the browser has not sent the matrix, Python will say so instead of inventing activity.
The connectome, the signs of the connections, the encoder, the connection strengths, the gains, the shifts, and FlyNet’s decoder stay fixed. Train changes only the scorer you create.
2. Load the chess examples¶
The file data/lichess_lab.csv has 60 real positions from Lichessstockfish_uci is the first move of the published Stockfish line. Each of those moves is legal on its own board.
The split column was assigned before any training, using row order only. Rows marked test are held out. Rows marked pool are the ones you may train on. This dataset has no game id, so positions from the same game could not be kept together. Details are in data/README.md.
# Click this cell, then press the play button (or Shift+Enter).
from pathlib import Path
import sys
import numpy as np
import pandas as pd
from IPython.display import display
folder = None
for candidate in [Path.cwd(), Path.cwd() / "chessfly"]:
if (candidate / "fly_scorer.py").exists():
folder = candidate
break
if folder is None:
raise FileNotFoundError("Could not find fly_scorer.py. Open this notebook from the chessfly folder.")
sys.path.insert(0, str(folder))
from fly_scorer import (
NARROW_METRIC,
check_training_rows,
compare_scorers,
create_move_scorer,
evaluate_scorer,
reset_scorer,
test_row_numbers,
train_scorer,
)
examples = pd.read_csv(folder / "data" / "lichess_lab.csv")
examples["split"] = examples["split"].astype(str)
preview = examples[["row_idx", "split", "stockfish_uci", "depth", "cp"]].copy()
preview.insert(0, "index", examples.index)
print(preview.to_string(index=False))
print()
print("Pool positions:", int((examples["split"] == "pool").sum()))
print("Test positions:", int((examples["split"] == "test").sum()))
missing = [
int(row.row_idx)
for row in examples.itertuples()
if row.stockfish_uci not in str(row.legal_uci).split()
]
print("Stockfish moves missing from that board's legal moves:", missing or "none")
print("Test indexes:", test_row_numbers(examples))
Question¶
The test indexes are already chosen. Why should those rows stay out of the list you train on?
Add your response here:
3. Load the fly network, then let Python receive the activity¶
Run the next cell. A box appears. Press Load the real network and wait.
The browser downloads the connectome and the weights, then runs all 60 positions. When the transfer works, the status line says that Python received the activity matrix and prints its shape. Then run the cell after the box. That cell is the Python side of the handoff. If you run it too early, it stops and tells you the activity is not here yet.
# Click this cell, then press the play button (or Shift+Enter).
# Then press Load the real network in the box below and wait.
from fly_activity import FlyNetwork
fly = FlyNetwork(examples)
display(fly)
print("Press Load the real network. Wait for: Python received the activity matrix.")
When the status line says Python received the activity matrix, run the next cell.
You should see a shape (60, 41692) and the word finite. The 60 rows are the chess positions, in the same order as the table above. The 41,692 columns are the readout neurons.
# Click this cell, then press the play button (or Shift+Enter).
activity = fly.get_activity()
print("Shape:", activity.shape)
print("Finite:", bool(np.isfinite(activity).all()))
print("Activity sum for index 0:", round(float(activity[0].sum()), 2))
print("This array is in Python. The browser computed it and sent these numbers across.")
Question¶
The activity sum is a total of neuron values. Which program computed the fly network: the browser, or this Python cell?
Add your response here:
4. Inspect one position¶
Pick a pool index. The default is 0. You can change it to another pool index from the table.
The cell prints the board, the Stockfish move, every legal move the scorer will be allowed to consider, and a short description of that row of activity. It does not print all 41,692 numbers.
# You can change example_index to another pool index.
example_index = 0
position = examples.loc[example_index]
legal_moves = str(position.legal_uci).split()
row = activity[example_index]
print("Index:", example_index, " Lichess row:", int(position.row_idx), " Split:", position.split)
print("Board:", position.fen)
print("Stockfish target:", position.stockfish_uci)
print("Stockfish move is legal here:", position.stockfish_uci in legal_moves)
print("Legal moves (", len(legal_moves), "):", " ".join(legal_moves))
print("Alphabetical baseline:", legal_moves[0])
print("Activity length:", row.shape[0])
print("Minimum:", round(float(row.min()), 4), " Maximum:", round(float(row.max()), 4))
print("Nonzero neurons:", int((row != 0).sum()))
Predict¶
Before any training, the scorer’s weights are zeros, so it picks the alphabetical baseline. For the position you just inspected, write that baseline move. After training, do you expect the model to match Stockfish on the test rows? Why or why not?
Add your response here:
5. Choose training examples¶
Edit the list below. Use indexes from the pool only. Index 4 is a test position, so it is not in the default list.
The check refuses test indexes. If you add one, the cell stops before any model is created.
# Edit this list. Keep every index in the pool.
train_rows = [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 13]
train_rows = check_training_rows(train_rows, examples)
print("Training rows:", train_rows)
print("Count:", len(train_rows))
print("Test rows stay fixed:", test_row_numbers(examples))
Question¶
What should the helper do if a test index is placed in train_rows?
Add your response here:
6. Create a fresh model¶
n_features is how many readout neurons your scorer may use. The helper keeps the neurons whose activity varies the most on your training rows only. Test rows do not vote. The move itself is described by 33 fixed numbers: start file, start rank, end file, end rank, and a constant. Those numbers are not trained.
With n_features = 16, the new model has 16 × 33 = 528 trainable numbers, all starting at zero. This object is model_v1. A later model will be a different object.
# You can change n_features to any whole number from 1 to 64.
n_features = 16
model_v1 = create_move_scorer(
activity,
examples,
train_rows,
n_features=n_features,
name="model_v1",
)
print(model_v1.summary())
print("Neuron indexes chosen from the training rows:", model_v1.neuron_index.tolist())
print("Those feature values at your inspected position:")
print(np.round(activity[example_index, model_v1.neuron_index], 3))
You should see weight length 0.0000. If you asked for 16 features, you should see 528 trainable numbers.
7. Train your model¶
Edit learning_rate and n_rounds, then run the cell. This is the step that changes model_v1’s numbers. The connectome does not change.
A larger learning rate takes a bigger step. More rounds repeat the step. The lines below show the weight length before and after each round. If you change the settings and run the cell again, the new length should differ from a run with the old settings, because training continues from the current weights. Call reset_scorer(model_v1) first if you want to start again at zero.
# Edit these two settings, then run the cell.
learning_rate = 0.5
n_rounds = 5
history = train_scorer(
model_v1,
activity,
examples,
train_rows,
learning_rate=learning_rate,
n_rounds=n_rounds,
)
for item in history[-n_rounds:]:
print(
f"Round {item['round']}: loss before the step {item['loss_before_step']:.4f}, "
f"training matches {item['train_matches']}/{item['train_total']}, "
f"weight length {item['weight_length_before']:.4f} -> {item['weight_length']:.4f}."
)
print("model_v1 weight length is now", round(model_v1.weight_length, 4))
You should see the weight length move away from zero. Training matches count only the rows in train_rows. They are not a test result.
8. Test¶
The test rows were not used to choose features or to fit. The table shows the alphabetical baseline, your model’s move, and the Stockfish move. Every model move should be legal on that board.
Read the training count and the test count separately. Matching the training rows does not show that the model will match the test rows.
# Click this cell, then press the play button (or Shift+Enter).
test_rows = test_row_numbers(examples)
train_table, train_summary = evaluate_scorer(model_v1, activity, examples, train_rows)
test_table, test_summary = evaluate_scorer(model_v1, activity, examples, test_rows)
print(NARROW_METRIC)
print(
"Training agreement:",
train_summary["agreements"],
"of",
train_summary["rows"],
)
print(
"Test agreement:",
test_summary["agreements"],
"of",
test_summary["rows"],
)
print()
print("Test positions")
display(test_table)
Question¶
Your model may agree with Stockfish on several training rows. Does that agreement mean the model is a strong chess player? What does the sentence about the metric say?
Add your response here:
9. Make a second model by changing one setting¶
model_v2 is a new scorer. Creating it does not reset model_v1. The cell below changes one setting: n_features is 8 instead of 16. The training rows, learning rate, and number of rounds stay the same.
The comparison uses the same test positions for both models.
# One changed setting: fewer activity features.
n_features = 8
length_v1_before = model_v1.weight_length
model_v2 = create_move_scorer(
activity,
examples,
train_rows,
n_features=n_features,
name="model_v2",
)
train_scorer(
model_v2,
activity,
examples,
train_rows,
learning_rate=learning_rate,
n_rounds=n_rounds,
)
print(model_v1.summary())
print(model_v2.summary())
print("model_v1 weight length before model_v2 trained:", round(length_v1_before, 4))
print("model_v1 weight length now:", round(model_v1.weight_length, 4))
print()
comparison = compare_scorers([model_v1, model_v2], activity, examples, test_rows)
display(comparison)
You should see two different parameter counts. model_v1’s weight length should be the same number before and after model_v2 trains.
# This resets model_v2 only.
print("model_v2 weight length before reset:", round(model_v2.weight_length, 4))
saved_v1 = model_v1.weight_length
reset_scorer(model_v2)
print("model_v2 weight length after reset:", round(model_v2.weight_length, 4))
print("model_v1 weight length after that reset:", round(model_v1.weight_length, 4))
print("model_v1 changed:", abs(model_v1.weight_length - saved_v1) > 1e-8)
Question¶
Reset cleared model_v2. What happened to the weights in model_v1?
Add your response here:
10. Optional: turn off connections in stored order¶
This section does not train a model. It changes the fly network inside the browser, then sends a new activity matrix to Python.
The button turns off the first 2,000,000 connections in stored order. That phrase means the first 2,000,000 entries of the published weight array. It is not a named brain region or a chosen circuit. Restore writes those same numbers back. The published website is not changed.
An activity sum adds the readout numbers. It is not a measure of intelligence or of chess performance. Your scorer’s weights stay whatever they are unless you call train_scorer again. Do not create a new scorer here if you want to keep the same features: a new scorer would choose neurons again.
Run the next cell, scroll back to the fly box, press the turn-off button, and wait until the status says Python received a new activity matrix. Then run the cell after that. The restore button puts the connections back and sends another matrix.
# Click this cell, then press the play button (or Shift+Enter).
# The ablation buttons appear on the fly box above.
fly.show_ablation = True
print("Scroll up to the fly box.")
print("Press: Turn off the first 2,000,000 connections in stored order.")
print("Wait until the status says Python received a new activity matrix.")
# Run this cell only after the status says Python received a new activity matrix.
weights_before = model_v1.weights.copy()
activity_after = fly.get_activity()
print("Row 0 activity sum before:", round(float(activity[0].sum()), 2))
print("Row 0 activity sum after:", round(float(activity_after[0].sum()), 2))
print("model_v1 weights changed:", not np.allclose(model_v1.weights, weights_before))
print(NARROW_METRIC)
after_table, after_summary = evaluate_scorer(
model_v1,
activity_after,
examples,
test_rows,
)
print(
"Test agreement on the new activity:",
after_summary["agreements"],
"of",
after_summary["rows"],
)
display(after_table)
Question¶
Did the activity sum for index 0 change? Did model_v1’s weights change? What does an activity sum tell you, and what does it leave out?
Add your response here:
Sources¶
ChessFly model card and weights: huggingface
.co /mlabonne /chessfly. The hosted files are loaded in the browser from mlabonne -chessfly .static .hf .space /data/. The worker is fetched from that site and patched in memory so Python can receive readout floats. This repository does not copy the worker, the connectome, or the weight file. If the published program no longer contains the expected lines, the loader stops.
FlyWire public-release guidelines, CC BY-NC 4.0: flywire
.ai /guidelines. The model card says those terms apply to anything derived from the graph. Activity is computed at runtime and is not saved here. Positions: Lichess
/chess -position -evaluations, CC0. The lab file is data/lichess_lab.csv. How it was sampled is indata/README.md.Helper code:
fly_scorer.pytrains your scorer.fly_activity.pyis the browser handoff.Playing a game stays in ChessFly: a fruit-fly brain map that plays chess.