From Protein Language Models to CoFoldArena: How to Evaluate Predictions

The previous ESM-2 tutorial turned amino acid sequences into vectors. The next question is how to evaluate a model when the desired output is a protein complex. A useful starting point is CoFoldArena, where you can compare predicted antibody–antigen structures.

This guide connects protein language models to that evaluation task, shows how to inspect the leaderboard, and uses a short Python example to explain its metrics. Website scope and controls were checked on September 20, 2026; rankings may change after publication.

1. Separate representations from structural predictions

ESM-2 provides learned sequence representations. ESMFold adds a structure prediction architecture that uses ESM-2 representations to produce coordinates. That is one concrete connection between language modeling and folding, documented in the official ESM repository.

Treat the output you need as the starting point for evaluation:

These are different questions. A co-folding leaderboard does not, by itself, rank the usefulness of every protein language model. Nor can an embedding similarity score replace a structural comparison.

2. Set up a meaningful CoFoldArena comparison

You need a browser for the website and Python 3 for the example below. No model download or GPU is needed.

At the time of checking, CoFoldArena offers antibody–antigen evaluation, antibody MSA controls, Fab/VHH format filters, and release-date windows. Protein–ligand evaluation is marked as coming soon. Start with these steps:

1. Choose the antibody format relevant to your work.
2. Record the antibody MSA setting. MSA means multiple sequence alignment.
3. Choose a release-date window and record the target count.
4. Compare named model versions on that same view.

Keep a note with the access date, filters, model versions, and results. A saved score without its selected target set is hard to interpret later.

The methodology describes a growing PDB-based panel, downloadable predictions, a 1,024-token target limit, and single-seed inference. It also flags a training-cutoff exception for Protenix-v1-20250630. Enable uncertainty to inspect bootstrap intervals and rank ranges.

My practical reading: treat this as evidence under specified conditions. Check training dates and related sequences separately; a recent structure release alone does not establish independence from training data.

3. Read DockQ before reading the rank

DockQ compares a predicted complex with a reference structure. Its quality bands are:

The leaderboard reports mean DockQ and fractions meeting these thresholds. The threshold columns overlap: a high-quality prediction also meets the medium and acceptable cutoffs. Do not add the percentages together. CoFoldArena leaderboard

A mean DockQ of 0.50 is not a 50% probability of binding. DockQ evaluates structural agreement and needs a reference; you cannot compute the same ground-truth score for a new complex whose structure is unknown. Keep a model’s confidence estimate separate from measured agreement with a reference. DockQ usage

4. Reproduce the difference between a mean and a success rate

Save this as score_demo.py. The values are invented to demonstrate the arithmetic; they are not downloaded results or a comparison of real models.

from statistics import mean

# Invented scores for arithmetic only; these are not CoFoldArena results.
scores = {
    "model_a": [0.90, 0.85, 0.10, 0.05],
    "model_b": [0.50, 0.50, 0.45, 0.45],
}
print("model     mean   acceptable   medium   high")
for model, values in scores.items():
    rates = [sum(x >= t for x in values) / len(values)
             for t in (0.23, 0.49, 0.80)]
    print(f"{model:9} {mean(values):.3f}  {rates[0]:9.1%}"
          f"  {rates[1]:7.1%}  {rates[2]:5.1%}")

Run it with the Python standard library:

python3 score_demo.py

Expected output:

model     mean   acceptable   medium   high
model_a   0.475      50.0%    50.0%  50.0%
model_b   0.475     100.0%    50.0%   0.0%

Both models average 0.475. Model A has two high scores and two failures; Model B has four acceptable-or-better scores but none in the high band. The average hides this difference.

For a workflow that needs usable starting structures across many targets, Model B’s pattern may be preferable. For one that can tolerate failures while seeking a few very accurate structures, Model A’s pattern may be more useful. This is an interpretation of the toy example, not a recommendation about any model on the live leaderboard.

5. Inspect individual cases and resolve confusing results

The methodology says predictions can be inspected in the browser or downloaded. Use that access to examine several successes and failures, rather than selecting only a visually convincing result.

For a small evaluation notebook, I would record the target identifier, model version, selected settings, score, and a brief observation about the interface. Compare several targets before changing your workflow.

Next step

Choose one CoFoldArena view, save its settings, and inspect a few target-level results before choosing a model to evaluate further. Keep sequence representations, structural accuracy, and experimental binding evidence distinct throughout the workflow.