Protein Language Models for Antibodies: Choose Models and Test Them Fairly

An antibody language model can turn a sequence into useful features. The harder question is whether those features help answer your research question: recovering missing residues, predicting expression, or estimating binding to a particular antigen.

This guide follows the ESM-2 embedding tutorial. It explains how to choose an antibody model and build a defensible evaluation, with a small Python check you can run before training. You need basic Python; the example requires no model download or GPU.

1. Choose the output before choosing the model

Start by writing down the quantity you want to predict. Here is a practical way to separate the tasks:

These are suggested evaluation choices, not a universal benchmark. An embedding is a numerical representation; similarity between two embeddings is not automatically a probability that the antibodies share a target. Likewise, reconstructing a missing amino acid tests sequence recovery, not experimental binding.

2. Compare general, antibody-specific, and paired models

ESM-2 supplies general protein representations and makes a useful baseline. Use the same downstream labels and evaluation split when comparing it with an antibody-specific model.

AbLang offers separate heavy- and light-chain models, residue and sequence encodings, and sequence restoration. It is a concrete example of adapting language modeling to antibody data. AbLang-2 is a separate model: its authors explicitly address the tendency to prefer germline residues and train with both paired and unpaired sequences. Keep their names and checkpoints distinct. AbLang-2 repository

For conventional antibodies, the heavy- and light-chain variable domains contribute to the binding site. IgBert and IgT5 use paired-chain training. Their study evaluated embeddings with supervised regression: paired models led its binding-energy benchmarks, while the general protein model ProtT5 led its expression benchmark. Those findings support task-specific comparisons; they do not establish a winner for every antibody dataset.

My starting recommendation is to compare a simple baseline, a general protein model, and one antibody model matched to your input format. With authentic heavy/light pairs, include a paired model. Follow each checkpoint’s documented chain order and separator tokens; concatenating arbitrary strings does not create native pairing information.

3. Keep sequence scores separate from property predictions

A masked language model estimates a residue from surrounding sequence context. A sequence plausibility score summarizes how well a sequence fits that model. Its interpretation depends on the masking and scoring procedure. It is not a measured dissociation constant or a calibrated binding probability.

This distinction matters when a model receives only antibody sequence. Without target information or a separately defined target-specific training task, its score cannot distinguish your intended antigen from every alternative antigen. A score may still correlate with a measured property in a particular dataset; test that association on held-out data before using it to rank candidates.

The germline bias studied in AbLang-2 is one reason to inspect what a score rewards. Frequent sequence patterns and useful task-specific behavior need not receive the same ranking.

4. Set up the evaluation before extracting embeddings

The following workflow is a practical recommendation for your own comparison:

1. Keep one record per observed heavy/light pair, including assay, target, sequence provenance, and replicate identifiers. For single-domain antibodies, record that format explicitly.
2. Define what generalization means. New variants of a known lineage, new lineages, and new antigens require different held-out sets.
3. Keep related records together. Use curated lineage labels where available, or define and document sequence clustering. Do not treat an arbitrary sample ID as a lineage label.
4. Extract features with a recorded checkpoint, tokenizer, chain representation, and pooling rule. Exclude padding and special tokens from residue pooling.
5. Fit a small supervised baseline. Fit scaling, feature selection, and hyperparameter tuning using training data and internal validation only.
6. Compare models on the same untouched test set. For regression, inspect absolute error and rank correlation; for rare binders, inspect precision and recall at a useful selection threshold.

If the intended use is prediction for new antigens, hold out antigens too. A lineage split alone does not test that capability. Separately inspect possible overlap with model pretraining data; a clean split in your own table cannot rule it out.

5. Run a small leakage check

The example uses invented pair IDs and lineage labels. It demonstrates one bookkeeping check; it does not run a language model, assign biological lineages, or measure model performance.

Create a directory and confirm Python 3 is available:

mkdir antibody-evaluation-demo
cd antibody-evaluation-demo
python3 --version

Save this as check_split.py. It uses only the Python standard library:

import argparse


def check_split(train, test):
    for field in ("pair_id", "lineage"):
        overlap = {row[field] for row in train} & {row[field] for row in test}
        if overlap:
            raise ValueError(f"{field} overlap: {', '.join(sorted(overlap))}")


def main():
    parser = argparse.ArgumentParser(description="Check a synthetic antibody split")
    parser.add_argument("--show-leak", action="store_true")
    args = parser.parse_args()
    # Invented metadata only: no sequences, measurements, or model predictions.
    rows = [
        {"pair_id": "p1", "lineage": "family_a"},
        {"pair_id": "p2", "lineage": "family_a"},
        {"pair_id": "p3", "lineage": "family_b"},
        {"pair_id": "p4", "lineage": "family_b"},
        {"pair_id": "p5", "lineage": "family_c"},
        {"pair_id": "p6", "lineage": "family_c"},
    ]
    if args.show_leak:
        train, test = rows[::2], rows[1::2]
    else:
        train = [r for r in rows if r["lineage"] != "family_c"]
        test = [r for r in rows if r["lineage"] == "family_c"]
    try:
        check_split(train, test)
    except ValueError as error:
        parser.exit(1, f"FAIL: {error}\n")
    print(f"PASS: train={len(train)}, test={len(test)}; no pair or lineage overlap")


if __name__ == "__main__":
    main()

Run the grouped split, then the deliberately leaky split:

python3 check_split.py --help
python3 check_split.py
python3 check_split.py --show-leak

The second command prints:

PASS: train=4, test=2; no pair or lineage overlap

The last command exits with status 1 and prints:

FAIL: lineage overlap: family_a, family_b, family_c

Every pair ID in the leaky split is unique, yet each lineage appears on both sides. That is the gap this example catches. To use the check with real records, first establish meaningful groups and keep a pair ID stable across repeat measurements.

A passing result only verifies those two fields. It does not detect identical sequences stored under different IDs, near-duplicates across lineage labels, assay batch effects, or pretraining contamination. Audit those separately.

6. Troubleshooting

Choose models by the property and generalization task you can actually test. Preserve chain pairing, check related records before fitting a predictor, and keep sequence plausibility separate from experimentally measured behavior.

Sources and repository documentation checked September 21, 2026. The cover is conceptual artwork, not an experimental molecular structure.