Symbiont Classifier

From word counting to a language model: you generate your training data with the simple method, fine-tune a BERT on it — and catch the Leech where word lists are blind: at the repeated idea.

What it's about

In the base part you read five roles — Bee, Ant, Butterfly, Capybara, Leech — from words alone. That is a coarse tool: word lists are a hypothesis, not understanding. Someone can say "idea" without being creative, and be very creative without ever using the word. Better methods — those built on language models like BERT — understand meaning in context. Exactly one of them you build here.

And you solve the base part's most stubborn problem: the Leech. With Felix we got only a hint (the self-positioning words), but missed the actual mechanism — that his "ideas" are, in content, repetitions of others' ideas. To recognise that, you must measure meaning-similarity over time. Exactly this is what modern language models can do.

Step 1 — Generate training data with the simple method

We collect 100 to 500 anonymised chats, apply the word-list classifier and get pseudo-labels automatically. Not perfect, but good enough as a teacher model.

# five roles, each with a small word list (from the base part)
wordlists = {
    "Bee":       ["idea", "imagine", "what if", "could we", "proposal"],
    "Ant":       ["plan", "list", "done", "deadline", "who does", "organise"],
    "Butterfly": ["beautiful", "feel", "wonderful", "dream", "mood", "aesthetic"],
    "Capybara":  ["fine", "no stress", "whatever", "relaxed", "all good", "chill"],
    "Leech":     ["my parents", "showed them", "impressed by", "as i said before"],
}

def wordlist_guess(text):
    text = text.lower()
    scores = {role: sum(w in text for w in ws) for role, ws in wordlists.items()}
    best = max(scores, key=scores.get)
    return best if scores[best] > 0 else None      # None = uncertain -> drop

# messages = list of many anonymised chat lines (strings)
pseudo = [(m, wordlist_guess(m)) for m in messages]
pseudo = [(m, r) for m, r in pseudo if r is not None]   # keep only sure labels
print("pseudo-labels:", len(pseudo))

Step 2 — Fine-tune a BERT model

With the pseudo-labels we train a pre-trained model (bert-base-uncased) on the symbiont classification. Thanks to transformers that is about 30 lines.

from datasets import Dataset
from transformers import (AutoTokenizer, AutoModelForSequenceClassification,
                          TrainingArguments, Trainer)

roles = list(wordlists)                          # fixed order -> numbers
to_id = {r: i for i, r in enumerate(roles)}

tok = AutoTokenizer.from_pretrained("bert-base-uncased")
def encode(b): return tok(b["text"], truncation=True, padding="max_length", max_length=64)

ds = Dataset.from_dict({"text":  [m for m, _ in pseudo],
                        "label": [to_id[r] for _, r in pseudo]})
ds = ds.map(encode, batched=True).train_test_split(test_size=0.2)

model = AutoModelForSequenceClassification.from_pretrained(
    "bert-base-uncased", num_labels=len(roles))

args = TrainingArguments("bert-symbiont", num_train_epochs=3,
                         per_device_train_batch_size=16, eval_strategy="epoch")
Trainer(model, args, train_dataset=ds["train"], eval_dataset=ds["test"]).train()

# the fine model now understands context, not only words
test = tok(["Should we think about this in a completely different way?"],
           return_tensors="pt", truncation=True, padding=True)
print(roles[int(model(**test).logits.argmax())])

Step 3 — Catch the Leech with semantic similarity

In addition we compute for each message the semantic similarity to all earlier messages of other people (with Sentence-BERT). Whoever regularly produces highly similar posts without citing is a Leech candidate.

from sentence_transformers import SentenceTransformer, util

sbert = SentenceTransformer("all-MiniLM-L6-v2")

# chat = list of (author, text) in chronological order
vectors = sbert.encode([t for _, t in chat], convert_to_tensor=True)

for i, (author, text) in enumerate(chat):
    earlier = [j for j in range(i) if chat[j][0] != author]   # only OTHERS, only EARLIER
    if not earlier:
        continue
    similar = util.cos_sim(vectors[i], vectors[earlier])[0]
    highest = float(similar.max())
    if highest > 0.70:                     # high similarity without citation -> suspicion
        source = chat[earlier[int(similar.argmax())]][0]
        print(f"Leech suspicion: {author} repeats {source}  (similarity {highest:.2f})")

In Felix's case this method clearly flags his Monday-18:42 post as taking over Mira's Monday-14:03 idea — exactly the take-over the word list missed.

Worksheet

When it sticks

Food for thought

• This is the leap the whole book is about: from counting to understanding. A word counter sees surface, a language model sees meaning — and yet this one too stays fallible.
• The Leech detector is powerful and delicate at once. It delivers a suspicion, not a verdict — and a suspicion about a person demands special care. Whoever treats it like proof misuses the tool.
• Weak supervision shows a general truth of AI: a coarse, honest method can train a finer one — but it also passes on its blind spots. Whoever builds models must know what they believe of their teacher.

Extension

• Test against real labels. Label 50 messages by hand and compare: how much better is the fine-tuned BERT really over the plain word list?
• The Butterfly. Build a second similarity test that measures stylistic (not content) closeness and check whether it finds the "aesthetically writing" person the word list misses.
• Independent convergence. Construct a chat in which two people arrive at the same idea independently. How often does your Leech detector falsely fire — and how could you soften that?