Plant Emotions with AI

From the signal comes an image, and a pre-trained network reads it: can you tell from the plant signal whether the person beside it was happy? Pure inference — you train no network of your own.

What it's about

If a plant reacts to light — does it also react to us? In a self-study over eleven days a person sat at their desk, half a metre from a Tradescantia. A simple sensor picked up the plant's electrical signal; at the same time a Happimeter smartwatch on the wrist continuously estimated the person's mood — happy, stressed, low. So two traces lie side by side: here the plant signal, there the person's mood.

The question is delicate and therefore exciting: does the plant signal really hold something about the person? Instead of believing it, you turn it into an image and let an AI look — and then check honestly whether its guess beats pure chance.

A little background

Again an image from the signal. The procedure is familiar: a signal over time becomes a spectrogram — an image of its frequencies — and then the same image-recognising machinery reads it that read cat sound, voice and heartbeat. New is only this: we do not train the network ourselves but take a ready one that has already learned from millions of images, and use its "view" as features. That is the core of transfer and inference.

Why the plant is slow. The plant signal changes over seconds to minutes, not milliseconds. So per mood reading we look at a window of a few minutes before it — long enough for the slow rhythms of the plant to show in the spectrogram.

Cut windows and build spectrograms

For each Happimeter reading we take the plant signal of the last three minutes, put it on an even 1 Hz grid and make a spectrogram. The full code is on GitHub.

import numpy as np, pandas as pd
from scipy import signal, interpolate

raw   = pd.read_csv("plant_raw.csv",   parse_dates=["timestamp"]).sort_values("timestamp")
reads = pd.read_csv("predictions.csv", parse_dates=["timestamp"])   # Happimeter readings

EPOCH = pd.Timestamp("1970-01-01")                                  # fixed zero point
t    = ((raw["timestamp"] - EPOCH) / pd.Timedelta(seconds=1)).to_numpy()   # seconds
volt = raw["voltage"].to_numpy()

WINDOW = 180                             # seconds of plant signal before each reading
GRID   = np.arange(0, WINDOW, 1.0)       # unify to 1 Hz

def spectrogram(end_s):
    m = (t >= end_s - WINDOW) & (t < end_s)
    if m.sum() < 120:                    # too few points -> skip
        return None
    w = interpolate.interp1d(t[m] - (end_s - WINDOW), volt[m],
                             bounds_error=False, fill_value="extrapolate")(GRID)
    w = (w - w.mean()) / (w.std() + 1e-9)          # normalise
    _, _, S = signal.spectrogram(w, fs=1.0, nperseg=32, noverlap=16)
    return np.log1p(S)                             # image of the slow rhythms

specs, y = [], []
for _, r in reads.iterrows():
    S = spectrogram((r["timestamp"] - EPOCH) / pd.Timedelta(seconds=1))
    if S is not None:
        specs.append(S)
        y.append(int(r["happy"] >= 2))            # 1 = happy, 0 = less
y = np.array(y)
print(len(specs), "windows  |  happy balance:", np.bincount(y))

The pre-trained network reads — pure inference

Now the core: a ResNet, pre-trained on millions of images, turns each spectrogram into a feature vector — its "view" of the image. It learns nothing here; we only let a plain nearest-neighbour method guess the mood from these features.

import torch, torch.nn as nn
from torchvision.models import resnet18, ResNet18_Weights
from PIL import Image
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import cross_val_score

weights = ResNet18_Weights.IMAGENET1K_V1         # loads the weights once (internet needed)
net = resnet18(weights=weights); net.fc = nn.Identity(); net.eval()
prepare = weights.transforms()

def features(S):                                 # spectrogram -> image -> ResNet view
    a = (S - S.min()) / (np.ptp(S) + 1e-9)
    img = Image.fromarray((a * 255).astype("uint8")).convert("RGB").resize((64, 64))
    with torch.no_grad():
        return net(prepare(img).unsqueeze(0))[0].numpy()

X = np.array([features(S) for S in specs])       # pure inference: the net is not trained

chance = max(np.bincount(y)) / len(y)
score  = cross_val_score(KNeighborsClassifier(5), X, y, cv=5).mean()
print("hits (5-fold):", round(score, 2), " |  chance line:", round(chance, 2))

Worksheet

When it sticks

Food for thought

• Pure inference is a powerful lever: a network that has never seen a plant signal still delivers useful features — because a spectrogram is, after all, an image. The same procedure, from tomato to stock market.
• This is the chapter where the whole book is put to the test. A responsible person does not believe an astonishing claim just because it is exciting — not even when it comes from a professor writing a book about it. You measured for yourself: a weak trace for "happy", barely anything for "stress".
• And the boundary stays: even a real correlation is not yet understanding. The trace probably runs via the heartbeat and via scents — not via a consciousness of the plant. Wonder need not fear sobriety.

Extension

• Look at spectrograms. Draw one spectrogram each from a "happy" and a "not happy" window side by side. Do you see a difference with the naked eye — or only the network?
• Other emotions, other windows. Vary the target (stress, depression) and the window length. Which combination shows a signal above chance at all — and which none?
• The real plant network. Instead of a photo network you can use Peter's own pipeline trained on plant signals (github.com/pgloor/hiddenbiosignals, folder plant_spectrograms) and compare the results.