Dev log: teaching Hypha to see

Every grower knows the moment. You open the tent, look at a block you've babied for three weeks, and something looks off. Is that a bruise or the start of a blotch? Is that fuzz mycelium or mold? Is this cluster ready, or will it be better tomorrow?

Hypha already keeps a photo journal for every grow, and a lot of you are great about using it. Right now those photos are a record of what happened. What I've been building for the last couple of weeks is a way for them to help while it's happening: you add a photo, and Hypha takes a look too.

This is a development log, not a launch post. The photo check is still in development: it runs in our test environment, it isn't in the app yet, and below I'll be specific about what has to get better before it is.

Three questions per photo

The first version answers three questions about every photo:

1. Is this a grow photo at all? A gate, so the rest of the model never has to form an opinion about your cat or a screenshot of a receipt.
2. Does anything look contaminated? Clean or contaminated, as a probability.
3. What stage is the fruit at? Pins, young, ready to harvest, or past peak.

Under the hood it's one large pretrained vision model that turns the photo into a long list of numbers (an embedding), plus three tiny classifiers that read those numbers. I'll get to why it's shaped like that in a minute.

The data we chose not to use

The obvious training set would be the photos growers upload to Hypha. We aren't using them. Training on your photos would need our Terms of Service and privacy policy to say so, and probably an explicit opt-in. Neither exists yet. Running a model on your photo to give you an answer is one thing. Quietly feeding your grow into our training set is another.

So I started where everyone starts: public datasets. I catalogued 150 of them (from Kaggle, Roboflow, Zenodo, Mendeley, Dataverse and GitHub), checked every license, and pulled in the ones we could use. That came to about 102,000 photos. After removing duplicates (the same oyster photos turn up in a lot of datasets) there were 89,809 left. Most are oysters and shiitake, with button mushrooms, king oysters, enoki, beech, reishi and agar plates mixed in.

Public data has a catch, though: everyone labels differently. One dataset's "contaminated" is another dataset's "bad photo." So we labelled our own.

Two humans, a lot of evenings

Kaylee and I built some simple review pages and labelled photos by hand: 11,195 decisions across 5,090 photos. The pages were blind. They never showed an AI's guess, because it's far too easy to click "agree" on a suggestion late at night, and a label that's really just a model agreeing with itself teaches the next model nothing.

Then we checked ourselves. Each of us re-labelled a blind sample of photos the other had already done, without knowing which ones they were or what the other person had said. It was humbling:

• Obvious contamination held up. When one of us called a photo clean or contaminated, the other gave the opposite answer on only about 1 in 12 photos. Most of our disagreements lived in the grey zone of "looks off" and "can't tell."
• Fruiting stage was fuzzy. We were within one step of each other 97% of the time, but where I said "young," Kaylee often said "ready to harvest."
• Colonization was a coin flip. Our agreement on how colonized a bag looked from a single photo was statistically no better than chance (Cohen's kappa of 0.03). Kaylee consistently saw less colonization than I did.

That last result changed the roadmap. I wanted a "should I break and shake this?" feature. But if two people who spend a lot of time staring at grain can't agree from one photo, a model trained on our labels won't do any better. It'll just be confidently arbitrary. Colonization is going to need a better signal than a single photo, and that's a problem for a future post.

Picking a model without fooling myself

Here's the trap with public datasets. Each one comes from a single farm or lab, with one camera, one lighting setup and one kind of tub. A model can score brilliantly by learning "photos from this dataset are the contaminated ones" instead of learning what mold looks like. It looks great in testing and falls apart on your shelf.

So every candidate is scored by leaving one source out: train on everything else, test on a dataset it has never seen, and repeat for each source. A separate evaluation set of held-out sources (12,376 photos) stays locked away. It's never used to make a choice, only to report numbers at the end.

Inside those rules I let the GPU grind. I tried seven backbones (DINOv2 in three sizes, two SigLIP 2 models, NVIDIA's C-RADIO and Meta's DINOv3) across crops, pooling and regularization settings, plus a few rounds of full fine-tuning. That's roughly 600 trials per question, all on a single desktop GPU. Fine-tuning didn't beat the simpler approach.

The current pick is boring in the best way: DINOv3 ViT-L/16, frozen, with a plain logistic-regression head for each question. It isn't the top scorer on every task. C-RADIO edged it on contamination, but the gap was within the noise and C-RADIO costs about six times the compute. One forward pass serves all three questions.

It runs on ordinary CPUs inside our own infrastructure, at about 0.8 seconds per photo in our benchmarks, and no third-party AI service sees the image. A frozen backbone has one more advantage while we're still developing: improving a head doesn't mean re-running the big model, so we can iterate quickly.

Where it stands today

These are the numbers from the current development build, measured on photos and sources the model didn't train on.

• The gate is in good shape, with one blind spot. It recognizes 99% of grow photos and rejects over 99% of the things that aren't. The exception is harvested mushrooms: picked mushrooms sitting on a tray or a table get through most of the time, because they look a lot like mushrooms on a block. (To be fair, they are mushrooms.)
• Contamination is promising, not ready. On datasets it never trained on, it scores an AUC of about 0.84. That means that if you show it one contaminated photo and one clean photo at random, it ranks the contaminated one as more suspicious about 84% of the time. The score swings a lot by source, from 0.68 on the hardest to 0.95 on the easiest. On a fresh batch of our own labels it caught 5 of 13 contaminated photos. Thirteen is a small sample, but five isn't a number I'd put in front of a grower.
• Stage isn't good enough to show yet. Overall accuracy is 57%. It's decent at "ready to harvest" (81%) and close to useless at pins (5%). Worse, a model that answered "ready to harvest" every single time would beat it on our main metric. So stage stays hidden while we work on it.

That's why the photo check isn't telling anyone anything yet. I'd rather ship nothing than a confident, wrong opinion about your grow.

How it'll help growers

The goal isn't a robot that grades your mushrooms. It's a second pair of eyes that never gets tired.

• Catching trouble early. Contamination spreads, and the earlier you spot it, the more of your room you save. What we're aiming for is a gentle "this one looks off, take a closer look" on the photo you just added. Not a verdict.
• Harvest timing. Oysters go from perfect to past peak fast. A nudge when a block looks ready, alongside your schedule, means less guessing.
• Photos plus sensors. This is the part I'm most excited about. Long stems with tiny caps is the classic sign of too much CO2, and cracked, dry caps usually mean low humidity. But a photo alone can't tell high CO2 from low light, or low humidity from a fan blowing straight onto the caps. If you run a sensor, Hypha also has your zone's CO2 and humidity history. A model that spots the symptom, plus the sensor log that explains it, can tell you why and what to change. We've started labelling those symptoms, but there's no model for them yet.

What's next

Roughly in order:

1. Shadow mode in production. The model scores photos quietly and nobody sees a result, while we check its thresholds against photos from our own grows and from growers who explicitly opt in.
2. A smarter gate that knows a harvest photo when it sees one.
3. A golden set: a carefully labelled, blind test set the model has never touched. Nothing in the app will ever say "likely contamination" until the model passes it.
4. A better way to measure stage, so "always say ready" can't win.
5. Labels from growers, if they want to help. Down the road, we'd like growers who opt in to be able to tag their own photos in the app ("first pins," "something's wrong here") and help teach it. Nobody's photos will go into training without that opt-in.

I'll keep posting updates here as it moves along. If you have questions about how any of this works, or opinions about what it should flag first, write to us at team@demetermycolabs.com. I read all of it.

— Derick

---

Built with DINOv3. The image backbone is Meta's DINOv3 (ViT-L/16), used under the DINOv3 License. Thanks also to the many researchers and growers who published their mushroom photo datasets openly. None of this works without them.