BioDex - I built a plant-collector simulation to help you touch grass

The BioDex plant-collector simulation game allows users to capture real plants using their device camera and transform them into collectible cards featuring scientific names, rarity levels, and unique attributes. The app is designed to be offline, meaning no internet connection is required to use it. Key features include camera-based plant capture, learning scientific and common names of flora, a rarity system (S-rank, rare, common), and offline gameplay.

BioDex helps nature enthusiasts, students, and casual gamers explore and build their own flora deck while playing offline. The app is built using Kotlin, Jetpack Compose, CameraX, and the Imageomics BioCLIP 2.5 ViT-H/14 image encoder. The model was trained to put photos of living things and their names into the same vector space, enabling users to identify plants by their photos.

The full teacher model is too large to run on a phone, so only the image encoder half runs on-device. The text half of the model has already done its job, as the 4,271 plant names were turned into embeddings ahead of time and shipped as a table. When a user captures a photo, the app crops and normalizes it to 224 x 224, runs a forward pass to get a vector, and compares it against the table with cosine similarity to find the closest match.

If no match is found with a similarity score of 0.45 or higher, BioDex informs the user that it cannot identify the plant. The app also generates cards with common names and short field notes written by a language model during the build process.