A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis

A 10-year-old AI model, fine-tuned on a dermatology dataset, outperformed two advanced language models in diagnosing skin cancer. CNN outperformed GPT-6 Astra and Claude Fable 5.1 in identifying harmful lesions, correctly diagnosing 62% of the 200 biopsy-proven lesions. The CNN was significantly more accurate in diagnosing harmless lesions like moles and seborrheic keratoses, achieving a 62% accuracy rate.

In contrast, the frontier LLMs achieved a 42% accuracy rate for the same task. The CNN was able to identify 13 of the 30 melanomas, while the LLMs only identified 8 and 11 respectively. The CNN was also able to name the benign lesions correctly, correctly identifying 37 of 50 moles and 26 of 50 seborrheic keratoses. The LLMs were less accurate in this regard, only correctly identifying 16 and 13 respectively.