{
  "id": 10639536,
  "title": "What it takes to remove the monitor and keyboard: decoders, projection and phosphenes",
  "url": "https://urgent.news/2026/09/29/what-it-takes-to-remove-the-monitor-and-keyboard-decoders-projection",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-29T07:37:26.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/_76130e67067eab4c8510/what-it-takes-to-remove-the-monitor-and-keyboard-decoders-projection-and-phosphenes-1og1"
  },
  "original_language": "en",
  "account": "What it takes to remove the monitor and keyboard: decoders, projection, and phosphenes\n\nThe question was whether AI could send signals into a brain so that an image appears on someone's retina. Retina is the input side of vision, not the output. An image formed in the visual cortex requires two independent problems: getting intent into the AI and getting the AI's output into the vision.\n\nDecoding speech from brain activity is the current approach. The system records from motor areas, not free thought, with 256 intracortical electrodes in total. A recurrent network trained with CTC loss outputs phoneme probabilities, a 3-gram language model turns phonemes into words, resulting in 62 words per minute with a 9.1% word error rate.\n\nAnother approach uses surface electrodes to record neural features, which are binned into short time windows. Large language model rescoring further improves accuracy to 97.5% for 8.4 months at about 32 words per minute. Phosphene generation is another method, where a stimulation pipeline maps electrodes to phosphene production, simulates perception, and tunes per-patient.\n\nRetinal projection is an optics problem, where a beam focused at the pupil center, called a Maxwellian view, keeps the image sharp regardless of the eye's focus. QD Laser's retinal scanning display uses RGB lasers and a MEMS mirror, while Meta's Orion and Ray-Ban Display use waveguides. Eye tracking latency affects the exit pupil tracking gaze, with foveated rendering needed for wider views.\n\nCortical stimulation encodes target images by mapping electrodes to phosphene production, using a differentiable simulator to predict percepts, training an encoder network to map frames to stimulation, and adjusting currents to generate a perceptual image. Dynaphos is a PyTorch simulator for step 2, with limited evidence showing identification of letters and object outlines in blind participants.",
  "summary": "The question that started this: can AI send signals into a brain so that an image shows up on someone's retina? I wanted to see AI output with my own eyes and skip the keyboard. I have not built anything yet, so this post is a technical survey of the pieces and a plan for what can be reproduced in software. Figures come from papers and official announcements, and I mark the ones I could only…",
  "key_points": [
    "Decoding speech from brain activity involves motor areas and 256 intracortical electrodes.",
    "Phosphene generation maps electrodes to phosphene production for per-patient tuning.",
    "Retinal projection uses optics to maintain image sharpness regardless of eye focus."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}