I built my friend a voice-based recipe note-taking app

I developed a browser-based app that allows users to record audio and convert it into recipe notes or memos. This application was created for my friend, Arnesh, who enjoys cooking and frequently needs to take notes while his hands are occupied. The app can be accessed at https://boisterous-gnome-e40fb5.netlify.app/ and its code is available on GitHub at https://github.com/XDut/recipe-keeper.

The app operates entirely within the browser, utilizing the browser's microphone APIs to capture audio input. It then employs the Transformers.js library with the open Xenova/whisper-tiny model for speech-to-text conversion. Once the audio is transcribed, the text is parsed into structured recipe information, including ingredients, steps, timing, and other relevant details.

To maintain a clean and easily extendable codebase, I structured the project into distinct modules. These modules cover areas such as audio recording, transcription, recipe parsing, recipe storage, rendering, UI state, and utilities. This design approach simplifies the overall understanding and future expansion of the application.

The user experience is straightforward: audio input is captured, transcribed using the Whisper Model, and the resulting recipe memo is displayed, downloadable as JSON, or printed. For testing purposes, sample audio is provided, enabling users to experience the app without the need to record their own audio.

The motivation behind this project stemmed from the personal and sensitive nature of cooking instructions. Recipes may encompass family traditions, personal voice recordings, or other content that users might prefer to keep private, avoiding uploading to third-party servers. By adopting a local, browser-based solution, the app addresses these concerns effectively.

Utilizing an open model through Transformers.js allows the core transcription functionality to run locally within the user's browser, negating the necessity for a proprietary speech-to-text API. Consequently, there is no requirement to establish a separate backend solely for transmitting someone's cooking audio to a server and awaiting the transcription results.

Perhaps most importantly, this project underscores the potential of AI technologies to create practical applications without necessitating a cloud service infrastructure. By harnessing the power of a compact open model running within the browser, the app successfully transforms a straightforward voice recording into a valuable tool.