{
  "id": 15453083,
  "title": "YouTube transcripts for RAG: fetch, chunk and embed captions in 30 lines of Python",
  "url": "https://urgent.news/2026/10/18/youtube-transcripts-for-rag-fetch-chunk-and-embed-captions-in-30",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-18T17:17:34.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/projectdev/youtube-transcripts-for-rag-fetch-chunk-and-embed-captions-in-30-lines-of-python-4125"
  },
  "original_language": "en",
  "account": "To perform Retrieval-Augmented Generation (RAG) with YouTube transcripts, three key components are required: an Apify Actor that fetches and returns timestamped transcripts, a chunking mechanism to segment the transcript into smaller pieces while preserving the start time, and a local open-source embedding model to convert the chunks into vectors for search and retrieval.\n\nTo begin, ensure that you have Python 3.11 or newer installed on your system. Additionally, you will need an API token from the Apify platform, which allows you to interact with the YouTube Transcript Scraper actor. The cost of using the actor is $0.001 per transcript delivered, equating to $1.00 for processing 1,000 videos. Note that videos without captions, private or unavailable videos, bad links, and playlist/channel listings are free—only transcripts that are returned to your dataset incur a cost. The free Apify plan typically has ample monthly usage credits to handle several hundred videos easily.\n\nTo set up the environment, install the necessary Python packages: `apify-client`, `sentence-transformers`, and `numpy`. If PyTorch is not already installed, it will be required for the embedding model and will consume a significant portion of your disk space.\n\nFirst, connect to Apify using the provided API token:\n```python\nimport os, json\nimport numpy as np\nfrom apify_client import ApifyClient\nfrom sentence_transformers import SentenceTransformer\n\nclient = ApifyClient(os.environ['APIFY_TOKEN'])\n```\n\nNext, fetch the transcripts for the specified YouTube video links. For this example, we'll use the channel of the popular educator, 3Blue1Brown:\n```python\nrun = client.actor('highbrow_fame/youtube-transcript-fast').call(\nrun_input={\n'videos': ['https://www.youtube.com/@3blue1brown'],\n'language': 'en',\n'maxVideosPerSource': 20\n}\n)\n```\n\nThe transcript data is returned as a structured dataset containing video information, including timestamps, segment durations, and textual content. This dataset is essential for the subsequent chunking and embedding steps. Each video record includes a list of segments, where each segment contains start time, duration, and the corresponding text.\n\nFinally, the pipeline includes a chunking function that slices the transcript into smaller segments, each no larger than 60 seconds of speech. This ensures that the chunks are manageable for vector embedding and efficient search. The chunks are then embedded using a local SentenceTransformer model, enabling semantic search capabilities within the video content without requiring the video files themselves.",
  "summary": "Disclosure: I built the Apify Actor used in this tutorial. This post was written with AI assistance. A talk, a lecture series or a product channel is a knowledge base that happens to be stored as video. To search it or feed it to a language model you need the captions as text, split into pieces, each piece turned into a vector. This tutorial does that with three pieces: an Apify Actor that…",
  "key_points": [
    "Install Python 3.11+, Apify-client, sentence-transformers, and numpy packages",
    "Fetch YouTube transcripts via Apify API token, segment into 60-second chunks",
    "Embed chunks with SentenceTransformer for semantic search and retrieval"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}