Ten YouTube Comments, One Commenter: What a Reply Budget Does to Your Sample
I run a visa agency in Bali. The part of my week that has nothing to do with visas goes into Apify Actors, and one of them pulls YouTube comments — one video, a channel, a playlist, or the top results of a search, one row per comment with likes, reply count, the pinned and hearted flags and the commenter's channel. People use comment data for sentiment: pull the last twenty videos of a…
The article discusses how YouTube comments can be scraped using Apify Actors, revealing that the sample size may not accurately represent the audience's sentiment. Two experiments on the same video, "Me at the zoo", with 10 comments each, showed vastly different results depending on whether replies were included. When replies were excluded, the comments were diverse and full-length.
However, when replies were included (default), the dataset was dominated by short, repetitive replies that did not reflect the broader audience's opinions. The article highlights the importance of understanding how comment sampling works, as the actor collects comments depth-first, meaning it follows replies before moving on to the next top-level comment.
This can lead to a skewed sample, with only a small number of comments dominating the dataset. To get a more accurate representation of audience sentiment, one should either exclude replies or adjust the sample size accordingly to account for the additional rows generated by including replies.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
