PipeWise: Turning r/Plumbing Into a Content Engine
A plumbing subreddit is a corpus of every question homeowners are too embarrassed to ask a plumber. Thousands of posts, each one a real problem with a real fixture and usually a photo. As raw text it's noise. Structured, it's a map of what a plumbing business should be writing about, ranked by how often people actually need the answer. PipeWise is the pipeline that does that structuring: scrape →…
PipeWise is a system that transforms the content of r/Plumbing, a subreddit dedicated to plumbing questions, into a valuable content engine for plumbing businesses. The system scrapes posts from the subreddit, enriches the data using a local machine learning model, stores the information in a structured format, clusters the data based on questions, ranks the clusters based on various factors such as frequency, evergreen nature, answer quality, and seasonality, and generates blog posts, video scripts, and FAQ blocks for the top-ranked clusters.
The system is built in four stages - scrape, enrich, store, and generate. It uses a local model for the majority of the work and only calls a paid API for high-value tasks. The system is designed to be flexible and modular, with each external interface injected as a function to allow for easy testing and mocking. It currently covers two products - a knowledge base and a content engine - with plans to expand into a market intelligence tool and a diagnostic assistant.
While there are some known issues with the clustering algorithm and the way embeddings are handled, the system is functional and produces useful rankings on day one.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
