Don't classify. Hallucinate!
Don't classify. Hallucinate! I still have quite a bit of older content on my blog that I never got round to tagging. My blog has 1,856 tags - likely too many to feed to an LLM in one go and say "which of these tags match the following content". Doug Turnbull has a neat solution. Tell the model to output tags without any details of the existing vocabulary, then use vector embeddings against the…
Doug Turnbull has devised an innovative approach to tackle the challenge of classifying content without relying on pre-existing tags. Instead of attempting to feed a large set of tags to a large language model (LLM) and ask it to match them with new content, Turnbull suggests a different strategy.
The model should generate its own set of tags for a given query, without being constrained by any existing vocabulary. Turnbull provides a clear example of what these novel tags could look like. For instance, when given the product classification query "brown coffee table," the model might suggest tags such as "Furniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables Décor & Pillows & Blankets / Throw Pillows Furniture / Bedroom Furniture / Dressers & Chests Kitchen & Tabletop / Kitchen Organization / Food Storage & Canisters School Furniture and Supplies / School Chairs & Seating / Stackable Chairs Baby & Kids / Toddler & Kids Bedroom Furniture / Kids Beds."
To assist the model in generating more accurate and relevant tag suggestions, Turnbull recommends including an example of the desired tag format in the prompt. This approach allows the LLM to make more informed guesses about suitable classifications, rather than simply regurgitating existing tags. By empowering the model to create its own set of tags, Turnbull's method offers a fresh solution to the age-old problem of content classification.
Written by urgent.news from Simon Willison's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
