Change.org says it is investing $100M of its own money to rebuild its core petitions platform with AI and has launched an AI copilot beta for petition creators (Dan Primack/Axios)

Running 100B+ MoE Models on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp

Tuần này trên Hacker News có một bài hơn 600 điểm: chạy một model MoE 125B tham số trên một con RTX 4090 mà vẫn đạt tốc độ sinh token rất đáng nể.

• A 125-billion parameter MoE model runs on a 24GB VRAM RTX 4090 using llama.cpp
• llama.cpp offers two expert offloading methods: --n-cpu-moe and -ot flags
• Quantizing KV caches to 8-bit reduces VRAM usage significantly

1h ago ·dev.to

Quiz Master: AI Quiz Learning Companion

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend What I Built I built Quiz Master, an AI-powered quiz application designed to make learning more interactive, engaging…

• Quiz Master is an AI-powered quiz application.
• Created by Divya Hodke for interactive learning.
• Open innovation model makes AI accessible.

1h ago ·dev.to

Singapore a high-value market for ChatGPT ads, but Indonesia, Vietnam offer scale: analysts

Ads are shown to users on its free tier and Go plan, which costs S$11 a month in the city-state

1h ago ·businesstimes.com.sg