Singapore a high-value market for ChatGPT ads, but Indonesia, Vietnam offer scale: analysts

Change.org says it is investing $100M of its own money to rebuild its core petitions platform with AI and has launched an AI copilot beta for petition creators (Dan Primack/Axios)

Change.org was founded as a venture-backed organization, before a 2021 restructuring that saw investors like Reid Hoffman …

1h ago ·axios.com

Running 100B+ MoE Models on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp

Tuần này trên Hacker News có một bài hơn 600 điểm: chạy một model MoE 125B tham số trên một con RTX 4090 mà vẫn đạt tốc độ sinh token rất đáng nể.

• A 125-billion parameter MoE model runs on a 24GB VRAM RTX 4090 using llama.cpp
• llama.cpp offers two expert offloading methods: --n-cpu-moe and -ot flags
• Quantizing KV caches to 8-bit reduces VRAM usage significantly

1h ago ·dev.to

"Your Ai coach"

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend What I Built Code How I Built It Why Does Open Innovation Matter? Prize Categories

1h ago ·dev.to