Tencent Hy4 770B โอเพนซอร์ส Apache 2.0 กับ IQuest-Q1 เปิดใช้แค่ 15B

Tencent, a Chinese tech giant, has recently opened-sourced its Hy4 preview model, which boasts 770,000 million parameters and can handle a context length of 1 million tokens. This model follows the Apache 2.0 license and is available on the company's GitHub page. The model architecture consists of 78 layers, with the first layer being a feed-forward neural network (FFN) and the remaining 77 layers employing the MoE (Mixture of Experts) approach.

Each layer in the MoE layers has 256 experts, with a single expert per token. The model also includes an MTP (Mixed Precision Training) layer with 10 billion parameters, which is used for latent token decoding. In terms of performance, Tencent's Hy4 preview achieved a score of 2.99 in a closed-test against 163 internal experts, outperforming GLM 5.3 (2.92) and Kimi K3 (2.94) by 46.8%, 12.8%, and 40.4%, respectively.

However, the model still exhibits some issues, such as slower thinking times on complex tasks and a tendency to repeat self-investigation. IQuest-Q1, another open-source model from Tencent's IQuest Lab, is a MoE model with approximately 320,000 million parameters but only uses about 15 billion parameters per token. The model architecture consists of 88 layers, with 256 experts and 8 experts per token.

It supports a context length of 524,288 tokens and has two specialized layers for training and inference. IQuest-Q1 requires at least 8 GPUs and recommends using SGLang or vLLM for deployment. The model is licensed under a modified MIT license, which requires the model name to be displayed on the UI when used commercially.