Kimi K3 vs GLM-5.3 vs DeepSeek-V4: Architecture, Benchmarks and Deployment

The report provides an in-depth analysis of three large language models: Kimi K3, GLM-5.3, and DeepSeek-V4-Pro. These models are compared based on their architecture, performance benchmarks, and deployment options.

Kimi K3 utilizes a hybrid approach combining 69 KDA layers with 24 Gated MLA layers, allowing for long context handling up to 1M tokens. DeepSeek-V4-Pro incorporates Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) techniques, and features shared experts for its configuration. GLM-5.3, on the other hand, builds upon the GLM-5.2 base model and enhances it through post-training.

The models exhibit varying parameter counts, with Kimi K3 boasting 2.8 trillion parameters, GLM-5.3 at approximately 753 billion, and DeepSeek-V4-Pro at 1.6 trillion. The active parameters for Kimi K3 are measured at 104 billion, while DeepSeek-V4-Pro utilizes a mixed precision approach with FP4/F8 formats.

When comparing benchmark results, it is important to note that the values come from the model cards rather than independent reproductions. For instance, Terminal-Bench 3.0 scores for Kimi K3, GLM-5.3, and DeepSeek-V4-Pro are 17.4, 28.3, and not reported respectively. These scores are specific to certain task versions and may not indicate a regression or superiority across all tasks.

In terms of deployment, the choice of model depends on input modality and available hardware kernels. Each model has its unique requirements, such as DeepSeek-V4-Pro's native FP4/F8 mix and Kimi K3's MXFP4 format. The report emphasizes the need to consider total weights for MoE capacity, API input/output handling, and cost measurement when deploying these models.