Running the Mistral Large 4 open weights yourself starts with one hard requirement: enough fast memory to hold roughly one trillion parameters. On October 29, 2026, Mistral released the weights of its improved Mistral Large 4 (nicknamed Le Chonk) on Hugging Face. Only about 52B parameters are active per token, yet the official FP8 checkpoint alone is about 1.06 TB, so eight data-center GPUs is the realistic entry point. As of October 2026, this guide uses the official announcement, model card, vLLM recipe and license text to explain the architecture, hardware sizing, quantization options and the commercial-use rules. The focus is the self-hosting decision rather than API features.

How the Mistral Large 4 open weights are built: 1T total, 52B active

The name 1T-A52B means about one trillion total parameters and about 52B active per token. The model card is more precise: 49B active parameters per token, or 52B once embeddings and output layers are included. It is a fine-grained mixture-of-experts (MoE) model with 128 routed experts, of which 4 are chosen per token, plus 1 shared expert used by every token. The vLLM recipe adds that it has 72 layers, MLA attention with a DSA sparse indexer, cross-layer KV sharing and a Pixtral-style vision encoder.

The common mistake is to treat the active count as the model size. Because the router can select any expert for the next token, every expert must stay loaded. Think of it this way: memory scales with the 1T total, while compute and decoding speed scale closer to the 52B active. That is the payoff of MoE. On hardware that can hold the weights, it generates tokens much faster than a dense 1T model would.

Read the context figure carefully. The checkpoint advertises a million positions, but the vLLM recipe validated 256k and sets --max-model-len to that value, stating that serving at 1M is not validated. The checkpoint also ships a DSpark speculative-decoding draft head that proposes three tokens at a time. Mistral measured up to 1.9x higher throughput with it, while noting that gains depend on hardware and workload.

Hardware you need: checkpoint sizes and validated GPUs

The default weights in the official repository are block-scaled FP8. The table combines the vLLM recipe's checkpoint list with the community GGUF builds from ggml-org. Sizes refer to weight files only; real serving also needs room for the KV cache, activations and runtime overhead.

Per-GPU numbers make the sizing clearer. Split across eight GPUs, the recipe says FP8 needs about 128 GiB of weights per GPU and NVFP4 about 76 GiB. That rules out a single node of 80 GB H100s for FP8, and even 141 GB H200s were validated with 16 GPUs (TP16 or PP2 x TP8). NVFP4 targets Blackwell, so on Hopper hardware plan around FP8 and multi-node serving.

Software readiness

As of the October 2026 recipe, no regular vLLM release or nightly wheel includes Mistral Large 4 support, so you use the dedicated vllm/vllm-openai:mistral-large-4 image, or the ROCm image on AMD, along with mistral_common[image] 1.13.0 or later. The model card recommends vLLM and also documents SGLang, Transformers from the main branch and llama.cpp. Tool calling and separated reasoning output need --tool-call-parser mistral and --reasoning-parser mistral in vLLM. Fine-tuning guides exist for Axolotl, Unsloth and TRL.

Picking a quantization

• Quality first: use official FP8 or NVFP4. The recipe measured NVFP4 at 1.09 to 1.18x faster per token than FP8 at TP8, with about 1.5x the KV capacity.
• Hardware-constrained: start with GGUF Q4_K_M (635 GB) and drop to Q2_K or IQ2_XXS only if you must.
• The 2-bit builds compress the routed experts hard while keeping attention, shared experts and embeddings at Q8_0. Do not assume published benchmark scores carry over.
• For image input, download the roughly 3.4 GB mmproj vision encoder alongside the GGUF.

The Mistral license: commercial use and the MaaS condition

Mistral Large 4 is not Apache 2.0. It ships under the Mistral Ecosystem Open-Weight License, which the license file describes as a modified MIT license. It grants rights to use, copy, modify, distribute and sell the model, and to further train it and create derivative works. The copyright and permission notice must be included in all copies or substantial portions.

The key restriction is section 2. If you or any affiliate operate a Model-as-a-Service business, you must sign a separate commercial license with Mistral before using the model or its derivatives for any commercial purpose. The license defines MaaS as giving a third party inference or post-training access, for example via an API or hosted endpoint, in a way that lets that party exercise meaningful control over inputs, parameters or training data.

The license also provides the model and its output as is, without warranty, and the model card says not to use it in ways that infringe third-party rights, including intellectual property. The terms above reflect the initial commit on October 29, 2026. Check the current LICENSE file before deploying, and get legal advice if your business model sits near the MaaS line.

Self-host Mistral Large 4 locally or use the API?

1. 1

   Write down why you need the weights

   Data residency, fine-tuning on proprietary data and offline operation are strong reasons. If none apply, the case for self-hosting weakens quickly.

2. 2

   Price the API first

   Mistral lists $0.70 per million input tokens, $0.07 per million cached input tokens and $2.10 per million output tokens. Estimate monthly usage, then compare it with renting or buying an 8-GPU server, power and the engineers who keep it running.

3. 3

   Match checkpoint to hardware

   Blackwell points to NVFP4, Hopper H200 to FP8 across two nodes, AMD MI350 to MXFP4. Unified-memory machines can test the low-bit GGUF builds, but benchmark speed before committing.

4. 4

   Pilot on a small scope

   Start from the recipe's baseline command with context at or below 256k and measure quality and latency on real prompts. The model card itself suggests the Mistral API if local serving underperforms.

5. 5

   Review license and security

   Confirm whether your offering counts as MaaS, then lock down endpoint authentication, logging and access control. Self-hosters also have to apply guardrail updates themselves.

Serving engines, quantized files and community measurements change quickly after an open-weight launch. The theddari AI Telegram trends page and the blog trends dashboard can help you follow channel discussion and hands-on write-ups as they appear. Community numbers come from mixed setups, so confirm them against the official recipe and your own tests.

Page Reference

Trendstheddari.com›TrendsTelegram · AILive posts from AI channels

Page Reference

Trendstheddari.com›TrendsBlog DashboardNaver blog activity rankings

Risks and limits before you deploy

• Most benchmark figures come from Mistral or its partners, and the full technical report is still listed as coming soon. Independent evaluations are worth waiting for.
• Mistral says it observed potential uplift for real cyberattacks and will keep modifying guardrails. On your own infrastructure, tracking those updates is your job.
• Serving at 1M context, DSpark with pipeline parallelism and several ROCm features are not yet validated or supported in the recipe.
• The download is around a terabyte, so plan storage, transfer time and version management.

In short, the Mistral Large 4 open weights are a serious option for organizations that need control over data and customization, not a model for casual desktop use. Size memory for 1T, expect speed closer to a 52B model, line up checkpoint, hardware and license, and start with a small pilot.