Lounge
Connecting…
Connecting…
View all

Mistral Large 4 Open Weights: Hardware and License Guide

Mistral Large 4 open weights: what the 1T-parameter, 52B-active MoE needs in GPU memory, which FP8, NVFP4 or GGUF build fits, and what the license allows.

AILatest7 min read55 viewsPublished Updated
Contents7 sections
  1. How the Mistral Large 4 open weights are built: 1T total, 52B active
  2. Hardware you need: checkpoint sizes and validated GPUs
  3. The Mistral license: commercial use and the MaaS condition
  4. Self-host Mistral Large 4 locally or use the API?
  5. Risks and limits before you deploy
  6. Frequently asked questions
  7. Sources and references

Key takeaways

  • Mistral Large 4 is a mixture-of-experts model with about 1T total parameters and 49B active per token (52B counting embeddings and output layers); the weights went public on October 29, 2026.
  • Only about 52B parameters do work per token, but all 128 experts must stay in memory: the official FP8 checkpoint is about 1.06 TB and the NVFP4 build about 610 GB.
  • The vLLM recipe validates data-center setups such as 8x GB300, B300 or B200, 16x H200 across two nodes, and 8x AMD MI350X.
  • Community GGUF files shrink it to 635 GB (Q4_K_M), 381 GB (Q2_K) or 298 GB (IQ2_XXS), small enough for a 512 GB unified-memory machine, with quality and speed trade-offs.
  • The modified MIT license allows commercial use and fine-tuning, but a company running a Model-as-a-Service business needs a separate commercial license from Mistral.

Running the Mistral Large 4 open weights yourself starts with one hard requirement: enough fast memory to hold roughly one trillion parameters. On October 29, 2026, Mistral released the weights of its improved Mistral Large 4 (nicknamed Le Chonk) on Hugging Face. Only about 52B parameters are active per token, yet the official FP8 checkpoint alone is about 1.06 TB, so eight data-center GPUs is the realistic entry point. As of October 2026, this guide uses the official announcement, model card, vLLM recipe and license text to explain the architecture, hardware sizing, quantization options and the commercial-use rules. The focus is the self-hosting decision rather than API features.

How the Mistral Large 4 open weights are built: 1T total, 52B active

The name 1T-A52B means about one trillion total parameters and about 52B active per token. The model card is more precise: 49B active parameters per token, or 52B once embeddings and output layers are included. It is a fine-grained mixture-of-experts (MoE) model with 128 routed experts, of which 4 are chosen per token, plus 1 shared expert used by every token. The vLLM recipe adds that it has 72 layers, MLA attention with a DSA sparse indexer, cross-layer KV sharing and a Pixtral-style vision encoder.

The common mistake is to treat the active count as the model size. Because the router can select any expert for the next token, every expert must stay loaded. Think of it this way: memory scales with the 1T total, while compute and decoding speed scale closer to the 52B active. That is the payoff of MoE. On hardware that can hold the weights, it generates tokens much faster than a dense 1T model would.

Total parameters

Official figure
About 1T
What it means for self-hosting
All weights must fit in memory

Active parameters

Official figure
49B per token (52B with embeddings and output)
What it means for self-hosting
Drives compute per token and speed

Experts

Official figure
4 of 128 routed + 1 shared
What it means for self-hosting
Expert parallelism matters for throughput

Context window

Official figure
1,048,576 tokens (1M)
What it means for self-hosting
vLLM validated up to 256k

Modalities

Official figure
Text and image in, text out
What it means for self-hosting
Vision encoder may be a separate file

Reasoning

Official figure
low, medium, high (default high)
What it means for self-hosting
Per the GGUF notes, reasoning cannot be turned off

Read the context figure carefully. The checkpoint advertises a million positions, but the vLLM recipe validated 256k and sets --max-model-len to that value, stating that serving at 1M is not validated. The checkpoint also ships a DSpark speculative-decoding draft head that proposes three tokens at a time. Mistral measured up to 1.9x higher throughput with it, while noting that gains depend on hardware and workload.

Hardware you need: checkpoint sizes and validated GPUs

Hardware you need: checkpoint sizes and validated GPUs

The default weights in the official repository are block-scaled FP8. The table combines the vLLM recipe's checkpoint list with the community GGUF builds from ggml-org. Sizes refer to weight files only; real serving also needs room for the KV cache, activations and runtime overhead.

Official FP8 (default)

Weight size
About 1.06 TB
Validated or stated target
8x GB300 or B300, 2 nodes of 8x H200, 8x MI350X

Official NVFP4

Weight size
About 610 GB
Validated or stated target
GB300 (one 4-GPU tray or 8 GPUs), 8x B200 or B300

AMD MXFP4 (Quark)

Weight size
About 578 GB
Validated or stated target
MI350 Series, one replica per 4 GPUs

GGUF Q8_0

Weight size
1.12 TB
Validated or stated target
Near-lossless

GGUF Q4_K_M

Weight size
635 GB
Validated or stated target
4-bit quant

GGUF Q2_K

Weight size
381 GB
Validated or stated target
Fits in 512 GB of unified memory

GGUF IQ2_XXS

Weight size
298 GB
Validated or stated target
imatrix-calibrated, fits in 512 GB

Per-GPU numbers make the sizing clearer. Split across eight GPUs, the recipe says FP8 needs about 128 GiB of weights per GPU and NVFP4 about 76 GiB. That rules out a single node of 80 GB H100s for FP8, and even 141 GB H200s were validated with 16 GPUs (TP16 or PP2 x TP8). NVFP4 targets Blackwell, so on Hopper hardware plan around FP8 and multi-node serving.

Software readiness

As of the October 2026 recipe, no regular vLLM release or nightly wheel includes Mistral Large 4 support, so you use the dedicated vllm/vllm-openai:mistral-large-4 image, or the ROCm image on AMD, along with mistral_common[image] 1.13.0 or later. The model card recommends vLLM and also documents SGLang, Transformers from the main branch and llama.cpp. Tool calling and separated reasoning output need --tool-call-parser mistral and --reasoning-parser mistral in vLLM. Fine-tuning guides exist for Axolotl, Unsloth and TRL.

Picking a quantization

  • Quality first: use official FP8 or NVFP4. The recipe measured NVFP4 at 1.09 to 1.18x faster per token than FP8 at TP8, with about 1.5x the KV capacity.
  • Hardware-constrained: start with GGUF Q4_K_M (635 GB) and drop to Q2_K or IQ2_XXS only if you must.
  • The 2-bit builds compress the routed experts hard while keeping attention, shared experts and embeddings at Q8_0. Do not assume published benchmark scores carry over.
  • For image input, download the roughly 3.4 GB mmproj vision encoder alongside the GGUF.

The Mistral license: commercial use and the MaaS condition

Mistral Large 4 is not Apache 2.0. It ships under the Mistral Ecosystem Open-Weight License, which the license file describes as a modified MIT license. It grants rights to use, copy, modify, distribute and sell the model, and to further train it and create derivative works. The copyright and permission notice must be included in all copies or substantial portions.

The key restriction is section 2. If you or any affiliate operate a Model-as-a-Service business, you must sign a separate commercial license with Mistral before using the model or its derivatives for any commercial purpose. The license defines MaaS as giving a third party inference or post-training access, for example via an API or hosted endpoint, in a way that lets that party exercise meaningful control over inputs, parameters or training data.

Internal chatbot or coding assistant

MaaS condition under the text
Does not apply (internal use)
What to check
The model is not offered to outside parties

End-user product with the model embedded in features

MaaS condition under the text
Excluded from the definition
What to check
Users cannot freely drive the raw model

Relaying requests to a model hosted elsewhere

MaaS condition under the text
Excluded from the definition
What to check
You are not hosting it yourself

Selling a general inference API

MaaS condition under the text
Separate agreement needed for MaaS operators
What to check
Affiliates count too

Using it through Mistral's own products

MaaS condition under the text
Does not apply
What to check
Those products have their own terms

The license also provides the model and its output as is, without warranty, and the model card says not to use it in ways that infringe third-party rights, including intellectual property. The terms above reflect the initial commit on October 29, 2026. Check the current LICENSE file before deploying, and get legal advice if your business model sits near the MaaS line.

Self-host Mistral Large 4 locally or use the API?

  1. 1

    Write down why you need the weights

    Data residency, fine-tuning on proprietary data and offline operation are strong reasons. If none apply, the case for self-hosting weakens quickly.

  2. 2

    Price the API first

    Mistral lists $0.70 per million input tokens, $0.07 per million cached input tokens and $2.10 per million output tokens. Estimate monthly usage, then compare it with renting or buying an 8-GPU server, power and the engineers who keep it running.

  3. 3

    Match checkpoint to hardware

    Blackwell points to NVFP4, Hopper H200 to FP8 across two nodes, AMD MI350 to MXFP4. Unified-memory machines can test the low-bit GGUF builds, but benchmark speed before committing.

  4. 4

    Pilot on a small scope

    Start from the recipe's baseline command with context at or below 256k and measure quality and latency on real prompts. The model card itself suggests the Mistral API if local serving underperforms.

  5. 5

    Review license and security

    Confirm whether your offering counts as MaaS, then lock down endpoint authentication, logging and access control. Self-hosters also have to apply guardrail updates themselves.

Serving engines, quantized files and community measurements change quickly after an open-weight launch. The theddari AI Telegram trends page and the blog trends dashboard can help you follow channel discussion and hands-on write-ups as they appear. Community numbers come from mixed setups, so confirm them against the official recipe and your own tests.

Risks and limits before you deploy

  • Most benchmark figures come from Mistral or its partners, and the full technical report is still listed as coming soon. Independent evaluations are worth waiting for.
  • Mistral says it observed potential uplift for real cyberattacks and will keep modifying guardrails. On your own infrastructure, tracking those updates is your job.
  • Serving at 1M context, DSpark with pipeline parallelism and several ROCm features are not yet validated or supported in the recipe.
  • The download is around a terabyte, so plan storage, transfer time and version management.

In short, the Mistral Large 4 open weights are a serious option for organizations that need control over data and customization, not a model for casual desktop use. Size memory for 1T, expect speed closer to a 52B model, line up checkpoint, hardware and license, and start with a small pilot.

Frequently asked questions

Can I run Mistral Large 4 on a single RTX 4090 or 5090?
Not in any practical sense. The smallest published GGUF, IQ2_XXS, is about 298 GB, far beyond the 24 GB or 32 GB on a consumer card. Offloading most layers to system RAM can technically load the model, but generation becomes very slow and 2-bit quantization may reduce quality. For experimentation, the Mistral API or OpenRouter is usually the cheaper path.
Why does a 52B-active model need memory for 1T parameters?
Because the router picks 4 of 128 experts per token, and the selection changes token by token. Any expert may be needed at any moment, so the full set of weights has to be resident in fast memory. The active count mainly determines compute per token and therefore speed. You also need extra memory for the KV cache, which grows with context length and concurrency.
Does an internal company chatbot need a commercial license?
Under the license text, internal use that does not make the model available to third parties is outside the Model-as-a-Service condition. End-user products where the model is embedded in specific features are also excluded from the definition. Selling a general inference API where customers control inputs, parameters or training data is the case that requires a separate agreement. If your setup sits near that boundary, read the license with legal counsel.
Will it run on a Mac Studio with 512 GB of unified memory?
The ggml-org GGUF repository says the Q2_K (381 GB) and IQ2_XXS (298 GB) files fit in 512 GB of unified memory. Fitting is not the same as being useful, so measure tokens per second and leave headroom for long contexts. Image input also requires the separate vision encoder (mmproj) file of about 3.4 GB.

Sources and references

  1. 1Mistral AI: Mistral Large 4 (Le Chonk) is now open weight(opens in a new tab)mistral.ai
  2. 2Hugging Face: mistralai/Mistral-Large-4-1T-A52B model card(opens in a new tab)huggingface.co
  3. 3Mistral Ecosystem Open-Weight License(opens in a new tab)huggingface.co
  4. 4vLLM Recipes: Mistral-Large-4-1T-A52B(opens in a new tab)recipes.vllm.ai
  5. 5ggml-org: Mistral-Large-4-1T-A52B-GGUF(opens in a new tab)huggingface.co