Models

Are all open-weight AI models open source?

Short answer: not necessarily.

Open weights means the trained model parameters are available to download and use under a license. You can run it locally.
Open source usually means the source code, training code, data, and license give you the freedom to study, modify, and redistribute the whole thing.
Many “open” AI models are open-weight but not fully open-source. Their licenses may restrict commercial use, derivatives, or redistribution. Others use Apache 2.0 or MIT, which are much closer to open source — but even then, the training data and full training pipeline may not be released.

Free Open Weights AI Models Examples:

• Qwen3.8-27B — Apache 2.0 weights
• Flint — Apache 2.0 weights
• DeepSeek V4.1 Flash — MIT-licensed weights
• Llama / Gemma — open-weight, but with custom community licenses

So: open weights are great for local AI; but “open source” is a stronger claim. Always check the license before assuming you can commercialize, fine-tune, or redistribute a model.

We also have AI models that are free to download and paid to use commercially.

Commercial Open weights AI Models Examples:

• Cohere Command R / R+  — CC-BY-NC 4.0 : Commercial use requires a paid license
• Mistral Medium / Large — Mistral Commercial License : Commercial/on-prem deployment
• FLUX.1 [dev] — Non-commercial license : Commercial use requires a paid license

Important distinction

Open weights = you can download the model.
Open source = you can study, modify, and redistribute the code and often the model.
Commercial license = you can use it in a business or paid product.
A model can be downloadable and still not be commercially usable for free.

So if you’re building a commercial local product, check the license first. “Downloadable” does not automatically mean “free to use commercially.”

AI Model Size Classes: From Tiny to Trillion-Parameter Giants

Size is the other big dividing line in the downloadable model world. A 1B model can run on a phone; a 500B mixture-of-experts model can need a workstation with a serious amount of RAM. The examples below are all models you can download and run locally — licensing varies, so check the section above before using one commercially.

What “size” actually means

Parameter count is the headline number, but it is not the whole story:

• Dense models use all their parameters for every token. A 27B dense model behaves like a 27B model.
• Mixture-of-experts (MoE) models have many total parameters but activate only a small subset per token. A 30B-A3B model has 30B total parameters and about 3B active. It can run like a small model while carrying more knowledge.
• Quantization changes the memory footprint. A 27B model at 4-bit is roughly 17GB; the same model at 8-bit is much larger.

That is why a 125B MoE model can sometimes run on a laptop while a 125B dense model would be hopeless.

Size classes at a glance

Parameter counts are approximate. A 27B model and a 30B model are usually in the same practical class.

Dense vs. MoE in practice

A MoE model can be much faster than a dense model of the same total size, but it still needs enough memory to hold the experts it uses.

What runs on what

Rough guidance for 4-bit quantized models:

Approximate 4-bit weight sizes:

• 4B — around 2.5GB
• 8B — around 5GB
• 14B — around 9GB
• 27B — around 17GB
• 70B — around 40GB
• 405B — around 230GB
• 552B — around 300GB

Add context cache, the operating system, and whatever else is running. A model that “fits” in theory may still crawl if you give it a 64K context on a 24GB machine.

Small models are not toys anymore

The 1–4B class has improved dramatically. Models like Qwen3-4B, Llama 3.2 3B, Gemma 3 4B, and Phi-3.5-mini can handle summaries, simple coding, local chat, basic RAG, tool calls, and small agent workflows.

They will not match a 27B model on hard reasoning, but they are fast, cheap, and private.

The 7–14B sweet spot

For many home users, 7–14B is the practical sweet spot. You get noticeably better reasoning and coding than the tiny models, while still running on a 16–24GB machine.

Good examples:

• Mistral 7B
• Llama 3.1 8B
• Qwen3-8B
• Qwen3-14B
• Phi-4 14B
• Gemma 3 12B

This is the class where local AI starts to feel genuinely useful for daily work.

The 24–32B workstation class

This is where things get interesting for enthusiasts. A 27B-class model like Qwen3.8-27B or Gemma 3 27B can handle serious coding, long documents, and multimodal tasks — if you have the memory.

You need roughly:

• 24GB VRAM for a tight Q4 fit
• 32GB for comfort
• 48GB+ for long context or higher precision

This is also the class where Macs with unified memory become attractive.

The 70B+ class

Once you move past 30B, you are in workstation or server territory.

• Llama 3.3 70B — a strong general model, but needs around 40GB at Q4
• Qwen2.5-72B — excellent quality, similar hardware demands
• DeepSeek-R1-Distill-Llama-70B — reasoning-focused

At this size, quantization and context length matter as much as raw parameter count.

The 200B+ MoE era

The newest trick is massive MoE models with small active parameter counts:

• DeepSeek V4.1-Flash — 552B total, about 16B active
• Qwen3.8-Flash-Next — 125B total, about 6B active
• Qwen3-235B-A22B — 235B total, about 22B active

These models can run on high-RAM workstations by keeping hot experts in memory and streaming the rest from SSD. They are slower than small models, but they bring frontier-adjacent quality to local hardware.

Which size should you choose?

• Phone / tiny device: under 1B
• Old office laptop: 1–4B
• 16GB laptop: 7–9B
• 24GB GPU: 14B comfortable, 27B tight
• 32GB: 27–32B
• 64GB+: 70B at low bit or larger MoE
• 128GB+: 70B at Q4, big MoE experiments
• Multi-GPU: 200B+ MoE

Bottom line

The downloadable model world now has a model for almost every size of hardware. The trick is not chasing the biggest number — it’s matching the model class to your RAM, your patience, and the job you actually need done.

Start small. Measure. Then move up a class only when you have a reason.

Dense or MoE? Choosing a Local Model

Every local model decision comes down to one question: how do I get the best answers out of the hardware I already own? Two architectures answer it in opposite ways, and understanding the difference saves you from downloading a 30 GB model that runs worse than a 5 GB one.

Two ways to spend your memory

A dense model uses all of its parameters for every token it generates. An 8B dense model does roughly 8 billion parameter reads per token, every single time, with no shortcuts. That makes its cost easy to reason about: memory scales with size, speed scales with size, and quality scales with size.

A mixture-of-experts (MoE) model splits its feed-forward layers into many separate "expert" subnetworks and adds a small router that picks the best two to four experts for each token. Only those experts do any work. Mixtral 8x7B holds 46.7B parameters but activates roughly 12.9B per token; Qwen3-30B-A3B holds 30.5B and activates 3.3B.

The trade, in one line: MoE lets you pay for a big model in memory and run a small model in compute.

Side by side

Where dense models win

They fit. A 7–9B dense model at 4-bit quantization takes about 5 GB, which means it runs on almost any laptop made in the last five years and leaves room for context.

They are easy to fine-tune. A LoRA on a dense 8B model is a weekend project with well-worn tooling. The same job on an MoE is a research problem: the router can collapse onto a handful of experts, and rare experts receive too few tokens to learn from.

They degrade gracefully. Push a dense model down to 3-bit or 2-bit and you get a dumber model that still behaves like a model. Push an MoE that far and specific experts fall apart, producing failures that look like bugs rather than reduced intelligence.

They are predictable. No routing decisions means no surprises. For deterministic pipelines, evaluation suites, and anything you need to reason about, that consistency is worth real quality points.

Where MoE models win

Quality per unit of compute. A 30B MoE with 3B active parameters generates tokens at roughly the speed of a 3B model while answering closer to a 30B one. On hardware that is short on compute but long on memory — Apple Silicon with 32 GB or more, a single 24 GB GPU, a workstation with lots of RAM — that is the best deal available.

Fast generation. Token-by-token decoding is limited by how many bytes you read from memory per token, not by raw arithmetic. MoE reads only the active experts, so it moves far less data per token than a dense model of the same total size. The effect is dramatic on memory-bandwidth-limited machines.

Long outputs and agent loops. Any workload that generates a lot of tokens — agents, long-form drafting, chain-of-thought — benefits most, because the savings apply to every single token.

Multiple users. In a server setting, different requests route to different experts, so one model can serve high throughput without every request paying for the full parameter count.

The catch: you still load everything

This is the part that trips people up. MoE saves you compute, not memory. Every expert has to be in RAM or VRAM before the model can respond, because the router might pick any of them.

That means a 30B MoE needs the footprint of a 30B dense model — around 18 GB at 4-bit — even though it computes like a 3B one. If it doesn't fit, you are offloading experts to system RAM or disk, and the speed advantage evaporates. A half-offloaded MoE is often slower than a dense model a third of its size, and it will not even be much smarter.

How to choose

The mistake is comparing an MoE to a dense model with the same active parameters. They are not competitors. Compare it to the largest dense model that actually fits in your memory.

• 8—16 GB of RAM, integrated graphics: dense, 7–9B at 4-bit. A 30B MoE does not fit and offloading will disappoint you.
• 16—24 GB (unified memory or a 12–16 GB GPU): dense 14B at 4-bit, or a small MoE like gpt-oss-20b if it fits with room for context.
• 32—64 GB unified memory: the MoE sweet spot. A 30B-A3B model fits comfortably and gives you near-32B-dense quality at roughly 3B-dense speed.
• 64 GB+ or multiple GPUs: larger MoE models become genuinely practical, and the compute savings compound.

Two rules of thumb cover most of it:

1. MoE memory ≈ dense model of the same total size.
2. MoE speed ≈ dense model of the same active size.

If both of those facts work for your machine, choose MoE. If only the second one does, choose dense.

Bottom line

Dense models are the dependable default: they fit in less memory, tolerate quantization, fine-tune cleanly, and behave the same way twice. MoE models are the better buy when memory is abundant and compute is the bottleneck — they buy you a larger model's knowledge at a smaller model's speed, as long as you can hold all of it at once.

Pick the architecture your RAM can actually feed, then pick the biggest model inside that budget. That beats chasing parameter counts in either direction.

What are abliterated AI models?

Abliterated AI models are open-weight models that have had their refusal behavior deliberately reduced or removed by editing the model’s internal weights — not just by prompting it differently.

The technique is called abliteration (from “ablation”). Researchers identify the internal direction associated with refusing requests, then remove or suppress that direction. The result is a model that is much less likely to say “I can’t help with that,” even for prompts the original model was trained to reject.

They’re popular for local experimentation, roleplay, red-teaming, and testing model behavior without refusal layers. Common names look like:

• Llama-3-8B-Instruct-abliterated
• Mistral-7B-Instruct-abliterated
• Qwen3.8-27B-abliterated

Important caveats:

Abliterated does not mean fully uncensored. The model may still refuse some things.
Safety alignment is weakened, so it can produce harmful or inappropriate content.
Quality can degrade — reasoning, coherence, and instruction-following may suffer.
Licenses and platform rules still apply.
Don’t deploy one publicly without guardrails.
Simple version: a normal instruct model has a built-in “I shouldn’t answer that” reflex. An abliterated model has had that reflex dampened at the weight level. It’s a research and hobbyist tool, not a free pass around safety or legality.

How do uncensored and abliterated AI models differ?

“Uncensored” is the broad label: any model trained or modified to refuse less, often through fine-tuning on unfiltered data. “Abliterated” is a specific technique: researchers identify the model’s internal refusal direction and remove or suppress it directly in the weights, usually with little or no extra fine-tuning. So every abliterated model is effectively uncensored, but not every uncensored model is abliterated — some are uncensored because they were fine-tuned on permissive data, while abliterated models had the refusal behavior surgically ablated.