Home / Directory / AI infrastructure / Runtimes, compilers & systems software / Groq
Groq
LPU-centered inference cloud and systems stack (GroqCloud) optimized for low-latency token generation, with public API pricing on select open models. Nvidia licensed Groq’s inference technology and hired its founder and most of its leadership in December 2025, and GroqCloud continues to operate under Groq as an independent company.
Product teams that need very low-latency LLM inference via API and can standardize on GroqCloud-supported models.
Training clusters, edge silicon OEMs, or buyers that require self-serve pricing on every frontier Llama SKU.
Verdict
Groq fits when inference speed on GroqCloud is the buying reason: LPU-backed serving and a developer console that feels like an API runtime more than a GPU lease. Published prices cover selected open models (for example GPT-OSS tiers); several Llama SKUs are enterprise quote-led. Treat this as a systems+runtime buy, not a general GPU cloud. Skip it for training or if you must pin unsupported models.
Score Breakdown
How Groq scores in the categories that matter to its buyers.
Pricing
GroqCloud publishes per-million-token rates for selected production models in console docs; several Llama SKUs are enterprise contact-sales. Prices below are USD from Groq model docs as of September 2026.
| Plan / SKU | Meter | Price (USD) | What stands out |
|---|---|---|---|
| GPT-OSS 20B | per 1M tokens | $0.075 in / $0.30 out | Self-serve production |
| GPT-OSS 120B | per 1M tokens | $0.15 in / $0.60 out | Self-serve production |
| Whisper Large V3 Turbo | per audio hour | $0.04 | Speech transcription |
| Llama 3.1 8B Instant | per 1M tokens | Contact sales | Enterprise plan |
| Llama 3.3 70B Versatile | per 1M tokens | Contact sales | Enterprise plan |
The Field at a Glance
Where Groq ranks among Runtimes, compilers & systems software vendors we reviewed, by Overall Score and relative typical engagement cost.
Groq leads this runtimes peer set at 7.9, ahead of FriendliAI (6.8) and d-Matrix (6.7). Relative typical engagement cost is lower for API token meters than for dedicated inference accelerators or GPU-second engines.
Use-case matrix
| Use case | Fit | Notes |
|---|---|---|
| Low-latency LLM API inference | Strong | This is GroqCloud’s core job. |
| Open-weight model serving (supported set) | Strong | Fast when the model is on the supported list. |
| Custom model on your GPUs | Mixed | FriendliAI runs models on your own GPUs, which is a different setup. |
| Large-scale training | Poor | Groq focuses on inference. |
| Edge AIPU silicon | Poor | GroqCloud is a cloud runtime and does not sell edge chips. |
| Every Llama SKU self-serve | Mixed | Some Llama SKUs are enterprise-only. |
Review Excerpts
Excerpts from public reviews and forum discussions of Groq, plus representative buyer views drawn from public information. Paid reviews and pay-for-play sites were excluded.
“Production models on GroqCloud expose OpenAI-compatible endpoints with published token speeds often in the hundreds of tokens per second on supported SKUs.”
“GPT-OSS 20B and 120B still show clear self-serve input/output rates on the public models table, which keeps experimentation cheap relative to reserved GPUs.”
“When will Groq actually tell us what their rate limit is?!”
“Several Llama SKUs moved to contact-sales pricing, which frustrates teams that budgeted against the older public token card.”
Representative buyer view
“Responses come back fast enough that latency stopped being the thing our users complained about.”
Representative buyer view
Where customer reviews couldn't be found, our analysts attempted to give a voice to buyer sentiment, drawing on real information from review sites, forums, and product documentation.
Based on: GroqCloud’s model documentation and Groq’s announcement of the Nvidia licensing agreement
Company Facts
- Founded
- 2016 (CB Insights)
- Headquarters
- Mountain View, California, United States (CB Insights)
- Total funding raised
- $4.26B (CB Insights)
- Key investors
- Disruptive, NVIDIA, Tru Arrow Partners, Palo Alto Growth Capital, Infinitum Partners (CB Insights)
- Website
- groq.com
Key Leadership
Jonathan Ross
Founder
Methodology
This page is an independent evaluation of Groq for buyers comparing options in runtimes, compilers & systems software. AI Industry Reviews accepts no sponsorships, advertising, or pay-for-placement fees. Groq did not pay for this review.
What we scored
The headline number is an Overall Score on a 0-10 scale. Eight criteria fall under it in two groups.
- Buyer outcomes (for runtimes, compilers & systems software)
- Low-latency inference runtime
- GroqCloud developer UX
- Model catalog depth
- Compiler / systems stack
- Company & commercial
- Innovation & product leadership
- Project management & communication
- Pricing
- Contract fairness
Pricing measures whether the price looks fair for the value delivered, including packaging and renewal friction that show up in real buying cycles.
Score Composition
| Input | Weight | What it covers |
|---|---|---|
| Reviews | 40% | A proprietary read of what practitioners say about likes, complaints, and day-to-day use, including public review sites, forums, and private chat rooms. Paid reviews and pay-for-play sites are out of scope. |
| Product | 35% | Hands-on look at screens and workflows. |
| Pricing | 15% | Whether the price looks fair for what you get. |
| Docs & training | 10% | Docs, tutorials, and training material. |
How we balanced the evidence
The Overall Score is the simple average of the eight criteria. Recommendation language follows that score and the use-case matrix above.
Scope
Groq is graded here as runtimes, compilers & systems software. Criteria scores can move as more review volume and product checks are added.