Muse Glimmer vs Qwen 3.8-27b on an Intel Arc Pro B70

Why this ran on our own hardware

One of our pipelines turns a sales qualification sheet into an internal win story. The sheet carries the customer’s name, named contacts and their email addresses, a revenue band, transaction volumes, and a frank account of what is broken inside their finance operation. The output is internal only.

We are not willing to paste that into somebody else’s API. Not because any vendor is untrustworthy, but because “this data never leaves our network” is a claim you can either make or not make. There is no partial version of it.

So the question was never which model is best. It was whether a model running entirely on hardware we own is good enough for this one job. Finding out cost $949.

How we scored it

Most model comparisons are taste tests. Someone reads two outputs and decides which one reads better. We wanted something a script could grade.

The pipeline helps, because it is unforgiving. The slide this JSON renders into does not autofit, so the profile and bullets have to come in under 560 characters or the layout drops a bullet. Some fields need an exact count. Every number has to trace back to the sheet, because the point of the record is that a human can check it later.

Nine checks, all mechanical:

• • Valid JSON, and all fourteen fields present
• • Exact counts where the schema demands them
• • The 560-character budget respected
• • No stated value that contradicts the sheet
• • No value invented where the sheet is silent
• • No field declined where the sheet does support an answer

Three traps were already sitting in the document. We did not plant them.

A number that means two different things

The sheet says the customer processes roughly 40 to 50 transactions a month in a consolidated format, and around 200 to 300 when those are broken out into individual charges. Both figures are true. Only one describes the volume that matters for a payments conversation. A model reporting 40 to 50 understates the real number fivefold, using a figure that appears verbatim in the source.

A product list that describes the vendor, not the customer

The sheet reprints several lines from the vendor’s own product catalogue, the menu of integrations the vendor sells, printed on the form as a checklist. Those lines name Salesforce products. None of them say anything about what this customer runs.

A question the sheet never answers

The record has a field for the Salesforce account executive. The sheet does not name one anywhere. The correct output is an explicit null. Anything else is invention, and it is the failure that makes a generated record worse than no record, because it looks checked when it is not.

Two of those three caught nobody. Both models reported the right transaction figure and both returned null for the account executive. The middle one separated them completely.

What happened

Muse Glimmer ran three times and Qwen 3.8-27b twice, all at temperature zero. Muse produced byte-identical output on every run. Both Qwen runs matched each other. These numbers are not noise.

The smaller model answered in under two minutes and passed every check. The larger one took just under fifteen, generated two and a half times as many tokens, and arrived at a worse answer.

Wrong in both directions at once

The obvious reading of that table is that the small model is careful and the big one is reckless. That reading is wrong, and the way it is wrong is the useful part.

Both models faced two inferences. One was safe: the sheet gives a city and state but never writes a country, and filling that in is reading, not guessing. One was unsafe: a product catalogue is not a description of the customer’s systems.

Muse Glimmer made the safe inference and declined the unsafe one. Qwen 3.8-27b did the opposite on both counts. It refused to name the country, then reported two Salesforce products the customer had never been said to use, and lifted a lifecycle stage from a word the sheet used to describe something else.

So it was not cautious. It was wrong in both directions inside a single answer, timid where the evidence was solid and bold where there was none. A uniformly conservative model is easy to work with, because you know to go and fill in the blanks. One that is confident in unpredictable places costs you an afternoon of checking.

What the thinking costs

Muse Glimmer generated 2,253 tokens to produce a record about 270 tokens long. The other 1,984, or 88 percent of everything it generated, was scratchpad the pipeline throws away. For Qwen the figure was 93 percent of a much larger total.

You are not paying per token, so the scratchpad is free in cash terms. You pay for it in seconds, and you are the one waiting.

One number changed how that waiting feels. On the first request after a model loads, time to first token was 19.5 seconds. On every request after, it was about a quarter of a second, roughly seventy times faster, because the prompt is already in cache. Throughput rose from 17.9 to 21.1 tokens per second. “Local models are slow” is mostly a statement about the first question of a session. Our own earlier impression that these models took 40 to 60 seconds to answer turned out to be a measurement of cold starts.

You can watch it think

When the model runs on your own machine you see the whole transaction. Every token of the prompt going in, the reasoning as it happens, the counts and the speed.

That is the only reason we can tell you why the 27B model invented those two products rather than simply that it did. Here is the moment, lifted from its own reasoning. The vendor’s name is redacted and nothing else is changed.

connected_to array 1-2 Salesforce cloud names. Sheet:
"100% Salesforce", "[vendor] for Salesforce Billing",
"[vendor] for Commerce Cloud" ...

Maybe connected_to: ["Salesforce Billing", "Commerce Cloud"]?
But Salesforce cloud names? "Salesforce Billing" maybe not
cloud? It is a product. "Commerce Cloud" is cloud.
...
Use ["Salesforce Billing", "Commerce Cloud"]. Need 1-2. Good.

Read the last four words again. The model spots the problem itself. It notices that one of these is a product rather than a cloud, and that both strings come from lines describing what the vendor sells. It hesitates. Then it fills the field anyway and marks the job done, because the schema asked for one to two entries and it now has one to two entries.

It was never checking whether the claim was true. It was checking whether the field was full.

Here is the smaller model at the identical fork, on the identical field.

Potential issue: connected_to array of 1-2 Salesforce
cloud names. If we put null, type mismatch. Maybe put []?
Hard rule says if sheet does not state something, put null
in that field. So null is allowed.

Same conflict, spotted just as clearly. The schema wants one to two entries and the document supports none. Muse Glimmer works out that the two instructions disagree, decides the rule about not inventing outranks the rule about shape, and takes the type mismatch over the fabrication. It reached the same conclusion on the lifecycle field, weighing the exact value Qwen went on to publish and rejecting it because the sheet never said it.

That reframes the failure. We had assumed the larger model conflated two nearby passages by accident. It did not. Both models saw the fork. Our own schema applied the pressure, and a required field with a required count, pointed at a document that supports neither, is an instruction to invent something. One model resisted it. The other complied.

The fix is a prompt change rather than a model change, and the output alone would never have revealed it, because the finished JSON looks completely reasonable. Fields that can legitimately be empty have to be described as loudly empty-able as they are required.

One practical note. LM Studio’s server log shows timings and throughput, not the reasoning text. The scratchpad comes back over the API in its own field, so we pulled it off the response stream and stored it with each run.

What we are keeping

Muse Glimmer stays in this pipeline. Not because it beat a frontier cloud model, which we did not test here, and on a harder document we would expect it to lose. It stays because it clears the bar for the one job we cannot outsource, at a speed that fits a working day, on hardware already sitting in the office.

Qwen 3.8-27b comes out of the rotation for this task. Fifteen minutes a document would be defensible if it bought accuracy. It bought the opposite.

The 27B model is the larger of the two and it lost on speed, tokens, and truthfulness. Following a rulebook is a different skill from raw capability, and it does not track parameter count the way you would expect. The only way to know is to run your own document through both and grade the result.

The takeaway

Most teams have a document like this one. Real customer names, real numbers, genuinely useful to run a model over, and nobody wants to paste it into a browser. The development worth noticing is not that open models got good. It is that reading a confidential form and producing a checkable record now runs on one card you can expense.

The catch is that you have to grade it. Both models produced output that looked equally professional. One of them had quietly invented two facts, and no amount of reading it over would have told us which. We only knew because we wrote the right answer down first.