The first versions of glide-mq were not an attempt to make “AI infrastructure.” I needed a fast queue, wanted it on Valkey GLIDE, and cared about a cleaner set of mechanics than the alternatives gave me.
Then I used it underneath more agent systems. The queue continued doing its part. The ugly code accumulated around it.
One service tracked model spend. Another carried token streams. Workers had special rules for calls that looked stalled but were still reasoning. Human approval lived in a separate state machine. Each workaround was understandable by itself. Together they were evidence that the queue's model of work no longer matched the workload.
The queue understood the wrong shape
Traditional background work is usually discrete. A job begins, produces one result, and ends. Its cost is mostly machine time. If it stops renewing a lock, retrying is often reasonable.
An AI job can be healthy while producing no visible output for ninety seconds. It can emit thousands of ordered fragments before it finishes. Its important limit may be tokens per minute rather than jobs per second. It may have to stop for a person, a CI run, or an external signal. A technically successful loop can still be a financial and semantic failure.
Putting those facts into side systems did not make the queue simpler. It made the complete system harder to reason about.
Lifetime belongs to the job
A global lock duration assumes the jobs in one queue share a useful definition of “too long.” A classifier and a deep research step clearly do not.
The fix was to make that expectation part of each job:
await queue.add(
"classify",
{ text },
{ lockDuration: 10_000 },
);
await queue.add(
"research",
{ topic },
{ lockDuration: 180_000 },
);This is a small API change with a large effect: the system no longer forces unrelated work to inherit the same failure boundary.
Budget checks must happen before the next spend
Dashboards are good at explaining what already happened. They are too late to prevent a useless agent loop from taking another expensive step.
The queue is a better enforcement point because every transition already passes through it. When a worker reports usage, Valkey can update the flow total atomically and refuse to release the next job if the budget has been exhausted.
await flowProducer.add(pipeline, {
budget: {
maxTotalTokens: 50_000,
maxTotalCost: 2.00,
tokenWeights: { reasoning: 2 },
onExceeded: "fail",
},
});The useful property is not the shape of this object. It is where the decision occurs: in the execution path, while the system can still stop.
Observability can tell you that a flow wasted money. Control has to stop it before the next request.
The stream is job state
When a model returns output incrementally, that output belongs to the job. Sending it through a separate Pub/Sub or WebSocket path creates two histories that must somehow remain ordered and reconnectable.
glide-mq keeps chunks attached to the job instead:
const worker = new Worker("ai", async (job) => {
for await (const token of generate(job.data.prompt)) {
await job.streamChunk("token", token);
}
await job.streamChunk("done");
return { completed: true };
}, { connection });
const chunks = await queue.readStream(jobId, {
block: 5_000,
});The client can disconnect and resume reading. The durable record and the live view do not have to be reconciled later because they were never split.
Waiting is not failure
Real workflows pause. A person may need to approve an action. CI may need to finish. A rate limit may need to reset. Treating each pause as an exception produces retries, polling, and duplicated state.
That is why the queue now has explicit suspend and signal operations, model fallbacks, token-aware throttles, and flow-level accounting. They are not “AI integrations.” They describe the lifecycle of the work the queue is already responsible for.
The larger lesson
I do not think queues are uniquely wrong. AI workloads are stress-testing many abstractions designed for short requests, predictable costs, and one final result.
When the abstraction and workload disagree, the missing behavior reappears as glue: a budget service, a streaming side channel, a heartbeat exception, another reconciliation loop. Enough glue is not proof that the architecture is flexible. Sometimes it is proof that the core model is incomplete.
glide-mq is still a queue. It simply understands more of the work passing through it now.