Annotation

Priya Desai

@priya_edge

The floor is the tell. If 8.1 GB was actually usable they'd publish those decode speeds instead of marking it as below floor and paging, but the marketing pitch needs a low number so someone on a base Mac clicks through before realizing the inference cost sits between zero utility and watching their machine swap. They're pricing disk space for attention before they price RAM for performance.

The model is 104 GB at 4-bit; slotstream streams it from SSD and runs it in whatever memory you give it, down to an 8.1 GB planned floor. One Swift binary with the commonly used Ollama and OpenAI chat/generate endpoints.
GitHub - carloslfu/slotstream: Run Qwen3.8-Flash-Next (125B MoE, 104 GB at 4-bit) on Macs with a fraction of that RAM by streaming experts from SSD. MLX + Swift, Ollama-compatible API.github.com
2
2

Replies 2

Ranked by votes, then newest

Sign in to reply.

  • Nina Hart@nina_scroll·Sep 2

    pricing disk space for attention is perfect because the pitch is always we can run it not we ran it and timed what happens when your M1 Air spends four seconds per token loading expert shards off a drive that was mid tier in 2021. that's not even vaporware, that's calling technically boots a feature set when the demo was always shot at 64GB and now the floor is a substitute for anyone saying here's the chart at 16.

    0
    Milo Grant@milo_scribble·Sep 3

    lol yeah technically boots when the actual move is we needed a floor that rhymes with accessibility so suddenly 8.1 GB becomes the number you print instead of admitting four seconds per token is a patience benchmark, not a product. You don't write planned floor unless the whole point was someone screenshots the repo before they run it and by then the star count did its job, which beats saying the Ollama compatibility was always aimed at the README not the task.

    1

Fair use dispute

Believe this breaches fair use? File a claim for review.