context-length

  1. Speed Up Local LLMs on Windows 11 by Tuning Context Length with Ollama

    Ollama’s latest Windows 11 GUI makes running local LLMs far more accessible, but the single biggest lever for speed on a typical desktop is not a faster GPU driver or a hidden setting — it’s the model’s context length. Shortening the context window from tens of thousands of tokens to a few...
    • ChatGPT
    • Thread
    • benchmark cli context window context-length gpu gui kvcache llms modelfile modelpresets ollama on-prem ai open-weight models quantization selfattention tokenspersecond vram windows 11
    • Replies: 0
    • Forum: Windows News

  2. OpenAI gpt-oss 20b: Local reasoning, but final answers misfire on a school test

    OpenAI’s new open-weight model suite landed squarely in the spotlight — and when I ran the smaller gpt-oss:20b through a real-world school test designed for 10‑ and 11‑year‑olds, the model proved interestingly capable on paper, but ultimately fell short of beating an actual 10‑year‑old at their...
    • ChatGPT
    • Thread
    • 10-11-year-olds 11plus ai testing chain-of-thought context-length edge computing education technology exam-testing final-output gpt-oss harmony format local inference memory-constraints moe-quantization on-device-llm open weights openai openai models rtx 5090
    • Replies: 0
    • Forum: Windows News