A laptop screen displays a futuristic panel of AI robots discussing ethics, safety, and governance.
Andrej Karpathy gave us the phrase "vibe coding." He also published a small open-source project that turns one chatbot question into a panel discussion among several AI models. MakeUseOf's Jayric Maning ran a more developed descendant of that project and wrote it up under a headline that says Karpathy "never trusts a single AI." That headline overreaches, and the setup is worth trying, but it needs a few caveats before you run it.

A laptop screen displays a futuristic panel of AI robots discussing ethics, safety, and governance. What Karpathy actually published​

The headline is an interpretation. Karpathy's repository doesn't say how often he uses a council, and it doesn't say he distrusts single-model answers. MakeUseOf says it tested Karpathy's original LLM Council project, and the repository README is more modest than the headline. It describes the project as a simple local web app that looks like a chat interface. The README says it was "99% vibe coded" as a Saturday hack, and Karpathy states he won't support or improve it. He offers it as inspiration only.

The README describes three stages:

  1. First opinions. Every model in the council gets the query on its own, and the answers appear in a tab view.
  2. Review. Each model sees the others' answers with the model identities hidden. It ranks them for accuracy and insight. The anonymity is there so a model can't play favorites.
  3. Final response. A designated chairman model compiles the answers and rankings into one reply.

Why MakeUseOf used The AI Counsel instead​

MakeUseOf's author calls The AI Counsel by Jacob Ben-David the best of the several Karpathy-derived projects they tried. It's MIT-licensed and public. It adds more than the original:

  • Council mode. This is the original three-stage flow, with execution modes. Chat Only runs stage 1. Chat + Ranking runs stages 1 and 2. Full Deliberation runs all three and is the default.
  • Iterative debate. The README describes council mode running multiple rounds, from one to five, with critique modes and early stopping when the models converge.
  • Advisors mode. Named personas such as The Skeptic, The Strategist and The Ethicist debate a question over configurable rounds. They then deliver a structured verdict and action plan.
  • Integrations. Besides the web interface, the project ships an MCP server for AI coding tools, which the MakeUseOf piece also mentions.

The README gives a rule of thumb for choosing a mode. Use Council for direct answers, factual questions and "give me the best response" synthesis. Use Advisors when the question involves real tradeoffs, risk, strategy, ethics or a decision. MakeUseOf's test case was an ethics-and-law dilemma, so Advisors mode may suit that kind of question even better. The author only describes running the council flow.

The setup, and where Windows readers should be careful​

The repository documents this sequence:

Code:
git clone [GitHub - jacob-bd/the-ai-counsel: The AI Counsel — Collective AI Intelligence through multi-model deliberation and persona-driven debate · GitHub](https://github.com/jacob-bd/the-ai-counsel.git)
cd the-ai-counsel
uv sync
npm install --prefix frontend
./start.sh

The README lists the prerequisites as Python 3.10+, Node.js 18+, and uv. MakeUseOf mentions Git, uv and Node.js but leaves out the Python version. Once the script runs, open [url]http://localhost:5173[/url]. Then configure your API keys (or subscription OAuth logins) in Settings.

The Windows catch: ./start.sh is a shell script. MakeUseOf tells readers to open a terminal from the Windows right-click menu and paste the commands. The repository's instructions don't establish that a native PowerShell or Command Prompt window can run that script. I found no Windows-specific launcher in the main repository. A Hebrew-language fork does list .bat and .command launchers, but that is a separate project, and I haven't verified it.

If ./start.sh fails in your shell, there are two reasonable routes:

  • Use a Unix-like environment. Git Bash or WSL are the usual options. This is general industry practice, not something the project documents.
  • Run the two servers by hand. The README's own manual option uses two terminals. In the first, run uv run python -m backend.main. In the second, change to the frontend folder and run npm run dev. I haven't tested this on Windows either.

How you know it worked: both servers start, and [url]http://localhost:5173[/url] loads the interface.

"Free" covers the software, not the model calls​

MakeUseOf says free OpenRouter models worked in its test, so readers can try the council before paying. That is the author's experience on one day. It doesn't establish that any particular model is free now, reliable, or fit for every stage. Provider terms and availability change.

Karpathy's README is blunter than the article. It tells users to buy OpenRouter credits or set up automatic top-up. The code is free to copy, but hosted model calls may carry usage limits or charges. Three or four models answering, ranking each other and being synthesized adds up quickly.

MakeUseOf reports that two single-pass questions used about 710,000 tokens. Treat that as one author's anecdote. The article doesn't name the models, prompt sizes or pricing, so it's not a benchmark. The AI Counsel's README does describe run-cost reporting, which includes input/output token splits and per-model breakdowns. If you try this, use that reporting to measure your own bill instead of guessing.

Privacy and credentials​

A "local app" only means the interface and backend run on your machine. Prompts still go to whichever hosted provider you configure, so don't paste confidential code, customer data or internal documents into it unless you've checked that provider's data handling. The project stores conversations locally in a data/ directory.

Credentials need care too. In file-storage mode, API keys and OAuth tokens are held as plaintext in data/credentials.json, with restricted file permissions. Practical steps:

  • Don't commit the data/ directory to Git.
  • Don't put it in a synced folder or an unencrypted backup you'd hesitate to share.
  • Rotate any key you think has been exposed.
  • Be careful on shared PCs.

The start script also exposes the frontend and backend on your network, with a YOUR_IP:5173 address alongside localhost. On an office or public network, check that you're comfortable with that before leaving it running.

What a council can and can't tell you​

In MakeUseOf's test, three models answered independently. Two refused to add a patrol-alert flag that identified likely protest organizers. A third leaned toward a limited alert. The chairman recommended declining in writing, documenting the reasoning and escalating to oversight. The scenario and the ~80 percent accuracy figure are the author's invented prompt, and this is one author's run, not independent evidence.

The useful part was the visible dissent. Reading the minority answer showed the tradeoff the other two had rejected. MakeUseOf's author admits the main weakness: similar models tend to give similar answers, so a council of near-clones is a single opinion with extra cost. Models can also share training data and blind spots. An anonymous ranking can produce confident consensus without making the answer correct.

So don't treat the chairman's verdict as verification. Its value is in the surfaced disagreement, which you can then check against real expertise and your own judgment. On questions with legal, security or compliance consequences, accountability still sits with a person.

Who should bother​

MakeUseOf's author says not to use it for daily prompts, citing cost, latency and setup overhead. I agree. The cases that fit are high-stakes ones, such as:

  • security design reviews
  • architecture tradeoffs
  • ambiguous-policy questions where one confident chatbot answer would worry you

For everything else, one chat window is faster and cheaper.

Bottom line: The idea is sound and the code is open, but the headline claims more than the sources support. Budget for model calls. Keep the plaintext credential file out of version control. Expect to troubleshoot start.sh on native Windows. Judge the council by whether its disagreements teach you something, not by how authoritative the final answer sounds.

 

References

  1. The guy who coined vibe coding never trusts a single AI, and you can copy his setup for free MakeUseOf 2026-11-20T11:30:17+00:00
  2. GitHub - jacob-bd/the-ai-counsel: The AI Counsel — Collective AI Intelligence through multi-model deliberation and persona-driven debate · GitHub github.com
  3. GitHub - karpathy/llm-council: LLM Council works together to answer your hardest questions · GitHub github.com