About
Promptfoo is an open-source CLI for evaluating and red-teaming LLM applications. It runs systematic test suites that compare prompts, models, and RAG/agent configurations side by side, checks outputs with assertions or model grading, and scans for security and safety vulnerabilities. It is best for teams shipping LLM features who want repeatable quality checks wired into CI instead of eyeballing outputs.
# Global install (npm)
npm install -g promptfoo
# Scaffold a starter eval, run it, and view results
promptfoo init --example getting-started
promptfoo eval
promptfoo viewCopy individual commands
- Global install (npm):
npm install -g promptfoo - Scaffold a starter eval, run it, and view results:
promptfoo init --example getting-started promptfoo evalpromptfoo view
Also available via brew install promptfoo, pip install promptfoo, or ad hoc with npx promptfoo@latest. You will need an API key for the model provider under test (e.g. OPENAI_API_KEY).
More in AI Tools
View all
PocketBase
An open-source realtime backend in a single file — SQLite, auth, file storage, realtime, admin dashboard.
NocoDB
A free, self-hostable Airtable alternative — turn any database into a smart spreadsheet with APIs and automations.
Gitea
A painless self-hosted Git service in a single Go binary — repos, PRs, issues, wiki, registry and CI/CD.