nemo-evaluator-sdk
Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.
Install
npx skills add davila7/claude-code-templates@nemo-evaluator-sdkRuns in your terminal. Adds the skill globally for Claude Code, Cursor, Codex and others; add -g -y to skip prompts.
What it does
NeMo Evaluator SDK - Enterprise LLM Benchmarking Quick Start NeMo Evaluator SDK evaluates LLMs across 100+ benchmarks from 18+ harnesses using containerized, reproducible evaluation with multi-backend execution (local Docker, Slurm HPC, Lepton cloud). Installation: Set API key and run evaluation: View available tasks: Common Workflows Workflow 1: Evaluate Model on Standard Benchmarks Run core academic benchmarks (MMLU, GSM8K, IFEval) on any OpenAI-compatible endpoint. Checklist: Step 1: Configure API endpoint For self-hosted endpoints (vLLM, TRT-LLM): Step 2: Select benchmarks Add tasks to…
Excerpt from the skill's own SKILL.md. Read the full file on GitHub before installing: skills run with your agent's permissions.
View source on GitHubBefore you install
Skills are plain text instructions the agent follows, sometimes with scripts. Check the source, prefer repositories with many installs and stars, and read any script it ships.
Categories
Related skills
- jira
davila7/claude-code-templates
Use when the user mentions Jira issues (e.g., "PROJ-123"), asks about tickets, wants to create/view/update issues, check sprint status, or manage their Jira workflow. Triggers on keywords like "jira", "issue", "ticket", "sprint", "backlog", or issue key patterns.
1.1K installs 31KAI & agentsProductivity - conversation-memory
davila7/claude-code-templates
Persistent memory systems for LLM conversations including short-term, long-term, and entity-based memory Use when: conversation memory, remember, memory persistence, long-term memory, chat history.
1.0K installs 31KAI & agents - crewai-multi-agent
davila7/claude-code-templates
Multi-agent orchestration framework for autonomous AI collaboration. Use when building teams of specialized agents working together on complex tasks, when you need role-based agent collaboration with memory, or for production workflows requiring sequential/hierarchical execution. Built without LangChain dependencies for lean, fast execution.
996 installs 31KAI & agents - langchain
davila7/claude-code-templates
Framework for building LLM-powered applications with agents, chains, and RAG. Supports multiple providers (OpenAI, Anthropic, Google), 500+ integrations, ReAct agents, tool calling, memory management, and vector store retrieval. Use for building chatbots, question-answering systems, autonomous agents, or RAG applications. Best for rapid prototyping and production deployments.
976 installs 31KReact & Next.jsAI & agentsGit & GitHub - modal-serverless-gpu
davila7/claude-code-templates
Serverless GPU cloud platform for running ML workloads. Use when you need on-demand GPU access without infrastructure management, deploying ML models as APIs, or running batch jobs with automatic scaling.
972 installs 31KDeploy & DevOpsDataPython - context7-auto-research
davila7/claude-code-templates
Automatically fetch latest library/framework documentation for Claude Code via Context7 API
736 installs 31KReact & Next.jsDocumentationResearch