Orchestra-Research/AI-Research-SKILLs
serving-llms-vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
Before you install
- Last update
- Files in folder
- 5
- Scripts
- No script files found
- SKILL.md size
- 8.8 KB (about 2,257 tokens of context each time it loads)
- License
- MIT
- Source on GitHub
- Orchestra-Research/AI-Research-SKILLs
Skills are plain text instructions the agent follows, sometimes with scripts. Check the source, prefer repositories with many installs and stars, and read any script it ships.
Install
npx skills add Orchestra-Research/AI-Research-SKILLs@serving-llms-vllmRuns in a terminal with Node.js 18 or newer. Add -y to skip the prompts and -g to install for every project instead of this one.
Install for your agent
npx skills add Orchestra-Research/AI-Research-SKILLs@serving-llms-vllm --agent claude-code -y- Project install writes to
.claude/skills/serving-llms-vllm- Global install writes to
~/.claude/skills/serving-llms-vllm
Start a new session afterwards so the agent picks the skill up. In Claude Code you can also call it directly with /serving-llms-vllm.
What it does
vLLM - High-Performance LLM Serving Quick start vLLM achieves 24x higher throughput than standard transformers through PagedAttention (block-based KV cache) and continuous batching (mixing prefill/decode requests). Installation: Basic offline inference: OpenAI-compatible server: Common workflows Workflow 1: Production API deployment Copy this checklist and track progress: Step 1: Configure server settings Choose configuration based on your model size: Step 2: Test with limited traffic Run load test before production: Verify TTFT (time to first token) 100 req/sec. Step 3: Enable monitoring…
Excerpt from the skill's own SKILL.md. Read the full file on GitHub before installing: skills run with your agent's permissions.
Categories
Related skills
- faiss
Orchestra-Research/AI-Research-SKILLs
Facebook's library for efficient similarity search and clustering of dense vectors. Supports billions of vectors, GPU acceleration, and various index types (Flat, IVF, HNSW). Use for fast k-NN search, large-scale vector retrieval, or when you need pure similarity search without metadata. Best for high-performance applications.
- mlflow
Orchestra-Research/AI-Research-SKILLs
Track ML experiments, manage model registry with versioning, deploy models to production, and reproduce experiments with MLflow - framework-agnostic ML lifecycle platform
- tensorrt-llm
Orchestra-Research/AI-Research-SKILLs
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
- peft-fine-tuning
Orchestra-Research/AI-Research-SKILLs
Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.
958 installs 14K - autoresearch
Orchestra-Research/AI-Research-SKILLs
Orchestrates end-to-end autonomous AI research projects using a two-loop architecture. The inner loop runs rapid experiment iterations with clear optimization targets. The outer loop synthesizes results, identifies patterns, and steers research direction. Routes to domain-specific skills for execution, supports continuous agent operation via Claude Code /loop and OpenClaw heartbeat, and produces research presentations and papers. Use when starting a research project, running autonomous experiments, or managing a multi-hypothesis research effort.
- ray-train
Orchestra-Research/AI-Research-SKILLs
Distributed training orchestration across clusters. Scales PyTorch/TensorFlow/HuggingFace from laptop to 1000s of nodes. Built-in hyperparameter tuning with Ray Tune, fault tolerance, elastic scaling. Use when training massive models across multiple machines or running distributed hyperparameter sweeps.