A full-stack LLM development toolkit from NVIDIA covering synthetic data generation, multi-backend inference, model training, and 11-category benchmark evaluation, scaling from single GPU to tens-of-thousands-GPU Slurm clusters.
Nemo Skills (CLI command ns) is a research infrastructure for the full LLM lifecycle, open-sourced by the NVIDIA-NeMo organization under Apache-2.0 (Python ≥ 3.10). Centered on a "synthetic data generation → model training → multi-dimensional evaluation" loop, it has produced notable datasets and models including OpenMathInstruct-2 (14M pairs), OpenMathReasoning, OpenCodeReasoning, and Nemotron-Math-v2.
The inference layer supports four backends—TensorRT-LLM, vLLM, sglang, and Megatron—and unifies OpenAI / NVIDIA NIM API providers via LiteLLM, enabling single-config switching from local single-GPU to tens-of-thousands-GPU Slurm clusters. The evaluation layer covers 11 benchmark categories: math (natural language / formal), code, science knowledge, instruction following, long context, tool calling, multilingual, speech/audio, VLM, and robustness/speculative decoding/external evaluation—supporting self-hosted LLM-as-a-Judge and cross-Slurm-job parallel evaluation. The training layer integrates NeMo-RL and verl frameworks. Additional built-in capabilities include agentic inference (parallel thinking, ToolManager, DirectPythonTool, MCP protocol), data decontamination, and LibTrace pipelines.
Installation follows a layered design: full install includes all features; the core/ sub-package retains only inference and evaluation (suitable for non-cluster environments); the tools/ sub-package provides only the tool runtime. The CLI is built on Typer, prompt templates are managed via YAML configuration, and cluster configurations are unified across local, containerized, and Slurm modes through ns setup + cluster_configs/. The project is stated to be strictly for research purposes and not an official NVIDIA product.