WEIGHTS-BIASES

W&B ML experiment tracking + LLM Weave platform.

Definition

W&B products: (1) Experiments: tracking ML/LLM training runs + hyperparameters + metrics + system metrics (GPU + CPU + memory) + artifacts, web UI rich visualizations (line charts + scatter plots + parallel coordinates + tables + image + audio + video + 3D models + HuggingFace text), wandb.init + wandb.log Python SDK. (2) Sweeps: hyperparameter optimization, grid search + random search + Bayesian Optimization, agents distribute across machines. (3) Reports: collaborative reports embed runs + plots + Markdown narrative, shareable links public/private. (4) Artifacts: version datasets + models + code + checkpoints, lineage tracking. (5) Model Registry: model versions + stages (Staging + Production) + aliases, replace MLflow Model Registry for W&B users. (6) Tables: interactive tables explore eval results + predictions vs ground truth, drill-down. (7) Weave (launched 2024): LLM observability platform alongside Experiments, traces LLM calls (OpenAI + Anthropic + Cohere + custom) + evaluations + datasets, similar Phoenix Arize + LangSmith competitor. (8) Launch: remote compute orchestration (run experiments on AWS/GCP/Azure GPUs from local). Customers: OpenAI (used internally pre-training models GPT series), Anthropic (Claude), Cohere, Stability AI, Hugging Face, NVIDIA, Toyota, BMW.

Origin

W&B founded 2017 by Lukas Biewald (founder CrowdFlower, ex-Yahoo + Powerset) + Chris Van Pelt + Shawn Lewis ; Series C $135M 2021 + Series D $135M 2022 + Series E $200M 2024 (Coatue, $1B+ valuation) ; ~1M+ users 2024.

Example in context

OpenAI internal team uses W&B during GPT-4 pre-training: massive training runs ~25K NVIDIA A100 GPUs months duration, W&B tracks training loss + perplexity + downstream task metrics + system metrics (TFLOPS utilization + temperature + memory), Reports collaborative shared engineering + research teams.

Related terms

• MLflow — OSS alternative.