MLFLOW-LLM

MLflow LLM tracking + eval + serving Databricks.

Définition

MLflow LLM features : (1) MLflow Tracking LLM : log LLM runs with prompts + responses + parameters (temperature + max_tokens + top_p) + metrics (latency + token counts + cost estimate), MLflow Trace API (auto-instrument LangChain + LlamaIndex + OpenAI + Anthropic clients) for distributed tracing. (2) MLflow Models LLM : package LLM models for deployment (mlflow.openai.log_model, mlflow.transformers.log_model HuggingFace, mlflow.langchain.log_model entire chain, etc.), serve via mlflow models serve or deploy Databricks Model Serving. (3) MLflow Evaluate : evaluate LLM applications via mlflow.evaluate(data, model_type='question-answering') with metrics : exact_match, token_count, ari_grade_level (readability), flesch_kincaid_grade_level, toxicity (HuggingFace toxicity model), perplexity, custom metrics support via make_metric. (4) MLflow Deployments Gateway : unified API gateway access LLM providers (OpenAI + Azure OpenAI + Anthropic + Cohere + AI21 + AWS Bedrock + GCP Vertex AI + custom), token tracking + rate limiting + caching. (5) MLflow Prompt Engineering UI : Databricks Notebook UI for iterative prompt experimentation, side-by-side prompts comparison. (6) Integration : Databricks Mosaic AI (ML platform), AzureML (similar via Microsoft), AWS SageMaker (MLflow Open Source compatible).

Origine

MLflow initial release juin 2018 par Databricks (Matei Zaharia + Spark team) ; MLflow LLM extension 2.4 mai 2023 ; MLflow 2.10 fevrier 2024 LLM enhancements ; ~17000 GitHub stars 2024.

Exemple en contexte

Databricks customer enterprise AI engineering team uses MLflow LLM Tracking + Evaluate : prompts iterated in Prompt Engineering UI, evaluations run via mlflow.evaluate against eval dataset (1000+ QA examples), tracks 50+ experiments versions prompts + models (GPT-4 vs Claude vs Llama 70B fine-tuned), best version deployed via Databricks Model Serving + MLflow Deployments Gateway.

Termes liés

• Weights & Biases — alternative ML tracking.