Moniotor/Logging/K8S #
| Name | Doc | Github | Note | Next |
|---|---|---|---|---|
| OpenTelemetry | Docs | Github | OpenTelemetry is a collection of APIs, SDKs, and tools for generating, collecting, and exporting telemetry data (traces, metrics, and logs). | Getting Started with OpenTelemetry |
| Prometheus | Docs | Github | Prometheus is a systems and service monitoring system. | |
| Thanos | Docs | Github | Thanos is a set of components that can be composed into a highly available Prometheus setup. | |
| Zabbix | Docs | Github | Zabbix is an enterprise-class open-source monitoring solution for tracking the status of various network services, servers, and other network hardware. | |
| Grafana | Docs | Github | Grafana is an open source, feature rich metrics dashboard and graph editor for Graphite, InfluxDB, Prometheus and other time series databases. | |
| Loki | Docs | Github | Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. | |
| containerd | Docs | Github | containerd is an industry-standard container runtime with an emphasis on simplicity, robustness, and portability. It manages the complete container lifecycle, including image transfer, storage, container execution, networking, and low-level runtime integration through OCI runtimes such as runc. It is the default container runtime used by Kubernetes (via CRI). | Need Real Install |
| runc | Docs | Github | runc is a lightweight, portable, OCI-compliant container runtime. It is responsible for creating and running containers by directly interacting with Linux kernel features such as namespaces and cgroups. runc is the default low-level runtime used by containerd and Kubernetes environments. | Need Real Install |
| Docs | Github | |||
| Docs | Github | |||
| Docs | Github | |||
| Docs | Github | |||
| Docs | Github | |||
| Docs | Github |
MLOps #
| Name | Doc | Github | Note |
|---|---|---|---|
| mlflow | Doc | Github | 侧重模型实验追踪 (Tracking) 和生命周期管理 |
| MLRun | Doc | Github | 编排式的 MLOps 框架，强于实时数据处理与自动化 |
| Airflow | Doc | Github | 通用的工作流调度平台，基于 Python DAG |
| Argo | Doc | Github | 云原生 (K8s) 工作流引擎，适合 CI/CD 和数据流水线 |
| Kubeflow | Doc | Github | 基于 K8s 的端到端机器学习平台，生态完整但相对复杂 |
| Databricks | Doc | Github | 统一数据分析平台，由 Spark 创始团队打造，深度集成 MLflow |
| DVC | Doc | Github | 机器学习版本控制工具，类似“数据界的 Git” |
| BentoML | Doc | Github | 专注于模型打包、部署与高性能模型服务的框架 |
| Ray | Doc | Github | 分布式计算框架，常用于大规模模型训练和在线推理 |
https://mlops.swiss-ai-center.ch/syllabus/
https://mlops.swiss-ai-center.ch/tools/
https://ml-ops.org/content/end-to-end-ml-workflow
https://mlops-guide.github.io/
https://www.systemdesigner.net/ml-systems/ml-fundamentals
GPU资源调度与性能优化 #
Kubernetes GPU资源调度优化：基于vGPU虚拟化技术实现算力利用率倍增
Reference #
Azure MLOps (v2) Solution Accelerator