Skip to content

Kubeflow + MLFlow

tl; dr; A combinator stack that provides Kubeflow and MLFlow.

Introduction

Kubeflow is an open-source MLOps platform that combines Jupyter hosting, ML pipelining, and hyperparameter tuning. It is packaged into a single UI to help data scientists train their ML models.

Kubeflow Pipelines (KFP) in particular, has emerged as one eminent ML pipelinging technology, mainly thanks to the managed hosting in various clouds.

Its opinionated ML-specific API helps data scientists and ML engineers develop robust, repeatable pipelines.

This stack adds MLflow for model management and makes it easy to log models to MLflow from kubeflow notebooks and pipelines.

Kubeflow Version

This installation uses Kubeflow version 1.2, which is now out of date.

Status and Recommendations

⚠ For Testing Only ⚠

This installation method is not recommended for use. It required a lot of work-arounds that are not suitable for production use. Please refer to the official documentation for production installation instructions.

Test Drive

The fastest way to get started is to use the test drive functionality provided by TestFaster. Click on the "Launch Test Drive" button below (opens a new window).

💻 Launch Test Drive 💻

Usage

Prerequisites

Start by preparing your Kubernetes cluster using one of the infrastructure components or use your own cluster.

Component Usage

module "kubeflow_mlflow_stack" {
  source  = "combinator-ml/stack-kubeflow-mlflow/k8s"
  # Optional settings go here
}

See the full configuration options below.

Instructions

Kubeflow is big, so it can take some time to start. Once it does connect to the istio ingress gateway service.

Once you see the login screen, the username is admin@kubeflow.org and the password is 12341234.

Requirements

Name Version
terraform >= 0.13
helm >= 2.2.0
k8s >= 0.9.1
kubernetes >= 2.3.2

Providers

Name Version
k8s >= 0.9.1
kubernetes >= 2.3.2
null n/a
time n/a

Modules

Name Source Version
kubeflow combinator-ml/kubeflow/k8s 0.0.2
mlflow combinator-ml/mlflow/k8s 0.0.3

Resources

Name
k8s_manifest
kubernetes_cluster_role_binding
null_resource
time_sleep

Inputs

Name Description Type Default Required
mlflow_namespace (Optional) The namespace to install into. string "mlflow" no

Outputs

No output.