Cloud and GPU Providers

Overview

Data center operators, regional and sovereign clouds, and GPU clouds sell compute to many customers. Waldur is the part their customers deal with: a catalog with price plans, ordering, organizations and projects, metering and monthly invoices. Underneath, the provider keeps the platforms it already runs, such as OpenStack, VMware, OpenNebula, Kubernetes or Slurm.

  • A catalog with price plans: fixed monthly, usage-based, one-time and prepaid terms, chosen per plan, with discounts and credits.
  • GPU capacity metered by type: GPU-hours on Slurm, GPU and vGPU allocations on OpenStack, GPU quotas on Kubernetes, each with its own price.
  • LLM access sold by the token: an API key per customer on a LiteLLM or Envoy AI Gateway proxy, with budgets, rate limits and token usage metered for billing.
  • Organizations, projects and roles for every customer, with orders approved by the customer, the project or the provider.
  • Monthly invoices in PDF, CSV or XLSX, with optional EU VAT, or usage taken into an existing billing system through the API.
  • Self-service and automation: the portal, a REST API, SDKs for Python, TypeScript and Go, a Terraform provider and an Ansible collection.
  • One support line through the built-in service desk, or Jira Service Management, Zammad or OpenText SMAX.

In the portal

A customer's view: GPUs in the catalog by accelerator, the price per GPU-hour, LLM access by the token, and one monthly invoice.
A GPU category with one offering per accelerator, and LLM inference next to them.
A GPU category with one offering per accelerator, and LLM inference next to them.
An H100 offering's price plan, metered per GPU-hour.
An H100 offering's price plan, metered per GPU-hour.
One monthly invoice with GPU-hours per accelerator and LLM tokens, line by line.
One monthly invoice with GPU-hours per accelerator and LLM tokens, line by line.

Recorded on a demo deployment: the organizations, people and numbers are made up.

How Waldur fits

Waldur runs next to the infrastructure, on the provider’s own servers, installed with Helm or Docker Compose. Clouds with an API are provisioned directly. Clusters and other services are run by the Waldur Site Agent at the provider’s site, which only connects out to Waldur.
  • OpenStack and VMware vCenter: tenants, virtual machines, volumes and networks.
  • OpenNebula, Kubernetes, OKD and Rancher: virtual machines, namespaces and projects with quotas.
  • Slurm: accounts with CPU- and GPU-hour limits, and usage per project.
  • Storage and registries: Ceph S3 object storage and Harbor, through the Waldur Site Agent.
  • Any other service as an offering fulfilled by the provider's staff, or automated with custom scripts.

What stays with other systems

  • Scheduling and partitioning GPUs, such as MIG, vGPU or fractional GPUs, is done by the platform underneath. Waldur meters and bills the result.
  • Bare-metal provisioning and colocation (racks, power, connectivity) are not automated. They can be sold as offerings fulfilled by the provider’s staff.
  • Waldur issues invoices but takes no card payments, and one deployment uses one currency.

Further reading

Implementation

A rollout usually starts with one platform and a few offerings, and adds services once billing works.

Project phases

  1. Proof of concept

    Deploy Waldur and connect one platform, such as an OpenStack region or a GPU cluster. Define the first offerings and price plans.

  2. Billing

    Decide how usage is invoiced, with Waldur's monthly invoices or through the existing billing system, and test it with a month of real usage.

  3. Customer sign-in

    Connect the identity providers customers use, through Keycloak, OpenID Connect, SAML or LDAP, and set up organization onboarding.

  4. Production launch

    Open the portal to customers. Set up monitoring, backups and the support channel.

  5. More services

    Add GPU, Kubernetes, storage or LLM offerings, and managed services fulfilled by staff.

Selling cloud or GPU capacity?

Contact us