Defining instance types

Last updated 6 months ago

Defining instance types

On a cloud installation or processing site, instance types are usually those of the underlying cloud provider. Although the article above can also be applied to cloud instances, it is mainly targeted at on-premise installations where resources are physical nodes and the cluster is of fixed size.

Clusterone scheduling

On an on-premise installation, you can configure your instance-types to represent your underlying hardware, or in more granular types. For example, assume you have the following machines:

name

vCPUs

memory

GPUs

CUDA version

tatooine

16

32GB

0

None

naboo

32

64GB

4

9.2

Then you can define instance types to exactly match the physical machines

  • cpu: (16vCPUs, 32GB, No GPU, No CUDA)

  • gpu: (32vCPUs, 64GB, 4, 9.2)

You can also define instance types that only use a subset of a given machine, and let Kubernetes compute resource manager and Clusterone's scheduling and orchestration manage the mapping of jobs to the physical machines. In Kubernetes language, this is equivalent to specifying resource limits for a pod, then letting Kubernetes decide which node it will be pinned to.

In the example above, we could define those instance type:

  • cpu-8: (8vCPUs, 16GB, No GPU, No CUDA)

  • cpu-16: (16vCPUs, 32GB, No GPU, No CUDA)

  • gpu-1: (16vCPUs, 32GB, 1, 9.2)

  • gpu-2: (16vCPUs, 32GB, 2, 9.2)

  • gpu-4: (32vCPUs, 64GB, 4, 9.2)

The following instance types could then run concurrently:

On tatooine

  • 2 cpu-8

    OR

  • 1 cpu-16 (saturating the number of vCPUs available)

On naboo

  • 2 gpu-1 (saturating the amount of memory available, there might be a smarter instance type definition to use all GPUs)

    OR

  • 2 gpu-2 (saturating the amount of GPUs available)

    OR

  • 2 gpu-4 (saturating the amount of GPUs available)

Note that the appropriate CUDA drivers have to be installed on the nodes.

Read more about Kubernetes resource management and what a Kubernetes CPU unit is.

Defining an instance type in the administration dashboard

Given they match those conditions, any instance type can be defined on Clusterone. Navigate to the Jobs / Instance Types section and hit "Add Instance Type".

To define an instance type, input:

  • Instance Type: the name of the instance that will be the unique name of the instance and used in the API and command line. It can only contain alphanumeric characters and "-".

  • Number of GPUs

  • Number of CPUs: corresponding to CPU units‚Äč

  • Memory

  • CUDA version: should correspond with CUDA version installed on the machine

  • Instance Name: a user friendly name that will be displayed in help texts

  • Show this instance type for ps and Show this instance type for worker: indicates whether or not the instance will be available for a parameter server or a worker

  • Processing site: the processing site on which the node resides, in doubt copy from a preexisting instance type.

  • Jobmaster queue: the job queue used for this instance type, in doubt copy from a preexisting instance type.

  • Show this instance type for all users: indicates whether this instance type is available to all users or Admin users only. This is useful for testing before making the instance type available to all your users.

Don't hesitate to reach out through Zendesk or support@clusterone.com for help in setting up instances.