Get Started

Last updated 4 days ago

This guide will get you started with Clusterone. We'll start with a hands-on example on this page to get you working in no time. Don't worry if the commands on this page don't make sense to you, you'll learn what they mean on the next pages.

To keep things simple, we'll show you how to use Clusterone using a ready-to-run demo of a self-driving car simulation.

Ready to get going? Let's dive in!

Set up

Before we begin, make sure you have all your gear ready:

  • A Clusterone account. Join the waitlist if you don't have one yet!

  • A GitHub account. You can register here.

  • Make sure you have Python 2.7 or 3.6 installed

  • Install the Clusterone Python package. It's as easy as pip install clusterone.

  • (Skip if you are using If you have a Clusterone Enterprise installation, point the CLI to your installation with just config endpoint <>

The Clusterone command line interface, called just, is installed automatically with the Clusterone Python package. Clusterone also provides a graphical web interface, the Matrix.

Linking your GitHub account allows you to access your GitHub repositories from within Clusterone. To do this, you need to create a GitHub access token and add it to your Clusterone account.

On GitHub

Log into your GitHub account and navigate to the Personal Access Tokens page in the developer settings. Generate a new token and grant it the following permissions:

  • repo

  • admin:repo_hook

Copy the token when it's created.

On Clusterone

Log into your Clusterone account and open the Matrix. On the Account page, select the Keys tab. Click the Add GitHub OAuth Token button and paste the access token you created above. Click Save to store the token.

Perfect, you have successfully linked your GitHub account to Clusterone.

For more information on linking GitHub to Clusterone, see here.

Create a Project and Run Code

To create a GitHub project after adding your GitHub token, you sometimes need to refresh the page. We are aware of this issue and currently working on fixing it!

Create a project

Log into the Matrix and toggle the switch on the left to show your projects. Click the Add Project button. In the wizard, select Link GitHub Repository and type clusterone/self-driving-demo on the following page to find the repository. Click the button at the bottom right to create the project.

Create a Job and Run it

Open a command line and log into your Clusterone account:

just login

It's possible that the following just create job command fails when running it shortly after creating the project. It may take up to a few minutes to retrieve all data from GitHub. If your just create job command fails, just grab a coffee or tea and try again.

Next, create a job:

just create job distributed --project self-driving-demo --module main_tf \
--datasets tensorbot/self-driving-demo-data --ps-type c4.2xlarge \
--worker-type c4.2xlarge --name first-job

Finally, all that's left to do is starting the job:

just start job -p self-driving-demo/first-job

That's it! You can monitor the status of your job on the command line using just get events. More elaborate monitoring is available on the Matrix. The web interface also offers automatic integration with TensorBoard.

The dataset we're using for this demo is already uploaded to Clusterone. If you'd rather create your own dataset, the data is available at this AWS S3 bucket:‚Äč

Learn More

Now that you have run your first job on Clusterone, it's time to learn more about the platform.

On the following pages, we'll show you more about creating projects and datasets, and how to run them on Clusterone. Finally, we will take you for a tour around Clusterone's graphical interface and show you how to access TensorBoard right from your project.

For more examples on how to run distributed code on Clusterone, also see our MNIST example.

Join our Slack to get support and tips from the community.