This guide will get you started with Clusterone. We'll start with a hands-on example on this page to get you working in no time. Don't worry if the commands on this page don't make sense to you, you'll learn what they mean on the next pages.
To keep things simple, we'll show you how to use Clusterone using a ready-to-run demo of a self-driving car simulation.
Ready to get going? Let's dive in!
Before we begin, make sure you have all your gear ready:
A Clusterone account. Join the waitlist if you don't have one yet!
A GitHub account. You can register here.
Make sure you have Python 2.7 or 3.6 installed
Install the Clusterone Python package. It's as easy as
pip install clusterone.
(Skip if you are using clusterone.com) If you have a Clusterone Enterprise installation, point the CLI to your installation with
just config endpoint <https://your-clusterone.com>
The Clusterone command line interface, called
just, is installed automatically with the Clusterone Python package. Clusterone also provides a graphical web interface, the Matrix.
Linking your GitHub account allows you to access your GitHub repositories from within Clusterone. To do this, you need to create a GitHub access token and add it to your Clusterone account.
Log into your GitHub account and navigate to the Personal Access Tokens page in the developer settings. Generate a new token and grant it the following permissions:
Copy the token when it's created.
Log into your Clusterone account and open the Matrix. On the Account page, select the Keys tab. Click the Add GitHub OAuth Token button and paste the access token you created above. Click Save to store the token.
Perfect, you have successfully linked your GitHub account to Clusterone.
For more information on linking GitHub to Clusterone, see here.
Log into the Matrix and toggle the switch on the left to show your projects. Click the Add Project button. In the wizard, select Link GitHub Repository and type
clusterone/self-driving-demo on the following page to find the repository. Click the button at the bottom right to create the project.
Open a command line and log into your Clusterone account:
Next, create a job:
just create job distributed --project self-driving-demo --module main_tf \--datasets tensorbot/self-driving-demo-data --ps-type c4.2xlarge \--worker-type c4.2xlarge --name first-job
Finally, all that's left to do is starting the job:
just start job -p self-driving-demo/first-job
That's it! You can monitor the status of your job on the command line using
just get events. More elaborate monitoring is available on the Matrix. The web interface also offers automatic integration with TensorBoard.
Now that you have run your first job on Clusterone, it's time to learn more about the platform.
On the following pages, we'll show you more about creating projects and datasets, and how to run them on Clusterone. Finally, we will take you for a tour around Clusterone's graphical interface and show you how to access TensorBoard right from your project.
For more examples on how to run distributed code on Clusterone, also see our MNIST example.
Join our Slack to get support and tips from the community.