AWS

See our full guide on deploying to AWS with EC2 here.

Recommended EC2 Instance Configurations

• Instance Type: g5.4xlarge or larger (24GiB CPU / 64GiB CPU)
• AMI (Amazon Machine Image): Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.1.0 (Ubuntu 20.04) 20231205 (ami-09c3540e22564aeba)
• Configure Storage: >256GiB Root volume
• Network Settings: Create a security group with these Inbound Security Group Rules
  • SSH: TCP, port 22
  • HTTP: TCP, port 80
  • HTTPS: TCP, port 443

If you’re planning to run your Takeoff on a AWS EC2 instance, we recommend choosing a g5.4xlarge instance in order to inference a 7B model.

You will also need to choose an AMI. We recommend Deep Learning OSS Nvidia Driver AMI GPU PyTorch AMI as this comes with the relevant Nvidia cuda drivers installed.

In addition, make sure you have at least 256 GiB of storage as you will need plenty of storage for the Takeoff image and downloaded models.

The ports 22, 80 and 443 also have to be exposed for you to ssh and send HTTP/HTTPS requests.

Connecting to your instance

1. Make sure you have your keypair needed to authenticate with your instance.

2. Locate your private key file and run this command to ensure your key is not publicly viewable. Otherwise, you will not be able to connect to your instance.

   chmod 400 "key.pem"

3. Connect to your instance with this command. The public DNS should be something like ec2-12-00-00-40.eu-west-2.compute.amazonaws.com

   ssh -i "key.pem" ubuntu@<PUBLIC_DNS>

   Alternatively, add the configuration to your ssh config file to use it with your IDE:

   Host <NAME>
       HostName <PUBLIC_DNS>
       User ubuntu
       IdentityFile "key.pem"

   Once you are connected, continue following Launching Takeoff.

   To use the Frontend UI later on, make sure you forward the ports 3000 and 3001.