Version: Next

Azure Kubernetes Service (AKS)

Member of Technical Staff, TitanML

How to Deploy Takeoff Stack in Azure AKS Table of Contents

This guide walks you through deploying the Takeoff Stack in Azure AKS. As an example, we will deploy a dummy_model with an embedding model jina-v2-code-embed.

Note:
This guide only uses a CPU instance and a dummy model for testing purposes. For more information on deploying GPU instances and other models, contact us.

Prerequisites

Ensure you have the following:

Access to the Azure Console
Azure CLI: For interacting with Azure from the terminal.
kubectl: For interacting with the Kubernetes cluster.
helm: For deploying Helm charts.

Step 1: Set Up an Azure AKS Cluster

Navigate to Azure Kubernetes Service (AKS) in the Azure portal.
Click "Create a Cluster".
Configure the following:
- Cluster Name: Provide a unique name for your cluster.
- Region: Select your preferred region.
- Node Pool: Configure your node pool (similar to a node group in EKS).
Leave other configurations as default and click "Create".

Step 2: Connect to the Cluster

Log in to your Azure account from the terminal:
```
az login
```
Configure Kubernetes credentials for the cluster:
- Replace <YOUR_CLUSTER_NAME> and <YOUR_RESOURCE_GROUP> with your actual cluster name and resource group.
```
az aks get-credentials --resource-group <YOUR_RESOURCE_GROUP> --name <YOUR_CLUSTER_NAME> --overwrite-existing
```

You should see output similar to this:

Merged "azure-takeoff-cluster" as current context in /home/<your-user>/.kube/config

Step 3: Prepare the Namespace and Secrets

Create the Namespace
- Use takeoff as the namespace (or choose a different name):
```
kubectl create namespace takeoff
```

Create Secrets

Docker Credentials: Ensure you provide your .dockerconfig.json file.

kubectl create secret generic takeoff-regcred --namespace takeoff \
  --from-file=.dockerconfigjson=<PATH_TO_YOUR_DOCKERCONFIG.JSON> \
  --type=kubernetes.io/dockerconfigjson

Takeoff Secrets: Replace <LICENSE_KEY> and <TOKEN> with your actual values:

kubectl create secret generic takeoff-secrets --namespace takeoff \
  --from-literal=LICENSE_KEY=<LICENSE_KEY> \
  --from-literal=TAKEOFF_ACCESS_TOKEN=<TOKEN>

Step 4: Deploy the Takeoff Stack Using Helm

Navigate to the Helm chart directory:
```
cd pantheon/hades/pantheon-helm/
```

Install the Takeoff Stack:

helm install takeoff-stack ./ --namespace takeoff -f values.yaml -f overwrites/azure_aks.yaml

Step 5: Test and Validate the Service

Port-Forward the Service

Forward the service port to your local machine:

kubectl port-forward service/takeoff-controller-svc 3000:3000 3001:3001 --namespace takeoff

Access the Frontend
- Open your browser and navigate to:
  - http://localhost:3000 for the main frontend.
  - http://localhost:3001 for the management frontend.
Validate the Model with a Test Request
- Use curl to send a request to the service and verify the model is working:
```
curl -X POST http://localhost:3000/generate -N \
  -d '{"text": ["How are you?"], "consumer_group": "generate"}' \
  --header "Content-Type: application/json"
```
- If successful, the model should return a response confirming it is loaded and functioning.

Prerequisites​

Step 1: Set Up an Azure AKS Cluster​

Step 2: Connect to the Cluster​

Step 3: Prepare the Namespace and Secrets​

Step 4: Deploy the Takeoff Stack Using Helm​

Step 5: Test and Validate the Service​