Version: Next

Azure Kubernetes Service (AKS)

This guide walks you through deploying the Takeoff Stack in Azure AKS. As an example, we will deploy a dummy_model with an embedding model jina-v2-code-embed.

Note:
For Azure, we do not have GPU quotas available. Therefore, this guide uses a CPU instance and a dummy model for testing purposes.

How to Deploy Takeoff Stack in Azure AKS

Prerequisites

Ensure you have the following:

Access to the Azure Console
Azure CLI: For interacting with Azure from the terminal.
kubectl: For interacting with the Kubernetes cluster.
helm: For deploying Helm charts.

Step 1: Set Up an Azure AKS Cluster

Navigate to Azure Kubernetes Service (AKS) in the Azure portal.
Click "Create a Cluster".
Configure the following:
- Cluster Name: Provide a unique name for your cluster.
- Region: Select your preferred region.
- Node Pool: Configure your node pool (similar to a node group in EKS).
Leave other configurations as default and click "Create".

Step 2: Connect to the Cluster

Log in to your Azure account from the terminal:
```
az login
```
Configure Kubernetes credentials for the cluster:
- Replace <YOUR_CLUSTER_NAME> and <YOUR_RESOURCE_GROUP> with your actual cluster name and resource group.
```
az aks get-credentials --resource-group <YOUR_RESOURCE_GROUP> --name <YOUR_CLUSTER_NAME> --overwrite-existing
```

You should see output similar to this:

Merged "azure-takeoff-cluster" as current context in /home/<your-user>/.kube/config

Step 3: Prepare the Namespace and Secrets

Create the Namespace
- Use takeoff as the namespace (or choose a different name):
```
kubectl create namespace takeoff
```

Create Secrets

Docker Credentials: Ensure you provide your .dockerconfig.json file.

kubectl create secret generic takeoff-regcred --namespace takeoff \
  --from-file=.dockerconfigjson=<PATH_TO_YOUR_DOCKERCONFIG.JSON> \
  --type=kubernetes.io/dockerconfigjson

Takeoff Secrets: Replace <LICENSE_KEY> and <TOKEN> with your actual values:

kubectl create secret generic takeoff-secrets --namespace takeoff \
  --from-literal=LICENSE_KEY=<LICENSE_KEY> \
  --from-literal=TAKEOFF_ACCESS_TOKEN=<TOKEN>

Step 4: Deploy the Takeoff Stack Using Helm

Navigate to the Helm chart directory:
```
cd pantheon/hades/pantheon-helm/
```

Install the Takeoff Stack:

helm install takeoff-stack ./ --namespace takeoff -f values.yaml -f overwrites/azure_aks.yaml

Step 5: Test and Validate the Service

Port-Forward the Service

Forward the service port to your local machine:

kubectl port-forward service/takeoff-controller-svc 3000:3000 3001:3001 --namespace takeoff

Access the Frontend
- Open your browser and navigate to:
  - http://localhost:3000 for the main frontend.
  - http://localhost:3001 for the management frontend.
Validate the Model with a Test Request
- Use curl to send a request to the service and verify the model is working:
```
curl -X POST http://localhost:3000/generate -N \
  -d '{"text": ["How are you?"], "consumer_group": "generate"}' \
  --header "Content-Type: application/json"
```
- If successful, the model should return a response confirming it is loaded and functioning.

Azure Kubernetes Service (AKS)

Table of Contents​

Prerequisites​

Step 1: Set Up an Azure AKS Cluster​

Step 2: Connect to the Cluster​

Step 3: Prepare the Namespace and Secrets​

Step 4: Deploy the Takeoff Stack Using Helm​

Step 5: Test and Validate the Service​

Table of Contents