Kubernetes
See our full guide on deploying to Kubernetes here.
K8s onboarding​
To get set up with Kubernetes, create the following manifest, and save it as a file called manifest.yaml
.
In order to download the takeoff server, you’ll need to first create the contents of the secret. To do so, run the following from any machine with the docker daemon running.
If you don’t have a machine with docker available (it doesn’t have to be a production machine) - please reach out and we can create the kubernetes secret for you.
docker login -u takeoffusers
And at the prompt, enter the API token provided by your account manager.
Then, run
cat ~/.docker/config.json | base64
from your command line, to base64 encode your docker token. The resulting string of characters should be entered into the <to-create>
field in the following manifest.
apiVersion: v1
kind: Secret
metadata:
name: regcred
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: <to-create>
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: takeoff-pro-deployment
labels:
app: takeoff-pro
spec:
replicas: 1
selector:
matchLabels:
app: takeoff-pro
template:
metadata:
labels:
app: takeoff-pro
spec:
containers:
- name: takeoff-pro-container
image: tytn/takeoff-pro:0.19.1-gpu
resources:
limits:
nvidia.com/gpu: 1
env:
- name: TAKEOFF_MODEL_NAME
value: TitanML/llama2-7b-chat-4bit-AWQ
- name: TAKEOFF_DEVICE
value: cuda
imagePullSecrets:
- name: regcred
Once you’ve done so, run
kubectl apply -f manifest.yaml