Scaling [us_dod_disa_3_entities_sourceip_destinationip_network-3f10] With Kubernetes
Generated On: 2025-01-07 16:36:33 UTC
You can scale your solution with Kubernetes. To do so, will will need to apply the following YAML files to your Kubernetes cluster.
Tip
Refer to TML documentation for more information on scaling with Kubernetes.
Watch the YouTube Video: here.
You can also run the YAML files locally in a 1 node kubernetes cluster called minikube. Refer to Installing minikube
Important
Below assumes you have a Kubernetes cluster and kubectl installed in your Linux environment.
Attention
Make sure to STOP the TSS Container and other containers before running Kubernetes/Minikube.
If you get the following WARNING from Kubernetes:
Warning FailedScheduling default-scheduler 0/1 nodes are available: 1 Insufficient nvidia.com/gpu. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod. Make sure no other application is using the GPU. You can check by executing in terminal the command: nvidia-smi
Oherwise, Issue the commands below:
sudo apt update && sudo apt install -y nvidia-docker2
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
Based on your TML solution [us_dod_disa_3_entities_sourceip_destinationip_network-3f10] - if you want to scale your application with Kubernetes - you will need to apply the following YAML files.
YML File |
Description |
us_dod_disa_3_entities_sourceip_destinationip_network-3f10.yml |
This is your main solution YAML file. It MUST be applied to your Kubernetes cluster. |
secrets.yml |
You MUST store your passwords in base64 format in this file. For instructions on how to convert plain text passwords to base64 refer to instructions |
mysql-storage.yml |
This is storage allocation for MySQL DB. It MUST be applied to your Kubernetes cluster. |
mysql-db-deployment.yml |
This is the MySQL deployment YAML. It MUST be applied to your Kubernetes cluster. |
kafka.yml |
This is the Kafka deployment YAML. This is MANDATORY if using kafka locally or on-premise. |
privategpt.yml |
This is the privateGPT deployment YAML. This is OPTIONAL. However, it must be applied if using Step 9 DAG. |
qdrant.yml |
This is the Qdrant deployment YAML. This is OPTIONAL. However, it must be applied if using Step 9 DAG. |
nginx-ingress-us_dod_disa_3_entities_sourceip_destinationip_network-3f10.yml |
If you are scaling your TML solution you must apply the nginx-ingressus_dod_disa_3_entities_sourceip_destinationip_network-3f10.yml; this yaml is auto-generated for every TML solution. For more details see section Scaling with NGINX Ingress and Ingress Controller |
kubectl Apply command
Important
To apply the YAML files below to your Kubernetes cluster simply run this command:
kubectl apply -f kafka.yml -f secrets.yml -f mysql-storage.yml -f mysql-db-deployment.yml -f us_dod_disa_3_entities_sourceip_destinationip_network-3f10.yml
us_dod_disa_3_entities_sourceip_destinationip_network-3f10.yml
Important
Copy and Paste this YAML file: us_dod_disa_3_entities_sourceip_destinationip_network-3f10.yml - and save it locally.
Attention
MAKE SURE to update any tokens and passwords in the secrets.yml file:
GITPASSWORD (MANDATORY)
READTHEDOCS (MANDATORY)
KAFKACLOUDPASSWORD (OPTIONAL)
MQTTPASSWORD (OPTIONAL)
For instructions on how to do this, refer to instructions
################# us_dod_disa_3_entities_sourceip_destinationip_network-3f10.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: us_dod_disa_3_entities_sourceip_destinationip_network-3f10
spec:
selector:
matchLabels:
app: us_dod_disa_3_entities_sourceip_destinationip_network-3f10
replicas: 3 # tells deployment to run 1 pods matching the template
template:
metadata:
labels:
app: us_dod_disa_3_entities_sourceip_destinationip_network-3f10
spec:
containers:
- name: us_dod_disa_3_entities_sourceip_destinationip_network-3f10
image: maadsdocker/us_dod_disa_3_entities_sourceip_destinationip_network-3f10-amd64:latest
volumeMounts:
- name: dockerpath
mountPath: /var/run/docker.sock
ports:
- containerPort: 5050
- containerPort: 4040
- containerPort: 6060
env:
- name: TSS
value: '0'
- name: SOLUTIONNAME
value: 'us_dod_disa_3_entities_sourceip_destinationip_network-3f10'
- name: SOLUTIONDAG
value: 'solution_preprocessing_dag-us_dod_disa_3_entities_sourceip_destinationip_network-3f10'
- name: GITUSERNAME
value: 'smaurice101'
- name: GITREPOURL
value: 'https://github.com/smaurice101/raspberrypitss.git'
- name: SOLUTIONEXTERNALPORT
value: '5050'
- name: CHIP
value: 'amd64'
- name: SOLUTIONAIRFLOWPORT
value: '4040'
- name: SOLUTIONVIPERVIZPORT
value: '6060'
- name: DOCKERUSERNAME
value: 'maadsdocker'
- name: CLIENTPORT
value: '0'
- name: EXTERNALPORT
value: '39399'
- name: KAFKACLOUDUSERNAME
value: 'MUHRHBPKJYPROKBX'
- name: VIPERVIZPORT
value: '49689'
- name: MQTTUSERNAME
value: 'smaurice'
- name: AIRFLOWPORT
value: '9000'
- name: GITPASSWORD
valueFrom:
secretKeyRef:
name: tmlsecrets
key: githubtoken
- name: KAFKACLOUDPASSWORD
valueFrom:
secretKeyRef:
name: tmlsecrets
key: kafkacloudpassword
- name: MQTTPASSWORD
valueFrom:
secretKeyRef:
name: tmlsecrets
key: mqttpass
- name: READTHEDOCS
valueFrom:
secretKeyRef:
name: tmlsecrets
key: readthedocs
- name: qip
value: 'privategpt-service' # This is private GPT service in kubernetes
- name: KUBE
value: '1'
- name: step4maxrows # STEP 4 maxrows field can be adjusted here. Higher the number more data to process, BUT more memory needed.
value: '800'
- name: step4bmaxrows # STEP 4b maxrows field can be adjusted here. Higher the number more data to process, BUT more memory needed.
value: '-1'
- name: step5rollbackoffsets # STEP 5 rollbackoffsets field can be adjusted here. Higher the number more training data to process, BUT more memory needed.
value: '-1'
- name: step6maxrows # STEP 6 maxrows field can be adjusted here. Higher the number more predictions to make, BUT more memory needed.
value: '-1'
- name: step9rollbackoffset # STEP 9 rollbackoffset field can be adjusted here. Higher the number more information sent to privateGPT, BUT more memory needed.
value: '-1'
- name: step1solutiontitle # STEP 1 solutiontitle field can be adjusted here.
value: 'TML Entity Based Cybersecurity Solution'
- name: step1description # STEP 1 description field can be adjusted here.
value: 'This is an awesome real-time cybersecurity solution built by TSS'
- name: KUBEBROKERHOST
value: 'kafka-service:9092'
- name: KAFKABROKERHOST
value: '127.0.0.1:9092'
volumes:
- name: dockerpath
hostPath:
path: /var/run/docker.sock
---
apiVersion: v1
kind: Service
metadata:
name: us_dod_disa_3_entities_sourceip_destinationip_network-3f10-visualization-service
labels:
app: us_dod_disa_3_entities_sourceip_destinationip_network-3f10-visualization-service
spec:
type: ClusterIP
ports:
- port: 80 # Ingress port, if using port 443 will need to setup TLS certs
name: p1
protocol: TCP
targetPort: 6060
selector:
app: us_dod_disa_3_entities_sourceip_destinationip_network-3f10
Tip
In the solution YAML file above, you can adjust the replicas field. Currently, replicas: 3 for demonstration purposes.
secrets.yml
Important
You MUST store base64 passwords in this file and apply it to the Kubernetes cluster.
Refer to instructions.
###################secrets.yml
apiVersion: v1
kind: Secret
metadata:
name: tmlsecrets
type: Opaque
data:
readthedocs: <enter your base64 password>
githubtoken: <enter your base64 password>
mqttpass: <enter your base64 password>
kafkacloudpassword: <enter your base64 password>
mysql-storage.yml
Important
Copy and Paste this YAML file: mysql-storage.yml - and save it locally.
################# mysql-storage.yml
apiVersion: v1
kind: PersistentVolume
metadata:
name: mysql-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 20Gi
accessModes:
- ReadWriteMany
hostPath:
path: "/mnt/data"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteMany
resources:
requests:
storage: 20Gi
mysql-db-deployment.yml
Important
Copy and Paste this YAML file: mysql-db-deployment.yml - and save it locally.
################# mysql-db-deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
spec:
selector:
matchLabels:
app: mysql
strategy:
type: Recreate
template:
metadata:
labels:
app: mysql
spec:
containers:
- image: maadsdocker/mysql:latest
name: mysql
resources:
limits:
memory: "512Mi"
cpu: "1500m"
env:
- name: MYSQL_ROOT_PASSWORD
value: "raspberry"
- name: MYSQLDB
value: "tmlids"
- name: MYSQLDRIVERNAME
value: "mysql"
- name: MYSQLHOSTNAME
value: "mysql:3306"
- name: MYSQLMAXCONN
value: "4"
- name: MYSQLMAXIDLE
value: "10"
- name: MYSQLPASS
value: "raspberry"
- name: MYSQLUSER
value: "root"
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: mysql-pv-claim
---
apiVersion: v1
kind: Service
metadata:
name: mysql-service
spec:
ports:
- port: 3306
selector:
app: mysql
kafka.yml
This is the Kafka service needed by TML pods - if using Kafka locally or on-premise.
apiVersion: apps/v1
kind: Deployment
metadata:
name: kafka
spec:
selector:
matchLabels:
app: kafka
replicas: 1 # tells deployment to run 1 pods matching the template
template:
metadata:
labels:
app: kafka
spec:
containers:
- name: kafka
image: maadsdocker/kafka-amd64 # IF you DO NOT have NVIDIA GPU use: maadsdocker/tml-privategpt-no-gpu-amd64
env:
- name: KAFKA_HEAP_OPTS
value: "-Xmx512M -Xms512M"
- name: PORT
value: "9092"
- name: TSS
value: "0"
- name: KUBE
value: "1"
- name: KUBEBROKERHOST
value: "kafka-service:9092"
---
apiVersion: v1
kind: Service
metadata:
name: kafka-service
spec:
ports:
- port: 9092
selector:
app: kafka
privategpt.yml
Note
This YAML is Optional - Use Only If Step 9 Dag is used
Important
Copy and Paste this YAML file: privategpt.yml - and save it locally.
Note
By default this assumes you have a Nvidia GPU in your machine and so it using the Nvidia privateGPT container:
image: maadsdocker/tml-privategpt-with-gpu-nvidia-amd64
if you DO NOT have a Nvidia GPU installed then change image to:
image: maadsdocker/tml-privategpt-no-gpu-amd64
################# privategpt.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: privategpt
spec:
selector:
matchLabels:
app: privategpt
replicas: 1 # tells deployment to run 1 pods matching the template
template:
metadata:
labels:
app: privategpt
spec:
containers:
- name: privategpt
image: maadsdocker/tml-privategpt-with-gpu-nvidia-amd64 # IF you DO NOT have NVIDIA GPU use: maadsdocker/tml-privategpt-no-gpu-amd64
env:
- name: NVIDIA_VISIBLE_DEVICES
value: all
- name: DP_DISABLE_HEALTHCHECKS
value: xids
- name: WEB_CONCURRENCY
value: "3"
- name: GPU
value: "1"
- name: COLLECTION
value: "tml"
- name: PORT
value: "8001"
- name: CUDA_VISIBLE_DEVICES
value: "0"
- name: TSS
value: "0"
- name: KUBE
value: "1"
resources: # REMOVE or COMMENT OUT: IF you DO NOT have NVIDIA GPU
limits: # REMOVE or COMMENT OUT: IF you DO NOT have NVIDIA GPU
nvidia.com/gpu: 1 # REMOVE or COMMENT OUT: IF you DO NOT have NVIDIA GPU
ports:
- containerPort: 8001
tolerations: # REMOVE or COMMENT OUT: IF you DO NOT have NVIDIA GPU
- key: nvidia.com/gpu # REMOVE or COMMENT OUT: IF you DO NOT have NVIDIA GPU
operator: Exists # REMOVE or COMMENT OUT: IF you DO NOT have NVIDIA GPU
effect: NoSchedule # REMOVE or COMMENT OUT: IF you DO NOT have NVIDIA GPU
---
apiVersion: v1
kind: Service
metadata:
name: privategpt-service
labels:
app: privategpt-service
spec:
type: NodePort #Exposes the service as a node ports
ports:
- port: 8001
name: p1
protocol: TCP
targetPort: 8001
selector:
app: privategpt
qdrant.yml
Note
This YAML is Optional - Use Only If Step 9 Dag is used
Important
Copy and Paste this YAML file: qdrant.yml - and save it locally.
################# qdrant.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: qdrant
spec:
selector:
matchLabels:
app: qdrant
replicas: 1
template:
metadata:
labels:
app: qdrant
spec:
#hostNetwork: true
containers:
- name: qdrant
image: qdrant/qdrant
ports:
- containerPort: 6333
volumeMounts:
- mountPath: /qdrant/storage
name: qdata
volumes:
- name: qdata
hostPath:
path: /qdrant_storage
---
apiVersion: v1
kind: Service
metadata:
name: qdrant-service
labels:
app: qdrant-service
spec:
type: NodePort #Exposes the service as a node ports
ports:
- port: 6333
name: p1
protocol: TCP
targetPort: 6333
selector:
app: qdrant
Tip
The number of replicas can be changed in the cybersecuritywithprivategpt-3f10.yml file: look for replicas. You can increase or decrease the number of replicas based on the amout of real-time data you are processing.
Kubernetes Dashboard Visualization
To visualize the dashboard you need to forward ports to your solution deployment in Kubernetes. For this solution, the port forward command would be:
kubectl port-forward deployment/us_dod_disa_3_entities_sourceip_destinationip_network-3f10 6060:6060
After you forward the ports then copy/paste the viusalization URL below and run your dashboard.
http://localhost:6060/cybersecuritydetection.html?topic=iot-preprocess,iot-preprocess2&offset=-1&groupid=&rollbackoffset=400&topictype=prediction&append=0&secure=1
Scaling with NGINX Ingress and Ingress Controller
All TML solutions will scale with NGINX ingress to perform load-balancing. But, before you can use ingress - ingress MUST be enabled in Kubernetes cluster. Follow these steps:
Important
STEP 1: To turn on ingress in minikube type:
minikube addons enable ingress
minikube addons enable ingress-dns
STEP 2: In Linux Add tml.tss domain name to /etc/hosts file
Edit your /etc/hosts file
add an entry to /etc/hosts:
127.0.0.1 tml.tss
Save the file
STEP 2b: In Windows Add tss.tml domain name to C:\Windows\System32\drivers\etc
Edit your C:\Windows\System32\drivers\etc\hosts file (Note: You may need to COPY the hosts file to another directory, then edit the file, then copy it back to C:\Windows\System32\drivers\etc\hosts
add an entry:
127.0.0.1 tml.tssSave the file
copy it back to C:\Windows\System32\drivers\etc\hosts
STEP 3: In a new Linux terminal you MUST turn on minikube tunnel type:
minikube tunnel
STEP 4: Apply nginx-ingress-us_dod_disa_3_entities_sourceip_destinationip_network-3f10.yml to your kubernetes cluster. First you need to save it locally then apply it:
nginx-ingress-us_dod_disa_3_entities_sourceip_destinationip_network-3f10.yml
############# nginx-ingress-us_dod_disa_3_entities_sourceip_destinationip_network-3f10.yml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: tml-ingress annotations: nginx.ingress.kubernetes.io/use-regex: "true" nginx.ingress.kubernetes.io/rewrite-target: /$2 spec: ingressClassName: nginx rules: - host: tml.tss http: paths: - path: /viz(/|$)(.*) pathType: ImplementationSpecific backend: service: name: us_dod_disa_3_entities_sourceip_destinationip_network-3f10-visualization-service port: number: 80 --- apiVersion: v1 kind: ConfigMap apiVersion: v1 metadata: name: ingress-nginx-controller namespace: ingress-nginx data: allow-snippet-annotations: "true"kubectl apply -f nginx-ingress-us_dod_disa_3_entities_sourceip_destinationip_network-3f10.yml
You are now ready to run the Dashboard using Ingress load balancing.
Ingress Dashboard Visualization
Copy and paste this URL below in your browser and start streaming. Because you are now using INGRESS, Kubernetes will perform load balancing on the streaming data.
http://tml.tss/viz/cybersecuritydetection.html?topic=iot-preprocess,iot-preprocess2&offset=-1&groupid=&rollbackoffset=400&topictype=prediction&append=0&secure=1
Making Secure TLS Connection with gRPC
You can make secure connection through the NGINX controller to your TML service using TLS encryption. TML solutions utilizing gRPC have a built-in gRPC server for secure and fast connections.
Note
Note that the REST API service is unencrypted, but very useful if you don't have the need for security inside your VM.
Here are the Steps to follow.
Step 1: Get Server Certificates
To utilize the secure gRPC service you must have valid security certificates for each server. You can self-sign your own certificates by following the steps here: Steps to Self-Signing Certificates
Tip
You can download the all the certificates from the certs repo
Step 2: Apply the Server Certificates to Kubernetes Cluster
Attention
These certificates are for the tml.tss sever. Follow the steps to add this host to your /etc/hosts file: :ref:Scaling with NGINX Ingress and Ingress Controller`
If you have a different host, then you will need to re-generate these certificates for your new host, replace tls.crt and tls.key with your your new keys. Note these keys are in base64 format for security. To replace with your own hosts, just update the the san.cnf file
########## NOTE: tls certs are for tml.tss server only
# You can replace tls.crt and tls.key with your own server.crt and server.key
#############secret-tls.yml
apiVersion: v1
data:
tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUY5ekNDQTkrZ0F3SUJBZ0lVWFlPeUJia21qaUxMMDFiMkI2NVI5UllHbVNBd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1ZURUxNQWtHQTFVRUJoTUNRMEV4RURBT0JnTlZCQWdNQjA5dWRHRnlhVzh4RURBT0JnTlZCQWNNQjFSdgpjbTl1ZEc4eERqQU1CZ05WQkFvTUJVOTBhV056TVJJd0VBWURWUVFEREFsVFpXSmhjM1JwWVc0d0lCY05NalF4Ck1qRTVNVFF4T1RJeldoZ1BNakV5TkRFeE1qVXhOREU1TWpOYU1GVXhDekFKQmdOVkJBWVRBa05CTVJBd0RnWUQKVlFRSURBZFBiblJoY21sdk1SQXdEZ1lEVlFRSERBZFViM0p2Ym5Sdk1RNHdEQVlEVlFRS0RBVlBkR2xqY3pFUwpNQkFHQTFVRUF3d0pVMlZpWVhOMGFXRnVNSUlDSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQWc4QU1JSUNDZ0tDCkFnRUE3SEdwMFVVV2toejIzWVpRWE41VEVSR3JHYk43RzdQRWI3dEtRTWFObUtkVmFRQjlpRmVqQ3ZtUWxYNXUKcjZjMjF6VkMrVC8zZ0lQZmN3bW9SNzVDZGQ4dDhYRkVIOXRFSVBsZXYrbzVRNzFHajRqdnh1YTBBeU1KQlVuRwpNeEZTT2Qxek9heVlGSDFNMk94R0RUOU1RT3liZEFLd3pGU2YyZHlDcUx3cW0rWTJjWlIrbkFWOS81T1RaOVhiCktmVjJwWFNyRjVhcGovQTFCL3lDTlpDb284Nit4bFBFajRISGhxV1NvVkZxT1dhMlZ4dC9UQjlnKzFyL2hGOWEKV2g5UTZZNmRtK1FYUnhaWGxZR1pzdGVmbjFCcmY2cHdrWTRJZmtPd3RCb29EUnhUajg5U213bjdwaDJQczdFMQpvU2NoYkVkSDZLSjIxZ3JVSXEwZFMzaEZES2pYL2UrT2xIclk1bE42UmJqLzAwUjVhenRMcXFQWjFGNVdBTHNsCnp5MzV2SHNoR2dzVVRNVGVZLzQ1RmpMamJrLzdMckdOZWlhQUZXcFkrb0oyTUF6ZUN1QnlRRW95OWhrLzVjNTUKS0IyMVNzdU02M3Q0bElQV0VGNFBrbUFGNHFDNEZOMldLQ2E2RTgyWTFsdmxYQmNHL2dZalZ0ZTdmTzdaLy9MRQp6Vk5tOHJqRGVXTUR6eEdTYzIvVXQ3QzJKVkF6VlRlWmZ5Vm9TZWtMZmZ1S1FreWxRUERHMGVQRXR6UEEyYjI3CkE1dTdsaThaV1RMUEs3WW0xTXNPOHN5ZmNmTTI1c0had2pDVVBOdzNhbXl6blRtLzhtb2FheVR2RWZZaTlIVjYKejlNY3Y1dlltdFhDaDZxSm00RHNLbmdGMkNrd29DMldmQjRjVjRvRDE2RStETGNDQXdFQUFhT0J2RENCdVRBYwpCZ05WSFJFRUZUQVRnZ2QwYld3dWRITnpnZ2gwYld3dWRITnpNakFkQmdOVkhRNEVGZ1FVVVZ2aE5zS3RHaURVCkFQTXo0Ni8wV2s0QkpuWXdlZ1lEVlIwakJITXdjYUZacEZjd1ZURUxNQWtHQTFVRUJoTUNRMEV4RURBT0JnTlYKQkFnTUIwOXVkR0Z5YVc4eEVEQU9CZ05WQkFjTUIxUnZjbTl1ZEc4eERqQU1CZ05WQkFvTUJVOTBhV056TVJJdwpFQVlEVlFRRERBbFRaV0poYzNScFlXNkNGQk5aVGFPVStaQlVDa29LVVJzZ2o3QVN0ZFBzTUEwR0NTcUdTSWIzCkRRRUJDd1VBQTRJQ0FRQjAwUThodVJ2d3RvTzFkQjZKRzBHTjE1bXJGYUhyRlR4dnU3RFlPK0xWc2RtQmxoNVoKUFFSUVdjUUZaeFRvVmJZbldPY2hmY1VMK3padVo2QVdQOVp4RHljWjNrT3JFUm9EbjJzL0wzSk5sK0IyRGhmUQpwUytMeWdLMWt6RGhEeWI2dE0rSVdTRzY2RWdIZHF0U0dDM2czWXR0eTNEZTZidzFzSERwTkg3Q08wNndId1Y4CkdCVC9PempoS2U2M3RyaGxZZmtvM1IrQnFQTWd0YlN4MTlOOVBBY0FRbFgrQVJEckJ5a2tZZkExVVpKcml5aHQKRllZNXdmcmpJcTB5N0RtSDBYVkwxRkhNKzFCdFJrcnJxU1pSWHYybjRlTnFuMlRiUjZpdTdWS0pjdnVCYkYrMwpMRDdEZDBjdjRYSmxxektyUWtWR0R5T3M2bVMvcEhPcVhMNmNOREdpc1BpSFRRWkppcEFaUFpQQ3pXUjdjam1WCmkyVmZMN3NPTmlReXY5WFIyYzYrQndFbXk3bDg1dnBmVG9VaHN2R1lMemJjcndDM1lOL0d1WnQzQW5EQ1cyT04KV05lcldSTmhrYjFSdUFtWVREY21jb0t0OSt2RFY1M2NtK1BJUno1ZldhUUNXYjhXdDFuMjFkNitoRWtBdVcxRQppQmh0RFJYcy8xeUkxOEI1UlFGSU44ZkNQMnh5WWR1b0MzbldnTUR5NzRkcUNsT0hlTmhEUFIyM3NZNjR4bmhKCkRBcHh2dTBwT3crTno2bjcyU0o4Ky9TODlaRW1jQy9nWFNnOVZ0SEYzNDR6YzVma3ZiMFZyWDRqVHA3R1JLSTEKRlZOWkRBbml3TXN5eGJnUEJEMGEwb1gwTlY0OEJUTkdaTzNWRzVLdVpwN3BXYmxseHh6QldHWnVvZz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUpRd0lCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQ1Mwd2dna3BBZ0VBQW9JQ0FRRHNjYW5SUlJhU0hQYmQKaGxCYzNsTVJFYXNaczNzYnM4UnZ1MHBBeG8yWXAxVnBBSDJJVjZNSytaQ1ZmbTZ2cHpiWE5VTDVQL2VBZzk5egpDYWhIdmtKMTN5M3hjVVFmMjBRZytWNi82amxEdlVhUGlPL0c1clFESXdrRlNjWXpFVkk1M1hNNXJKZ1VmVXpZCjdFWU5QMHhBN0p0MEFyRE1WSi9aM0lLb3ZDcWI1alp4bEg2Y0JYMy9rNU5uMWRzcDlYYWxkS3NYbHFtUDhEVUgKL0lJMWtLaWp6cjdHVThTUGdjZUdwWktoVVdvNVpyWlhHMzlNSDJEN1d2K0VYMXBhSDFEcGpwMmI1QmRIRmxlVgpnWm15MTUrZlVHdC9xbkNSamdoK1E3QzBHaWdOSEZPUHoxS2JDZnVtSFkrenNUV2hKeUZzUjBmb29uYldDdFFpCnJSMUxlRVVNcU5mOTc0NlVldGptVTNwRnVQL1RSSGxyTzB1cW85blVYbFlBdXlYUExmbThleUVhQ3hSTXhONWoKL2prV011TnVUL3N1c1kxNkpvQVZhbGo2Z25Zd0RONEs0SEpBU2pMMkdUL2x6bmtvSGJWS3k0enJlM2lVZzlZUQpYZytTWUFYaW9MZ1UzWllvSnJvVHpaaldXK1ZjRndiK0JpTlcxN3Q4N3RuLzhzVE5VMmJ5dU1ONVl3UFBFWkp6CmI5UzNzTFlsVUROVk41bC9KV2hKNlF0OSs0cENUS1ZBOE1iUjQ4UzNNOERadmJzRG03dVdMeGxaTXM4cnRpYlUKeXc3eXpKOXg4emJtd2RuQ01KUTgzRGRxYkxPZE9iL3lhaHBySk84UjlpTDBkWHJQMHh5L205aWExY0tIcW9tYgpnT3dxZUFYWUtUQ2dMWlo4SGh4WGlnUFhvVDRNdHdJREFRQUJBb0lDQUQ1UnVSSWsxUTJlMzd4RWpnTGtRRjJqCjNBYVNwVlNJWGJLYldUZFlmZktwekJ1NFd0M29SMXQ1cXMrVU91VkdPL0NlSTdCaFdVRkF3TkRuenpoVm45dkUKZnE0QURoWWRhMGdMb2hzUVI1YWdtU3YweWtvUS9ZcEVIamtNR0ZiV2JtYzlCSVZEaGZRRWtKQXVPa3A4a0FNZQp1ZHhxWnlINy9nUGttSFdUM3VFblhOc3o2ZWtDazVLYzJZSEpQcEpCRmN3SFE1OGNnVVdrYUwzWm9wSXV0aHd5CnZscTBzbjZtbEtuYkV4bzh4TFFyYTh6cXZQTVo1Q3hyOENQNkkrelVDelg3OW5PanV6VHI0UnJSUldyN1pTR1AKQno1bmRITVF6aEZGa3hudE9QZzNxcGloYXVMZFR6d1oxNG5qbjhDQmVWQTZPMnhJQWUxcGZqOURoSkNqT3dOWQo5Y0RsYUEwVXpqOXcxTjdZdjk0RERDWWJIVVJweldQNXQ2TUo4ZFp3eHIxNmV6eG01ejlPaExTZzBzWWx6Z3k0CjA2YlRia2pIMlg1S05tZlNjK05saDh0T2t2QnRVSXgydUZvNkZWRFVhKzRKSUhyNVRtUUZMbS92OE5ha0puZWsKbkF0UGpSL0ZubzNGT1F0Ynpja2xJNVVXaVBHWWczQ3Y3Z3ZqYkVuUXJtT3B5Z3QxcFdWVnpTRnhCWjl1QmRxcgpod01USHlURHFxTDBtaGg4UXhnOGR2NllBR0srR3lSQmU3SDVENTdqVXE0eG5ML20zWVA2SERnV2x0YTg4MTVlCmtlalhkVUcvMk9qVmEwZmNxK2x2d3htRUJ3YXJueGw2VUVpTGhlb1JzSkFaZkl6L1hFMlhxYStnaWpwcDRRbFgKTkxNeXNQNWVYb0JxNXArOUQzZnBBb0lCQVFEN2wrMEx5ZFVkdERCdU5EQ0dUSGhBS1N1Z05HWDZyK2Fady9XOQpyVkZRYWJTdWhjYy9MMmxIUHNsN0NNamZrNVk5b1BxazZUYUJKR0NhMFFBWGc5bTBCR09CMUZhd203Z0F5YlIyCldJWlViaGxybjBBM29zU3J3YUVnc3BZUG5WRTlQOU9GUGdhWWoyd2NNSkVleENRb2t4QmlvOFQwSElGQklWQUUKTGExTmQvcFl0bUNYeGtncjlnWHh3QlpaZlJjbk93V0dkOVBvMDN6Y3NVOWVsSE1kOG1Db1dQbGtGcXBQN2VXKwp5VHo5Q3ovWUgrTTYyWFBOQldJM2VNRWpuMlBaUEp1WnFmcVN4cVcwMWJSdU1lbFgrd1dSeEdHK1hSS3l5cENlCkV4N0xTd3pNdk9QQVVkVVFrT3A0ZEFXZGFQeGNXcUFGVU5Cb1VTY2xGMmtOTlRVL0FvSUJBUUR3bGMrOHBxSDIKV2dtOW83RnltSFhHdldJdmIvMTBpRFpFZ1gzK3dTU1Y5SW5mNEtINDZvM29ER21tTWlYS2R3T1FYenVLTksycgpSSHNOZEl4VG05cnFxa3BNckZGaUplZ3JuTU43VFpON0pnaHE4SkNkR2JUb0xPYTNVTzJSWnRSZ0phOUVPZjhXClFBRmxqeTRnN294ZktudURORWZGNUFzTW5PaTdTWFNQSXpxdDBXMGdZcmdQaGd1YUFGSFIyMzNJRkRkeG02d0sKTGJkSDdmS09QQ1diVEpQVzQvQTRDaks3WVVKWERzUHFmbjBYT2JNWDlWSVNDLzM2M1JqbjQ1S3RuVUtJa2RCRgpIWkxLc1hvOU0wT2ZUb0dTdlNsV0g0SVZmTE81NTlsM2FFVHp2OXZQVzRqcmE1Rkc3TEV6a3FiSXRUWkEyemp2Cmgwa0JRMk4xMGZLSkFvSUJBRGJqckdtMy9QRGdFUGphRmdRV3h0MW9uZ1h6cUpRS3NFcTN2L05EenN1MlpCNzMKUE1NQ092dTZMUWJVb2M1MVNuL2prUXROZmdDcXlSQzlyRUYxR0pmM3BTWDhCM1c4WTJaNG14Qit1Ny9Meld2MwpjSEV5NTZsNU13Z0pMa2YxMEhXR2FVVldoT1hmMUh4SjlEODhGNDlxbGxhTzJEZFJ5TGxHNVVna0Z2MGh3ZEo4CjU1SDFSbVdnNVNjYSswVkd6emhWM2h5Nkk5ZFYzSlhoY1NsM1JhNHc1UG1WZjhOZ1ZvUGRxUlA0bjMrdFpwNW0KUnBMZVFpOW1qMGorNVZRNlAvUnpEcGQxeUI4aGk2RnFSbFVNT3BaaFE1UEx2bTlqcXVLcTR1WTUwYXdVa1pSUgpXWGJwNDR3YnNhdloxQ2ZGY2RsTVJFRWtvbk0vMFVSOFdRVHlxTTBDZ2dFQkFNUUVDMkZWRXBpNCt6NjdaQlJPCkM0ZUZQYjRRckp5SmJrMmFnNkZRbEJKcFR2eE05U3J0VC9sRVE3L1pFOWxGNW0xMmFmaE11MExUWkw2dHVyZFUKUUtUNVlkZmVmZUJOcWovK1ZYYmMyZEI0U0Z0NDdScFNtNGFmTHNzazhLcUs4WFgwdmp3RVZNVTRHT3M2SVFkTAoxS3FrM2tVa0QyWTRTcGhZTDNhSWZxTXd2TnBweTFPYm13TnEzNEQxeWJRRjlSRlRCMmxVd0hMNmxGM1NqTkUrClNCV2o2c0FtcnMyNTRXT3g5bThmNUpmbHZ0MXhjVzJQdnZKZE91MXR2cUVRVmEyR2QzTDErbzZWYmNnZm1jekwKTzhsTUdWNEpLT2kyZXpJdWkvQm42bExUYlhwN1V3ZzdOKzgza1FJTVRzUUtORUZMQTQwTUQvTjRjZzdKYlB2Tgp0cUVDZ2dFQkFNK1Y0VGp3N2xTblQzMUdyNzl6RFQ2bnIrYytOVE96VnBxNG4zcTVnbk5SUklKamN1NkxPRXhvCmRhSVFWZlcxU0hGUVowcXFKYXJoZmZlRnQ0OVd5SW5lcHY4TTJWNmFDMkFZdm1qazhuMHlHcUFqOE1zLzVBTTQKQ1lKRDJ1Tm1EZDI5TkxLTEdFbzNaNFE0c1MrMVY3aXRtbGw5V3lYQWNiUStienAvbEhaa0ZNNzFNUGtuN0ErcwpmOHhUekhYYVR5YjQ2b3pBMmxyQmVFQ2QrMGdyaVZLZG51V2pqT0Z5YldldHIyTXcyR05yWEF6RHpCa21Kc1JTCnhmbVFVSGw2OXlSc0M4Q05Hd3k3VGJxQ3gwOEJ5ZHAzWG9yR1ZyakpRak95Y0p2ODErQkl3V2YrcUp2THlFRWgKbTgvQjN4RkUyTkNITHRmVEtKcVFabDZEQkhXK2xBaz0KLS0tLS1FTkQgUFJJVkFURSBLRVktLS0tLQo=
kind: Secret
metadata:
creationTimestamp: "2024-12-19T15:56:35Z"
name: self-tls
namespace: default
type: kubernetes.io/tls
Step 3: Apply the secret-tls to Kubernetes Cluster
Save the Yaml in Step 2 locally, then apply it to the cluster.
kubectl apply -f secret-tls.yml
You are done! You know have a secure connection betweenn your client gRPC application and the TML solution.
Using gRPcurl to Write Data to the TML gRPC Server
gRPcurl is a utility for writing data to your gRPC solution.
Tip
You can install gRPCurl from here.
Important
You must download four (4) files to your local machines:
Run the gRPCurl Commands
Once your TML solution using gRPC is running in Kubernetes you can test it by sending data with these commands:
grpcurl -insecure -H "client-api-protocol: 1,1" -cacert ca.crt -import-path . -proto tml_grpc.proto tml.tss:443 list tmlproto.Tmlproto
grpcurl -insecure -H "client-api-protocol: 1,1" -cacert ca.crt -import-path . -proto tml_grpc.proto tml.tss:443 list
grpcurl -insecure -H "client-api-protocol: 1,1" -cacert ca.crt -import-path . -proto tml_grpc.proto tml.tss:443 describe tmlproto.Tmlproto.GetServerResponse
grpcurl -insecure -H "client-api-protocol: 1,1" -cacert ca.crt -import-path . -proto tml_grpc.proto tml.tss:443 describe .tmlproto.Message
grpcurl -insecure -H "client-api-protocol: 1,1" -cacert ca.crt -import-path . -proto tml_grpc.proto -msg-template tml.tss:443 describe .tmlproto.Message
Send data to the sever:
grpcurl -insecure -H "client-api-protocol: 1,1" -cacert ca.crt -import-path . -proto tml_grpc.proto -d '{"message":"admin yeah!!"}' tml.tss:443 tmlproto.Tmlproto/GetServerResponse
Kubernetes Pod Access Commands
To go inside the pods, you can type command:
kubectl exec -it <pod name> -- bash
Note: replace <pod name> with actual pod name..use this command to get the pod name
kubectl get pods -A
To list service pods type:
kubectl get svc -A
To list deployment pods type:
kubectl get deployments -A
To Horizontally AUTO-SCALE Deployments type:
kubectl autoscale deployment <deployment name> --cpu-percent=50 --min=1 --max=100
Important
The above command instructs Kubernetes to scale pods based on 50% CPU utilization to a minimum number of pods of 1 (small workload) to a maximum of 100 pods for large world loads. Of course, you can easily change these min and max numbers.
This auto-scaling is very important to scale up and down your solution, while efficiently managing cloud computing costs.
To list deployments being auto-scaled type:
kubectl get hpa -A
To delete the pods:
kubectl delete all --all --all-namespaces
To get information on a pod type:
kubectl describe pod <pod name>
Start minikube with NVIDIA GPU Access:
minikube start --driver docker --container-runtime docker --gpus all --cni calico --memory 8192
Note
Note you may need to type: ./minikube
Start minikube with NO GPU:
minikube start --driver docker --container-runtime docker --cni calico --memory 8192
DELETE minikube:
minikube delete
Tip
Adjust the --memory 8192 as needed.