Preface
As a cloud company, we usually build our container cluster on cloud platforms such as GCP or AWS. but sometimes we need to think about how to take to go solution to our customer. So, here comes a challenge - Scale-in our cloud platform and put into pocket (mmm I mean “minikube”)
An example GKE service
Here is a simple example which you (might) build in GKE.
DB: MongoDB with stateful set binding with google persistent disk. Use stateful set from (“Running MongoDB on Kubernetes with StatefulSets”) as an exampleWeb service(fooBar): a golang application which accesses mongo DB with the load balancer. Because fooBar is proprietary application which fooBar image store in GCR (Google Cloud Registry) not in docker hub.
How to migrate your service from GKE to minikube:
We will just list some note to let you know any tip or note you migrate your service to minikube.
1. Migrate Persistent Volume to minikube.
“Here” is a good article to understand StatefulSet. But however it use some resource you could not use in minikube, such as:
Let me put all yaml setting and give more detail.
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: mongo
spec:
serviceName: "mongo"
replicas: 3
template:
metadata:
labels:
role: mongo
environment: test
spec:
terminationGracePeriodSeconds: 10
containers:
- name: mongo
image: mongo
command:
- mongod
- "--replSet"
- rs0
- "--smallfiles"
- "--noprealloc"
ports:
- containerPort: 27017
volumeMounts:
- name: mongo-persistent-storage
mountPath: /data/db
- name: mongo-sidecar
image: cvallance/mongo-k8s-sidecar
env:
- name: MONGO_SIDECAR_POD_LABELS
value: "role=mongo,environment=test"
volumeClaimTemplates:
- metadata:
name: mongo-persistent-storage
annotations:
volume.beta.kubernetes.io/storage-class: "fast"
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 100Gi
The volumeClaimTemplates provide stable storage using PersistentVolumes provisioned by a PersistentVolume Provisioner. (Use “Google Persistent Disk” on GCP)
The “fast” will specific to use google persistent disk with “SSD” performance. So, in this case our Kubernetes volume claim will specific to google cloud persistent disk.
How we migrate to Google Persistent Disk volume into minikube?
There are two solutions for this:
- Use
hostPath directly.
- Still using volume claim but remove google persistent disk annotation.
In the second case, we could still use volumeClaimTemplates and just remove volume provider persistent disk annotation. It will select the best for our system. (for now, it is hostPath.
volumeClaimTemplates:
- metadata:
name: minikube-clain
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 10Gi
2. Using GCR in minikube
When you deploy GKE, you usually use GCR (Google Cloud Registry) to storage your docker container. If you want to move your project from GKE to minikube, you have solutions as follows:
- Build your own docker registry. But you will need handle follow things:
- Use
--insecure-registry if you use the self-signed key.
- Use public DNS to request CA key (It will against we want to move into minikube)
- Use minikube to connect to GCR directly.
- Minikube has add-on
minikube addons configure registry-creds
- Use temproary token solution (around ~30 mins), here is the reference.
kubectl delete secret gcr
kubectl create secret docker-registry gcr \
--docker-server=https://asia.gcr.io \
--docker-username=oauth2accesstoken \
--docker-password="$(gcloud auth print-access-token)" \
[email protected]
Here is the document from minikube.
Launch your service
Ok, that’s done. If you are wondering how to connect to your web service fooBar. You can just call minikube service fooBar.
Troubleshooting
using busybox to debug
It is very useful when you don’t know what’s happening on your pod, especially for networks storage or network.
cat <<EOF | kubectl create -f -
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: busybox
spec:
selector:
matchLabels:
app: busybox
replicas: 1
template:
metadata:
labels:
app: busybox
spec:
containers:
- name: busybox
image: busybox
command:
- sleep
- "86400"
EOF
kubectl exec -it <BUSYBOX_POD_NAME> -- nslookup redis.default
example:
kubectl exec -it busybox-7fc4f6df6-fhxnp -- nslookup redis.default
Destroy & Cleanup the minikube
Remember to remove the setting file when you uninstall minikube.
minikube delete
rm -rf ~/.minikube
Cannot start minikube
It might happen as follow reasons:
- minikube version change.
- docker version change.
- VM has already been destroyed by VirtualBox.
Starting local Kubernetes v1.8.0 cluster...
Starting VM...
E1124 22:16:44.080926 87887 start.go:150] Error starting host: Temporary Error: Error configuring auth on host: Temporary Error: ssh command error:
Solution: rm -rf ~/.minikube
