安裝 Mesos 在本地端透過 Vagrant 來安裝 Mesos 可以參考 Install Mesos via Vagrant． 心得: 雖然官方文件相當的清楚，但是 Vagrant 本身就有一些雷要踩．不論是 Vagrant SSH key insert failed. Vagrant Shared files between node Vagrant node IP 亂跑的問題 加上如果沒注意到記憶體跟 CPU 極有可能在 Mesos 裡面會出現無法 Scale Task 的狀況． 關於雷的部分，講明白點…. vagrant 的 private IP 不是每次 vagrant up 都會正確，經常會跑掉．得要 vagrant restart． vagrant 彼此間要共享檔案，可以透過 /vagrant 這個資料夾．不過 /vagrant 只是 guest OS 跟 host OS 溝通的共享資料夾，如果兩個 guest OS 要共享，還是得透過 host OS． 然而，共享並不是 real time 而是啟動的時候由 host 帶過去到 guest 的 /vagrant，如果要複製 guest file 到 host 是可以即時，如果要另外一個 guest 拿到，就得要重啟 guest． 安裝 Mesos CLI 流程 先安裝 pip curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py sudo python get-pip.py 在來安裝 virtualenv sudo pip install virtualenv 安裝 mesos.cli sudo pip install mesos.cli 在 GCE (Google Compute Engine) 安裝 DC/OS 參考這一篇 幾個東西要注意一下: 記得要先登入 Google Cloud 帳戶 gcloud auth login 在修改 https://github.com/dcos-labs/dcos-gce 安裝設定的 group_vars/all 的時候，記得以下資料: bootstrap_public_ip: 要參考原先設定 bootstrap 的 IP．必須確認該 IP 的格式與必須要確認一開始的 bootstrap 機器跟你要建立的 master 與 agent 是同個 subnet． subnet: 必須要設定一個新的 subnet (不能使用 default) 一定得建立一個新的 subnet ．...

Difference between Chubby and Zookeeper Consensus Algorithm: Chubby: Using Paxos consensus algorithm Zookeeper: Using ZAB (which is a modified algorithm of Paxos) Access Path: Chubby: Must through leader, follower don’t accept any command directly. Zookeeper: Can accept any command from all follower, but will go back to leader. Data Out-Of-Date: Chubby: No rish on out-of-date, because all data fro Leader. Zookeeper: Will get out-of-date data, if read without sync command. Sync command will force ask follower ask Leader first before return result to client. The tolerance of Quorum-Backup and Primary-Backup: Majority Quorums: For Paxos or ZAB, A leader election need meet majority quorum ( n/2 +1 ) . It means if n=2f at most the service can tolerance f service failed at the same time. ( Quorum-Backup Replica ). Primary-Backup All data must confirm by all follower confirmed. Much slower than “Quorum-Backup”. Kafka ISR (In-Sync Replica) In-Sync Replica is a...

使用 Google Kubernetes Engine (Container Enginer) 由於剛學完 Udacity 上面的 “Scalable Microservices with Kubernetes” 接下來的例子主要是課堂上有教的部分． 先使用 Container Enginer 來建立一個新的 Container Cluster (P.S. 你需要記住你的 GKE 名稱 (ex: mygke)) 透過 kubectl 來連接你的 Cotainer Cluster gcloud container clusters get-credentials mygke 建立一個具有 TLS 的 nginx 連線 建立 TLS 確認你已經有 key ls tls > ca-key.pem ca.pem cert.pem key.pem 使用 kubectl 建立一個 tls secret kubectl create secret generic tls-certs --from-file=tls/ > secret "tls-certs" created 顯示 tls secret 的內容 kubectl describe secrets tls-certs > Name: tls-certs > Namespace: default > Labels: <none> > Annotations: <none> > > Type: Opaque > > Data > ==== > ca-key.pem: 1679 bytes > ca.pem: 1180 bytes > cert.pem: 1249 bytes > key.pem: 1675 bytes 建立一個給 nginx proxu 的 config map kubectl create configmap nginx-proxy-conf --from-file=nginx/proxy.conf > configmap "nginx-proxy-conf" created 詳細了解 nginx-proxy-conf configmap kubectl describe configmap nginx-proxy-conf > Name: nginx-proxy-conf > Namespace: default > Labels: <none> > Annotations: <none> > Data > ====...

miniKube 單機版的 Kubernetes miniKube 是 Google 發布可以在單機上面跑 Kubernetes 的工具，安裝跟使用都相當簡單．由於會在本地跑一個 VM ，所以也不用擔心會被 Google Cloud 不小心付費的問題． 安裝 curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.6.0/minikube-darwin-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/ 啟動 minikube start 連接進該 VM minikube ssh 打開並且顯示 minikube 的 dashboard minikube dashboard 接下來，安裝 Kubernetes 安裝 Google Cloud SDK curl https://sdk.cloud.google.com | bash 安裝好之後，會有 gcloud, gsutil 但是還需要安裝 kubectl 透過 Google Cloud SDK 安裝 Kubernetes gcloud components install kubectl 簡單 Tutorial # Startup miniKube minikube start # Create a deployment kubectl run hello-minikube --image=gcr.io/google_containers/echoserver:1.4 --port=8080 # Expose it kubectl expose deployment hello-minikube --type=NodePort # Check pod kubectl get pod # Scale kubectl scale deployment hello-minikube --replicas=4 # Direct link to hello-minikube curl $(minikube service hello-minikube --url) How to Rolling Update on Kubernetes kubectl edit deployment hello-minikube Update your service version and apply it (save it back). You can also update this file via minikube dashboard Edit “deployment” Update “spec”->”spec”-> “image” version. Kubernetes 與...

Start Docker Swarm Mode Docker Swarm Mode is specific for Docker Swarm Version 2 which only enable after Docker 1.12. It is cluster management system for Docker. Install Docker-Machine: Better to prepare at least 3 docker-machine for swarm node. If you use Docker for Mac or Docker for Windows beta, you still need install `docker-machine. Install docker-machine: curl -L https://github.com/docker/machine/releases/download/v0.7.0/docker-machine-`uname -s`-`uname -m` > /usr/local/bin/docker-machine && \ chmod +x /usr/local/bin/docker-machine Run Docker Swarm Start to create 3 docker machine for cluster management. docker-machine create --driver virtualbox v1 docker-machine create --driver virtualbox v2 docker-machine create --driver virtualbox v3 Init cluster leader in v1 docker-machine ssh v1 For example if your v1 node IP address is 192.168.99.110. docker swarm init --listen-addr 192.168.99.110:2377 --advertise-addr 192.168.99.110 Init Other Worker Node v2, v3 Let v2, v3 join cluster as nodes. Login and Control v2. docker-machine ssh v2 docker swarm join --token SWMTKN-1-3h0ndq6j0agkl1inb7sd9gnrk1va4e0sggw74jsaj7xkx75c7n-31coul06qcdb7g411ww8jnurw 192.168.99.110:2377 > This node...

Keyspace keyspace == database in SQL CREATE KEYSPACE KEYSPACE_NAME WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }; then use KEYSPACE_NAME; in cqlsh before you create/alter/drop/select tables. Special Data Operation in Cassandra (Set/List/Map/Nested Type/Frozen/Tuple/JSON) Set data Table schema for set. CREATE TABLE users ( user_id text PRIMARY KEY, first_name text, last_name text, emails set<text> ); Insert data into table INSERT INTO users (user_id, first_name, last_name, emails) VALUES('frodo', 'Frodo', 'Baggins', {'[email protected]', '[email protected]'}); List data Table schema CREATE TABLE users ( user_id text PRIMARY KEY, first_name text, last_name text, top_places list<text> ); Data Insertion. INSERT INTO users (user_id, first_name, last_name, emails) VALUES('frodo', 'Frodo', 'Baggins', [ 'rivendell', 'rohan' ]); Map Data Table schema CREATE TABLE users ( user_id text PRIMARY KEY, first_name text, last_name text, todo map<timestamp, text>; ); Data Insertion. INSERT INTO users (user_id, first_name, last_name, emails) VALUES('frodo', 'Frodo', 'Baggins', { '2012-9-24' : 'enter mordor', '2014-10-2 12:00' : 'throw ring...