Difference between Chubby and Zookeeper

  • Consensus Algorithm:
    • Chubby: Using Paxos consensus algorithm
    • Zookeeper: Using ZAB (which is a modified algorithm of Paxos)
  • Access Path:
    • Chubby: Must through leader, follower don’t accept any command directly.
    • Zookeeper: Can accept any command from all follower, but will go back to leader.
  • Data Out-Of-Date:
    • Chubby: No rish on out-of-date, because all data fro Leader.
    • Zookeeper: Will get out-of-date data, if read without sync command. Sync command will force ask follower ask Leader first before return result to client.

The tolerance of Quorum-Backup and Primary-Backup:

Majority Quorums:

For Paxos or ZAB, A leader election need meet majority quorum ( n/2 +1 ) . It means if n=2f at most the service can tolerance f service failed at the same time. ( Quorum-Backup Replica ).

Primary-Backup

  • All data must confirm by all follower confirmed.
  • Much slower than “Quorum-Backup”.

Kafka ISR (In-Sync Replica)

In-Sync Replica is a backup policy which similar with Primary-Backup but in sychronous replication.

Server store two copy of replica data:

  • ISR (The latest one) store in Leader.
  • Follower store replica data might later than ISR.
  • Use Zookeeper to update all follower replica data from ISR.

In this case, if total server is n=f+1, we could tolerance number of f service failed. ( Primary-Backup replica )

trade-off:

  • Follower data critical late than leader, but in Kafka this case might be acceptable.
  • If follower cannot catch up data from leader, will drop ISR by leader.
  • If failed on leader, it has high possibility lost data if next leader. (Choose leader from non-ISR list, because all follower in ISR leave behind.)
  • Use Zookeeper to store ISR will introduce split brain issue. (two majority group issue).

Refer to Kafka ISR

Reference


Evan

Attitude is everything