강호영

2 Contents ZooKeeper Overview ZooKeeper’s Performance ZooKeeper’s Reliability ZooKeeper’s Architecture Running Replicated ZooKeeper ZooKeeper C Client Library Pilot Project: StreamMQ

3 ZooKeeper Overview (1/3) ZooKeeper? – A high-performance coordination service for distributed applications (naming, configuration management, synchronization, and group Services) – Used to implement consensus, group management, leader election, and presence protocols – Runs in Java and has bindings for both Java and C Features of ZooKeeper – Shared hierarchical namespace: consists of znodes (data registers) in memory – High performance: can be used in large, distributed systems – Reliability: keeps from being a single point of failure – Strict ordered access: sophisticated synchronization primitives can be implemented at the client (Zookeeper stamps each update with a number) – Replication: replicated itself over a sets of hosts called an ensemble

4 ZooKeeper Overview (2/3) Hierarchical Namespace: znode – name: a sequence of path elements separated by “/” – In znodes, stores coordination data: status information, configuration, location information, and etc. – znodes: maintain a stat structure (version numbers for data changes, ACL changes, and timestamps.) – ephemeral nodes: exists as long as the session that created the znode is active. (when the session ends, the znode is deleted.) ※ Data is kept in memory for good performance. Watches – If client sets a watch on a znodes, the watch will be triggered so that the client can receive a packet saying that the znode has changed when it changes.

5 ZooKeeper Overview (3/3) Guarantees – Sequential Consistency: Updates from a client will be applied in the order that they were sent. – Atomicity: Updates either succeed or fail. No partial results. – Single System Image: A Client will see the same view of the service regardless of the server that it connects to. – Reliability: Once an update has been applied, it will persist from that time forward until a client overwrites the update. – Timeliness: The clients view of the system is guaranteed to be up-to- date within a certain time bound. Simple API – create, delete, exists, get data, set data, get children, sync ( ※ sync: waits for data to be propagated.)

6 ZooKeeper’s Performance ZooKeeper is especially high performance in applications where reads outnumber writes, since writes involve synchronizing the state of all servers. ZooKeeper 3.2 Dual 2GHz Xeon Two SATA 15K RPM drivers - One drive was used as a dedicated ZooKeeper log device - The snapshots were written to the OS Drive Requests Size - Write requests: 1K writes - Read requests: 1K reads The ZooKeeper ensemble was configured such that leaders do not allow connections from clients 매우 빠른 속도로 동작하지만, 서버 수가 증가함에 따라 성능은 저하됨

7 ZooKeeper’s Reliability If followers fail and recover quickly. The leader election algorithm allows for the system to recover fast enough to prevent throughput from dropping substantially. As followers recover, ZooKeeper is able to raise throughput again once they start processing requests. Same situation benchmark as before Run a ZooKeeper service made up of 7 machines Keep the write percentage at a constant 30% The events marked in the figure ① Failure and recovery of a follower ② Failure and recovery of a different follower ③ Failure of the leader ④ Failure and recovery of two followers ⑤ Failure of another leader

8 ZooKeeper’s Architecture A ZooKeeper service is replicated over a collection of servers called an Ensemble. A client interacts directly with a single server at a time over a TCP connection. If the server fails, the client initiates a new TCP connection with another server.

9 Running Replicated ZooKeeper (1/2) Run a ZooKeeper service made up of 3 machines. (One Leader, Two Followers) ① Configure host file (/etc/hosts) : in all machines zookeeper1 leader zookeeper2 follower zookeeper3 follower2 ② Install zookeeper : in zookeeper1 machine wget tar xvfzp zookeeper tar.gz sudo mv zookeeper /usr/local/zookeeper cd /usr/local/zookeeper cp –rp zoo_sample.cfg zoo.cfg ③ Configure zoo.cfg : in zookeeper1 machine sudo vi zoo.cfg dataDir=/home/csos/storm/zk_data server.1= server.2= server.3= Follower Leader Follower ZooKeeper Service Ensemble

10 Running Replicated ZooKeeper (2/2) ④ Copy installation file to other servers: in zookeeper1 machine cd /usr/local tar cvfpz zookeeper.tar.gz./zookeeper copy to zookeeper2: scp –rp zookeeper.tar.gz zookeeper2:/usr/local copy to zookeeper3: scp –rp zookeeper.tar.gz zookeeper3:/usr/local ⑤ Extract installation file: in zookeeper2, 3 machines tar xvfzp zookeeper.tar.gz ⑥ Configure PATH: in all machines sudo vi /etc/profile export ZOOKEEPER_HOME=/usr/local/zookeeper export PATH=$PATH:$ZOOKEEPER_HOME/bin source /etc/profile ⑦ Configure zk_data/myid: all machines mkdir -p /home/csos/storm/zk_data cd /home/csos/storm/zk_data vi myid do typing “1”, “2” or “3” following server number written in zoo.cfg ⑧ Run ZooKeeper: all machines zkServer.sh start

11 ZooKeeper C Client Library (1/4) ZooKeeper C Client Library Overview – Provides a C Client Interface to ZooKeeper server. (C Binding Library) – API types: synchronous and asynchronous Synchronous API: provides a blocking flavor of zookeeper operations and runs its own event loop in a separate thread Asynchronous API: provides non-blocking operations with completion callbacks and relies on the application to implement event multiplexing on its behalf ( ※ Sync and Async APIs can be mixed and matched within the same application) – Libraries: zookeeper_st and zookeeper_mt zookeeper_st: provides the Async API and is NOT thread-safe. zookeeper_mt: support for both Sync and Async API.

12 ZooKeeper C Client Library (2/4) Client API APIContents Update/ Write Operation Create(znode, data, flags) znode: full path flag: REGULAR, EPHEMERAL setData(znode, data, version) Rewrites znode with data, if version is the current version number of the znode sync() Waits for all updates pending at the start of the operation to be propagated to the Zookeeper server that the client is connected to delete(znode, version) Delete the znode if the version is equal to the actual version of the node Read Operation exists(znode, watch) Returns true if the znode exists, false otherwise getData(znode, watch) Returns data stored at this znode getChildren(znode, watch) Returns the set of children znodes of the znode ※ watch: is a subscription to receive an information from the ZooKeeper when the znode is changed.

13 ZooKeeper C Client Library (3/4) Installation zookeeper C client library ① wget ② tar xvfzp zookeeper tar.gz ③ cd zookeeper-3.4.5/src/c ④ autoreconf –if ⑤./configure ⑥ make ⑦ make install Troubleshooting: fix compile error – Error: bad instruction `lock xaddl r4,r3,#0 – Solution: use alternative system function in mt_adaptor.c file

14 ZooKeeper C Client Library (4/4) Programming using the zookeeper API – Include the zookeeper header: #include – Use –DTHREADED compiler option to enable Sync API Single-threaded client: link zookeeper_st library with not using -DTHREADED Multithreaded client: link zookeeper_mt library with using –DTHREADED – Use the ‘-Wl,-rpath -Wl,LIBDIR’ linker flag to link installed libraries. Example source: cli_mt, cli_st where zookeeper/src/c/src – cli_mt: multithreaded, built against zookeeper_mt library – cli_st: singlethreaded, built against zookeeper_st library ※ Take a look at cli.c to understand how to use the Sync/Async API types.

15 Pilot Project: StramMQ (1/2) Tentatively named “StreamMQ” – StreamMQ aims to provide a way to process real-time stream signals from and to SoSp(Self-organizing Software-platform) devices through SoSp cluster network using ZeroMQ and ZooKeeper Technologies. – StreamMQ has to provide high speed real-time stream messaging services to SoSp mobile subscribers independent of their location and device without any interruption over wireless/wired heterogeneous networks. Use Cases – Ability to simulate stream signals to provide heavy load to network. – Ability to start/stop/pause/resume to process real-time stream signals. – Ability to change a source stream of SoSp devices at runtime. – Ability to save in SoSp Router and replay stream signals. (Construct distributed personal health record storage using NoSQL) – Ability to manage coordination service in ZooKeeper. (Construct private federation cloud to accept according to physical location) – Initiating processing of a set/batch of data based on a schedule – Periodically sending a temporary data to SoSp device

16 Pilot Project: StramMQ (2/2) Modules NameStatusRole & Responsibility 1StreamSignalsOngoing Simulates stream signals. (e.g. sine-curve) 2StreamProcessorOngoing Start/stop/pause/resume to process real- time stream signals. Changes a source stream of SoSp Device. Saves stream in SoSp Router. 3StreamSubscriberOngoing Receives stream signals independent of their location and device 4StreamManagerNot started yet Manages coordination service in ZooKeeper 5StreamSchedulerNot started yet Initiates processing of a set/batch of data based on a schedule 6StreamPeriodicSignalsNot started yet Periodically sends a temporary data to SoSp device through SoSp Routers

17 Thank you