1. IST346: Scalability

2. Agenda
Discussion
Content
Scalability
Wrap-Up

3. Discussion Questions
Explain Master / Slave architecture for redundancy.
What is a hot spare? Cold Spare?
What is a load balancer?
Define scaling for performance vs scaling for resilience?
What is the difference between latency and bandwidth?
What are the two ways one can test that a service can scale?
Give students 1-2 minutes for each question. Its important to provide them with the opportunity to answer the questions based on the reading, labs, and assigned homework.
6. Load testing (black box) and dataflow analysis (white box)

4. Lab Debrief
Lab G

5. What is Scalability?
The capability of a system to handle an increased amount of work.
We see this all the time:
Doing laundry at home versus the laundromat
Cooking for friends versus catering a party for 500 people
Walking to work versus riding a bike versus driving a car
Having friends help you with your wedding invitations versus doing them yourself.

6. Scaling Services: How do you address growth?
Vertical “Scale Up”
Horizontal “Scale Out”
Add more resources to an existing system running the service or split across layers.
Easier, but limited scale.
Single point of failure
Run the service over multiple systems, and orchestrate communication between them.
Harder, but massive scale.
Overhead to manage nodes.
The last thing to discuss before we get into devops is how services are scaled.
[CLICK, READ]
To be compatible with the devOps universe we need to build apps that scale out. More on this as we cover devOps next.

7. Trivial Vertical Scaling
Buy your way out of the problem.
Buy a faster computer
Invest in a faster internet connection
We do this all the time with our phones, etc…
Limited based on available technology.

8. Vertical Scale Through Layering
We re-design / program the application to support multiple users by breaking up / splitting the layers.
Presentation
Business Logic
Clients
Presentation
Business Logic
Data Access
Data
Network + Middleware
Scale Up
Monolithic
We scale up data-oriented applications by separating the layers of the application into their own independent services which communicate over the network.
This allows more users to access the application, yet introduces some design complexity.
ADVANCE SLIDE
When an application’s layers are segmented we must introduce middleware between the applications of each layer in order for them to communicate. For example we might use ODBC middleware so that the data access layer can communicate with the database itself. Or in the case of the diagram you see an ODATA REST API so that the business logic can communicate with the data access layer.
Data Access
Data
Server

9. Example Wordpress
We ran the Wordpress Blog/CMS in the lab as an N-tier application.
This was scaled Vertically across 3 containers.
We could have run the entire application from within a single container with all three services competing for resources!
Clients
Single System
Three Systems
NGINX +
WORDPRESS +
MYSQL
NGINX
WORDPRESS
MYSQL

10. Redundancy != Scalability
Redundancy is not scalability.
Scalability addresses performance of a system to do work
Redundancy addresses availability of a system to do work
Clients
Three Systems
NGINX
WORDPRESS
MYSQL
MYSQL (Replica)

11. Horizontal Scalability
Horizontal Scalability or Scaling out always involves sharing load among multiple instances of the same service.
Systems must be designed to scale this way
They also require a system to direct traffic to a specific instance (load balancer)
Clients
Load Balancer
Instance
Instance
Instance

12. Balancing Load Who does it? How it is done?
A Master service of the system to orchestrates load distribution (Hadoop, and most noSQL database systems work this way).
Another service orchestrates distribution. HAProxy, DNS, Zookeeper, etc.
Round Robin / Ring
Request-Based
Location-Based
Based on existing instance load.
Custom / Hybrid

13. Performance vs Scalability
Is the system slow for a single user?
You have a performance problem.
Is the system fast for an individual user, but slow under high load?
You have a scalability problem.

14. Check Yourself: Scaling SMTP
Brainstorm ways you can scale the SMTP (Simple mail transport Protocol – the service that sends ) at your large organization.
What might be a strategy to scale this service out and balance the load across 10 SMTP servers?

15. Group Activity
Scaling your life

16. Details of Group Activity
Divide into groups of 3
Come up with 3 activities you do every day then include solutions for how you can scale those activities to improve output.
Improvements can be time-saving or increasing volume / throughput.
For each solution, identify if it is scale up or scale out? Make sure you can explain / justify.

17. Your To-Do List What to work on for next class
ALWAYS CONSULT THE SYLLABUS
What to work on for next class

18. Exit Ticket
Share one thing you learned today that you didn’t know before class!

19. Questions?