Presentation on theme: "Best Practices for Optimizing Blackboard Learn Steve Feldman,"— Presentation transcript:
Best Practices for Optimizing Blackboard Learn Steve Feldman,
What Well Cover A deployment approach for the ages. How to make use of the new sizing guide. Optimizing the platform components.
Flexible and Scalable Application Deployment
An ideal deployment will contain… –Availability at every edge of the application environment Strategy: Physical distribution of load-balanced systems Strategy: Minimum DB recovery, not necessarily 0 downtime –Consumption of every possible machine resource Strategy: Virtualization provisioning –Techniques for improving user experience Strategy: Techniques and tools for achieving page-level SLAs –Large addressable memory spaces Strategy: 64-bit and large OS process space allocations
Flexible and Scalable Application Deployment An ideal deployment will contain… –Minimum Storage Recovery Time Strategy: Enterprise storage with Snapshot capabilities –Advanced monitoring for operations and planning Strategy: Measurement tools and analytics –Automation…Automation…Automation Strategy: Investment in repeatable, reliable automated processes.
Deployment: Availability VLEs are different beasts today then in the past. –Communities are bigger –Sessions last longer –Content is richer –Key point: Adoption is greater and users expect their sites up 24 x 7 x 365 Architecture is designed for many parallel instances of the product scaled in a horizontal fashion. –Distributed physical deployments –Virtualization is a key element Database failover more important than horizontal database scalability. –Emphasis on vertical database scalability
Deployment: Resource Utilization Moores law is in full effect –CPUs are getting faster with more cores –Memory is in abundance and cheap –Storage is grossly abundant Massive systems can be obtained at low cost, but cannot be saturated in stand-alone configurations. Virtualization offers the opportunity… –Deploy with availability in mind –Saturate system resources
Deployment: Improving Page Responsiveness Gzip…Gzip…Gzip… –All of our supported browsers handle gzip? –Reduces payload Improves lower latency connections like Cable, DSL and Dial-up –Minor overhead on the application layer (~2% to ~5%) Have the option to perform at the load-balancer layer –Most Bb deployments do not enable Gzip at all Even when enabled, some proxies and software packages mess-up the Accept Encoding Header Optimize your images –Page size really does matter –Reduce the size without reducing the quality
Deployment: Large Address Space As of Blackboard Learn Release 9.1 all supported/certified configurations include a 64-bit option. Pushing more processing to client and DB over the last few releases, but major memory management technique is to use more application caches. –Memory stays persistent longer –Less wasteful from a creation/destruction perspective, but puts greater demands on larger spaces. Most of our application testing focused on 4GB and 8GB JVM deployments on 6GB and 10GB OS spaces. –Limited testing at 16GB and 32GB
Deployment: Storage MTTR Reference architecture pushes for diskless boots in which ISCSI or NFS partition resides on an enterprise storage system. Both OS/VM partition and data partition served up from remote storage deployment designed for performance and scalability. –Make your hardware work from a CPU, Memory and Network perspective…save the Disk for the experts. Consider scenarios for reducing Mean Time to Recovery or Repair –Snapshot technology offering minutes for recovery
Deployment: Advanced Monitoring Measurement is the secret sauce for successful deployments. –Most reliable and scalable deployments measure beyond the server infrastructure Different types of measurements –System/Environmental measurements –Business measurements –Synthetic measurements Collecting is only part of the prize –Need to analyze the data to drive business decisions from the data.
Deployment: Automation Goal of moving to 100% unattended and fully automated deployment. Reduce MTTR and prevent disasters Automation requires intimacy…intimacy requires knowledge Use automation for –Configuration Management and Deployment –Maintenance –Repeatable tasks –Adaptive tuning –Minimize possibility of human error isioning_Whitepaper.pdfhttp://dev2ops.org/storage/downloads/FullyAutomatedProv isioning_Whitepaper.pdf
Sizing the Application: To Hyper- Thread or Not Applies to Intel deployments only –..delivers thread-level parallelism on each processor resulting in more efficient use of processor resourceshigher processing throughputand improved performance on multi-threaded software. –Intel Corporation Greatly improved in series processor Provides double worker thread capacity If its not turned on, stop what you are doing and enable it ASAP!
Moving Away from Clusters Tomcat clusters were introduced back in Blackboard Learn 7.X prior to the transcendence of server virtualization. –Only supported 32-bit configurations at the time, but systems were being shipped with 8GB, 16GB and 32+GB of RAM. –Needed a way to take advantage of memory, but were limited to a 1.7GB address space. –Recommending distributed deployment approaches as well. Still applies, but can be achieved differently. Clustering has its advantages, but also has its penalties. –Failover not as ideal as one would desire. Best approach is to scale up with 64-bit spaces and distributed JVMs across both virtual and physical configs
Sizing Using P.A.Rs PAR = Performance Archetype Ratios –Methodology for sizing based on units of work that can be applied to unit of configuration PARs assume a world of linear units –Add units of configuration to meet growing demands of unit of work. PARs based on (4) key resources: CPU, Memory, Disk and I/O and application interfaces (threads and connections). Used for making capacity decisions for sizing both virtual and physical components.
Optimizing the Web Server The web server in the Blackboard Learn configuration is nothing more than a gateway to the application container. –When clusters were more relevent, the web server acted as a pseudo load-balancer. Not many opportunities for optimization other than –KeepAlives –Interfaces –Compression It can become a bottleneck if not properly optimized –Better to have high ceilings from an interface perspective
Optimizing the JVM Java hotspot offers standard –X and non-standard –XX options for performance and behavior. –-X options are always guaranteed across releases and patches of Java. –-XX options must be used with caution as they are subject to change with any release of Java. -XX options should be tested and measured using the production safe arguments. Read the release notes of Java for performance updates –http://java.sun.com/javase/6/webnotes/ReleaseNotes.htmlhttp://java.sun.com/javase/6/webnotes/ReleaseNotes.html
Optimizing the JVM Cross-platform recommendation for using Concurrent Mark Sweep Collector –Best optimized for 64-bit address –Combine –XX:+UseConcMarkSweepGCwith –XX:+UseParNewGC Manually size New Space using –XX:NewSize and – XX:MaxNewSize options (1/4 to 1/3 total heap). –Consider Survivor Space ratios 4 or lower. Be careful about sharing –XX non-standard options across customers. –If you dont understand what the option does and its not recommended by Blackboard, best choice is to not use it.
Optimizing the Database: SQL Server # of data files makes no difference on SQL Server for Data and Transaction Allow the data/transaction files to grow as big as they want within reason. –Whats reason: 64GB –http://msdn.microsoft.com/en-us/library/ms143432(sql.90).aspxhttp://msdn.microsoft.com/en-us/library/ms143432(sql.90).aspx TempDB is completely different story –# of files = # of DB Threads –Set first X files to a uniform size, set last file to same size with auto-extension ON –Determine size need over time Separate volume for paging file
Optimizing the Database: SQL Server Be aware of MDOP: Max Degree of Parallelism –Setting to unlimited can have a negative affect on query performance unintentionally. AWE can and does work on 64-bit systems Configure READ_COMMITTED_SNAPSHOT Two nuggets of information: –Learn How to Use SQL DMVs –Study SQL Server Wait Events and Tuning
Optimizing the Database: Oracle Balance I/Os across multiple data files (~2 to 8GB per file). REDO is critical to performance a session/query level. –Be aware of how much REDO is being used over time. –NOLOGGING will disable, be we rarely use NOLOGGING TEMP is very complex and used for managing transient data. –One TEMP file is adequate –If latency exists on TEMP, consider introducing TEMP file groups SGA is important, but PGA can be your best friend or your worst enemy with high concurrency.
Optimizing the Database: Oracle Oracle DBO can be your friend –Must understand optimizer behavior –Willingness to read Cost Execution Plans Using Wait Events and Cost Execution Plans for tuning initiatives –Wait events are at a system, session and query level Importance of Statistics and Histograms –CBO is just guessing without properly set statistics and histograms. –CBO is dependent on your data.
Please provide feedback for this session by ing The subject of the should be title of this session: [INSERT TITLE HERE]