Download presentation
Presentation is loading. Please wait.
1
DevOps And the Agile DBA
Kellyn Pot’Vin-Gorman | Technical Intelligence Manager | September, 2017
2
Kellyn Pot’Vin-Gorman
Technical Intelligence Manager, Delphix Multi-platform DBA, (Oracle, MSSQL, MySQL, Sybase, Postgres…..) Oracle ACE Director, (Alumni) Oak Table Network APEX Women in Technology Award, CTA 2014 STEM education with Raspberry Pi and Python President, Rocky Mtn Oracle User Group Liaison, Denver SQL Server User Group Author, blogger, (
3
DevOps DevOps is a short for DEVelopment and IT OPerationS. The term is used to refer to a set of practices that emphasize the collaboration and communication of both software developers and information technology (IT) professionals while automating the process of software delivery and infrastructure changes. It aims at establishing a culture and environment where building, testing, and releasing software can happen rapidly, frequently, and more reliably DevOps derives from both development and operations, groups that DBAs often have a foot in each of. There is a high focus on collaboration, geared on methodologies, process and practice. The goal is to release more frequently, more successfully and with less bugs.
4
How the DBA is Impacted by DevOps
Database Administrators heavily influence decisions. Adverse to bleeding edge, high risk. Introduction of the cloud, especially and SaaS, (Software as a Service). Demand for decreased development cycles introduce risk, DBAs are seen as a roadblock to this. Still, DBA skills are in high demand towards DevOps initiatives…
5
History of DevOps First called “Agile Infrastructure” in 2008
2009, the first time DevOps was coined in Belgium Has taken over most of the fellow terms of WinOps and in some cases, Agile. Introduced at relational database events in 2010, but rarely was given much credit until the last five years in the DBA community. Agile 2008 conference, Andrew Clay Shafer and Patrick Debois discussed "Agile Infrastructure” The term DevOps was popularized through a series of "devopsdays" starting in 2009 in Belgium
6
DevOps Relationship to Agile
Agile and DevOps aren’t one in the same, but as it’s well known, DevOps came out of Agile’s success. Agile= culture, where DevOps focuses more on the organization changes.
7
Don’t Confuse the Two… Agile’s success pushed the need for DevOps
Agile and DevOps have similarities, but Agile focuses more on culture and DevOps focuses on organizational changes to create success. DevOps uses the success of release implementation as a scoring method.
8
Common Terms, (Alphabetically)
Agile Build Automation Configuration Management Continuous Delivery/Rapid Deploy Functional Testing/Unit Testing/Test Driven Development Lean/Scrum Monitoring/Alerting Packages/Containers/Virtualization Release Coordination/Orchestration
9
Build Automation DBAs are a natural fit, due to their history with scripting and automation experience for backups, monitoring and alerting. Build automation is the process of automating the creation of a software build and the associated processes including: compiling computer source code into binary code, packaging binary code, and running automated tests.
10
Configuration Management
Concept & Development Testing & Verification Configuration management (CM) is a systems engineering process for establishing and maintaining consistency of a product's performance, functional, and physical attributes with its requirements, design, and operational information throughout its life. A DBA’s desire for low risk and stability assists here as we desire routine that results in expected outcomes. Operations & Support Deployment to Production
11
Continuous Delivery DBAs are much slower to embrace continuous delivery. The risk is high, with it’s “fail first, fail fast” approach. Continuous delivery (CD) is a software engineering approach in which teams produce software ... incremental updates to applications in production. A straightforward and repeatable deployment process is important for continuous delivery.
12
All Tiers of Specific Environment
Rapid Deploy Software Content Enablement Service All Tiers of Specific Environment Need or Focus Area This is another area that introduces risk, so DBAs can be very adverse to it, but the focus on a single feature, helps minimize the impact and often isolate any issues.
13
Release Coordination/Orchestration
A demand for release tools and orchestration was at times a direct result to DBAs hesitation to risk production with a release. This helped automate and bypass the DBA to get code and features into production. Release Orchestration is the use of tools like XLRelease which manage software releases from the development stage to the actual software release itself.
14
Source/Version Control
“A component of software configuration management, version control, also known as revision control or source control, is the management of changes to documents, computer programs, large web sites, and other collections of information.” I’m going to add to this definition with Data version control.
15
The Goals of Each are the Same
Shorter Delivery Cycles Higher Success Rate on Releases Shorten Time to Address Issues Continued Improvement in Communication Increased Revenue
16
For the DBA, Why Virtualization
A technical approach in which users and applications do not use physical machines, but simulated systems running on actual, “real” hardware. Virtualization can be used to eliminate resource usage and enable savings for databases, network, file and application management, along with server infrastructure. In computing, virtualization means to create a virtual version of a device or resource, such as a server, storage device, network or even a database. The framework divides the resource into one or more execution environments. For data, this can result in a golden copy or source that is used for a centralized location and removal of duplicated data. For read and writes, having unique data for that given copy, while duplicates are kept to singular.
17
Many Types of Virtualization
Each Virtual Database takes up around 5-10Gb upon creation, (dependent upon parameters) PRODUCTION Database/App Tier 1 TB Storage Pool for Delphix DEV PATCH TEST QA TEST Point out the engine and size after we’ve compressed and de-duplicated. Note that each of the VDBs will take approximately 5-10G vs. 1TB to offer a FULL read/write copy of the production system It will do so in just a matter of minutes. That this can also be done for the application tier! ▶ ▶ ▶ Virtualize and Deployed▶ ▶ ▶ Read AND Write Read From Production 600GB
18
Why Database Virtualization is Different
DBA is “only as good as their last backup”. Many database tools take considerable time to recover. DevOps is often about automation- automate an “undo” for development and testing that includes data rewind. Include a self-service tool that will allow for rewind without DBA intervention.
19
Data Holdup to Development Cycle
Timeline in a Project Database Refresh Testing Cycle Development Production Release Data Holdup to Development Cycle Waterfall- 5 Releases January February March April Agile with Virtualization- 12 Releases January February March April Over 80% of time is waiting for RDBMS, (relational databases) to be refreshed. Developers and Testers are waiting for data to do their primary functions. This allows for faster and less costly migrations to the cloud, too. The Goal of Agile
20
Packages/Containers Containers offer the ability to isolate application code and/or the whole infrastructure stack into a package able entity to ease deployment, even from the same kernel. This is a powerful tool for DevOps to ease deployment for complex tiers, applications and multiple data stores. Package software into standardized units for development, shipment and deployment. A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings.
21
“Containerize” for Ease of Delivery
Delphix Virtualization Engine Same Storage Size as Production Deployed using Jenkins Reporting NFS iSCSI Testing NFS iSCSI Development NFS iSCSI Rsync (UNIX/Linux) Robocopy (Windows) Create “Container” We refer to a container as a template in our product.
22
Branching and Bookmarking
The ability to mark each iteration of development with a bookmark Simplify to lock and deliver while testing a consistent image via a virtual database, (VDB) If a something goes wrong, the ability to “bookmark”, (and subsequent snapshot) to deliver to development to address. This is a cornerstone to developers and testers, so as DBAs, we know the pain when a developer comes to us to flashback a database and before that, recover or logically recover, (import or datapump) independent objects. What is The developer/tester could do this for themselves?
23
Using Jetstream Branch 1.1 Release 1.2 Release 1.3 Branch 1.3
This is the interface for Developers and testers- they can bookmark before important tasks or rewind to any point in the process. They can bookmark and branch for full development/testing needs. Release 1.2 Release 1.3 Branch 1.3
24
DevOps and Source Control
This may appear to be a traffic disaster of changes, but for developers with Agile experience, a “sprint” looks just like this. You have different sprints that are quick runs and merges where developers are working separately on code that must merge successfully at the correct intersection and be deployed. Versioning with source control is displayed at the top, using Virtual images. You can see each iteration of the sprints. In the middle section is the branches of that occur during the development process. A virtual can be spun from a virtual, which means that it’s easier for developers to work from the work another developer has produced. Stopping points and release via a clone is simply minutes vs. hours or days.
25
Maturity The maturity of the DevOps environment will decide how silo’d or how blended the role you’ll have in DevOps vs. your standard role as a DBA.
26
Collaboration Practices/Tools
Methods- Scrum, KanBan, Extreme Programming, Continuous Delivery Hybrid approaches combine, such as Scrumban and KanPlan Collaboration- Jira, SlackStorm, Pivotal Tracker, HipChat, Rally, (now Agile Central) Deployment- RapidDeploy, ElasticBox, JuJu, Otto Automation- Jenkins, Chef, Ansible Build Tools- Maven, Gradle, Make, Packer, MSBuild Methods provide a format or guide to work from. Hybrid approaches often implement best. Collaboration methods ensure that communication continues when team members return to their desks Deployment tools help with documenting and lessons learned Build tools help with automation and orchestration
27
Agile Methodologies Scrum- A lightweight PM framework. Benefits include a flexible set of management and project protocols and strong collaboration focus. Lean/Kanban- Strong focus on team and deliverables. Efficiency and ability to see the “whole”. Extreme Programming, (XP)- Now the most popular, with continuous delivery key. Crystal- A lightweight agile framework. A removal of hurdles, including bureaucracy, obstacles and other impediments to the development cycle. Feature Driven Development, (FDD)- client and architecture centric software development, focused on the feature deliverable. Scrum focuses on features, bug fixes and backlog debt. Serves very large teams, including those 800+ Lean’s goal is to eliminate all waste, over demand on resources and ability to deliver faster and more effectively each time. XP is one of the most controversial due to the ability to deliver even to large companies every 1-3 weeks. Very disciplined approach. Crystal is often known under Crystal Clear, Yellow Orange and others.
28
What is Scrum? An Agile Framework Scrum Framework consists of:
A Product owner creating a wish list and the “sprint” begins Sprint planning and backlog is created. Team sets up schedule and beings to have daily scrum standups, (commonly 5 minutes) Scrum master keeps team focused and collaborating, keeps track of status Product is released Sprint ends with feedback and lessons learned Next sprint begins
29
Example of a Scrum Sprint
30
Lean Kanban Considered a very “visual” development process, based off of grocery store shelf stocking. Uses standardized cues and refined processes Goal to reduce waste and maximize value Most often uses sticky notes and whiteboard to create a picture of the work to complete, what’s in process and what’s done. Visualize Work Limit Work in Process Focus on Flow Continuous Improvement
31
Picture is Worth a 1000 Words
Often uses a whiteboard with sticky notes…
32
Extreme Programming Not THAT Extreme!
33
This Extreme Programming, (XP)
Code may come first in XP, but testing must already exist to know what the successful outcome will be. Code is written by pairs of programmers, allowing for better collaboration. Believes in the power of doing, vs. extensive planning. Failure is expected. Always build foundations that can be built on later. Rarely specialize- everyone develops, tests, designs, etc.
34
XP at a Glance
35
Crystal Alistair Cockburn said, “Crystal is a family of human- powered, adaptive, ultra light, ‘stretch-to-fit’ software development methodologies.” People are the most important aspect of Crystal. Development is self-sufficient and self-organizing, (very different from Scrum) Very lightweight Very little documentation, overhead management or reporting.
36
Feature Driven Development
Like Rapid Deploy, more focused on delivering one feature as the product.
37
DevOps Tools for the DBA
38
The Periodic Table of DevOps Tools
39
Ansible Learn about Playbooks
Powerful collection of steps, (aka plays) Plays consist of a list of tasks Handlers allow for remote operations YAML Syntax Security configurations based on hosts and users Ability to run as secondary users, (privilege escalation) Huge support community and best practices are mature
40
Example of an Ansible Call
- name: Transfer and execute a script. hosts: server remote_user: test_user sudo: yes tasks: - name: Transfer the script copy: src=test.sh dest=/home/test_user mode=0777 - name: Execute the script command: sh /home/test_user/test.sh
41
Jenkins Jenkins is an open source automation server-
Easiest executed via a jetty servlet Has an easy to access URL, (commonly host:8080) Automation is done via individual projects. Has numerous plugins to support environments Supports multiple scripting languages, including shell, batch commands and Ant scripts. Supports robust revision control and compatible with version control systems like CVS and Subversion. Supports notifications like qmail and sendmail. Ant is another java based built tool that’s part of Apache open-source project. Similar to Make and written in XML.
42
Jenkins Groovy Script Example
// prepare variables for subscript execution: String subscript = "cdmserver-monitor-instance-status.groovy"; println "executing subscript ${subscript}" // variables to be passed to the subscript status = (action == "start" ? "started" : "stopped") def s = ScriptlerConfiguration.getConfiguration().getScriptById(subscript) File scriptSrc = new File(ScriptlerManagment.getScriptDirectory(), s.getScriptPath()); This Groovy script executes another script, making it valuable in environments that already have a number of mature scripts in place that should be reused in automation.
43
Jenkins Plugin for Delphix
44
Puppet Review, automate, deploy and invest in all your software.
Control and enforce consistency across environments and platforms. Share, test and enforce changes on-premise and in the cloud. Similar to Puppet Chef Jenkins
45
Docker Virtualization is important, this is OS level
Resource level isolation at the kernel level Technology such as OverlayFS, allows for containers, which removes overhead of other virtualization technologies. Docker Alternatives: Open Container Initiative, (OCI) Kubernetes, (Google) CoreOS and rkt Apache Mesos Canonical and LXD
46
GitHub Git is an open-source version control system that was started by Linus Trovalds If you aren’t using it now, start. Yes, as a DBA… Great option to highlight your scripting work GitHub Alternatives Subversion CVS Mercurial
47
Creating a GitHub Repository Project
48
Github Repository Example
49
So, Know Your Tools Incredible variation of tools and knowing tool classifications doesn’t mean that the tools are alike. Many groups are often comparing one feature of a given tool.
50
From the DBA Perspective
DevOps include the following focus areas for Database Administrators: Automating repeatable and complex processes Ensuring the databases and data stores aren’t the bottleneck to the development cycle and critical data is protected. Locating heterogeneous tools/scripting languages that can support multiple tiers of technology. Understanding that automation does not lessen your value. Embracing new skills to add more value to the business and to your career.
51
Summary- DBA Focus Areas
Update skills, (scripting, monitoring, automating) to remove bottlenecks Enhance monitoring utilities that take infrastructure and development into overall view Recognize a need to become more skilled in scripting and become knowledgeable of DevOps tools. Data is EVERYWHERE! Recognize the integral part Database Administrators play in DevOps. DBA Skills DevOps
52
Thank you! kellyn@delphix.com http://dbakevlar.com http://delphix.com
@DBAKevlar Free eBook Download:
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.