Download presentation
Presentation is loading. Please wait.
Published byEdwina Muriel Adams Modified over 6 years ago
1
Part 2 – OpsWorks Stacks and Opsworks for Chef Automate
© 2017 Amazon Web Services, Inc. and its affiliates. All rights served. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon Web Services, Inc.
2
What Is OpsWorks Stacks?
Integrated application management solution for ops-minded developers and IT admins Model, control and automate applications of nearly any scale and complexity Management Console, SDKs, or CLI No additional cost Integrated application management solution for ops-minded developers and IT admins You can model, control, and automate application stacks of nearly any scale and complexity. You only pay for the resources you use – OpsWorks itself is free.
3
Why AWS OpsWorks? SIMPLE PRODUCTIVE FLEXIBLE POWERFUL SECURE
Easy to use, get started quickly and improves productive PRODUCTIVE Reduces errors with conventions and scripted configuration FLEXIBLE Simplifies deployments of any scale and complexity POWERFUL Reduce cost and time with automation SECURE Enables control with fine grained permissions Why should you use OpsWorks? SIMPLE to use built-in support for common frameworks and tasks, Makes teams more PRODUCTIVE Reduces errors with conventions and scripted configuration. FLEXIBLE Simplifies deployments and supports a wide variety of application architecture POWERFUL You can reduce cost and engineering time with automation and an event-driven configuration system with support for 1) customizable deployments,2) rollback, 3) patch management, 4) auto scaling, and more And finally, it’s SECURE You can use the default security settings that come with OpsWorks, as well as extend those and use fine grained permissions to control who has access to your stack, and what they can do with that access.
4
Improve Productivity Scalable infrastructure Flexible architecture
Deploy often Staging environments AWS OpsWorks gives us the tools we need to automate operations. We can scale Monster World, one of the largest Facebook games, to millions of users without ever needing more than two backend developers. Jesper Richter-Reichhelm head of engineering Let’s talk about typical 3 use cases that we see with OpsWorks customers. The first is using automation to improve productivity. Wooga is a gaming customer. Games draw an unpredictable number of players, making it difficult to calculate the number of instances needed at any given time. But Wooga uses OpsWorks to create high degree of automation the enable the rapid deployment of apps, so they’re able to rapidly adjust to increased usage. It gives them solid reliability. If a game is offline, players will move on to the next. Finally, OpsWorks is a solution that can work for small, agile teams, as you can see here Wooga runs a lean engineering organization.
5
Improve Reliability Git Jenkins OpsWorks Deploy Code Build Test
Check in – Build & Test Tests pass – Deploy Git Jenkins OpsWorks Deploy Code Build Test Provision Monitor The second use case is integrating a development pipeline to improve reliability. You can automate your entire build process – first deploying into a staging environment that mirrors production, and then when your integration tests pass, deploy to production. You can easily spin up parallel production environments and migrate traffic incrementally so that it’s easy to roll back if there are problems. So, let’s say you have a Git repo with your code. When new code is checked in, kick off the build process using something like Jenkins. When build is done, run through automated tests. Then script the OpsWorks CLI to change the app’s version and deploy to prod. (Or clone your stack, deploy to the new stack, and flip DNS when ready)
6
Improve Control Versioned config
Control anything you can script with Chef recipes Lock down console access Change and deployment logs Crashlytics uses AWS OpsWorks to support the rapid growth of our mobile crash reporting solution. With AWS OpsWorks, we can focus on the development and growth of our service and not spend developer cycles on infrastructure and operational tasks. Jeff Seibert CEO of Crashlytics The third use case is using both recipes and permissions to improve control. We’ll talk more about Chef in a moment, but OpsWorks can use Chef recipes to control anything you can script. With our Identity and Access Management tools, you can specify users or groups of users that are able to take actions against your stack. So you may have one team responsible for dev / test, but another team responsible for pushing those changes to production.
7
OpsWorks Security Let’s take a quick look at how OpsWorks Security works for users. You define the users you want to have access to your stacks, and what level of access those users have. AWS OpsWorks is the only service that integrates IAM with instance logon/credentials.
8
OpsWorks Security We have some built in security policies for you, or you can customize the level of access to reflect your business requirements.
9
OpsWorks Security Once you’ve decided on who should have what responsibility, you can easily make those changes.
10
OpsWorks Concepts Decide how to scale: manually, with 24/7 instances, or automatically, with load-based or time-based instances. A stack represents the cloud infrastructure and applications that you want to manage together. A layer defines how to setup and configure a set of instances and related resources. Then deploy your app to specific instances and customize the deployment with Chef recipes. Layers: App Server, Load Balancer, Database Layer, Caching Layer,
11
Modeling in AWS OpsWorks: Stacks
In OpsWorks you organize your projects / stages / environments in Stacks.
12
Creating a Stack Via the AWS Management Console Via the AWS CLI
Via an AWS SDK $ aws opsworks create-stack As with our other AWS services, OpsWorks gives you several options for creating your stack.
13
Modeling in AWS OpsWorks: Layers
Layers inside the Stack are used as blue prints to configure and manage your instances.
14
10 Built-In Layers Java App Server HAProxy AWS Flow (Ruby + SWF)
Memcached Ganglia MySQL HAProxy Static Web Server Rails App Server PHP App Server Node.js App Server Open Source Cookbooks available at OpsWorks has 10 built-in layers, and one custom layer, where our customers can define their own resources that make up a layer
15
Modeling in AWS OpsWorks: Instances
Instances are blueprints and might be different EC2 instances over time. We support the direct integration with other services like ELB, our elasticLoad Balancer
16
Scaling AWS OpsWorks: 24/7 Instances
17
Scaling AWS OpsWorks: Time-based Instances
18
Scaling AWS OpsWorks: Load-based Instances
OpsWorks integrates with CloudWatch to give you detailed layer metrics. If you notice heavy CPU or memory load, you can setup triggers to add “load-based” instances to your layer.
19
Scaling AWS OpsWorks: Load-based Instances
20
Back to Automation: The OpsWorks Agent
talks with The core of OpsWorks is a software agent we put on every server you start. That little piece of software talks to our service and can execute commands on the instances. In addition to that it also sends a heartbeat from the server and host level metrics to OpsWorks.
21
AWS OpsWorks Agent Installed on every instance
Can execute different commands on an instance Sends keep alive messages for auto healing Sends 15 host-level metrics every minute The core of OpsWorks is a software agent that exists on every server you start. The agent talks to our service lets you execute commands on the instances. In addition to that it also sends a heartbeat from the server and host level metrics to OpsWorks. Host-level metrics sample: cpu metrics, load metrics, memory metrics, and active processes
22
AWS OpsWorks and Chef Lifecycle Events Recipes Chef is an open-source framework that automates software deployment and configuration. Whenever a change happens on your stack, or upon request, all instances are notified and recipes are run. Metadata OpsWorks triggers events during the application lifecycle, such as when an instance is provisioned or an application is deployed. And it does this by using Chef. Chef is an open-source framework that automates software deployment and configuration. It does this in the form of “recipes”. Chef recipes are attached to these lifecyle events. You can also specify metadata that is passed into the recipes, so that you don’t have to hardcode things like passwords inside your configuration code. Let’s take a look at an example.
23
Chef Recipes in Action Recipe + Metadata = Command
execute “mysql-connect” do command “/usr/bin/mysql -u#{node[:deploy][:myphpapp][:database][:username]} -p#{node[:deploy][:myphpapp][:database][:password]} #{node[:deploy][:myphpapp][:database][:database]} … Recipe + Metadata = Command “deploy”: { “myphpapp”: { “database”: { “username”: “root”, “password”: “abcxyc”, … “/usr/bin/mysql -uroot –pabcxyz myphpapp … So this is a recipe that might connect your app to a MySQL server. The recipe would be brittle if we hard coded information like the username and database name, so we define metadata and pass it into the recipes. So when you take a recipe, and add in the meta data, the system sees the command as if you’d hard-coded it straight into your script.
24
App Environment Variables
user = ENV[“payment_provider_user“] password = ENV[“payment_provider_password“] PaymentGateway.new(user, password) Additionally, you can use the Console to set environment variables. Here, we have specified values for a user and password, and our apps now have access to those variables.
25
AWS OpsWorks Agent: Custom Recipes
Custom recipes can be found in OpsWorks cookbooks, like these AWS provided recipes on Github, or in other 3rd party repos
26
AWS OpsWorks Agent: Custom Recipes
…and those recipes can be used to create custom layers in your stack.
27
Lifecycle Events The commands the agent reacts on describe important events in an instances life. There is the setup event to get a server from the plain vanilla state to a become a web server, or a database, or whatever you want it to be. The configure event, which updates the servers configuration on changes in the Stack. This allows you to react to changes so that you can, for example scale up or switch databases when the primary crashes. The deploy and undeploy events you use to get code on and off the machines. The shutdown event to trigger a script to ship your last logs before the instance is gone or similar clean up things.
28
Setup Event Sent when instance boots Includes deploy event
Use for initial installation of software & services A recipe that kicks off when the instance boots, and it’s useful for Use for initial installation of software & services. You might use it to automatically update your system, install patches, or start services.
29
Configure Event Sent to all instances when any instance enters or leaves online state Use for making sure the configuration is up-to-date So if your autoscaling were to add a new instance, the configure event would run on all your instances, ensuring that they were up to date and matched each other.
30
Deploy Event Sent when you deploy via UI/API; part of each setup.
Use for custom deployment
31
Undeploy Event Sent via UI/API when apps are deleted
Use to remove apps from running instances
32
Shutdown Event Sent when an instance is shut down ~45s to execute
Use for clean shutdown
33
Debugging 1) Create a Stack 2) Add a layer 3) Start layer 4) Create simplejsp.jsp file in a simplejsp directory, archive, upload 4a) zip –r simplejsp.zip simplejsp 4b) aws s3 cp simplejsp.zip s3://bbvademo/ --acl public-read 5) Add the app
34
opsworks-agent-cli list_commands list the commands get_json show the JSON used run_command re-run a command show_log show the current or an old log For that you just SSH on to an instance And there you would use the opsworks-agent-cli The opsworks-agent-cli is a convenience wrapper around our agent So how would you use it to develop a cookbook
35
Agent CLI – list what happened
# list event history (up to 10 events) # opsworks-agent-cli list T10:26:58 setup T10:55:13 configure T14:59:05 execute_recipes T18:25:53 deploy T18:34:11 configure T10:05:23 configure T14:43:43 configure You would probably start listing the most recent events that happened on the machine to figure out which kind of event and therefore which recipes you want to change
36
Agent CLI – get the JSON to debug
# show JSON events (latest or by type/timestamp) # opsworks-agent-cli get { "ssh_users": { "2066": { "name": "thomas", "public_key": "ssh-rsa AAAAB3NzaC1...", "sudoer": true ... You can also look up the JSON we did hand over for each even by using the get command
37
Make a Change to your Cookbooks
# list current agent folder and cookbooks # ls /opt/aws/opsworks/current cookbooks # OpsWorks cookbooks berkshelf-cookbooks # Cookbooks from Berkshelf site-cookbooks # Your cookbooks merged-cookbooks # Resulting cookbooks for run ... To do the actual change you would change into opt / aws /opsworks/ current / site-cookbooks As this is the place were your own cookbooks live Than just use vi or some other editor to do the change on your cookbook
38
Agent CLI – re-run an event
# run last setup event (changed cookbooks, same JSON) # opsworks-agent-cli run setup # or select specific date # opsworks-agent-cli run T14:43:43 [ :06:41] INFO [opsworks-agent(17817)]: About to re-run 'setup' from T14:43:43 Waiting for process Finished Chef run with exitcode 0 After you saved your change you can use the opsworks-agent-cli again to re-run the last event of a specific type or you can also pass on a timestamp to re run a specific older event if you like After that you will see the Chef log appear in you console and you can check live on that instance if your change was successful
39
Testing
40
Infrastructure Deserves Tests
Infrastructure as code Test your code Test your infrastructure But how? What you probably really want is a similar fast experience but also write actual tests for your cookbooks Because if infrastructure as code is true Infrastructure should have tests For this we chose one of many tools out there – Test Kitchen.
41
Test Kitchen “Test Kitchen is a test harness tool to execute your configured code on one or more platforms in isolation.” Test Kitchen is a test harness tool to execute your configured code on one or more platforms in isolation. So test kitchen is Configuration Management-Neutral and not bound to Chef. A driver plugin architecture is used which lets you run your code on various providers and virtualization technologies such as Amazon EC2, Vagrant, Docker, and many more. We currently build a driver directly for OpsWorks we hope to release soon.
42
Test Kitchen – Workflow
Destroy Create Converge Setup Verify So how does a test kitchen run look like A test starts by Cleaning up potential left overs and destroying still running instances Than it Create as instance in OpsWorks as we will use the OpsWorks driver Converge does the chef actual run Setup – setups the used test environment Verify – uploads the test to your OpsWorks instances and run the tests Destroy again cleans up running instances You can repeat any step as often as you like So how is test kitchen set up
43
AWS OpsWorks for Chef Automate
[ COPY FROM WEBSITE ]
44
Benefits Fully Managed Chef Server Programmable Infrastructure
Scaling Made Easy Support from Active Chef Community Simple to Manage Hybrid Environments
45
Fully Managed Chef Server
Easy to use Chef on AWS Automatic backups Automatic updates & upgrades Chef minor versions OS updates Security updates AWS OpsWorks for Chef Automate makes it easy to use Chef sever on AWS. There is no need for Chef server provisioning and installation. OpsWorks for Chef Automate automatically updates the Chef server software and creates backups of the server for you. You can use the AWS Management Console, AWS Command Line Interface (CLI), or AWS SDKs to provision a ready-to-use Chef server.
46
Support from Active Chef Community
Use Chef community cookbooks or create your own Supports a wide variety of applications out-of-the-box Control every aspect of your host’s configuration Interface with your Chef server using native Chef tools - Knife command-line tool or ChefDK AWS OpsWorks for Chef Automate supports the latest version of Chef server and Chef Automate. You can use any Chef community-built tools or cookbooks with your Chef server. With OpsWorks for Chef Automate, you can interface with your Chef server using native Chef tools, such as the Knife command-line tool or Chef Development Kit (ChefDK).
47
Programmable Infrastructure
Define configuration using Chef recipes written in a Ruby DSL Maintain and version your configuration Ensure consistent configuration and compliance With Chef server, you can define configurations for your servers in a format that you can maintain and version just like your application source code. Chef server ensures your servers are consistently configured and maintained, and handles complex operational tasks when there are interdependencies across your servers.
48
Get visibility into the state of your nodes
Chef Visibility – a view into convergence, compliance, cookbooks, recipes and more Premium Chef Features With AWS OpsWorks for Chef Automate, you receive the full Chef Automate platform which includes premium features that you can use with Chef server, like Chef Workflow, Chef Visibility, and Chef Compliance View into the state of your cookbooks, recipes, roles and most importantly your nodes, Whether what is defined in the recipe meets the state of the node. Chef calls converge status. Data warehouse platform – use through the UI to get insights into the state of your environment.
49
Go beyond Configuration Management
Chef Workflow – A continuous delivery pipeline for infrastructure and applications Move a change in your host or application configuration from the developer’s workstation all the way to production. Has dashboards that let you track each change as it goes through the pipeline
50
Go beyond Configuration Management
Chef Compliance - Discovery and analysis of compliance risks across environments Chef Compliance lets you generate customizable reports to identify: Compliance issues, security risks, and even outdated software. Aggregated in the Visibility console Write your own compliance rules, or you can get started quickly by using built-in ones. Use predefined rule-sets for existing security frameworks
51
Any Questions?
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.