1. Taking ActiveRecord to the Next Level Blythe Dunham blythe@snowgiraffe.com http://snowgiraffe.com

2. Goal Leverage advanced MySQL functionality with ActiveRecord

3. Disclaimer!!!! Premature Optimization

4. ActiveRecord 101 What's going on under the covers?

5. ActiveRecord 101 class User < ActiveRecord::Base end CREATE TABLE `users` ( `id` int(11) NOT NULL auto_increment, `name` varchar(255) default NULL, `password` varchar(255) default NULL, `email` varchar(255) default NULL, `created_at` datetime default NULL, `updated_at` datetime default NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 Database Table ff Active Record Model

6. ActiveRecord 101 with Animals! class Animal < ActiveRecord::Base end CREATE TABLE `animals` ( `id` int(11) NOT NULL auto_increment, `name` varchar(255) default NULL, `password` varchar(255) default NULL, `email` varchar(255) default NULL, `species_id` int(11) default NULL, `created_at` datetime default NULL, `updated_at` datetime default NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 Database Table ff Active Record Model

7. Creating a Record animal = Animal.new animal.name = 'Jerry Giraffe' animal.password = 'jerry' animal.save! INSERT INTO `animals` (`name`, `updated_at`, `species_id`, `password`, `email`, `created_at`) VALUES('Jerry Giraffe', '2009-03-15 00:48:28', NULL, 'jerry', NULL, '2009-03-15 00:48:28')

8. Updating a Record animal.name = 'Jerry G' animal.save! UPDATE `animals` SET `updated_at` = '2009-03-15 03:01:06', `name` = 'Jerry G' WHERE `id` = 1

9. Finding a Record jerry = Animal.find :first, :conditions => ['name = ?', 'Jerry G'] SELECT * FROM `animals` WHERE (name = 'Jerry G') LIMIT 1 #shortcut Animal.find_by_name 'Jerry G'

10. Representing Relationships Species name Animal name email password fav_beer updated_at created_at species_id

11. CREATE TABLE `animals` ( `id` int(11) NOT NULL auto_increment, `name` varchar(35) NOT NULL, `email` varchar(40) default NULL, `fav_beer` enum('Guiness','Pabst','Redhook','Chimay') default 'Pabst', `created_at` datetime default NULL, `updated_at` datetime default NULL, `password` varchar(25) character set latin1 collate latin1_bin NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 Representing Relationships(DDL) CREATE TABLE `species` ( `id` int(11) NOT NULL auto_increment, `name` varchar(255), PRIMARY KEY (`id`), ) ENGINE=InnoDB AUTO_INCREMENT=9 DEFAULT CHARSET=utf8

12. Representing Relationships (AR) class Animal < ActiveRecord::Base belongs_to :species end class Species < ActiveRecord::Base has_many :animals end

13. Representing Relationships (AR) jerry.species SELECT * FROM `species` WHERE (`species`.`id` = 1) species.animals SELECT * FROM `animals` WHERE (`animals`.species_id = 1)

14. Representing Relationships (AR) giraffe = Species.find_by_name 'giraffe' giraffe.animals << jerry SELECT * FROM `species` WHERE (`species`.`name` = 'giraffe' ) LIMIT 1 UPDATE `animals` SET `species_id` = 1, `updated_at` = '2009-03-19 23:15:54' WHERE `id` = 7

15. Migration Set limits Set default values Identify NOT NULL columns Use enumerated columns Custom DDL Add (unique) indices Foreign Keys are great Primary Key Modifications

16. Migration 101 ruby script/generate scaffold Animal name:string password:string email:string fav_beer:string class CreateAnimals < ActiveRecord::Migration def self.up create_table :animals do |t| t.string :name t.string :password t.string :email t.string :fav_beer t.timestamps end def self.down drop_table :animals end

17. Set Limits Default String is VARCHAR(255) create_table :animals do |t| t.string :name, :limit => 35 t.string :password, :limit => 25 t.string :email, :limit => 40 t.string :fav_beer, :limit => 40 t.timestamps end

18. Numeric Type Limits t.integer :mysmallint, :limit => 2 "Smart types" determines numeric type for MySQL :limit Numeric TypeColumn Size 1tinyint1 byte 2smallint2 bytes 3mediumint3 bytes 4, nil, 11int(11)4 bytes 5 to 8bigint8 bytes

19. Set columns to NOT NULL create_table :animals do |t| t.string :name, :limit => 35, :null => false t.string :password, :limit => 25, :null => false t.string :email, :limit => 40 t.string :fav_beer, :limit => 40 t.timestamps end

20. Set default values create_table :animals do |t| t.string :name, :limit => 35, :null => false t.string :password, :limit => 25, :null => false t.string :email, :limit => 40, :default => nil t.string :fav_beer, :limit => 40 :default => 'Pabst' t.timestamps end

21. Remove unneeded columns create_table :animals do |t| t.string :name, :limit => 35, :null => false t.string :password, :limit => 25, :null => false t.string :email, :limit => 40, :default => nil t.string :fav_beer, :limit => 40 :default => 'Pabst' t.timestamps end

22. Enumerated Column Plugin create_table :animals do |t| t.string :name, :limit => 35, :null => false t.string :password, :limit => 25, :null => false t.string :email, :limit => 40, :default => nil t.enum :fav_beer, :default => 'Pabst' :limit => %w(Chimay Pabst Redhook) t.timestamps end

23. Think about the table parameters create_table :animals, :options => 'ENGINE=MyISAM' do |t| t.string :name, :limit => 35, :null => false t.string :password, :limit => 25, :null => false t.string :email, :limit => 40, :default => nil t.enum :fav_beer, :default => nil :limit => %w(Chimay Pabst Redhook) t.timestamps end

24. Custom DDL create_table :animals do |t| t.string :name, :limit => 35, :null => false t.string :email, :limit => 40, :default => nil t.enum :fav_beer, :default => nil :limit => %w(Chimay Pabst Redhook) t.timestamps end #case sensitive password (encrypted) execute "ALTER TABLE `animals` ADD `password` varchar(25) character set latin1 collate latin1_bin NOT NULL"

25. Create (Unique) Indices create_table :species do |t| t.string :name, :null => false, :limit => 25 end add_index :species, :name, :unique => true, :name => 'uk_species_name'

26. ActiveRecord Uniqueness class Species < ActiveRecord::Base validates_uniqueness_of :name end Doesn't Guaranty Data Integrity!

27. I Heart Foreign Keys Referential Integrity

28. The AR Way: Foreign Keys class Species < ActiveRecord::Base has_many :animals, :dependent => :nullify end

29. The Rails Way: Foreign Keys class Species < ActiveRecord::Base has_many :animals, :dependent => :nullify end Dependent Value SQL Equivalent: :nullify => ON DELETE SET NULL :delete_all => ON DELETE CASCADE :destroy => No SQL equivalent. Every association is instantiated and and callbacks are executed before destruction

30. Redhills Foreign Key Migration Plugin to the rescue! add_column :animals, :species_id, :integer, :references => :species, :name => 'fk_animal_species', :on_delete => :set_null, :on_update => :cascade ALTER TABLE `animals` ADD `species_id` int(11); ALTER TABLE animals ADD CONSTRAINT fk_animal_species FOREIGN KEY (species_id) REFERENCES species (id) ON UPDATE CASCADE ON DELETE SET NULL;

31. CREATE TABLE `animals` ( `id` int(11) NOT NULL auto_increment, `name` varchar(35) NOT NULL, `email` varchar(40) default NULL, `fav_beer` enum('Guiness','Pabst','Redhook','Chimay') default 'Pabst', `species_id` int(11) default NULL, `created_at` datetime default NULL, `updated_at` datetime default NULL, `password` varchar(25) character set latin1 collate latin1_bin NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 Primary Keys CREATE TABLE `species` ( `id` int(11) NOT NULL auto_increment, `name` varchar(255), PRIMARY KEY (`id`), ) ENGINE=InnoDB AUTO_INCREMENT=9 DEFAULT CHARSET=utf8

32. Modify the Rails Primary Key Change type with change_column MySQL Migration Optimization Plugin create_table :animals, :primary_key => "special_key", :primary_column => { :type => :integer, :limit => 2, :precision => :unsigned, :scale => 3 } do |t| end CREATE TABLE `animals` (`special_key` smallint(3) UNSIGNED NOT NULL auto_increment PRIMARY KEY) ENGINE=InnoDB;

33. Advanced ActiveRecord Insert and update options Import bulk data Finder Options Tools: plugin AR-Extensions

34. ActiveRecord on Steroids: ar- extensions plugin Additional Create and Update options save(options={}) save!(options={}) create(args, options={}) create!(args, options={}) Options :ignore :on_duplicate_key_update :keywords :reload :pre_sql :post_sql

35. :ignore Standard ActiveRecord: Create the record if it does not already exist unless Animal.find_by_name('Jerry G') Animal.create!(:name => 'Jerry G', :password => 'jerry') end

36. :ignore Ignore duplicates! One query, less code, fewer queries! Animal.create!({:name => 'Jerry G', :password => 'jerry'}, {:ignore => true})

37. :on_duplicate_key_update Update the record if it exists, if not create a new one. A lot of code to upsert and performs two SQL queries! jerry = Animal.find_by_name 'Jerry G' jerry ||= Animal.new(:name => 'Jerry G') jerry.password = 'frenchfry' jerry.save!

38. :on_duplicate_key_update jerry = Animal.new :name => 'Jerry G', :password => 'frenchfry' jerry.save! :on_duplicate_key_update => [:password, :updated_at] INSERT INTO animals (`name`, `updated_at`, `species_id`, `password`,`email`, `created_at`) VALUES('Jerry G', '2009-03-15 06:17:51', NULL, 'frenchfry', NULL, '2009-03-15 06:17:51') ON DUPLICATE KEY UPDATE `animals`.`password`=VALUES(`password`), `animals`.`updated_at`=VALUES(`updated_at`)

39. Reloading the instance AR Data can become inconsistent with DB after an IGNORE, UPDATE, or ON DUPLICATE KEY UPDATE reload executes more queries For UPDATE the duplicate is automatically reloaded jerry.email = 'jerry@snowgiraffe.com' jerry.save! :on_duplicate_key_update => [:password, :updated_at], :reload => true, :duplicate_columns => [:name]

40. More Customization jerry.save(:keywords => 'LOW_PRIORITY', :pre_sql => '/*Pre comment*/', :post_sql => "/*#{__FILE__} #{__LINE__}*/") /*Pre comment*/ UPDATE LOW_PRIORITY `animals` SET `created_at` = '2009-03-15 06:13:48', `species_id` = NULL, `email` = NULL, `password` = 'frenchfry', `updated_at` = '2009-03-15 06:45:38', `name` = 'Jerry G' WHERE `id` = 7/*animal_controller.rb 147 */

41. Import (Bulk Insert) Instead of one-by-one, insert a ton of records fast

42. Import (Bulk Insert) Standard way: Insert each animal one by one Animal.create!(:name => 'dolly dolphin', :password => 'dolly') Animal.create!(:name => 'big birdie', :password => 'birdie') and so on….

43. Fast Import: One INSERT animals = [ Animal.new(:name => 'dolly dolphin', :password => 'dolly'), Animal.new(:name => 'big birdie', :password => 'birdie')] Animal.import animals INSERT INTO `animals` (`id`,`name`,`email`,`fav_beer`,`created_at`,`updated_at`,`password`) VALUES (NULL,'dolly dolphin',NULL,'Pabst', '2009-03-20 00:17:15','2009-03-20 00:17:15','dolly'), (NULL,'big birdie',NULL,'Pabst', '2009-03-20 00:17:15','2009-03-20 00:17:15','birdie') ON DUPLICATE KEY UPDATE `animals`.`updated_at`=VALUES(`updated_at`)

44. Fastest Import: fewer columns columns = [ :name, :password ] values = [['dolly dolphin', 'dolly'], ['big birdie', 'birdie']] options = {:validate => false, :timestamps => false} Animal.import columns, values, options INSERT INTO `animals` (`name`,`password`) VALUES ('dolly dolphin','dolly'),('big birdie','birdie')

45. Insert Select Standard: Query and Insert one by one Species.find(:all).each do |s| SpeciesZoo.create!(:species_id => s.id, :zoo_id => zoo.id, :extra_info => 'awesome') end Executes a query for each species INSERT INTO `species_zoos` (`zoo_id`, `id`, `species_id`, `extra_info`) VALUES (1, 3, 3, 'awesome') INSERT INTO `species_zoos` (`zoo_id`, `id`, `species_id`, `extra_info`) VALUES (1, 3, 2, 'awesome') And so on…

46. Insert Select Import SpeciesZoo.insert_select( :from => :species, :select => ['species.id, ?', zoo], :into => [:species_id, :zoo_id]) One INSERT statement INSERT INTO `species_zoos` ( `species_id`, `zoo_id` ) SELECT species.id, 1 FROM `species`

47. Temporary Tables Not so good for slave replication Can be used as a sandbox then imported into a real table with ar-extensions gem Animal.create_temporary_table do |t| t.create!(:name => 'giraffe', :password => 'goo') Animal.insert_select( :from => t, :select => [:name, :password, :fav_beer], :into => [:name, :password, :fav_beer], :on_duplicate_key_update => [:password, :fav_beer]) end

48. Customizing Find Additional finder options :keywords :pre_sql :post_sql :index_hint

49. Customizing Find Animal.find(:all, :conditions => ['name = ?', 'Jerry G'], :keywords => 'HIGH_PRIORITY', :pre_sql => '/*Pre comment*/', :post_sql => 'FOR UPDATE /*After the fact*/', :index_hint => 'USE INDEX (uk_animal_name)' ) /*Pre comment*/ SELECT HIGH_PRIORITY * FROM `animals` USE INDEX (uk_animal_name) WHERE (name = 'Jerry G') FOR UPDATE /*After the fact*/

50. Need more? Get dirty with find_by_sql sql = Animal.send :finder_sql_to_string, :conditions => ['name = ?', 'Jerry G'] sql.gsub! /WHERE/, 'where /* Dirty hand */' Animal.find_by_sql sql

51. More: find_union & count_union Animal.find_union( {:conditions => ['animals.name like ?', 'Jerry%']}, {:conditions => ['species.name = ?', 'giraffe'], :include => :species} ) (SELECT `animals`.* FROM `animals` WHERE (animals.name = 'Jerry G')) UNION (SELECT `animals`.* FROM `animals` LEFT OUTER JOIN `species` ON `species`.id = `animals`.species_id WHERE (species.name = 'giraffe'))

52. Finder Issues: Speed and Memory paginate - less loaded into memory :select option - Data is retrieved faster when fewer columns are selected

53. Paginated Finders Rails 2.3.2 includes :batch_size option Animal.find_each(:batch_size => 2) do |animal| #do something end Will Paginate Plugin page = 1 begin animals = Animal.paginate :per_page => 2, :page => page animals.each{|animal| …do something… } end while (page = animals.next_page) Paginating Find Plugin

54. :select Data is retrieved faster when fewer columns are selected Animal.find :first, :select => 'name'

55. :include hates :select But :select is ignored with eager loading (:include)! Animal.find :first, :select => 'animals.name, species.name', :include => :species, :conditions => ['species.name like ?', 'giraffe'] SELECT `animals`.`id` AS t0_r0, `animals`.`name` AS t0_r1, `animals`.`email` AS t0_r2, `animals`.`fav_beer` AS t0_r3, `animals`.`created_at` AS t0_r4, `animals`.`updated_at` AS t0_r5, `animals`.`password` AS t0_r6, `animals`.`species_id` AS t0_r7, `species`.`id` AS t1_r0, `species`.`name` AS t1_r1 FROM `animals` LEFT OUTER JOIN `species` ON `species`.id = `animals`.species_id WHERE (species.name like 'giraffe') LIMIT 1

56. Alternatives to Eager Loading Eager loading for sparse :include can be time consuming Use :join instead of :include Eager Load Plugin Rails 2.3.2 Query Cache helps ActiveRecordContext Plugin PiggyBack Plugin

57. :join instead of :include Eager loading is slow in Rails and can be slow on the database. Use an (inner) :join instead of :include animal = Animal.find :first, :select => 'animals.name, species.name as spec_name', :joins => :species, :conditions => ['species.name like ?', 'giraffe'] animal.spec_name == 'giraffe'

58. Force it with Eager loading plugins Eager loading is slow in Rails and can be slow on the database. eload-selecteload-select plugin

59. Help from Rails 2 Query Cache Animals of the same species are only loaded once ActiveRecord::Base.cache { Animal.find(:all).each {|a| a.species } } Animal Load (1.8ms) SELECT * FROM `animals` Species Load (0.3ms) SELECT * FROM `species` WHERE (`species`.`id` = 2) CACHE (0.0ms) SELECT * FROM `species` WHERE (`species`.`id` = 2) Species Load (0.3ms) SELECT * FROM `species` WHERE (`species`.`id` = 1) CACHE (0.0ms) SELECT * FROM `species` WHERE (`species`.`id` = 1)

60. ActiveRecordContext Plugin ActiveRecord::Base.with_context do animals = Animal.find :all Species.prefetch animals animals.each {|a| a.species } end Animal Load (0.4ms) SELECT * FROM `animals` [Context] Storing Animal records: 1, 2, 3, 4, 5, 6, and 7 Species Load (0.4ms) SELECT * FROM `species` WHERE (`species`.`id` IN( 2,1 )) [Context] Storing Species records: 2 and 1 [Context] Found Species #2 [Context] Found Species #1

61. Piggyback Plugin Delegate records with :has_one and :belongs_to associations Great for accessing extension tables with TEXT or BLOB

62. Piggyback Plugin Uses joins to delegate records from :has_one and :belongs_to associations class Animal < ActiveRecord::Base belongs_to :species piggy_back :species_name, :from => :species, :attributes => [:name] end animal = Animal.find :first, :piggy => :species_name animal.species_name == 'giraffe' SELECT animals.*, species.name AS species_name FROM `animals` LEFT JOIN species ON species.id=animals.species_id LIMIT 1

63. Avoiding Deadlock Deadlock Retry Deadlock Retry plugin - retries query up to 3 times Batch Operations (AR-Extension plugin) Animal.delete_all(['name like ?','giraffe%'], :batch_size => 50)

64. Reading from a Replica Masochism plugin

65. Export Data ar-fixtures: Export entire table to yaml ar-extensions: export to csv ar-dumper: export (paginated) to yaml, xml or csv

66. Cache! Show me the money! Query Cache Memcache Static Record Cache plugin

67. ActiveRecord plugins and gems AR-Extensions http://www.continuousthinking.com/tags/arext http://www.continuousthinking.com/tags/arext Piggy Back http://railsexpress.de/svn/plugins/piggy_back/trunk/README http://railsexpress.de/svn/plugins/piggy_back/trunk/README Eager Loading http://www.snowgiraffe.com/tech/?p=329 http://www.snowgiraffe.com/tech/?p=329 Active Record Context http://svn.techno- weenie.net/projects/plugins/active_record_context/ http://svn.techno- weenie.net/projects/plugins/active_record_context/ Will Paginate http://github.com/mislav/will_paginate/tree/master http://github.com/mislav/will_paginate/tree/master Deadlock Retry http://agilewebdevelopment.com/plugins/deadlock_retry http://agilewebdevelopment.com/plugins/deadlock_retry Paginating Find http://www.railslodge.com/plugins/287-paginating-find http://www.railslodge.com/plugins/287-paginating-find Static Record Cache http://github.com/blythedunham/static_record_cache/tree/master http://github.com/blythedunham/static_record_cache/tree/master

68. Other Useful Plugins Migration Plugins Redhills Consulting Foreign Key Migrations http://agilewebdevelopment.com/plugins/foreign_key_migrations http://agilewebdevelopment.com/plugins/foreign_key_migrations Enum Column http://enum-column.rubyforge.org/ http://enum-column.rubyforge.org/ MySQL Migration Optimizer http://github.com/blythedunham/mysql_migration_optimizer/tree/master http://github.com/blythedunham/mysql_migration_optimizer/tree/master Replica Plugins Masochism http://github.com/technoweenie/masochism/tree/master http://github.com/technoweenie/masochism/tree/master Active Delegate http://www.robbyonrails.com/articles/2007/10/05/multiple-database- connections-in-ruby-on-rails http://www.robbyonrails.com/articles/2007/10/05/multiple-database- connections-in-ruby-on-rails Export Plugins Ar-fixtures http://github.com/topfunky/ar_fixtures/tree/master http://github.com/topfunky/ar_fixtures/tree/master Ar-dumper http://github.com/blythedunham/ar_dumper/tree/master http://github.com/blythedunham/ar_dumper/tree/master

69. Questions? Thanks for attending! I hope you are on the next level. Slides available at: http://snowgiraffe.comhttp://snowgiraffe.com

70. Pics Biz card: http://www.twozdai.com/http://www.twozdai.com/ Toolbox: http://www.flickr.com/photos/29802022@N07/2971774302/http://www.flickr.com/photos/29802022@N07/2971774302/ Cash: http://www.flickr.com/photos/gnerk/2466566500/http://www.flickr.com/photos/gnerk/2466566500/ Emoo cow: http://blog.petegraham.co.uk/2007/05/15/animals-with-emo-haircuts/http://blog.petegraham.co.uk/2007/05/15/animals-with-emo-haircuts/ Giraffe in Car: http://www.norrishallstudio.com/assets/img/products/switchplates/Giraffe_in_Car_SO.jpghttp://www.norrishallstudio.com/assets/img/products/switchplates/Giraffe_in_Car_SO.jpg Leafcutters: http://www.makingthelink.co.uk/Leafcutter%20Ants%20Corcovado.jpghttp://www.makingthelink.co.uk/Leafcutter%20Ants%20Corcovado.jpg Puppy: http://www.flickr.com/photos/todorrovic/2287792473/http://www.flickr.com/photos/todorrovic/2287792473/ Dump truck: http://farm3.static.flickr.com/2065/2197925262_bd2726c3fa.jpg?v=0http://farm3.static.flickr.com/2065/2197925262_bd2726c3fa.jpg?v=0 Ignore sign: http://www.flickr.com/photos/alexsuarez/2504638107/http://www.flickr.com/photos/alexsuarez/2504638107/ Tigers: http://www.flickr.com/photos/sharynmorrow/19981568/http://www.flickr.com/photos/sharynmorrow/19981568/ Single Leafcutter: http://www.flickr.com/photos/mattblucas/2176783448/http://www.flickr.com/photos/mattblucas/2176783448/ Giraffe with tongue: http://www.flickr.com/photos/ucumari/2570608134/http://www.flickr.com/photos/ucumari/2570608134/ Giraffe in a can: http://www.flickr.com/photos/10159247@N04/2877489356/http://www.flickr.com/photos/10159247@N04/2877489356/ Beaver: http://www.flickr.com/photos/krrrista/2286455954/http://www.flickr.com/photos/krrrista/2286455954/ Mini Pig: http://www.richardaustinimages.com/http://www.richardaustinimages.com/ Giraffe herd: http://www.flickr.com/photos/acastellano/2260928018/http://www.flickr.com/photos/acastellano/2260928018/ Giraffe skin: http://www.flickr.com/photos/tomorrowstand/1806095442http://www.flickr.com/photos/tomorrowstand/1806095442 Dirty hands: http://www.flickr.com/photos/dragonhide/2372544373/http://www.flickr.com/photos/dragonhide/2372544373/ Blythe twins: http://www.flickr.com/photos/petitewanderlings/434252916/http://www.flickr.com/photos/petitewanderlings/434252916/ Dead lock: http://www.flickr.com/photos/fpsurgeon/2453544236/http://www.flickr.com/photos/fpsurgeon/2453544236/ Foreign Keys: http://www.flickr.com/photos/zeeny79/347753999/http://www.flickr.com/photos/zeeny79/347753999/ Gorilla: http://www.flickr.com/photos/shuttershrink/425613091/http://www.flickr.com/photos/shuttershrink/425613091/ Speed of light: http://www.flickr.com/photos/laserstars/908946494/http://www.flickr.com/photos/laserstars/908946494/ Fat Giraffe: http://funnies.com/funny-picture-fat-giraffe.htmhttp://funnies.com/funny-picture-fat-giraffe.htm