1. Code-First.NET and MongoDB The Ubiquitous Blog Example Philly ALT.NET User Group 2010-11-16 John C. Zablocki Development Lead, MagazineRadar Adjunct, Fairfield University

2. Agenda MongoDB Basic Concepts MongoDB Shell NoRM MongoDB C# Driver MongoDB Design Considerations In Depth: Meringue Case Study: RateMySnippet Questions?

3. MongoDB Concepts Schema-less documents stored in collections Schema-less documents stored in collections Documents are stored as BSON (Binary JSON) Documents are stored as BSON (Binary JSON) JavaScript used to query and manipulate documents and collections JavaScript used to query and manipulate documents and collections Each document in a collection has a unique BSON ObjectId field named _id Each document in a collection has a unique BSON ObjectId field named _id Collections belong to a database Collections belong to a database

4. Installing MongoDB on Windows Download the binaries from mongodb.org Download the binaries from mongodb.org Extract to Program Files directory (or wherever) Extract to Program Files directory (or wherever) Create a directory c:\data\db Create a directory c:\data\db Run mongod.exe from the command line with the -- install switch Run mongod.exe from the command line with the -- install switch  See http://bit.ly/aed1RW for some gotchas http://bit.ly/aed1RW To run the daemon without installing, simply run mongod.exe without arguments To run the daemon without installing, simply run mongod.exe without arguments Run mongo.exe to verify the daemon is running Run mongo.exe to verify the daemon is running

5. MongoDB and.NET A funny thing happened on the way to Philadelphia...

6. MongoDB - Shell The MongoDB interactive JavaScript shell (mongo.exe) is a command line utility for working with MongoDB servers The MongoDB interactive JavaScript shell (mongo.exe) is a command line utility for working with MongoDB servers Allows for CRUD operations on collections Allows for CRUD operations on collections May be used for basic administration May be used for basic administration  Creating indexes  Cloning databases Also useful as a test-bed while building apps Also useful as a test-bed while building apps

7. MongoDB - Shell /*Connect to a server:port/database (defaults are localhost:27017/test ):*/ mongo.exe localhost:27017/AltNetGroup //Switch database: use CodeCamp //View collections in a database: show collections //create an index on Name field db.Posts.ensureIndex({ Name : 1 }); //copy one database to another db.copyDatabase("CodeCamp", "AltNetGroup")

8. MongoDB - Shell //create a document var post = { Title: "On Installing MongoDB as a Service on Windows" } //insert a document, if the collection doesn't exist it's created db.Posts.insert(post); //verify that the document was created db.Posts.find(); //write a query to find a post with a valid title var query = { Title: { $ne: null} } //use that query to find the post var post = db.Posts.findOne(query); //this line will actually set the content after pressing enter post.Content = "When installing MongoDB as a service on Windows..."

9. MongoDB - Shell //update the content to include an author using collection update method db.Posts.update( { Title : "On Installing MongoDB as a Service on Windows" }, { Author : "John Zablocki" } ) //check that the post was updated db.Posts.findOne() //where'd my document go? updates are in place, replacing entire documents! //need to use the $set operator to update partial documents - let's start over //first remove the new document. Notice how remove takes a func as an argument. //find and findOne also accept functions as arguments db.Posts.remove(function (e) { return this.Author == "John Zablocki" }) //rerun the first statements up to but not including the db.Posts.update(... db.Posts.update({ Title: "On Installing MongoDB as a Service on Windows" }, { $set: { Author: "John Zablocki" } }) //verify that the update worked db.Posts.findOne()

10. MongoDB - Shell //add a tag to the post db.Posts.update({ Title: "On Installing MongoDB as a Service on Windows" }, { $push: { Tags: "mongodb" } }) //look for that tag db.Posts.findOne() //add two more tags db.Posts.update({ Title: "On Installing MongoDB as a Service on Windows" }, { $pushAll: { Tags: ["windows", "nosql"] } }) //add another post db.Posts.insert( { Author : "John Zablocki", Title : "On MapReduce in MongoDB", Tags: ["mongodb", "nosql"] }) //verify that last insert worked db.Posts.findOne(function (e) { return this.Title.indexOf("MapReduce") != -1; })

11. MongoDB - Shell //add a "like" counter to the post. The booleans arguments tell update //not to insert if the document doesn't exist and to update all documents, //not just one respectively db.Posts.update({ Author: "John Zablocki" }, { $set: { Likes: 0} }, false, true) //increment the likes counter for the mapreduce article db.Posts.update({ Title: /mapreduce/i }, { $inc: { Likes: 1} }) //check that the counter was incremented db.Posts.findOne({ Title: /mapreduce/i }).Likes

12. MongoDB – Shell /* Create the map function This function creates a view that looks like { “mvc”, [1, 1] }, { “norm”, [1] } */ var map = function() { if (!this.Tags) { return; } for (var index in this.Tags) { emit(this.Tags[index], 1); } };

13. MongoDB – Shell MapReduce /* Create the reduce function conceptually, reduce gets called like: reduce("mvc", [1, 1]); reduce("norm", [1] /* var reduce = function(key, vals) { var count = 0; for (var index in vals) { count += vals[index]; } return count; };

14. MongoDB – Shell MapReduce /* Run the mapreduce command on the Posts collection using the map and reduce functions defined above store the results in a collection named Tags */ var result = db.runCommand( { mapreduce : "Posts", map : map, reduce : reduce, out : "Tags" }); db.Tags.find()

15. Something Completely Different

16. NoRM

17. NoRM Developed by Andrew Theken among others Developed by Andrew Theken among others Active community with reliable support Active community with reliable support  I received help even as I prepared this slide! Support for typed and untyped collections, MapReduce, Property Mapping, LINQ, Expando, nearly all CRUD operations Support for typed and untyped collections, MapReduce, Property Mapping, LINQ, Expando, nearly all CRUD operations Will eventually sit on top of the 10gen officially supported driver Will eventually sit on top of the 10gen officially supported driver

18. NoRM – The Basics //Connections managed with IDisposable pattern //Connect to test database on localhost using (IMongo mongo = Mongo.Create("mongodb://localhost/test")) { Console.WriteLine(mongo.Database.DatabaseName); } //Mongo instance has MongoDatabase property /*MongoDatabase has GetCollection methods for accessing MongoCollection instances*/ //CRUD operations performed on collections

19. NoRM - CRUD //Class properties map to document structure public class Artist { //ObjectId property mapped to _id field in document public ObjectId Id { get; set; } public string Name { get; set; } //IList property will be mapped to JavaScript Array private IList _albums = new List (0); public IList Albums { get { return _albums ; } set { _albums = value; } }

20. NoRM - CRUD var artist = new Artist() { Name = "The Decembrists" }; //Inserting a document into a typed collection mongo.Database.GetCollection ("Artists").Insert(artist); //Updating (replacing) a document in a typed collection artist.Name = "The Decemberists"; mongo.Database.GetCollection ("Artists").Save(artist); //Updating a nested collection mongo.Database.GetCollection ("Artists").UpdateOne( new { Name = "The Decemberists"}, new { Albums = M.Push("Picaresque") } );

21. NoRM - CRUD //Find all documents in a typed collection var artists = mongo.GetCollection ("Artists").Find(); Console.WriteLine(artists.FirstOrDefault().Name); //Query with a document spec var artist = mongo.GetCollection ("Artists").FindOne( new { Name = "The Decemberists" }); Console.WriteLine(artist.Albums.Count); //Count the documents in a collection long count = mongo.GetCollection ("Artists").Count(); Console.WriteLine(count);

22. NoRM - MapReduce //Add a Tags collection added to Artist class private IList _tags; public IList Tags { get { return _tags; } set { _tags = value; } } //Add some tags – use Set not PushAll, since Tags was null mongo.Database.GetCollection ("Artists").UpdateOne( new { Name = "The Decemberists" }, new { Tags = M.Set(new List () { "Alternative", "Folk Rock }) });

23. NoRM - MapReduce //Create map and reduce functons string map = @"function() { if (! this.Tags ) { return; } for (index in this.Tags) { emit(this.Tags[index], 1); } }"; string reduce = @"function(previous, current) { var count = 0; for (index in current) { count += current[index]; } return count; }";

24. NoRM - MapReduce //MapReduce class is responsible for calling mapreduce command MapReduce mr = mongo.Database.CreateMapReduce(); //Represents the document passed to the //db.runCommand in the shell example MapReduceOptions options = new MapReduceOptions("Artists") { Map = map, Reduce = reduce, OutputCollectionName = "Tags" }; MapReduceResponse response = mr.Execute(options); var collection = mongo.Database.GetCollection ("Tags"); Console.WriteLine(collection.Count());

25. NoRM - LINQ //LINQ provider exposed via AsQueryable method of MongoCollection var artists = mongo.Database.GetCollection ("Artists").AsQueryable(); //Find items in typed collection var artistsWithDecInName = from a in mongo.Database.GetCollection ().AsQueryable() where a.Name.Contains("Dec") select a; Console.WriteLine("First containing Dec in name: " + artistsWithDecInName.First().Name);

26. NoRM - LINQ //Find artists without pulling back nested collections //Note use of Regex name search var artistsWithoutAlbums = from a in mongo.Database.GetCollection ().AsQueryable() where Regex.IsMatch(a.Name, "ber", RegexOptions.IgnoreCase) select new { Name = a.Name }; Console.WriteLine(artistsWithoutAlbums.First().Name); //Find artists with a given tag var artistsWithFolkRockTag = mongo.Database.GetCollection ("Artists").AsQueryable().Where(a => a.Tags.Any(s => s == "Folk Rock")); Console.WriteLine(artistsWithFolkRockTag.First().Name);

27. MongoDB C# Driver 10gen developed and supported Consists of two primary components, a BSON serializer and the MongoDB driver Support for typed and untyped collections, MapReduce, and all CRUD operations Support for typed and untyped collections, MapReduce, and all CRUD operations Currently lacking a LINQ provider Currently lacking a LINQ provider Released about two weeks ago as version 0.7! Released about two weeks ago as version 0.7!

28. MongoDB C# Driver – The Basics private static MongoDatabase _mongoDatabase = null; static Program() { //MongoConnectioSettings used to create connection strings MongoConnectionSettings settings = new MongoConnectionSettings(); settings.Address = new MongoServerAddress("localhost", 27017); //MongoServer manages access to MongoDatabase MongoServer mongoServer = new MongoServer(settings); //MongoDatabase used to access MongoCollection instances _mongoDatabase = mongoServer.GetDatabase("AltNet"); }

29. MongoDB C# Driver - CRUD var artist = new Artist() { Name = "The Decembrists" }; //Inserting a document into a typed collection _mongoDatabase.GetCollection (COLLECTION).Insert(artist); //Updating (replacing) a document in a typed collection artist.Name = "The Decemberists"; _mongoDatabase.GetCollection (COLLECTION).Save(artist); //Updating a nested collection _mongoDatabase.GetCollection (COLLECTION).Update( Query.EQ("Name", "The Decemberists"), new BsonDocument("$pushAll", new BsonDocument("Albums", new BsonArray() { "Castaways and Cutouts", "Picaresque", "Hazards of Love", "The Crane Wife" })) );

30. MongoDB C# Driver - CRUD //Find all documents in a typed collection var artists = _mongoDatabase.GetCollection (COLLECTION).FindAll(); Console.WriteLine("Artist name: " + artists.FirstOrDefault().Name); //Query with a document spec var artist = _mongoDatabase.GetCollection (COLLECTION).FindOne( new BsonDocument { { "Name", "The Decemberists" } }); Console.WriteLine("Album count: " + artist.Albums.Count); //Count the documents in a collection long count = _mongoDatabase.GetCollection (COLLECTION).Count(); Console.WriteLine("Document count: " + count);

31. MongoDB C# Driver - CRUD var artists = _mongoDatabase.GetCollection (COLLECTION); //Find items in typed collection var artistsStartingWithThe = artists.Find(Query.Matches("Name", new Regex("the", RegexOptions.IgnoreCase))); Console.WriteLine(artistsStartingWithThe.First().Name); //Find artists without pulling back nested collections var artistsWithDecInTheName = artists.Find( Query.Matches("Name", "Dec")).SetFields("Name"); Console.WriteLine(artistsWithDecInTheName.First().Name); ////Find artists with a given tag var artistsWithIndieTag = artists.Find(Query.In("Tags", "Indie")); Console.WriteLine( artistsWithIndieTag.First().Name);

32. MongoDB C# Driver - MapReduce //Add some tags _mongoDatabase.GetCollection (COLLECTION).Update( Query.EQ("Name", "The Decemberists"), new BsonDocument("$pushAll", new BsonDocument("Tags", new BsonArray() { "Folk rock", "Indie" })) ); var artist = new Artist() { Name = "Sunny Day Real Estate", Albums = new List () { "How it Feels to be Something On", "Diary" }, Tags = new List () { "Indie", "Emo" } }; _mongoDatabase.GetCollection (COLLECTION).Save(artist);

33. MongoDB C# Driver - MapReduce _mongoDatabase.GetCollection (COLLECTION).Save(artist); //Create map and reduce functons BsonJavaScript map = @"function() {...}"; BsonJavaScript reduce = @"function(previous, current) {... }"; var result = _mongoDatabase.GetCollection (COLLECTION).MapReduce(map, reduce, MapReduceOptions.SetKeepTemp(true).SetOutput("Tags"));

34. In Depth: Meringue

35. Design Considerations Your object graph is your data model Don't be afraid to store data redundantly  Your graph might be redundant Not everything has to fit in 1 document Don't be afraid to store aggregate statistics with a document.

36. Object Graph as Data Model Generally speaking, most MongoDB drivers will serialize an object graph as a single document  The relationships of your classes creates an implied schema!  Migrating this schema is not trivial if you are trying to deserialize properties that did not or no longer exist Consider use cases carefully to avoid inefficiently structured documents Projection queries will be your friend

37. Redundant Data is OK Optimize documents for quick reads and writes Your application layer will have to maintain referential integrity! If every time you access a Post document, you need some of an Author document's data, store that data with Post Design simple classes for this redundant data for reusability (see AuthorInfo in Meringue)

38. Do Not Stuff it In 1 Document Nothaving formal relationships does not mean throwing away relationships Consider a user and his or her logged actions  The user would likely have a User class/doc with properties for name, email, etc.  User actions are generally write heavy and read out of band.  Don't clutter user documents - create a separate collection for user actions

39. Don't Be Afraid of Extra Data The schema-less nature of documents makes it easy to store meta data about that document – particularly aggregate data Consider a blog post with a rating feature  Each rating would be stored as a nested document of the post  Rather than compute vote totals and averages real time, simply add these properties to the document and update on writes

40. Case Study: RateMySnippet.com

41. Final Thoughts Eat food. Not too much. Mostly plants. - Michael Pollan

42. Final Thoughts Write Code. Not too much. Mostly C#. - John Zablocki

43. The Meringue Stack ASP.NET MVC  MVCContrib MongoDB  Mongo C# Driver Spring.NET  Recoil AutoMapper

44. Links http://dllHell.net - my blog http://dllHell.net http://www.CodeVoyeur.com - my code http://www.CodeVoyeur.com http://www.linkedin.com/in/johnzablocki http://twitter.com/codevoyeur http://mongodb.org - Official MongoDB site http://mongodb.org http://bitbucket.org/johnzablocki/meringue http://bitbucket.org/johnzablocki/codevoyeur- samples