1. MANIPULATING DATA http://www.flickr.com/photos/maggiew/6145245962/

2. To manipulate data, you need a reference to the data… Welcome to Object keys Full Key = app engine application ID + separator + entity kind + separator + entity group + separator + entity ID key – Actually, each part is hashed before concatenation Typical ways to specify the entity ID key – You set it yourself as a String (or Long) – You let GAE automatically assign a Long – You let GAE automatically set an encoded String – You set it yourself as a custom Key object

3. Setting your own String as a key package edu.oregonstate.mobilecloud.lectures.clouddatastore; import javax.jdo.annotations.PersistenceCapable; import javax.jdo.annotations.Persistent; import javax.jdo.annotations.PrimaryKey; @PersistenceCapable public class Test2Simple { @PrimaryKey @Persistent private String title; public String getTitle() { return title != null ? title : ""; } public void setTitle(String title) { this.title = title; } 1.Declare @PrimaryKey 2.Before trying to makePersistent, be sure to set the value 3.Any existing entity of that type with that key is overwritten 1.Declare @PrimaryKey 2.Before trying to makePersistent, be sure to set the value 3.Any existing entity of that type with that key is overwritten

4. Letting GAE automatically assign a Long key package edu.oregonstate.mobilecloud.lectures.clouddatastore; import javax.jdo.annotations.IdGeneratorStrategy; import javax.jdo.annotations.PersistenceCapable; import javax.jdo.annotations.Persistent; import javax.jdo.annotations.PrimaryKey; @PersistenceCapable public class Test3Members { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Long id; public Long getID() { return id; } 1.Declare @PrimaryKey 2.Specify IDENTITY generator type 3.Do not try to assign the ID when instantiating object 1.Declare @PrimaryKey 2.Specify IDENTITY generator type 3.Do not try to assign the ID when instantiating object

5. Listing all entities of a certain kind <% PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class); List allTest3 = (List )query.execute(); for (Test3Members obj : allTest3) { out.write(" "+obj.getDepartment()+obj.getCoursenum() +": difficulty "+obj.getDifficulty()+" "); out.write(" "+obj.getDescription()+" "); out.write(" "); } } finally { pm.close(); } %> 1.Create Query 2.Execute 3.Cast to List of objects 1.Create Query 2.Execute 3.Cast to List of objects test5.jsp

6. Retrieving a subset of entities based on a value <% PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class, "department == :dd && coursenum >= :cn"); List allTest3 = (List )query.execute("CS", 400); for (Test3Members obj : allTest3) { out.write(" "+obj.getDepartment()+obj.getCoursenum()+": difficulty "+obj.getDifficulty()+" "); out.write(" "+obj.getDescription()+" "); out.write(" "); } query.closeAll(); } finally { pm.close(); } %> 1.Create Query with parameters 2.Bind parameter values on execute 3.Cast to List of objects 4.Close query when done 1.Create Query with parameters 2.Bind parameter values on execute 3.Cast to List of objects 4.Close query when done test5a.jsp

7. Retrieving an object by key <% PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class); query.setOrdering("id desc"); List allTest3 = (List )query.execute(); long maxId = allTest3.get(0).getID(); Test3Members highest = pm.getObjectById(Test3Members.class, maxId); out.write(highest+"..."+highest.getID()); query.closeAll(); } finally { pm.close(); } %> 1.Use getObjectById to retrieve 2.If object might not exist, catch JDOObjectNotFoundException or JDOException 1.Use getObjectById to retrieve 2.If object might not exist, catch JDOObjectNotFoundException or JDOException test5b.jsp

8. Deleting entities test5c.jsp <% PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class, "coursenum >= 600"); List toDelete = (List )(query.execute()); pm.deletePersistentAll(toDelete); query.closeAll(); } finally { pm.close(); } %> 1.Retrieve list of objects to delete (which seems inefficient?!?!) 2.Invoke pm.deletePersistentAll 1.Retrieve list of objects to delete (which seems inefficient?!?!) 2.Invoke pm.deletePersistentAll

9. By the way… Query has a closeAll() method. – close the query when you’re done with results Query has a deletePersistentAll() instance method. It is painfully slow in some SDKs. – If you use this method, be sure to assess how fast it is on very large sets of objects before you rely on it.

10. Now what if two people want to manipulate an object at the same time? … @PersistenceCapable public class Test1Object { @PrimaryKey @Persistent private String title; … @Persistent private TreeSet categories = new TreeSet (); public void addCategory(Integer category) { categories.add(category); } <% String title = request.getParameter("title"); if (title == null) out.write("provide a title"); else { PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Test1Object obj = null; try { obj = pm.getObjectById(Test1Object.class, title); } catch (JDOObjectNotFoundException doesNotExist) { obj = new Test1Object(); obj.setTitle(title); } obj.addCategory(Math.round(Math.rand()*100)); pm.makePersistent(obj); } finally { pm.close(); } %>

11. You need a transaction when your servlet needs to… Update an entity if it already exists, or create a new one otherwise Compute a new member variable value based on an existing value Modify multiple entities in a consistent way … Do anything where your servlet could potentially be confused if gets two simultaneous requests/hits from users

12. Setting up a simple transaction Transaction trans = pm.currentTransaction(); trans.begin(); Test1Object obj = null; try { obj = (Test1Object)pm.getObjectById(Test1Object.class, title); } catch (JDOObjectNotFoundException doesNotExist) { obj = new Test1Object(); obj.setTitle(title); } pm.makePersistent(obj); trans.commit(); 1.Tell the current transaction to begin tracking operations 2.Do your operations 3.Commit your changes 4.(On uncaught exception, changes are rolled back automatically) 1.Tell the current transaction to begin tracking operations 2.Do your operations 3.Commit your changes 4.(On uncaught exception, changes are rolled back automatically) test2.jsp

13. Transactions on GAE A transaction places an optimistic lock all of the entities in a certain group If any other transaction tries to modify entities in that group, then – The first transaction to commit is successful – The second (and later) transactions fail on commit In practice, contention can cause commits to fail… so just retry Very very similar to DBMS optimistic locking, which we all know and love, except for this concept of entity groups

14. Entity groups Groups partition the set of all entities in GAE Every entity is in exactly one group Every group has at least one entity Every group is very similar to tree – It has a root (which is used to identify the group) – Every entity has a parent (which for roots is the entity itself – this is where the tree analogy breaks) Overall, therefore, the datastore is a forest

15. In other words… A transaction can lock one, and only one, tree in the forest Oregon State CSECE CS361 CS496 Univ Oregon Info SysArt History IS434 IS210 Univ Calif Berkeley CSECE CS364 CS351 CS364

16. Entities in a group are stored in the same place(s) in distributed file system Remember: Your data values are copied (replicated) to many different servers. Storing entities that are locked together (a group) helps to improve performance Downside: an entity’s group cannot be changed after the entity is first saved Odd side-effect: entities are only affected by a transaction if they are retrieved by implicit specification of a file system location, e.g., retrieval by key or via a query that references an ancestor filter (e.g., parent key)

17. How to set up entity groups Option 1: Use JDO Relationships to specify that the child is dependent on the parent Option 2: Use custom JDO Key object that identifies the parent entity for a child entity

18. Option 1: Use JDO Relationships This is different than a compound object (an object that has multiple parts) It is when you have two entities, one of which is a child of another The parent “owns” the child in JDO-speak

19. Example code snippet … @PersistenceCapable public class Test3JDOEmployer { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) @Extension(vendorName = "datanucleus", key = "gae.encoded-pk", value = "true") private String id; @Element(dependent="true") private HashSet employees = new HashSet (); … @PersistenceCapable(identityType = IdentityType.APPLICATION) public class Test3JDOEmployee { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) @Extension(vendorName = "datanucleus", key = "gae.encoded-pk", value = "true") private String id;

20. Example servlet code Test3JDOEmployer boss = new Test3JDOEmployer(); boss.setName("Employer "+empname); for (int empid = 1; empid <= 3; empid++) { Test3JDOEmployee worker = new Test3JDOEmployee(); worker.setName("Employee "+empname); boss.addEmployee(worker); } pm.makePersistent(boss);

21. In the servlet… In the servlet – Just initialize and store the parent (no explicit child save) – On load, just load the parent and access child via the parent as needed FYI: Child objects are not loaded until they are read or written, so loading the parent can be faster.

22. Option 2: Use explicit JDO Keys Which I’m not going to show you in detail because it is much more hassle to set up, and it doesn’t really add any more benefits. But here’s how you can do it: – Only list the keys of the child in the parent – When instantiating the child, pass the parent’s key – In the child’s constructor, use KeyFactory to construct a child key, based on the parent’s key – See GAE Book 324-325 for details (note typos)

23. Summary of important points Use a transaction if two simultaneous hits to the server could lead to inconsistency Each transaction can only operate on a single entity group – To put multiple entities in the same group, use a JDO parent-child relationship If you need a transaction and you need to modify multiple entity groups, use cursors and transactional tasks