1. Simple SIS Service (sakora-csv)
Dan McCallum – Unicon, Kim Eke – UNC,
Patty Wolfe - Unicon
Sakai Conference – Denver, Jun 17, 2010
© Copyright Unicon, Inc., This work is the intellectual property of Unicon, Inc. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of Unicon, Inc. To disseminate otherwise or to republish requires written permission from Unicon, Inc.

2. Concepts
sakora-csv at UNC
Usage
Gotchas
Other Options
File Formats

3. Concepts

4. SSSS Data Flow Sakai C S V WWW Sakai SIS DB C S V C S V Saka i FS
Course
Management
APIs
CM Entities
Upload Tool
WWW
SIS
Sakai DB
C S V
line
line
Scheduled
Job
C S V
C S V
Saka i FS

5. Sakai CM Entities

6. CM-to-Site Mappings Site Site Groups/Sections WS Setup Tool CM APIs
CM Entities
GroupProvider
Provider IDs
Site
Users+Roles
Site Groups/Sections

7. RoleResolvers
<util:map id="org.sakaiproject.coursemanagement.GroupProviderConfiguration">
<entry key="siteRoleResolutionOrder">
<list>
<value>Instructor</value>
<value>Teaching Assistant</value>
<value>Student</value>
</list>
</entry>
<entry key="officialInstructorToSiteRole" value="Instructor"/>
<entry key="enrollmentStatusToSiteRole">
<map>
<entry key="enrolled" value="Student"/>
<entry key="wait" value="Student"/>
</map>
<entry key="sectionRoleToSiteRole">
<entry key="I" value="Instructor"/>
<entry key="GSI" value="Teaching Assistant"/>
<entry key="S" value="Student"/>
<!– (to be continued…) -->
</util:map>

8. RoleResolvers
<util:map id="org.sakaiproject.coursemanagement.GroupProviderConfiguration">
<!– (continued…) -->
<entry key="courseOfferingRoleToSiteRole">
<map>
<entry key="CourseAdmin" value="Instructor"/>
<entry key="I" value="Instructor"/>
</map>
</entry>
<entry key="courseSetRoleToSiteRole">
<entry key="DeptAdmin" value="Instructor"/>
</util:map>

9. sakora-csv at UNC

10. Sakai Growth

11. sakora-csv at UNC
Sakai section awareness killer feature relative to Bb
Still feeding files to Bb
Sites provisioned by request – would like to remove search/request features in WS Setup
Coordinator and Tech Support memberships added at Course Offering level

12. Usage

13. Snapshots Send all “current” records every time
(previous file) – (latest file) = deletions
Applies to:
sessions.csv
people.csv
sectionMembership.csv
courseMembership.csv
(Currently) no way to ignore omitted memberships

14. Upload curl -F "username=sakora-csv-sub-admin" -F "password=f00b4r" \
-F \
-F \
-F \
-F \
-F \
-F \
-F \
-F \
-F "runJob=true" \

15. User Credentials Use HTTPS, obviously
Any super user (site.upd in !admin)
Batch will always be processed as admin

16. Directory Polling Schedule “SIS CSV Data Loader” job
Configure (need to do this anyway):
Currently no way to detect partial uploads

17. Monitoring No UI (patches welcome!)
Tail catalina.out
Watch SAKORA_LOG
Remember processing is async w/r/t upload
Most logging is at DEBUG level
log.config.count=n
log.config.n=DEBUG.net.unicon.sakora

18. Monitoring Fatal errors… Batches never retried
{upload.dir}/*-finished.csv – Batch completed w/o fatal error
{upload.dir}/*-failed.csv – Batch aborted w/ fatal error
Fatal errors…
DB, file system exceptions
Shutdown
Some file syntax errors
Batches never retried

19. Site Membership Syncing
Pre-2.6 needed SAK-14259
Not typically needed any longer b/c Site Info refreshes site membership (SAK-14336)

20. What About LDAP? You’re fine
Don’t send people.csv if using LDAP for authN and attributes
User IDs in people.csv and referencing files need to be EIDs
Usually net IDs

21. Gotchas

22. Where Did My Terms Go? To show up in WS Setup, a term must be:
“current”
Within roster.available.weeks.before.term.start
sessions.csv is the authority for “currentness”
…even for terms never sent in that file!

23. I Don’t See My Rosters in Tool X
Kernel and WS Setup patches
2.7
SAK-18636
2.6.2
SAK-17790
KNL-320
2.6.x

24. I Can’t Remove Instructors!
Creating user automatically added to new Site
Trumps “provided” membership pre-2.7
SAK-18536
KNL-403

25. Instructors Don’t Show up in Roster Tool
It’s a bug – official instructors aren’t saved properly
Actually… it’s the only reason KNL-403 works with sakora-csv at all

26. My Section Names Have Commas
Escape all fields if you can
Use quotes to escape quotes
EID, Title, Description, Category, parentEID, esEID, coEID
“FOO101.SP10.001”,”Intro to “”Foo”””,”T/T, Intro to “”Foo”””,“Lect”,””,””,”FOO101.SP10”
“BAR101.SP10.001”,”Intro to “”Bar”””,”M/W/F, Intro to “”Bar”””,“Lect”,””,””,”BAR101.SP10”

27. My Files are Getting Huge!
(Currently) no way to ignore omitted memberships
Make sure you have indexes in {sakora-csv-src}/sakora-csv-impl/impl/src/sql/
Can truncate SAKORA_LOG (but not other SAKORA_* tables)

28. Sakai is Ignoring the First Line in my Files!
Set
sakora-csv doesn’t actually use header rows as metadata even if hasHeader=true

29. I Sent an Empty File and Nothing Happened
E.g…. I just want to delete all records of type X
sakora-csv always ignores empty/missing files.
Trying to avoid accidental data deletion
Allows for partial batch uploads
Sometimes helpful. Sometimes annoying.

30. How do I Configure X in WS Setup?
wsetup.skipCourseSectionSelection
sitemanage.cm.courseset.categories

31. How do I Configure X in WS Setup?
titleEditableSiteType
wsetup.disable.joinable
roster.available.weeks.before.term.start

32. How do I Configure X in WS Setup?
SectionFieldProvider

33. CourseSet X Doesn’t Show Up In WS Setup Drill-Down Search
Only top-level CourseSets appear in pull-down
Use parentEID field in courseSets.csv reluctantly

34. Other Options

35. CM Synchronizer A fine option
Modified versions used at UDel, URI, probably others
Undocumented file format
Code intended as a demo
Reads entire document into memory
XML – might simplify your life, might create strife

36. SAKORA “Proper” IMS LIS reference implementation
Appealing if you’ve already purchased PeopleSoft SAIP
Standardized outcome reporting (but no Sakai impl yet)
Event-based
Less widely deployed than sakora-csv

37. Others Home-grown solutions abound Check sakai-dev

38. File Formats

39. File Formats Detailed field definitions in Confluence
Note attached template files

40. Questions? dmccallum@unicon.net kim_eke@unc.edu pwolfe@unicon.net