Data Aggregation & Exchange Using the Ed-Fi ODS / API

Data Aggregation & Exchange Using the Ed-Fi ODS / API
Chris Moffatt Director of Technology, Ed-Fi Alliance Ben Meyers Senior Software Engineer, Double Line Partners

Agenda Overview of the Ed-Fi ODS / API - 20 mins
Accessing the API – Hands On - 55 mins Accessing the REST API directly Using the Client SDK Bulk API & Tools BREAK – 15 mins Security - 45 mins Security Concepts Security Demo / Hands On (Claimsets and API Profiles) Extensibility & Customization - 45 mins Extensibility Concepts Extensibility Demo / Hands On (Student Transcript example and API Composites) © 2016 Ed-Fi Alliance

Overview

In the Beginning…. The Ed-Fi ODS / API v2
In the Beginning….The Ed-Fi ODS / API v2.0 Grew Out of Tennesse DOE Field Implementation (also AZ and NE) TN Extended Source: TN DOE presentation

Ed-Fi Technology Components Version 2

Platform Fundamentals
An Open Source, Customizable System Made to be Secure Made to be Extended Code Generation Wherever Possible Built for Agile Development and Continuous Integration Support for Transactional & Bulk Modes © 2016 Ed-Fi Alliance

The Ed-Fi ODS / API is a highly customizable system The source code distribution has a complete set of features and can easily be set up to run on a development machine or test environment Some analysis, planning, and development work are required to put the complete system into production for an enterprise. See Platform Dev. Guide - Extensibility & Customization , Platform Dev. Guide – Configuration & Ed-Fi Exchange An Open Source, Customizable System Made to be Secure Made to be Extended Code Generation Wherever Possible Built for Agile Development and Continuous Integration Support for Transactional & Bulk Modes © 2016 Ed-Fi Alliance

Technology Stack A few things to note:
The Ed-Fi ODS / API platform may run on a Microsoft technology stack – but clients and consumers of the platform can be written in practically any language for any modern operating system. See the API Client Developers' Guide for details. The Ed-Fi ODS / API platform can run in a variety of server environments, including on-premises hardware or cloud-based platforms like AWS and Azure. See the Deployment section in this documentation for details. Many ODS / API implementers use the solution as a data source for the Ed-Fi Dashboards. As noted in the diagram above, the dashboards use the ODS as a data source, but leverage their own data access layer to pull information from a performance-optimized data mart. This direct approach may be suitable for some kinds of business intelligence systems as well. © 2016 Ed-Fi Alliance

An Open Source, Customizable System Built from the ground up to provide developers and agencies with a solution to keep data secure and private More on this in the session on security Made to be Secure Made to be Extended Code Generation Wherever Possible Built for Agile Development and Continuous Integration Support for Transactional & Bulk Modes Made to be Secure Everyone working with student information wants to know that their data is secure, and that students’ information is kept private. The Ed-Fi ODS / API was built from the ground up to provide developers with a solution to keep data secure and private More on this in the session on Security © 2016 Ed-Fi Alliance

An Open Source, Customizable System Made to be Secure Out of the box, the ODS / API core data model covers a wide swath of information related to the K–12 domain, with a focus on student achievement. The data model is easily extended to handle information specific to your environment. More on this in the session on extensibility Made to be Extended Code Generation Wherever Possible Built for Agile Development and Continuous Integration Support for Transactional & Bulk Modes © 2016 Ed-Fi Alliance

An Open Source, Customizable System Made to be Secure Made to be Extended Much of the data access code and API surface have architectural patterns defined, so the ODS / API leverages code generation techniques throughout the system. The Visual Studio solution includes several T4 templates to generate many of the C# data-related objects based on the data mode Code Generation Wherever Possible Built for Agile Development and Continuous Integration Support for Transactional & Bulk Modes © 2016 Ed-Fi Alliance

Architecture When the ASP.NET WebAPI framework receives the JSON payload, it is deserialized into the C# Resource class model. The data is then copied to the Entity model which matches the structure of the ODS database

Architecture EdFi_Bulk

An Open Source, Customizable System Made to be Secure Made to be Extended Much of the data access code and API surface have architectural patterns defined, so the ODS / API leverages code generation techniques throughout the system. The Visual Studio solution includes several T4 templates to generate many of the C# data-related objects based on the data mode Code Generation Wherever Possible Built for Agile Development and Continuous Integration Support for Transactional & Bulk Modes © 2016 Ed-Fi Alliance

An Open Source, Customizable System Made to be Secure Made to be Extended Code Generation Wherever Possible The solution ships with unit tests and integration tests that provide assurance that the system functions as intended. Under Agile development methodologies, code introduced to fix existing issues or provide new features should have accompanying test coverage, and should not break any pre-existing tests “Some assembly required” for deployments beyond developer machine. See Platform Dev. Guide - Deployment Built for Agile Development and Continuous Integration Support for Transactional & Bulk Modes © 2016 Ed-Fi Alliance

Logical Configuration

Load Balanced Deployment

An Open Source, Customizable System Made to be Secure Made to be Extended Code Generation Wherever Possible Built for Agile Development and Continuous Integration Bulk loading is useful for initial loading of data and solutions where data is updated in batches (e.g., for organizations that feed data on a nightly schedule) A transactional model is useful once you have data in the system. Individual records and fields can be updated in real-time (or near real-time) by client applications Support for Transactional & Bulk Modes © 2016 Ed-Fi Alliance

Platform Tools Swagger Documentation Site
Sandbox Administration Portal Security Configuration Tool © 2016 Ed-Fi Alliance

Swagger Documentation Site

Sandbox Administration Portal

Security Configuration Tool

Accessing the API – Hands On

Break – 15 mins

Security

Security Concepts Security Authorization Concepts Profiles
Authentication – Identifying the API caller Authorization – Deciding whether to service a caller’s request Authorization Concepts Resources, Taxonomy, Claim Sets, Authorization Strategies Profiles

Authentication The ODS / API uses 2-legged OAuth2 for authentication
Before accessing the resources in an ODS / API platform, client applications need to obtain an access token from the API platform This access token is validated on every call made to the API as a representation of the client's application key and secret. This document briefly outlines the steps involved. Clients and platforms talk to each other over HTTPS

Authorization - Concepts
Resources – The endpoints exposed by the API Taxonomy – The organization of Resources into a hierarchy Claim Sets – A named set of claims given to an API consumer (aka Role) Authorization Strategies – Domain-specific approaches for authorizing requests for specific Resources.

Authorization - Resources

Authorization - Taxonomy
The surface area of the API is large 82 “Domain” Resources 181 Types/Descriptors Resources Taxonomy introduced to facilitate authorization management We refer to these as “Resource Claims” Authorization Strategies (and CRUD permission) flow down

Authorization - Taxonomy
Out-of-the-Box Higher Order Resource Claims people – Students, Staff, Parents educationOrganizations – SEAs, LEAs, ESCs, Schools, etc. assessmentMetadata – Assessments, Assessment Families, etc. educationStandards – Learning Objectives and Standards relationshipBasedData – Other API Resources with relationships to EdOrgs or People primaryRelationships – Association Resources that must be established before access to relationshipBasedData can be granted systemDescriptors – Descriptors that are managed by the API host managedDescriptors – Descriptors that are managed by the API consumers types – All standard Ed-Fi “enumeration” Resources (corresponding to the XML Schema)

Authorization – Claim Sets
Claim – An assertion from a trusted source about something Claim Set – A named collection of Claims assigned to a caller (aka “Role”) Resource Claims in the Ed-Fi ODS API are logical, not physical Claims aren’t issued and transmitted between parties (e.g. SAML, JWT) Claims are “issued” based on the EdFi_Security database and the caller’s access token Cached locally in each Ed-Fi ODS API host process until token expires

Authorization – Claim Sets
Claim Sets enable the assignment of CRUD permissions to Resources Example: Education Organizations Claim Set: SIS Vendor (Read-only)

Authorization - Strategies
Enable domain-specific logic to be applied to authorization decisions once simple CRUD checks have passed. NoFurtherAuthorizationRequired – No additional logic is applied NamespaceBased – Authorization is based on ownership via Namespace AssessmentMetadata – Authorization is based on ownership via Namespace of the related Assessment or Assessment Family RelationshipsWithEdOrgAndPeople – Authorization is based on relationships the caller has with People and Education Organizations ODS/API v2.1.1 – Introduces support for Claim Set-specific authorization strategy overrides on resources Authorization=specific validations

Authorization - Strategies
RelationshipsWithEdOrgsAndPeople Example: SIS Vendor accessing a Student Relationships from the SIS Vendor’s associated LEA to the Student are verified in the Ed-Fi ODS In other words... LEA  School  StudentSchoolAssociation  Student

API Profiles - Overview
Background Feedback was received that Ed-Fi is large and unapproachable Some Ed-Fi implementations have the need to control who is reading and/or modifying particular parts of the API Resources Goals We wanted to make it easier to focus on a specific use case We wanted to make it easier to define and assign policies for reading and writing data without a need to modify the core model Solution A Profile definition identifies a subset of API Resources, and can also limit the data (properties, collections and collection items) that is available for reading and writing to hone in on a particular use case.

API Profiles - Overview
Profiles enable you to limit a caller’s ability to read or write: Specific resources Specific attributes on a resource SIS vendor can read the entire Student resource Nutrition system can read and write food service eligibility. Others can read other attributes, but not food service eligibility. Specific collections on a resource Specific items within a collection Special Ed system needs to read and write addresses. Library system can only read addresses. Certain systems may need to read all identification codes except the SSNs or driver’s license numbers.

Student Resource Profile: Nutrition Content Type: Write
Content Type: Read

Profiles – Content Types
Content Types (or “media types”) are: An Internet standard Used to describe the type of data being transmitted (e.g. text/html, application/json, application/vnd.ms-excel, etc.) In the Ed-Fi ODS API, special content types identify Requests that are using a Profile The convention for Profile-based content types is: application/vnd.ed-fi.{resource}.{profile}.[readable|writable]+json

Profiles - Authorization
API resources are still secured using CRUD permissions based on a Claim Set. Profile(s) can be assigned to a Vendor application. Once a Profile has been assigned to a caller, the Profile-based Content Types must be used (i.e. “application/json” will not be accepted). application/vnd.ed-fi.student.nutrition.readable+json application/json 

Profiles - Authorization
Authorization = Security Policy Profile = Data Policy

Security – Hands On

Extensibility & Customization

Extensibility Fundamentals
The basic pattern for adding extensions to the Ed-Fi ODS / API data model is to develop and place customizations into the Ed-Fi-ODS-Implementation directory. When developers complete the development for an extension to the ODS / API data model, re-running the “initdev” process Implementing an extension involves making changes to the following components: XSD Schema Database Scripts API Metadata Financial / HR Food Service Dashboards © 2016 Ed-Fi Alliance

API Composite Resources
As part of the evolution of the Ed-Fi ODS / API from primarily supporting data collection use cases to a widening set of data integration use cases, a mechanism was needed for exposing Composite Resources Composite Resource A read-only resource which combines data from multiple standard API resources in a single call Reduces the “chattiness” of the API interactions and provide a simple data integration experience for API consumers Standard API Composite Resources Enrollment Assessment Customizable Composite Definition Expressed in XML and compiled into the application © 2016 Ed-Fi Alliance

Example: Enrollment Section Composite

Extensibility & Customization – Hands On

Wrap Up

Data Aggregation & Exchange Using the Ed-Fi ODS / API

Similar presentations

Presentation on theme: "Data Aggregation & Exchange Using the Ed-Fi ODS / API"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Data Aggregation & Exchange Using the Ed-Fi ODS / API

Similar presentations

Presentation on theme: "Data Aggregation & Exchange Using the Ed-Fi ODS / API"— Presentation transcript:

Similar presentations

About project

Feedback