1. The FIE = Data-based Problem Solving Texas Educational Diagnostician Association (TEDA) Annual Conference April 8, 2011

2. PIEMS October 2010 Snapshot Primary Disability Category NumberPrimary Disability Category Number AI7,569MR30, 490 AU29,536NEC3,776 DB128OHI54,436 DD446OI4,287 ED28,939SI88,958 LD187,296TBI1,336 MD5,576VI3,819 TOTAL = 446,592

3. Diagnostic Conclusions The determination of a disability category, (especially MR and LD) must be made through the use of professional judgment, including consideration of multiple information/data sources to support the eligibility determination. (TEA, §89.1040 Eligibility Criteria, Frequently Asked Questions) Information/data sources = statewide assessment results, formal evaluation test scores, RtI progress monitoring data, informal data (e.g., work samples, interviews, rating scales), anecdotal reports

4. Data-based Problem Solving In order to improve the diagnostic process, need to approach diagnosis and the FIE as Problem Identification & Analysis – process of assessment and evaluation for identifying and understanding the causal and maintaining variables of the problem

5. Problem-Solving: Identification and Analysis Problem Identification – define referral problem in specific, operational terms; only then can problem can be objectively assessed; think of problem as discordance between expected/desired levels of performance and current performance Problem Analysis – why is the problem occurring; functional assessment of academic and/or behavioral problems; problem analysis is the link between a well- specified problem and a problem solution

6. Data-Based Problem-Solving: Analysis In Analysis stage – – Gather Data and then form hypotheses – Hypotheses must be systematically tested, rejected or verified not just formulated – Systematic hypothesis testing must use low inference approaches (high inference approaches usually rely on theoretical within-person constructs that require assumptions that can’t be verified; e.g., observe a student and say he has low self-esteem) – Use analytic versus standard battery approach

7. Clinical Judgment A special type of judgment rooted in a high level of clinical expertise and experience – emerges directly from extensive data; based on clinician’s training, experience, and knowledge of person and environment Characterized by being: – Systematic (organized, sequential, logical) – Formal (explicit and reasoned) – Transparent (apparent and communicated clearly)

8. 4 Clinical Judgment Strategies Clarify and state precisely the question set Conduct/gather a thorough history Perform assessments Synthesize the data

9. Thinking Errors to Avoid Affective error – your “feelings” – e.g., incorrect stereotypes, misplaced empathy, or what you wish were true Anchoring Error – first bit of information “anchors” your mind on an incorrect decision Availability – happened recently or most dramatically Blind obedience - what authority said

10. Thinking Errors to Avoid Commission bias – Do something, anything Confirmation bias – find what you expect Diagnosis momentum – Piling on after an initial diagnosis Framing effects Mistakenly influenced by context Premature closure – deciding too soon Representativeness error – what is typically true

11. John O. Willis, 2010 Complex and difficult problems require carefully planned, very thorough, goal-directed comprehensive evaluations that will guide the examinees, parents, teachers, and other professionals in fully understanding the student’s strengths and needs as a basis for planning interventions that will ameliorate the problems and lead to improved functioning p. xxi, Forward to Mather & Jaffe (Eds.) (2010) Comprehensive Evaluations: Case Reports for Psychologists, Diagnosticians, and Special Educators

12. Time Consuming Process P-Time: Planning Time – once you have reviewed records need to plan assessment E-Time: Administration and Scoring of instruments, including conducting interviews, observations and other informal evaluation activities T-Time: Thinking Time – you must build this in to your evaluation I-Time: Interpretation/Analysis/Synthesis of Data T-Time: again! Prove hypotheses R-Time: Report-writing time

13. Determining the “Appropriateness” of the FIE (Etscheidt, 2003) “Adequacy” - Compliance with IDEA evaluation requirements – Use a variety of assessment tools – Use technically sound instruments – Utilize non-discriminatory procedures – Administer standardized tests validated for the specific purpose for which they are used – Administered by trained and knowledgeable personnel in accordance with instructions provided by the test producer

14. Determining the “Appropriateness” of the FIE (Etscheidt, 2003) “Scope” of the evaluation – Must be assessed in all areas of suspected disability – Should not use a limited selection of assessment tools – If there are reasons to suspect additional areas requiring assessment, then the scope of the evaluation must be broadened to include assessment in those areas – Must be comprehensive enough to address related services as well

15. Determining the “Appropriateness” of the FIE “Utility” - Provision of sufficient information to develop the IEP – Must determine/identify the educational needs of the child – Must discuss the impact of the child’s disability on educational performance – Must lead to the development of goals and objectives Reference: Etscheidt, S. (2003). Ascertaining the Adequacy, Scope and Utility of District Evaluations. Exceptional Children, 69(2), 227-247.

16. Do’s and Do not’s DoDon’t Have a MDTBe the only evaluator Follow the RIOT and ICEL Model; Be thorough Refer to “audit file” vs. conducting review; conduct a standard battery only approach Select most appropriate methods and instruments; Keep up with current literature and “best practice” Select instruments based on convenience; use outdated instruments; rely on information or procedures that are not current Administer procedures in standardized manner; Explain any violations Provide scores for tests not designed for the age of the student Score correctly; Write all responses verbatim Make errors; just put score versus what the student answered/did

17. Do’s and Do not’s DoDon’t Verify hypotheses and validateLeave “danglers” or just form hypotheses without proving them Write about student and not test; give examples of student’s performance Write about the test more than about the student Link conclusions and recommendations to data; make research-based recommendations Generate a conclusion without an adequate explanation; Make generic recommendations that are too broad/could apply to all students Proof FIE report for errors; Make sure report is finalized prior to giving it to parents Give out “drafts” Keep the data from which you based decisions Destroy test protocols

18. Anatomy of a Paragraph Identify cluster or indexThe Basic Reading Skills cluster measures single word decoding skills. Describe taskOn the LW ID test the student reads from a list of real words; on the WA test the student is presented with a list of nonsense words to read. Explain student performance with examples Jenny performed poorly on both tasks. She used a whole word approach to decode unfamiliar words (e.g., sentence=certainty; moustache=muscle), and when attempting to apply phonics to break down the word, could not do so correctly (e.g., bounties= bount teece). On nonsense words, Jenny had difficulty with short vowel sounds (e.g., nan=nane). This pattern indicates that Jenny has not acquired phonemic awareness skills consistent with age expectations. …

19. Typical Order for FIE Reason for Referral – Description of why the student in being referred for the FIE. Usually ends with a standard statement such as: The purpose of this FIE is to (a) determine the presence or absence of a disability condition(s), (b) identify and describe the student’s strengths/assets and weaknesses/needs, and (c) provide data that will assist the ARD committee in its deliberations regarding educational programming.

20. Typical Order for FIE Sociological including Educational History/ Educational Progression Review of Previous Evaluations (if applicable) Review of ARD or 504 documents including previous interventions and services At this point in the report, the chronology and background of student is clear and narrative takes you to current status. Also you have identified any cultural or experiential factors which may have an impact on decision making.

21. Typical Order for FIE Speech/Language/Communication (Articulation, Voice, Fluency, Intelligibility, Single-words to Language Use to higher level Language skills including pragmatics if applicable; Communication Samples) Physical/Developmental/Motor (Developmental Progression, Medical History, Vision, Hearing, Sensory if applicable, Medications, Motor skills - fine and gross; OT, PT, Audiological, etc. would go here if applicable) At this point in the report, we have data regarding language/communication, developmental, and medical status. Depending on the student, Physical could come before Language, especially if student has a particular condition that will impact your selection of instruments and interpretation of the actual test results.

22. Typical Order for FIE Intellectual/Cognitive (thorough evaluation of cognitive processes related to referral questions – e.g., if considering MR/ID focus on broad or general processes; for LD, careful selection of narrow abilities under these broad areas; developmental assessment would be here if applicable; also other processes not only on IQ measures need to be considered – e.g., phonological processing, executive function, attention, memory) Adaptive Behavior (can be informal or formal; need formal measures for MR/ID and AU; consider issues of informants and rating scales versus interview methods)

23. Sample intro statements for IQ &AB The following measures assess cognitive processing. IQ does not equate with innate or intellectual capacity but rather evaluates the various processes by which students learn. [If all processes are intact, learning should be intact. Conversely, if certain processes are deficient then there will be an impact (constraint) on learning.] Adaptive behavior refers to the degree to which the student is personally and socially self-sufficient. This level of independence is based on age expectations and is performance and not only ability based (e.g., a student may possess the ability to perform a skill but does not do so at an independent level, thus self-sufficiency is adversely affected).

24. What is cognitive processing? Cognitive Processes – higher level brain processes necessary for learning; processes involved in perception, thinking, reasoning, problem solving, learning, storing and retrieving information (Dehn, 2005) LD model – postulates that LDs are associated with cognitive processing deficits that significantly impair learning

25. Processing – Disability & Intervention Processing strengths and weaknesses associated with diagnosing LD, but Also important and associated with other disabilities If learning depends on processing, then having information about a student’s strengths and weaknesses can assist in design of interventions, selection of accommodations, and determination of modifications

26. Main Processing Components to Assess Attention processingProcessing speed Auditory processingShort-term memory Executive processingSimultaneous processing Fluid reasoningSuccessive processing Long-term retrievalVisual Processing Phonemic awarenessWorking Memory Planning

27. Typical Order for FIE Emotional/Behavioral/Social [discipline history, attention and task orientation, social relationships with peers and teachers, relating behavioral characteristics to developmental level, etc.; should include subheading for FBA and any specialized assessments if applicable (e.g., social skills)] If no significant emotional/behavioral issues, then this section is usually before IQ or Achievement. If there are significant emotional/behavioral difficulties, then this section usually follows IQ and Ach.

28. Typical Order for FIE Academic Achievement/Educational Performance Levels [4 types of data: Informal (e.g., interviews, grades), Criterion-referenced (e.g., benchmarks, TAKS), Curriculum-based (e.g., universal screening, RtI progress monitoring), Norm-referenced (standardized measures of academic achievement); important here is comparison of student to peers (e.g., rank) and expected criterion (e.g., DRA level should be 28, student is 10)] Depending on nature of referral and how you want data presented, the Academic section can come before the IQ section.

29. Typical Order for FIE Assistive Technology [must be “required”, thus must assess for this (e.g., writing sample with and without use of computer)] AT needs should be discussed given physical, cognitive, linguistic, and academic functioning levels; on most reports the AT section is at the end Vocational/Transition [when applicable, prevocational and vocational skills should be addressed; functional vocational evaluation would go here if applicable; measurement of interests and aptitudes; should also conduct skills analysis (e.g., cognitive, academic, behavioral, etc.)with transition goals and needs)]

30. Typical Order for FIE At this point in the FIE, we now have all the information to make decisions regarding diagnosis/classification and educational needs Conclusions Recommendations (all needs and related services must be addressed, e.g., AU supplement) – Lead-in statement (next slide) – Usually have categories for this section such as Educational Programming, Academic Skills, Behavior, Social, Transition, etc.) Assurances and MDT Signatures

31. Recommendations: Lead Statement This multidisciplinary evaluation is considered a valid representation of ____’s current levels of functioning in the areas assessed. The purpose of the FIE is to identify the presence or absence of a disability condition and provide information regarding the student’s needs within the educational setting. All determinations pertaining to eligibility, educational planning/programming, services to be provided, and placements remain the responsibility of the ARD committee.

32. XBA is Commonplace – Acknowledge the Procedure in Your Report The results presented in this report were compiled from tests that do not share a common norm group; however, test results have been interpreted following the cross- battery approach and integrated with data from other sources including educational records, parent/teacher interviews, behavioral observations, work samples, and other test findings to ensure ecological validity. Standardization was followed for all test administrations. No single test or procedure was used as the sole criterion for classification, eligibility or educational planning. Unless otherwise noted, the results of this evaluation are considered a reliable and valid estimate of [Student’s Name] demonstrated skills and abilities at this time. Adapted from D. Miller (2010)

33. References Essentials of Psychological Assessment Series. Hoboken, New Jersey: John Wiley & Sons. – Dehn, M.J. (2006). Essentials of Processing Assessment. – Flanagan, D.P. & Alfonso, V.C. (2011) Essentials of Specific Learning Disability Identification – Flanagan, D.P., Ortiz, S.O., & Alfonso, V.C. (2007). Essentials of Cross-Battery Assessment, 2 nd Ed. Intellectual Disability: Definition, Classification, and Systems of Supports. 11 th Edition of the AAIDD Definition Manual. (2010). Washington, DC: American Association on Intellectual and Developmental Disabilities. – Chapter 8 – Role of Clinical Judgment in Diagnosis, Classification, and Development of Systems Supports Other references noted on slides