1. Ventura County SELPA Pattern of Strengths and Weaknesses (PSW) Model: Psych TOT Training #2
Jenny Jones, Director, VCOE
Sandi Killackey, Social-Emotional Services Specialist, SELPA
Kim Charnofsky, Mental Health Facilitator, CVUSD

2. Agenda Checking in Moodle Access Planning as a Team
Evaluating Academic Strengths and Weaknesses
COMPARES

3. To assist assessment teams in their collaborative work during the assessment process, a Planning Worksheet was created (pages of the manual). On the first page of this worksheet, teams can indicate important dates related to the student’s timelines. In addition, the team can document the specific reason for referral to assist the team in focusing the assessment. The table in the middle of the page can be used in conjunction with the COMPARES document to document possible areas of focus for the assessment in the area of processing.

4. COMPARES
The Comprehensive Organizational Matrix of Processing-Achievement Relations, Evaluating Significance (COMPARES) is a tool that assessment teams can use in many points within the assessment process. Teams can use this tool when planning for their assessment as well as reference after the assessment has been completed.
The Overview of the COMPARES is a one page document quickly summarizing the strength of the relationship between the processing area and the academic area, along with a page reference where more complete information can be found.
While the COMPARES provides detailed information summarizing a wide range of research articles, the document is not comprehensive of all of the research that exists. In addition, new research is being completed all the time. However, the document does provide assessment teams with a great road map of the research that does exist at this time.
In the future, on the Ventura County SELPA website, there will be a more expansive version of the COMPARES as well as an annotated bibliography of the information within this document.

5. This is the second page of the planning document
This is the second page of the planning document. At the top of the page, assessment teams can mark which assessment approach the team will be utilizing when working with the specific student. The table allows the teams to determine which areas need to be assessed for this particular student, who will complete the assessment and which tools may be used to assess this area.
When you are doing training in the district, you may want to show this document and discuss ways that it can be used. When you are training, you may want your psychologists to read through the narrative in this section of the manual (pages 56-57) so that everyone has the background knowledge behind the purpose of this form. Please remember that SAI teachers and SLP were hopefully in attendance at the overview; however, not all of the groups have had their trainings in their district/county office. Therefore, the information on this planning sheet may have not been disseminated to all of the assessment team members at the time of your training.
On the Moodle site, both the pdf and a word document of this form is available for download. Assessment teams are welcome to use the word document to type directly into or to make changes that are needed for their specific teams.

6. How will you train: Planning as a Team
Have participants discuss in dyads and then share out some ideas of how they will train the information on the Planning as a Team section. How do they see their district utilizing this form?

7. Academic Assessment Standardized Other Academic Data Sources
List of tests in manual
Age vs. Grade Norms
Other Academic Data Sources
Minimum of 3
Guidelines for cut-off scores
Observation
Within the Evaluating Academic Strengths and Weaknesses section of the PSW Procedural Manual (starting on page 60 of the manual), there is a narrative and then some additional tools. The tools include a list of frequently used standardized academic achievement tests organized by the 8 academic areas outlined by CA Education code.
Within the narrative is a brief explanation of when to use age versus grade norms for academic tests. The narrative and Guidelines for Cut-off Scores also provides information about the needed other academic data sources to substantiate the standardized testing that was found.
Lastly, this section of the manual also provides some observation documents that assessment teams may want to utilize during the assessment. These observation documents can be used during observations, or in an interview with a teacher and/or parent.
Discuss what data sources you have within your district?

8. How will you train: Academic Assessment
Have participants discuss in dyads and then share out some ideas of how they will train the information on the Academic Assessment section. How will the psychologists collaborate with the SAI Teacher Trainer in their district on this topic?

9. COMPARES Comprehensive Organizational Matrix of Processing-
Achievement
Relations,
Evaluating
Significance
The COMPARES acronym was carefully designed to explain exactly what the tool is and does. It is a “comprehensive” tool in that it includes all of the processing areas and academic achievement areas specified in California’s Educational Code. It provides an “organizational” structure that makes it fairly simple to get the information you need, by displaying in a “matrix” format the research review of “processing-achievement relations,” and it uses a numeric key as a way of “evaluating the significance” of the relationship between processing and achievement areas.

10. PSW Premise Based on several core research-based principles:
Specific Learning Disabilities are characterized by neurologically-based deficits in cognitive processing (NASP, 2007). This conclusion is supported by a meta-analysis that found significant processing differences between students with SLD and students without SLD (Johnson, Humphrey, Mellard, Woods, & Swanson, 2010).
Research has demonstrated the existence of specific cognitive processes (Flanagan, Ortiz & Alfonso, 2013; Hale & Fiorello, 2004). Researchers are also in agreement that sound tools and measures exist to assess these cognitive processing areas (LDA, 2010).
Research has also found links between various cognitive processes and specific areas of academic achievement .
Kim
Specific Learning Disabilities are characterized by neurologically-based deficits in cognitive processing. This is a within learner trait. Research supports that students with SLD process information differently than neuro-typical students.
Research has demonstrated the existence of specific cognitive processes. E.g. auditory processing, visual processing, working memory, etc.
Researchers are also in agreement that sound tools and measures exist to assess these cognitive processing areas
Research has also found links between various cognitive processes and specific areas of academic achievement. See Comprehensive Organizational Matrix of Processing and Achievement Relations, Examining Significance (COMPARES) which can be found in the PSW Procedural Manual.
This slide was in the ToT #1 powerpoint, and this time we are emphasizing training on the COMPARES. The COMPARES is found in the PSW Procedural Manual, pages 85 through 113.

11. Birth of the COMPARES Subcommittees:
wrote definitions of processing areas;
reviewed existing grids used by other researchers and other psychologists;
overlaid three existing grids to see how they corresponded;
collected and reviewed bibliographies from the other grid-makers, where available.
Processing area definitions were the basis of the beginning of the COMPARES Glossary, which grew over time. The Glossary for the COMPARES follows immediately after the COMPARES in the PSW Procedural Manual, starting on page 114 and going through page 136.
The three most useful grids and bibliographies that were reviewed were those prepared by teams in Eugene, Oregon; Portland, Oregon; and the work of Dr. Milton Dehn, author of Essentials of Processing Assessment.

12. Grid Review Findings Yes-No Format
Oral Language & Listening Comprehension incomplete
Unique terminology
Differing meanings of basic terms
Inconsistent use of research-based terms
Not “California” processing areas
Grid conclusions were not consistent with one another
All of the grids used a “yes” “no” format for determining whether there was a significant relationship.
When the research team started reading articles, it turned out that the “yes-no” format of the other grids simplified reality too much. While existing research shows that some processing areas are clearly related to particular academic areas, and some are clearly not related, quite a few fall somewhere in between, and a number of areas are still under study. The “yes-no” format was not helpful for the cases where the relationships fell somewhere in between.
Some of the grids excluded oral language & listening comprehension or included it but indicated that research was underway and they could make no conclusions. Because there is debate in the field about whether oral language and listening comprehension should really be considered academic areas, or are processing areas themselves, some grids that were reviewed didn’t include these areas, or had not yet reviewed or summarized research in these areas.
Some of the grids used their own unique terminology. Their teams explained these terms were intended to make things easier to understand for parents. For example, instead of using the term, “Executive Functions,” one team used the term, “Mental Control.” Instead of using the term, “Processing Speed,” one team called it simply “Processing.”
The grids’ terms did not correspond to other grids’ terms, even for basic areas like “visual” and “auditory.” One grid included the categories of “auditory” and “visual,” and subsumed “phonological processing” under the category of “auditory,” and “orthographic processing” under the category of “visual” – whereas another grid called out “auditory processing” as separate from “phonological processing” – and another grid had a “visual-spatial” plus a separate “orthographic processing” area, and had no “auditory processing” per se, only “phonological awareness” and “language.”
Some of the grids reflected terms used in the research, others did not. Terms used in the research include, for example, “Rapid Naming Skills” or “Rapid Automatic Naming,” which has been studied quite a bit in recent years, looking at its relationship to various academic areas. One grid called this area out specifically, and also had a category for Long-Term Storage-Retrieval – another team did not have either of these categories, although had a general category called “memory” – and a third subsumed “rapid automatic naming” under the category of Long-Term Retrieval.
None of the grids corresponded exactly to California processing areas. Grids were not made by California teams, and none corresponded with our five areas of processing that we see on our SLD page: auditory, visual, cognitive abilities, sensorimotor, and attention.
Even when terms were reclassified to be more consistent, when overlaid, the grids did not show identical findings related to processing-achievement relations. Most interesting of all, even when the terms used by the other teams were re-classified to be more consistent, when they were overlaid, they did not produce a consistent set of conclusions about which processing areas are most related to which achievement areas.
Thus, a new grid, specific to California, inclusive of all of the processing and achievement areas, including sensible sub-areas that reflected the terms used in the research we were reading, and showing gradations in the relationships between processing and achievement areas was needed.

13. The Research Process
A team of volunteer school psychologists and graduate students were enlisted to read research articles.
Initially, people signed up to read about their “favorite” processing areas. Ultimately, people read in whichever areas were needed to complete the grid.
Initially, we planned to limit our review to published peer-reviewed journal articles in educational psychology and neuropsychology, but we expanded the research base that was reviewed to be more comprehensive.
Each researcher wrote a brief summary of what an article said about processing-achievement relations which became the basis of the findings in the COMPARES, and will be available online in the ABC.
The Team, originally called, “Team Matrix” and finally “The COMPARES Team,” was made up of your colleagues, school psychologists from several Ventura County districts, as well as from two graduate programs. Although each COMPARES Team member had a chance to sign up for her “favorite” processing area, it quickly became clear that this was a huge task and that everyone would read whatever was needed to fill in blank spots on the grid.
Although the team started out searching data bases with published peer-reviewed journal articles in educational psychology and neuropsychology, we needed to ultimately look in related fields and specialized areas, including reading from the literature in speech/language pathology, occupational therapy, optometric science, and fMRI studies. We also reviewed recent textbooks from well-respected researchers in the field, since these documents integrated recent research and provided citations to support their research. We also reviewed well-designed correlation studies, and a number of reviews or synthesis works, including meta-analyses.
The summaries that people wrote will be posted online soon, in a document called the ABC, or “Annotated Bibliographic Citations.” In the References section of your PSW Procedural Manual, beginning on page 160 in the hard copy manual, you will see that some of the articles have an “ABC” in parentheses after them – this means that the article was read by our team, is summarized in the ABC, and the information from the article was factored into the findings on processing-achievement relations in the COMPARES. Some of the citations in the References section say “KEY” after them – this means the work was relied upon as a “key” research source. The other abbreviation you will see there is “HT,” which means that the work can be found in our ABC, but is excerpted from a key source, which was the work of Jim Hanson and Monet Templeton (hence HT), in the Draft Meeting of the Minds document from 2013 called Guidance for the Identification of Specific Learning Disabilities: Meeting of the Minds Committee for Specific Learning Disabilities, Using Patterns of Strengths and Weaknesses Methodology and Research.

14. The COMPARES Key
A key was created so that our Team of researchers could evaluate the strength of the processing-achievement relations in a consistent manner. The key evolved as we read and needed to come up with categories that reflected our findings. The final COMPARES Key is on page 94 of the hard copy of the PSW Procedural Manual and a discussion of how to use it is found on page 92 in the Manual. Here is an excerpt from page 92:
How to Interpret the COMPARES Key of Rating Symbols:
The Key uses a five-point scale to rate the relationship between processing areas and academic achievement areas, based on existing reviewed research.
• Relationships that have a rating of “four” will suggest to the practitioner that there is strong convincing evidence of processing-achievement relations.
• Scores of “three” suggest convincing evidence, but may not be unanimous among researchers, and/or may not have the explicit research base that a score of “four” would imply.
• Relationships marked with a “two” would need to be carefully considered by practitioners; if a finding of a more significant processing-achievement relationship for a particular child than the COMPARES research supports is to be considered, the team would need to carefully document the evidence.
• Relationships marked with a “one” indicates either weak or little relationship, or studies done without strong foundations.
• A null sign or blank in the COMPARES indicates that no research was discovered that supports the relationship at this point in time.
• On a few occasions, the rating differs depending on a student’s age, which is noted.

15. Ratings Reconciled & Gridded
All summaries and ratings from our team were compiled into one document of annotated bibliographic citations (ABC).
The study authors’ names, the study dates, and a brief one- or two-line summary of each study were all placed into the Expanded COMPARES grid in the appropriate boxes, with more recent study dates listed at the top, going down in order to earlier studies. (next slide)
Ratings for each section were reconciled. Studies that received “stand-alone” ratings by our team of researchers were considered in light of the body of readings from all of our researchers on that topic, and final ratings were determined.
Final key numbers were placed into the appropriate boxes of the grid and the author references and dates were removed from the final version of the COMPARES, which is in your procedural manual.
The document of Annotated Bibliographic Citations or “ABC” contains research summaries written by nine different team members. In addition to these brief summaries, which range from a short paragraph to more than a page, there are even more-distilled summaries in the Expanded COMPARES, with just a line or two of the key points, written after the study authors’ names and publication dates of their research. A sample is on the next slide.
Because different studies often used differing terms and sometimes reached different conclusions, after our researchers submitted their summaries, we needed to consider entire sections together. For example, we might have had two researchers reading separately on the same processing-achievement relations in the area of phonological processing and reading fluency, and each rated their studies as a “2” because the research seemed to be preliminary, without other studies supporting them. In reviewing the section together, it might become clear that there actually had been several studies with consistent findings, so the rating would have been increased to a “3.”

16. Expanded COMPARES Sample
The Expanded COMPARES is going through final editing and formatting. Once online, teams will be able to use this grid to quickly see which studies the ratings are based upon, and to read in more detail what the research says about these relationships. Above is a sample from the Expanded COMPARES, where you can see how Phonological Processing relates to Basic Reading Skills (reading decoding) and Reading Fluency, with the authors’ names, study dates, and a line summarizing the research.

17. Processing Areas’ Relations to Each Other
Processing areas overlap, as no part of the brain works in complete isolation
Some processing areas are more related to each other than others
“Processing clusters” are described in Dehn’s Essentials of Processing Assessment
Memory Cluster
Executive Cluster
Language Cluster
Visual-Motor Cluster
Before practicing with the COMPARES, there are a few more points to remember. These are found in the PSW Procedural Manual, on pages 86 through 93.
Multiple brain structures are involved in any one function, and measuring cognitive processes in isolation is challenging. Yet, we do it. Because some processing areas are more related to one another than others, we can think of processing “clusters,” as described in Dehn’s Essentials of Processing Assessment (pages 46-48). If there is a suspected weakness in one processing area in a cluster, it is a good idea to investigate the other areas, as well.

18. “Cognitive Abilities” as a “Processing Area”
California Ed Code includes “Cognitive Abilities” as a processing area, and explicitly lists Association, Conceptualization, and Expression as part of this area: Association=Memory Conceptualization=Fluid Reasoning, Problem-Solving Expression=Oral Expression, Language Processing California Ed Code does not exclude consideration of other “cognitive abilities” that are related, such as: Rapid Automatic Naming/Rapid Naming Skills Executive Functions Processing Speed/Perceptual Speed All of these are included in the COMPARES.
From page 88 of the PSW Procedural Manual (hard copy):
When California Education Code lists “Cognitive Abilities” as a processing area, the text
explicitly includes association, conceptualization, and expression. Definitions for these and
other processing-related terms are found within the glossary, but the interpretation of the simple
equivalents to these terms in the assessment vernacular would equate association with
“memory” of all kinds, conceptualization with “fluid reasoning” and “problem-solving,” and
expression with “oral expression” and “language processing.” These are also terms that are
used by major test publishers to define the factors that are being measured during a
psychoeducational battery.
In addition, the Education Code definition of “Cognitive Abilities” does not appear to specify
exclusion of other cognitive abilities that might be related to those processing areas that are
specifically mentioned. Therefore, the Cognitive Abilities section of the COMPARES includes
the processing abilities of Rapid Automatic Naming (RAN) (which taps into long-term
memory/storage and retrieval), Executive Functions (a “gateway” processing area that helps the
brain organize and use all of the other processing areas), and Processing Speed/Perceptual
Speed (measured as such during research projects and variously covering visual, auditory, or
sensory-motor speed).

19. “Attention” vs. ADHD
Studies tend to use students with Dx of ADHD rather than those who have been identified with an attention processing deficit.
More research is needed that specifically examines the components of attention, and how these are related to academic areas.
There is currently a lack of solid research demonstrating strong associations between attention and academic areas, with the exception of Math Calculation.
Careful observations and clinical judgment will be essential in cases where attention is suspected as being the primary cause of academic weakness(es).
One of the challenges in reviewing the literature on attention as a processing area is that most studies that examine attention use students with ADHD. While this is convenient for the researchers since someone has already provided a diagnosis, we don’t have access to information about the test subjects beyond their diagnosis – was their difficulty primarily attention? impulsivity? hyperactivity? – which makes it hard to interpret results.
As research emerges that evaluates attention by component parts (for example, focused attention, sustained attention, selective attention, alternating/shifting attention, or divided attention), it would be anticipated that a greater clarity of connection will emerge between the attentional components and the academic achievement areas.
On page 95 of the PSW Procedural Manual, you can see that at this point in the evolution of the research base, only Math Calculation has a research base that shows strong relations with attentional weakness. However, as assessment team members are aware, based on clinical experience and many hours of classroom observation, attention is a foundational processing area, and can impact every academic area when a student is not able to be engaged. Your observations and clinical judgment will prove essential in cases where attentional processing weaknesses appear to impact academic functioning.

20. Language: A Two Axis Item
“Language” comes under the category of “Cognitive Abilities” on the left or vertical axis of the COMPARES:
“Expression” refers to Oral Expression
“Language Processing” includes Crystallized Knowledge such as vocabulary
“Language” also appears on the top or horizontal axis of the COMPARES, since two academic achievement areas are language-based:
Listening Comprehension
Oral Expression
Language as a Process, Language as an Academic Skill (p. 89 of PSW Procedural Manual)
Language has the special distinction of being both a “process” and an “academic skill.” A
student may have a neuropsychologically-based weakness in processing incoming language or
in expressing herself through language, and/or a student may have an academic skills
weakness in Listening Comprehension and/or Oral Expression that could be caused by a variety
of cognitive processes (not just a weakness in language processing, per se) (Dehn, 2014a)
Students with these various challenges who are eligible for special education services may be
identified as having a Specific Learning Disability, and/or they may be identified as having a
Speech/Language Impairment. Either way, because of the unique status of language, there are
language-related categories on both axes of the COMPARES. In several cases, where a grid
intersects that would show where language processing is related to an academic achievement
area related to language, there are no citations or ratings, since it is evident that the two areas
are overlapping. Very few studies attempt to evaluate whether language processing is related
to Listening Comprehension and Oral Expression, since it is implicit that their relationships are
strong and not mutually exclusive.
The “crystallized knowledge” skills that include possessing general information, comprehending
the world around, and maintaining a trove of vocabulary words are not included in the
COMPARES as processing areas per se, since they are not thought to involve processing so
much as a store of knowledge, to be “used” by other processing areas during learning
(Dehn,2014a). However, because many research studies use vocabulary as an indicator of
language skills, there are some references to studies involving crystallized knowledge,
vocabulary, and “Gc” within the COMPARES, found in relation to the language categories.

21. Developmental Notes (Dehn 2014a)
 Mature Early After Gradual Development (plateau in early elementary school)
Auditory Processing
Fine Motor Processing
Long-Term Recall
Phonological Processing
Visual-Spatial Processing
Mature Early After Rapid Development (plateau in early elementary school)
Processing Speed
Mature Late After Gradual Development (maturation occurs during adolescence)
Attention
Executive Functions
Fluid Reasoning
Oral Language
Working Memory
Processing Development Changes as Students Grow (p. 92 of PSW Procedural Manual, hard copy)
The COMPARES includes “Developmental Notes” to remind users that, although all of the
processes begin to develop around the same time in early childhood, the pace of development
varies by processing area, and the primary process a student relies on for a particular task may
change over time (Dehn, 2014a)
Factor loadings (indications of what a subtest is primarily
measuring) for some processing subtests change over the course of development. For
example, a visual-spatial subtest designed to measure fluid reasoning in older students may
actually measure visual-spatial ability in a younger student more than it measures fluid
reasoning. The test performance of younger students typically relies on fewer processes than
that of older children.
Also, when an essential process is underdeveloped at the time of testing,
it may have undue influence on subtests designed to assess other processes. Thus, a young
student’s limited ability to sustain attention can have a strong influence across much of a
cognitive battery.
When considering which processes relate to a student’s academic achievement performance,
the student’s developmental stage and the timing of the maturation of processing areas should
be carefully considered. Where research supports the finding of a difference in significance
between a processing-achievement duo based on age differences, the COMPARES may list
two separate numbers, one for each age group studied. The practitioner should be sure to
consult the appropriate rating for the student’s age group.

22. All Brains Are Different
Clinician judgment and experience are essential in interpretation of assessment results using the COMPARES.
Brains differ by size and shape of the various structures, by connectivity, by gender, and of course the same brain changes over time. Research on processing areas deals with majority effects and levels of significance for groups of subjects. In the real world of assessments, we are dealing with individuals with unique brains. The COMPARES should not be used to exclude the possibility that, in an individual student, a particular processing weakness might affect academic performance in a way that is not consistent with known research findings. Because all brains differ, individual profiles may differ from the norm. Clinician judgment and experience are essential to interpretation.

23. When to use the COMPARES?
Determine and define the hypothesis about strengths/weaknesses at the outset of the case
During assessment as you test your hypothesis
As you complete your assessment and meet with the assessment team to share results and plan for the IEP meeting, to verify the research-based link between processing weakness(es) and academic weakness(es)
PSW Procedural Manual (p. 19 hard copy)
Overview of the Model shows the COMPARES being used to help define the hypothesis about a student, during hypothesis testing, and again at the end of assessment when you are meeting with your assessment team, to verify the link between the processing deficits and academic weaknesses.

24. Case Study #1: 3rd Grade Boy
Read the case of Jeff.
If you need to see processing area definitions, consult the glossary, which begins on page 115 of PSW Procedural Manual.
Circle the suspected areas, then compare your responses with your neighbor.
Consult the COMPARES as you discuss your hypotheses.
Be ready to share with the larger group.
See separate handout with Case Study #1.

25. Case Study #2: 7th Grade Girl
Read the case of Anisa.
Underline or highlight key points.
Do Anisa’s test scores make sense, given her history and the information in the COMPARES? Which of her current processing weaknesses link with her current academic weaknesses? Does it look like she will continue to qualify under PSW?
Discuss with your neighbor.
See separate handout with Case Study #2.

26. Case Study #3: 1st Grade Girl
Read the case of Callie.
Underline or highlight key points.
Based on the weak academics, the weak processing areas, the strong processing areas, your classroom observation, and the teacher’s report about the student, does Callie’s profile make sense when you consult the COMPARES? That is, is there a research-based link between the identified processing weaknesses and the identified academic weaknesses? If yes, what are the strongest links? If there are no strong links, is there a case to be made for linking one or more processing deficit(s) to an academic deficit(s)?
Discuss with your neighbor.
See separate handout with Case Study #3.

27. Case Study #4: 9th Grade Boy
Read the case of Zack.
Underline or highlight key points.
To explain Zack’s low academic scores, what additional processing area evaluation might you do, if any? Consult the COMPARES as you consider.
Discuss with your neighbor.
Finish by considering how and whether you will use case studies as you teache your colleagues about the COMPARES.
See separate handout with Case Study #4.

28. Jenny
Are there additional resources that the trainers need?

29. Thank you! Jenny Jones jjones@vcoe.org
Sandi Killackey
Kim Charnofsky