Presentation is loading. Please wait.

Presentation is loading. Please wait.

Copyright : Cognadev UK Ltd Dr Maretha Prinsloo The future of assessment.

Similar presentations


Presentation on theme: "Copyright : Cognadev UK Ltd Dr Maretha Prinsloo The future of assessment."— Presentation transcript:

1 Copyright : Cognadev UK Ltd Dr Maretha Prinsloo The future of assessment

2 Copyright : Cognadev UK Ltd Demands on assessment:  dynamic complexity  integrated and contextualised nature of human functioning  collective consciousness & the evolution of consciousness  ethical considerations  societal needs  technology  theoretical trends The future of assessment

3 Copyright : Cognadev UK Ltd The future of assessment Thus much room for improvement of current assessment methodologies

4 Copyright : Cognadev UK Ltd Psychometrics criticised for:  little innovation  commercial viability “We have a relatively stable, mature, market consisting of a few standard assessment formats … which are liberally branded by a variety of companies. These tests barely differ in content, accuracy, or utility; they differ solely in terms of their marketing.” (Barrett, 2011) Barrett regards conventional psychometrics as a “pathology of science” The future of assessment

5 Copyright : Cognadev UK Ltd Psychometrics: use of limited experimental designs weak quantification weaker assessment methodologies causal inference based on correlational models The future of assessment

6 Copyright : Cognadev UK Ltd Research focus on: the number of constructs the level at which these are to be conceptualized the structure of the model (single, multiple, hierarchical, facet, phases, interactive, cube, overlapping, linear-causal, quadrants…) the statistical methods to be used (correlations or correlations) the constructs e.g. IQ, personality and EQ researched via individual differences The future of assessment

7 Copyright : Cognadev UK Ltd The Scientific status of Psychology explanatory sciences descriptive sciences speculative sciences These fields are characterized by differences in: ontology (entities / object of study) epistemology (how we get to know) methodology (investigative techniques) The future of assessment

8 Copyright : Cognadev UK Ltd Psychology: a descriptive science with a weak rational basis and only some empirical grounding David Freedman (1991), a mathematician, recommends a “low-tech” approach which “relies on intimate knowledge of the subject matter and a meticulous research design …” “Simplicity after complexity” given robust design and rigour The future of assessment

9 Copyright : Cognadev UK Ltd Why do we want to test commercial viability statistical challenges posed pragmatic considerations of HR clinical applications educational development The future of assessment

10 Copyright : Cognadev UK Ltd The future of assessment The future of assessment: - Constructs - Methodology - Innovation - Dynamic modeling - Integration - Contextualisation - Internet - Statistics - Effectiveness of assessment - Legal & ethical - Practical utility - International collaboration - Psychophysiological

11 Copyright : Cognadev UK Ltd Future constructs: Intelligence → cognitive processing → wisdom → intuition Personality assessment → motivation → consciousness Integrity → compassion Leadership → intention The future of assessment

12 Copyright : Cognadev UK Ltd Future methodology Statistics not well suited to Psychology Johnson (1936): "Those data should be measured which can be measured; those which cannot be measured should be treated otherwise.” “Statistics has failed to demonstrate the true structure of the attributes it is studying. … It has failed to understand the difference between quantity and quality as distinguishable modes of being” (Grice, 2012) The future of assessment

13 Copyright : Cognadev UK Ltd Statistics in Psychometrics: averages group membership and category bell curve (not for random data / qualitative differences) standard deviation correlations null hypotheses (no proof) The future of assessment

14 Copyright : Cognadev UK Ltd Dilemma: meaning lost via testing and statistical analysis Resulted in: devising more complex analytical techniques without reconsidering the fundamental premises But, psychology operates in a subtle, complex, dynamic and interactive world, one that appears random because we don’t understand the dynamics involved Although statistical inference has not served psychology that well, formal analysis of data is important in any science The future of assessment

15 Copyright : Cognadev UK Ltd Statistics “Substantive methodological synergies” (Marsh et al 2009)  integrating multilevel and structural equation approaches  to control measurement and sampling error The future of assessment

16 Copyright : Cognadev UK Ltd A call for Innovation: An approach which caters for the dynamic and complex nature of human functioning is required Brad Woods (2011): very little progress except for Observation Oriented Modelling (OOM) – behaviour seen as goal directed and purposeful, not mechanical and responsive Barrett (2011, 2012): Innovations include: Cognitive Process Profile (CPP) Read et al’s (2010) neural net research in personality Gigerenzer and colleagues from the ABC group: "fast and frugal heuristics” The future of assessment

17 Copyright : Cognadev UK Ltd Dynamic modeling  Move from broad generalizations into more precisely modeling the individual  Not for psychometrics and statisticians, but for “scientist-philosopher-heuristics specialists” (Barrett)  Consider technology platforms and business intelligence / rules for dynamic assessment The future of assessment

18 Copyright : Cognadev UK Ltd Dynamic modeling to capture the output of a "complex", self- organizing, adaptive system may require a new kind of systems-modeling where trajectory mapping for an individual is the goal rather than "measurement" of psychological attributes The issue is thus not "psychometrics - measurability", but the "outcome-consequences" of a dynamic complex system (Barrett, 2011) The future of assessment

19 Copyright : Cognadev UK Ltd Dynamic modeling techniques: Neural net applications and algorithmically based, “if- then” expert systems CPP OOM The future of assessment

20 Copyright : Cognadev UK Ltd Integration of study fields and constructs  Human functioning is integrated (physiological, emotional, motivational, psychological, spiritual and collective factors)  Holistic assessment practices  Human functioning is contextualised  Integration of disciplines (e.g. psychology & neurology / sociology …)  Integration of nomological networks of concepts The future of assessment

21 Copyright : Cognadev UK Ltd Building of an integrated science  Across epistemologies: both emic (meaningful to actor) and etic (objective description) perspectives  Across disciplines  Across schools of psychology  Across constructs The future of assessment

22 Copyright : Cognadev UK Ltd Contextualisation of assessment results Human functioning involves individual, collective and contextual components Currently dealt with via: competency approach person-job-matching situational judgement methodology Challenge: cross-cultural bias The future of assessment

23 Copyright : Cognadev UK Ltd Internet options and delivery Spread to a wider audience The individual taking charge of own growth Social networks New functionalities e.g. alternative item generation The future of assessment

24 Copyright : Cognadev UK Ltd Ensuring the effectiveness of assessment Determining the actual value that tests add to the user Evidence of the utility of assessment is required to guide investment in psychometric research and application But how to measure it Maybe also systems modeling plus rigorous evidence-base designs The future of assessment

25 Copyright : Cognadev UK Ltd Ethical and legal implications Current pragmatic value of psychometrics Benefits of the test subject: self-assessment and self-development offerings, job applications Feuerstein: assessment should have a developmental impact Cross-cultural bias and fairness Addressing values of business leadership Intellectual property challenges The future of assessment

26 Copyright : Cognadev UK Ltd Practical utility of assessment Clinical purposes Legal field Moral-ethical matters Optimising the man-machine interface Advertising Learning disabilities Relationship counselling Self-development The future of assessment

27 Copyright : Cognadev UK Ltd International collaboration and group research Barrett suggests large scale international academic collaboration The future of assessment

28 Copyright : Cognadev UK Ltd Psychophysiological Medical imaging to integrate functional and structural aspects of mental functioning The future of assessment

29 Copyright : Cognadev UK Ltd Summary Theoretical integration New constructs, models and nomological networks New research methodologies (dynamic modeling) New delivery mechanisms New analytical techniques New purposes Contextualisation Practical everyday uses Ethical approach Measurement of impact The future of assessment

30 Copyright : Cognadev UK Ltd Conclusion The psychometrics paradigm thus needs to make way for something more: comprehensive relevant dynamic purposeful The future of assessment

31 Copyright : Cognadev UK Ltd Thank you for your valuable time 011 –


Download ppt "Copyright : Cognadev UK Ltd Dr Maretha Prinsloo The future of assessment."

Similar presentations


Ads by Google