Presentation is loading. Please wait.

Presentation is loading. Please wait.

The role of textual data in a statistical approach for the evaluation of the regulatory impact Simona Balbi, Germana Scepi, Giorgio Infante Università

Similar presentations


Presentation on theme: "The role of textual data in a statistical approach for the evaluation of the regulatory impact Simona Balbi, Germana Scepi, Giorgio Infante Università"— Presentation transcript:

1 The role of textual data in a statistical approach for the evaluation of the regulatory impact Simona Balbi, Germana Scepi, Giorgio Infante Università Federico II di Napoli NTTS 2009

2 outline of the proposal our aim to propose statistical tools for ex ante evaluation of public actions to propose statistical tools for ex ante evaluation of public actions, i. e. helping in identifying groups of interest characterized by different opinions concerning regulatory proposals, in order to identify costs and benefits of different alternative actions NTTS 2009 By introducing in Factorial Conjoint Analysis Textual Data Analysis tools (Balbi, Infante, Misuraca, 2008), it is possible to cluster judges, on the basis of their preference structure referred both to alternative public actions and their free description of a desired intervention

3 theoretical frame Regulatory Impact Analysis Regulatory Impact Analysis Factorial Conjoint Analysis Factorial Conjoint Analysis Textual External Information Textual External Information Characteristic words Characteristic words NTTS 2009

4 Regulatory Impact Analysis (RIA) "RIA encompasses a range of methods aimed at systematically assessing the negative and the positive impacts of proposed and existing regulation (OECD) oRIA oRIA contributes to the creation of an open, transparent, and empirically based regulatory system ( The European Policy Centre) oPeople consensusreducing the risks of regulatory failure oPeople consensus is a key factor for reducing the risks of regulatory failure oIn political processes actors in favor adverse oIn political processes, it is important to know the actors in favor and the actors adverse to specific interventions NTTS 2009

5 Factorial Conjoint Analysis NTTS 2009 Conjoint Analysis (C.A.) seems particularly appropriated since, starting form different regulatory stimuli, it allows to decompose several evaluation dimensions. It enables to estimate the partial utility coefficients of different factor-levels and to define groups of users on the basis of their response similarity A Factorial Approach to C.A.* allows to get the following purposes: i)to synthesize the individual judgments reconstructed directly by the model on the principal axes obtained by their linear combination, ii)to look for an optimal synthesis of such judgments according to the perceived benefits, iii)to build two-dimensional graphics (factorial maps, called perceptive maps) for the study of the existing relationships among designed stimuli (normative options), judgments and descriptive levels (core indicators), with the further possibility to underline the different evaluation structures expressed by several groups of judges. *Lauro Giordano Verde 1998

6 We project and administer a questionnaire for collecting opinions expressed in relation to a set of stimuli simultaneously described by the relevant dimensions (core indicators) g 1 g 2 ………………...….......g N JUDGES 1.A sample of citizens 2.A sample of experts S1S2S3SqS1S2S3Sq STIMULI OR DIFFERENT OPTIONS the factors should take into account organizational,financial,economic and social aspects, criticality and so on.. and so on.. ratings or ranking several criteria: expected benefits, expected public utility, strategic priority indirect net benefits, and so on…… Y Factorial Conjoint Analysis for RIA* NTTS 2009 *Scepi,Giordano Lauro 2008

7 usually, judges are asked to compare different potential alternatives (in order to ranking or scoring), by the so called full profile method (oldest but still used): a limited number of attributes is used to describe the product or service, but sufficient cards or treatments are shown to one respondent in order to compute his/her utilities coefficients Ex. Which model of Italian University, do you want to work in? Textual Information - Motivations NTTS 2009 PUBLIC STANDARDISE PRIVATE STANDARDISE PUBLIC With AUTONOMY PRIVATE With AUTONOMY

8 the data collection is a critical step in CA: it is difficult for judges comparing a huge number of potential actions, described in a complex way a small number of categorical variables (with a small number of categories) is rigid NTTS 2009 Textual Information - Motivations A free description of the desired public action can be useful in order to introduce elements do not consider in CA

9 1....... G 1......1...... v K 1.....1..... p Z 1...1... s Y 1..... L 1...1... s X data structure in Z we consider the p characteristics and behaviors of the G interviewed in K we consider the v terms used in the open question by the interviewed X and Y are classically used in the C.A., respectively: the experimental design and the preference data matrices NTTS 2009

10 the lexical table 1....... G 1......1...... v K before constructing the K matrix the open questions are pre-treated as usual in textual statistics by considering: normalization normalization lexicalization lexicalization lemmatization lemmatization a stop list is considered for eliminating the instrumental terms and a special threshold is introduced for the hapax and infrequent terms NTTS 2009

11 the strategy 1....... G 1......1...... v T 1.....1..... p Z 1...1... s Y 1...1... s 1..... LX K is tranformed in the presence/absence matrix T 1..... L 1......1...... v Z is projected as supplementary information after analysing the Q matrix NTTS 2009

12 the analysis the analysis can be seen in a factorial analysis framework by considering the decomposition of trough a singular value decomposition in this way we obtain a graphical representation of the terms and the C.A. level, in which we can also consider as supplementary points the information dealing with the interviewed individuals NTTS 2009

13 Reforming University in Italy in Italy there is a wide debate concerning with the university system in Italy there is a wide debate concerning with the university system a sample of 30 university professors, with different backgrounds and experiences, have been asked to describe their ideal university, before ranking different options in a full profile Conjoint Analysis a sample of 30 university professors, with different backgrounds and experiences, have been asked to describe their ideal university, before ranking different options in a full profile Conjoint Analysis the levels and the factors used in the C.A. have been chosen by considering different political proposals under debate the levels and the factors used in the C.A. have been chosen by considering different political proposals under debate NTTS 2009

14 conjoint analysis – full profile Management public/private public Teacher Recruitment private public Legal value of the certificate NO YES Formative Path autonomous standardise Formative Target professional cultural NTTS 2009

15 Ideal University: partial utilities public management public management public/private man public/private man public recruitment public recruitment private recruitment private recruitment standardize path standardize path autonomous path autonomous path cultural target cultural target private target private target legal value YES legal value YES legal value NO legal value NO NTTS 2009

16 Graphical representation of C.A. levels NTTS 2009 professional no legal value public legal value public/private cultural private recruitmentpublic recruitment standardizedautonomous

17 Graphical representation of personal characteristics NTTS 2009 > 9 years exp 4-9 years exp full professor associate professor <=3 years exp researcher

18 Graphical representation of words

19 the proposed strategy allows a final clustering of judges, based on Factorial CA results and described by characterizing words, as usual in textual data analysis the proposed strategy allows a final clustering of judges, based on Factorial CA results and described by characterizing words, as usual in textual data analysis here we have identifies 5 groups: G1 G1: working in a university connected with the external world (società, ruolo, sistema) also by financing (valutare, finanziare) G2 G2: working in a public university with public recruitment (garantire, pubblico) G3 G3: working in a university research oriented (ricerca) G4: G4: working in a university labour market oriented (di base, professionale, formativo) G5: G5: The fifth class is pragmatic: not reference to ideal worlds (potere, dovere), the characteristic words looks at results (produttivo, utile) but not at students (studente has frequency equal to 0)! Clustering judges


Download ppt "The role of textual data in a statistical approach for the evaluation of the regulatory impact Simona Balbi, Germana Scepi, Giorgio Infante Università"

Similar presentations


Ads by Google