Developing a coding scheme for content analysis A how-to approach

Literature review As is pretty much always the case, finding out what people have done in the past is the best way to prepare for your own research – Gives you the theoretical lay of the land – Helps you see how others have approached your own concerns – Helps you determine what measures have been used and how they have worked – Allows you to identify areas where additional study is needed

Constructs To carry out a successful study, you will need to identify constructs of significance for your research – Remember our discussion of explication? From the literature review and your own experience/interests, what are the constructs you want to measure? – Is there a theory here?

Constructs In order for your analysis to be valid, you must first specify your constructs carefully – What are the characteristics of your construct? What would be a case of sexism, for example? – (Note: perhaps ‘genderism’ would be better). What would not reflect sexism in your definition but someone else might think would? What would be an example of non- or anti-sexism?

An example: How would you define ‘Respect for authority’? What behavioral examples would reflect it? What examples would you say do not reflect it, but some folks would say did? What examples of lack of respect or disrespect for authority can you think of?

Based on your analysis of ‘respect for authority,’ what examples can you think of? Who are authorities? What behaviors show respect? Lack of respect or disrespect? What behaviors might someone else say show respect but you exclude from your list?

Now identify features of content that reflect your constructs What would we look for in romance films that would reflect the concept of genderism? – Males making important decisions – Women wearing skimpy clothing – Females showing greater sensitivity – Men always driving the cars – Etc. Note: There are a great number of potential indicators—you have to choose those that are the best indicators and are practical to score

Example If a student does what the teacher in a TV show says, is it an example of respect for authority? What else might be necessary? – “Whatever you say, you’re the boss.” – “I’ll do it to save my A, but that’s the only reason.” – “I really like you, but I think this is a dumb idea.”

Choosing indicators Content analysis is quantitative, so you need to develop categories that can be assigned numbers – Use indicators that can be translated reasonably into quantitative scores Avoid developing category schemes that call for value judgments and/or too much interpretation on the part of the coder

Objective v. subjective codes Subjective categories (“high, medium, low”; “better, worse”) are read quite differently by various coders so their use invites unreliability Try to construct objective categories as much as possible – “4 or more times” v. “often” – “Over 250 pounds” v. “large”

Good coding categories Categories should be: – Exhaustive – Mutually exclusive – Derived from a single classification principle – Independent – Adequate to answer the questions asked of the data

Exhaustive There should be a coding category that each recording unit can be placed in – Can use “other” or “none” categories to make the scheme exhaustive

Mutually exclusive Each recording unit should fit into only one category on a given scoring dimension

Derived from a single classification principle Must keep conceptually different dimensions of analysis separate – Code separately for each dimension Example: Character presented as “fearful and inactive” (separate fear and activity into individual scoring dimensions)

Independent Each category should be independent of other categories—seek an ‘absolute’ value for each category – This will be violated if your categorization scheme assigns units to categories according to their relative position on some dimension “more biased,” “scarier”

Adequate to answer the questions asked of the data Must cover the entire concept or nearly so Must exclude third variables/related concepts that are not supposed to be measured by the category scheme – Don’t let influences other than those you are studying creep in and affect your results Differences among categories must be meaningful – Large enough to matter – Dimension is appropriate

The precision tradeoff The more narrowly tailored and precise your categories, the better the test of your research questions or hypotheses However, the finer the distinctions you ask your coders to make, the more ‘mistakes’ you will generate – In this form of text analysis, when two coders put the same coding unit into different categories, at least one of the coders is ‘wrong’

Good practice Try not to make categories too narrow e.g., years old, 13-15, 16-18, etc. – Few instances – Coders have difficulty making such fine distinctions Cover enough features to make valid judgments – Use multiple indicators of each construct Use the indicators that are easiest to measure/code – Only go to less clear-cut, more complicated indicators after the straightforward ones have been exhausted

Think of a critic when you design your scheme What would someone who doesn’t believe genderism exists say when I have completed my study and present my findings? Someone who is adamant that genderism is rampant in the media? Anticipate and deal with the most significant critiques prior to finalizing your coding scheme