1. CS 594: Empirical Methods in HCC A Brief Introduction to Content Analysis
Dr. Debaleena Chattopadhyay
Department of Computer Science
debaleena.com
hci.cs.uic.edu

2. Content analysis
A quantitative approach to analyze unstructured or “qualitative” data, maybe text, audio, or video data.
Within the scope of human-centered computing, content analysis implies a series of specific steps aimed at ensuring systematic sampling, coding, and often counting of content to make quantitative inference.
Content analysis and GTM are often confused as the same. They are NOT—although they are often considered equivalent.
Content analysis is more straightforward and often “easier” than GTM.

3. Advantages and Disadvantages of Content Analysis
Content analysis is quantitative, systematic, and objective technique for describing the observable.
We count what is tangible and observable.
We cannot count abstract or latent (hidden) notions.
Any abstract concept, such as learning outcome, enjoyment, immersion, or group cohesion need to be operationalized in terms of variables that are observable, and thus can be measured.
Strength: emphasis on the systematic coding, counting, and analysis of content.
Limitation: Most effective if used for comparisons.
How we interpret the final counts directly affects the validity of the analysis.
Intercoder reliability gives a quantitative measure of reliability.

4. Steps of doing Content Analysis
Develop a hypothesis or research question about the HCI content.
Define the content to be analyzed.
Sample the universe of content. “Universe” has the same meaning for content as “population” does for people. Also called “coding frames”.
Select units for coding (or unit of analysis).
Develop a coding scheme.
Assign each occurrence of a unit in the sample to a code in the coding scheme.
Count occurrences of the coded units and report their frequencies.

5. Example
RQ: Do users ask for permission when sharing content on a open-control public display?
Content to be analyzed: videos of classes/ meetings
Sampling: all meetings in Continuum
stratified sampling— meetings of different sizes and types
Units for coding: content showed on a shared display
Coding scheme: verbal, non-verbal, explicit, implicit
Count Occurrences of the Coded Units and Report Their Frequencies

6. A simple coding sheet and summary data
Permission types
Tally
Count 1
verbal
III 3 2
Non-verbal
IIIIIII 7 … N
% of time verbal permission asked
5/50 = 10%
Explicitly asked for permission
10/50 = 5%
Implicitly asked for permission

7. Interaction analysis
Most commonly used in HCI; borrows from the tradition of content analysis of human interactions.
For example, three broad categories of group behavior are task-oriented, group-oriented, and self-centered.
Task-oriented individuals focus on the group’s work
Group-oriented individuals work to ensure that the group remains cohesive
Self-centered individuals may refuse to participate or at the other extreme may dominate discussions.

8. Things you may count Words Sentences Paragraphs Characters Semantics
Concepts
Themes
Images
Sequences

9. Coding scheme Apply an established code or develop your own
When developing your own coding scheme, you may use open coding or GTM.

10. Steps to create a coding scheme (when not using GTM)
Review literature
Identify constructs
Identify features of content that reflect your constructs
Choose indicators
Finalize coding categories

11. Good coding categories
Exhaustive: there should be a coding category that each recording unit can be placed in.
Mutually exclusive: Each recording unit should fit into only one category on a given scoring dimension
Derived from a single classification principle: keep conceptually different dimensions of analysis separate
Independent: Each category should be independent of other categories
Adequate to answer the questions asked of the data
should cover the entire concept or nearly so (content validity)
Should exclude spurious variables/related concepts that are not supposed to be measured by the category scheme
Differences among categories must be meaningful

12. Qualitative vs. Quantitative content analysis
Earlier definition of content analysis is as a quantitative research method.
to classify written or oral materials into identified categories of similar meanings
Criticized as oversimplification as a result of breaking down text etc. into quantifiable units in the analytic process
Hence qualitative content analysis was developed.
“a research method for subjective interpretation of the content of text data through the systematic classification process of coding and identifying themes or patterns”

13. Inductive and Deductive approaches to qualitative content analysis

14. GTM vs. Content Analysis
The use of grounded theory aims to generate a substantive theory that will explain a phenomenon in a specific context and suited to its supposed use. The emphasis in GTM is theory development (Strauss & Corbin, 1994). GTM is appropriate when no theory exists or when a theory exists that is too abstract to be tested.
Qualitative content analysis aims to “systematically describe the meaning” of materials in a certain respect that the researcher specified from research questions.
Both use coding, but content analysis does not focus on finding relationships among categories or theory building; instead, it focuses on extracting categories from the data.

15. Similarities Based on naturalistic inquiry
Flexibility of using multiple sources of data
Systematic steps in analysis
Seeking themes through coding process
Content to be coded into categories or themes
Naturalistic inquiry is an approach to understanding the social world in which the researcher observes, describes, and interprets the experiences and actions of specific people and groups in societal and cultural context.

16. Differences

17. Differences (cont.…)