1. Content and face validity of the Registered Nurses’ Attitudes towards Postgraduate Education (NATPGE) in Australia: A Pilot Study
Linda Ng¹,², Margo Pritchard¹,²,³, Anthony Tuckett², Stephanie Fox-Young², Victoria Kain²
¹ Grantley Stable Neonatal Unit, Perinatal Research Centre, Women’s and Newborn Services, Royal Brisbane and Women’s Hospital
² The University of Queensland
³ The University of Queensland Centre for Clinical Research.
Background
Nursing education is a dynamic process designed to enable nurses to competently meet the healthcare needs of society. Health system restructuring has been associated with diminishing postgraduate specialist nursing numbers worldwide. Valid instruments that monitor and evaluate nurses attitudes to gauge educational barriers and facilitators are a central component in planning effective education, and have been unavailable. We previously developed the Registered Nurses Attitudes Towards Post Graduate Education (NATPGE) survey-instrument to fill this gap, in phase one of the study.
Objective
To test the validity and reliability of the Registered Nurses Attitudes Towards Post Graduate Education (NATPGE) survey-instrument.
Methods
I. Content Validity Index (CVI)1 based on expert ratings of relevance was used as a method of quantifying content validity for the NATPGE survey-instrument. Four categories: relevance, simplicity, clarity and non-ambiguity were used to enable the appraisal of the tool’s phrasing and terminology as well as a section for recommendations for content modification and was used as follows to assess the:
NATPGE Content validity: four content experts (CE) who specialised in: specialist-nurse education, psychometric scales; development and analysis of instruments were selected to undertake judgment-quantification and agree on the final version of the NATPGE survey-instrument prior to testing its face validity.
NATPGE Face validity: a convenience sample of 25 Registered Nurses (RNs) were selected from four major Queensland tertiary hospitals to assess the instrument content readability and relevance.
II. Test-retest reliability: a random sample of 100 RNs from the Nurses and Midwives e-Cohort Study (NMeS)2 were invited to participate in a test-retest pilot as part of the process of assessing the reliability of the online NATPGE. To gauge the test-retest reliability, the instrument was administered at two different time points, 3 weeks apart, under similar conditions.
III. Demographic variables were collected on all participants.
Analysis
The content and face validity was assessed using descriptive statistics. For the test-retest reliability the 15 NATPGE questions were analysed on an item by item basis to calculate the intra-rater reliability using the weighted kappa (kw) statistic and its standard error (SE). The kw implicitly assumes that all disagreements are equally weighted as are all agreements. The reference values for the strength of agreement are in accordance with Altman3 ( as poor, as fair, as moderate, as good and as a very good agreement. Data were analysed using Stata 12 (StataCorp. 2011, TX: StataCorp LP.).
Results
Demographics of participants
The four CE and the RNs were similar in terms of their years of experience, postgraduate qualification and working in a specialty area (Table 1):
Content and Face Validity
Overall both the CE and the RNs ranked the NATPGE, using the CVI, as a realistic training platform that would be useful for evaluating RNs’ attitudes towards postgraduate education (Table 2). The comments received from the CE resulted in some minor changes to the wording of some items for better clarity and simplicity. No particular concerns were raised about any of the items by the CE. The CE was agreeable that the items were arranged in a positively and negatively worded sequence, which was intentional as to prevent response bias.
Test-retest reliability
Complete data is available and was analysed for 36 of the 100 (36%) sample of RNs who completed the test-retest reliability of the NATPGE instrument. Overall the results display an 80% fair to moderate kappa (kw = ) agreement; however, there is some variability (kw = 0.0 to 0.79) between the test and retest kw for each individual question (Graph 1).
Graph 1: Kappa Agreement and SE (test and retest of 36 Registered Nurses) for the 15 NATPGE instrument questions.
Conclusion
The present research indicates very good content and face validity and whilst the test-retest reliability overall was moderate, several individual questions did have poor kappa values. As such, we plan to refine the instrument, before its validation in a larger sample using factor analysis. This work is currently being undertaken.
   
Table 2. Content Validity Index results for the NATPGE
Registered Nurses Attitudes Towards Post Graduate Education (NATPGE) survey-instrument
Content Validity Index (CVI)
Categories:
Content validity
N=4
Face validity
N=25 %
1. A Registered Nurse needs to have a Post Graduate qualification related to their specialty practice in order to be called a specialist nurse
Relevancy
Simplicity
Clarity
Non-ambigous
100 75
2. A Registered Nurse needs to have experience in a specialty clinical setting before they can be called a specialist nurse
3. A Registered Nurse needs to have both a Post Graduate education qualification relevant to their specialty practice and experience in the specialty clinical setting before they can be called a specialist nurse
4. Post Graduate education is useful for nurses in specialty practice **
5. Post Graduate education in a specialty area enhances nurses’ careers
6. Undertaking Post Graduate education whilst working in a shift environment is stressful
7. Post Graduate education in a clinical area increases nurses’ confidence **
8. It is difficult to balance work, study and family whilst undertaking Post Graduate education
9. Post Graduate education is mainly for single people, it is not appropriate for people with family responsibilities **
10. Post Graduate education in a clinical setting improves nurses’ knowledge
11. There is no difference in practice between a Registered Nurse with a Post Graduate specialty practice qualification and a Registered Nurse without a Post Graduate qualification
12. A Registered Nurse when undertaking Post Graduate education needs support from their employer
13. A Registered Nurse with Post Graduate specialty practice education experiences increased job satisfaction **
14. Post Graduate education in a clinical specialty improves nurses’ skills
15. The cost of Post Graduate education is too high
** Questions that CE recommended for re-wording**
Table 1. Demographic variables of CE and RNs
Participants
Years of experience
Post-graduate qualification
Working in speciality
CE (n= 4)
20 to 35 years
100%
RNs (n=25)
5 to 30 years
80%
Acknowledgement: This work has been undertaken by Linda Ng who is currently a PhD candidate with the School of Nursing & Midwifery, Faculty of Health, The University of Queensland.
References
1. Polit, D., Beck, C., & Owens, S. (2007). Is the CVI an acceptable indicator of content validity? Appraisals and recommendations. Research in Nursing 30,
2. NMeS
3. Altman, D. G. (1991). Practical statistics for medical research. London: Chapman & Hall