1. 2018 Census content development Disability
Presentation 2a
2018 Census content development Disability

2. 2018 Census content review
The programme has undertaken an assessment and consultation on the content for the 2018 Census
This has been followed by our testing programme which informed the final content recommendations
A refresh of content was necessary given the previous ‘no change’ census models
Need for the information collected in the census to remain relevant, while retaining the value of longitudinal data
Opportunities:
Refreshing the content allows us to address important unmet information needs
Leveraging the benefits of an improved processing system, revised methodologies, administrative data and the online first collection model will allow us to produce higher quality outputs for customers
Challenges:
Managing expectations around the amount of new content we can include on the forms. Increasing the total amount of content will likely lower overall response rates and data quality
The high profile of the census and the level of detail that it can provide means that many customers may want information on a topic that is better suited in another survey collection

3. The stages to determine 2018 content
 2013 Census content
Public engagement and consultation
 Assessment of formal submissions
 Question development and cognitive testing
Engagement and consultation have played a major roles in ensuring that the census content remains relevant and reflect real world change
This census cycle we undertook engagement and consultation using three methods
Face to face seminars and workshops in five main centres - In addition to meetings with key agencies at the end of 2014, we ran face-to-face seminars and workshops in Auckland, Hamilton, Wellington, Christchurch, and Dunedin in May These seminars involved a small group of technical experts, and provided an introduction to the modernised 2018 Census and a general overview of census content. The workshops included a detailed run-through of the Preliminary View of 2018 Census Content for each topic. 
Public online engagement ran over nine weeks from April 2015 using the Loomio discussion forum. This is the first time Stats NZ has conducted live, supported, public online engagement. It was a unique opportunity for the public to have their say about census content and for us to learn to engage with the public in this way. 
Formal submissions - The formal submission period started two weeks after online engagement in May 2015 and closed at the end of June. Formal submissions started after online engagement to encourage people to discuss and develop their thinking with others on the discussion forum before making a formal submission. Where appropriate we encouraged people to represent the views of their organisation. For formal submissions, we encouraged people to fill in a form on the Stats NZ website, but also accepted and paper submissions. This approach was in line with our digital-first goal for the 2018 Census
  Continuing question development and public tests during
 Government Statistician makes final decision on 2018 Census content

4. Content determination
Does the proposed change add value to New Zealand's society and economy?
Is the census the most appropriate information source?
Does the proposed change reflect an enduring information need?
Will the proposed change produce quality information?
Is there continuity with previous census data?
Is data consistent with other data collections?
Is there general acceptance of the proposed change?
Would the proposal questions be easy for respondents to complete?
The content team reviewed submissions, and went through a content determination process. 
The topics included in the census must reflect legal requirements, current information needs, and our aims for the census.
The census must also reflect real-world change and maintain relevance. 
Regardless of whether census forms are competed online or on paper, it is still important to consider the overall amount of information being collected and overall respondent burden., The size of the paper forms and ability to fit the content onto these is an important factor. 
The census content and all topics raised during engagement and consultation were then evaluated against a set of criteria.
Scoring of submissions:
<60 – Currently recommending exclusion
60-69 – More content development and evidence (QD/content team/external)
70-74 – Currently recommended for inclusion, but some minor issues may need to be resolved
75+ - Currently recommended for inclusion

5. Content to questionnaire testing
Cognitive testing to assess whether new and changed content is suitable for a self-completed questionnaire
A number of topics did not progress past cognitive testing
We have thoroughly tested new and changed content including cognitive testing and mass completion testing with testing the questionnaire in two major tests in July 2016 and March 2017. 
Cognitive testing to assess whether new and changed content is suitable for a self-completed questionnaire
A number of topics did not progress past cognitive testing (stepfamilies, gender identity, volunteering, license to occupy)
Public content tests:
March 2016
Field focused test
Limited number of variables
Housing quality questions were tested on dwelling form
July 2016
Content focus
Wide scope of content tested: 7 page individual form
Two sample groups: paper and internet
8,000 households (paper forms)
External reference group – 23,000 people tested online forms
April 2017
Content further refined
Using intended paper size for 2018: 4 pages
While earthquake rescoped the test to not include test outputs, more focus put onto output and detail of data quality

6. Testing programme March 2016 July 2016 April 2017 Experimental design
22,000 households
Split our sample to test different approaches
Self response – 32 percent before field visits Response rate – 59% 64% took part online
March 2016
Two sample groups
8,000 households (paper forms)
External reference group – 23,000 people tested online forms
Content focus
20% response rate
July 2016
Rescoped after the November earthquake
20,00 households in Whanganui
36% response rate
76% took part online
Marketing campaign, communications & community engagement tested
April 2017
A number of small independent tests conducted together with larger tests in 2016 and (Already been reflected on from the content perspective in the first session).
Testing process is an ongoing one even though our field tests have been completed.
April 2017
We received 15,000 individual forms, and 7,000 dwelling forms. 
It took people approximately eight minutes to complete their individual form online.
Overall, 69% of online responses were completed on a desktop computer, 17% were by mobile and 14% were on a tablet.
Most people completed their census test forms on 4 April 2017, despite the risk of flooding in Whanganui that day.

7. Washington Group Short Set
Initially a matrix format was tested on paper due to uncertainty around space constraints, for the 2017 Census Test the format was revised given the space available
We have thoroughly tested new and changed content including cognitive testing and mass completion testing with testing the questionnaire in two major tests in July 2016 and March 2017. 
Cognitive testing to assess whether new and changed content is suitable for a self-completed questionnaire
A number of topics did not progress past cognitive testing (stepfamilies, gender identity, volunteering, license to occupy)
Public content tests:
March 2016
Field focused test
Limited number of variables
Housing quality questions were tested on dwelling form
July 2016
Content focus
Wide scope of content tested: 7 page individual form
Two sample groups: paper and internet
8,000 households (paper forms)
External reference group – 23,000 people tested online forms
April 2017
Content further refined
Using intended paper size for 2018: 4 pages
While earthquake rescoped the test to not include test outputs, more focus put onto output and detail of data quality

8. Other new content Usual residence one year ago
Housing quality – damp, mould, access to basic amenities
Travel to education - usual instead of census day - educational institution address
Revisions to other content eg. appliances used to heat dwelling, sector of landlord categories
All of these topics had quite a lot of work / testing done prior to the decision to exclude.
Sexual orientation
Data quality concerns
Sensitivity of questions
Stats NZ exploring further (GSS)
A third category for sex
Low quality data
Low prevalence
Paper form option
Gender identity
Data quality
Complexity of concept
Response behavior was variable
Second address/residence
Complex living situations
Burdensome and difficult for respondents
Data quality issues
Step-families
Difficulties in understanding ‘step’
Complex
Further testing and development work
Licence to occupy
Term not well understood
Confusion and errors

9. Other topics explored but excluded
Sexual orientation
A third category for sex
Gender identity
Second address / residence
Step-families
Licence to occupy
All of these topics had quite a lot of work / testing done prior to the decision to exclude.
Sexual orientation
Data quality concerns
Sensitivity of questions
Stats NZ exploring further (GSS)
A third category for sex
Low quality data
Low prevalence
Paper form option
Gender identity
Data quality
Complexity of concept
Response behavior was variable
Second address/residence
Complex living situations
Burdensome and difficult for respondents
Data quality issues
Step-families
Difficulties in understanding ‘step’
Complex
Further testing and development work
Licence to occupy
Term not well understood
Confusion and errors