Methodology

The Kansas Communities That Care (KCTC) Student Survey has been administered annually at no charge to school districts throughout the state since 1994 to students in the 6^th, 8^th, 10^th, and 12^th grades. Funded by Kansas Department for Aging and Disability Services (KDADS), the survey tracks teen use of harmful substances such as alcohol, tobacco, and other drugs. In addition, the survey provides a measure of teen participation in, perception of, and attitudes toward both prosocial and antisocial behavior at the peer, school, family, and community levels. It provides a measurable level of student risk and protective factors that influence the behavior, attitudes, and opinions of Kansas teens.  

Questionnaire

• The KCTC Student Survey Instrument, Versions, and Modules can be previewed at thisLINK. 
• Detailed information on KCTC Student Survey Reliability and Validity can be viewed at thisLINK. 

Technical Assistance

Technical assistance and support for the survey is provided by Greenbush – The Education Service Center, which has been a KDADS contractor since the KCTC Student Survey began. Greenbush has a full-time KCTC Survey Coordinator who has expertise and experience in conducting surveys. Throughout the annual survey cycle, the coordinator manages:  

• Survey registration and reporting processes.   
• KCTCdata.org website content and functionality.  
• Automated emails to KCTC contacts that provide timely survey information.  
• Individual technical assistance and general survey support, such as data interpretation.  

Additionally, as part of the Kansas Prevention Collaborative, coalitions and school districts have trained Community Support Specialists located throughout the state to promote and assist with KCTC survey protocols for guiding the use of data in local decision making.  

Administration protocols are sent to local KCTC contacts at least one month prior to administration and include survey preparation and administration instructions.  

Data Collection Procedures and Response Rates   

Survey data are collected and reported at the state, regional, county, district, building, and grade levels. Students typically complete the survey in approximately 27 minutes with trained school personnel. Standardized survey instructions are embedded by video at the beginning of the survey. School personnel also have access to a written copy of the instructions. Students hear standardized survey instructions viewed in an embedded video or read as transcribed text. Proctors may play the video or read the instructions to the entire group before starting the survey. Students may also independently view or read the instructions embedded in the survey in both English and Spanish. Additionally, other accessibility options are offered (e.g., access to screen reader technology, low vision).   

All KCTC procedures are designed to protect guardian and student rights and privacy, allowing for self-administered, anonymous, and voluntary participation. The KCTC Student Survey and related documents go through an annual Internal Review Board (IRB) process and approval. In addition, a standardized Annual Review Process is followed prior to submission to the IRB.  

The survey is completely anonymous   

• Students are not asked for names or other personally identifiable information.  
• No one, including school personnel, can connect a student’s name to responses.  
• Administration protocols suggest room set-up to ensure as much privacy as possible. During online administration, responses are entered into a website and when submitted are sent to a secure server. Responses cannot be saved on a computer hard drive or downloaded.  

Participation is entirely voluntary

• Guardians are informed within four months of the administration date and must give written consent for their child to participate.  
• Students may decline to participate in the survey on the day administered.  
• Students participating may skip questions and/or exit the survey at any point.  

Data are confidential and protected  

• District- and building-level data are password protected, where the superintendent manages who may have access to local results. County- and state-level data are publicly available. This information is used for state and county needs, including reporting outcomes, tracking trends, and obtaining grant funding.   
• Data are always reported as an aggregate or group, never at an individual level.   
• A minimum of 25% participation and at least 20 participants per aggregation level is required to report data at any given demographic (e.g., grade, school, district, county, ethnicity, sex).   
• Once a student submits their survey, data is sent to a secured database. Privacy is ensured through a sound technology infrastructure along with clear protocols and practices restricting access to key personnel. Greenbush is committed to ensuring the security of information and compliance with all state and federal compliance requirements. Reasonable physical, technical, and administrative safeguards in place are designed to prevent unauthorized access to or use of the information collected online. All data collected or processed does comply with the security controls, systems, process requirements, and standards as set forth by Greenbush. If you have any questions regarding Greenbush’s Data Privacy and Security Policies, please contact the Greenbush Technology Support Services Department at[email protected]. The data are secure and are not sold or given to any commercial group.  

The KCTC Student Survey is administered from September to January each school year. Schools identify the exact dates for administration based on their schedules. It is recommended that schools choose the same time each year to strengthen longitudinal comparisons. District registration requires the superintendent’s signature and assignment of a local KCTC contact(s) to receive communications and administration protocols. The KCTC contact works with teachers, counselors, or other survey facilitators to coordinate survey proctoring.   

Schools are responsible for obtaining informed guardian consent. An IRB-approved informed consent form, which includes a link to the survey questions, is sent to all districts and is also available on the website. Best practice has shown the most effective way to obtain informed consent is with the materials in the district enrollment packet.   

The KCTC is a census sample open to all public and private schools in Kansas. It provides local data to school districts to compare data at the building, district, county, and state levels.   

Response rate is calculated by dividing the number of surveys that pass the validity checks by the number of eligible students in the unit (grade, building, district, county, state). Responses are not weighted. Response rates vary from year to year. These variations are noted where applicable on reports at kctcdata.org.  

Data Processing and Analytic Methods  

Data are collected online through a secured server. Once the survey administration is closed, a data download process and standardized cleaning protocol is applied using SPSS to ensure data validity and reliability. Data are aggregated and reported at the building, district, county, regional and state level to display on the kctcdata.org website if inclusion criterion are met. District and building level data are password protected. County, regional and state data are available to the public. For each of these levels, subcategory aggregations are calculated for each grade surveyed (6^th, 8^th, 10^th, 12^th) middle school (6^th and 8^th grade) and high school (10^th and 12^th). Further demographic data are not currently available on the website but can be requested. SQL queries are used to populate various stakeholder reports.   

Data are reported as percentages to allow for comparisons across levels (e.g., district, county, state). Trends are calculated and reported on the website for evaluating changes over time. Statistical Analysts from the Learning Tree Institute at Greenbush use advanced statistics (e.g., correlations, risk ratios, chi-square) to help identify relationships between variables and to determine significant changes.   

Limitations

The KCTC Student Survey has several limitations. Some limitations occur only within an administration year and/or only for specific school districts or buildings. Those are noted where applicable on reports and on the KCTC Student Survey website. General limitations that are monitored and addressed annually include: 

• Like all self-reported data, the extent of underreporting or overreporting of behaviors and attitudes cannot be determined, although statistics consistently demonstrate that the data are reliable and valid. Question language may also cause prevalence to be under-reported in some cases. For example, the survey asks about the use of electronic cigarettes; however, students often do not consider ‘vaping’ as using an electronic cigarette. This may cause the survey responses to be lower than actual use. The KCTC Student SurveyAnnual Review Process reviews language as needed to consider adjustments to survey questions and/or wording.  
• Being a census approach, the data applies only to youth who took the survey. With high participation rates, state data have been shown to reflect student enrollment data representing the populations of interest. Caution should be used when response rates are low or are inconsistent from year to year.  
• Out-of-school (e.g., homeschool, drop-out) students are not available to be sampled. Their views may differ from students attending school.  

Survey Review Process

The KCTC Student Survey is reviewed annually, undergoing survey revisions as needed for language, cultural relevance, stakeholder needs, and community needs. The revision process starts with a review of pilot items and a Needs Assessment to determine areas of concern. Statistical Analysts from the Learning Tree Institute at Greenbush determine the next steps based on current best practices in survey research. Changes to survey question and response items can be accessed at the KCTC Question Response Wording Changes.  

Data Availability and Dissemination  

KCTC Student Survey data (1995–present) are available at kctcdata.org, as are additional resources. Availability of data depends on the level of survey participation, high data quality, and data-sharing policies. Variables are standardized to facilitate trend analyses and for combining data. School district and building data are only available through the district’s superintendent. The KCTC Team also performs statistical tests by topic and filters, and sorts the data by race/ethnicity, sex, grade, gender, and sexual orientation. Data requests and other KCTC Student Survey-related questions can be sent to [email protected].

Annual Survey Review Process

A KCTC Annual Survey Review Process provides the KCTC Team at Greenbush (Statistical Analysts, Database Developers, Project Managers, and support staff) an opportunity to investigate if the survey and its administration are current and relevant for the stakeholders of Kansans. Once the survey administration window closes each year, the team:  

1. Reviews all data and provides data for publication   
2. Reviews pilot items and provides data for publication   
3. Revises pilot items if applicable   
4. Conducts a Needs Assessment including:  
   • Internal queue checklist   
   • Client Satisfaction Survey   
   • Interviews if applicable, as determined by legislative changes, survey changes, stakeholder changes, etc.   
   • Summary of informal stakeholder (guardians, students, school administrators, community organizations, etc.) feedback submitted to the KCTC Team inbox and customer service phone line over the past survey cycle.   
5. Reviews Needs Assessment. This includes reviewing questions/comments/recommendations submitted by external sources.   
6. Recommends further analysis if needed.  
   • E.g., focus groups, roundtable, literature review  
   • Use predetermined selected criteria for accepting or declining the addition or deletion of questions  
7. Presents Needs Assessment results to the Kansas Department for Aging and Disability Services, Behavioral Health Services Commission, Prevention Team for review and determination of further action if needed.   
8. Submits all documents, including the survey, to the IRB (Institutional Review Board) for review and approval. The IRB is a review committee of at least five diverse people who read and analyze research materials to ensure the protection of participants. This also serves as final quality assurance.  

The KCTC Annual Review Process is a dynamic process that may change as the needs of stakeholders change. The following table outlines the review process:

KCTC Annual Review Process

Action	Person(s) Responsible	General Timeframe
1. Data and reports are reviewed, revised, tested, and published	The KCTC Team consists of Statistical Analysts, Database Developers, Project Managers, and support staff.	During the two consecutive months following survey closure.
2. Review pilot items and provide data for publication	Two Statistical Analysts	During the two consecutive months following survey closure.
3. Revise pilot items if applicable	Two Statistical Analysts	During the two consecutive months following survey closure.
4. Conduct Needs Assessment , including internal queue checklist, Client Satisfaction Survey, and interviews (if applicable, as determined by legislative changes, survey changes, stakeholder changes, etc. )	Project Manager; One Statistical Analyst  manages interviews	During the first two weeks following survey closure.
5. Review Needs Assessment and other external feedback	Two Statistical Analysts and Project Manager	During the first month following survey closure.
6. Recommend further analysis if needed based on selected criteria  (focus groups or roundtable, for example )	Two Statistical Analysts and Project Manager ; One to Two Statistical Analysts conduct focus groups or surveys, depending on project	During the two consecutive months following survey closure.
7. Submit to KDADS	Greenbush Liaison or other KDADS contact	During the two consecutive months following survey closure.
8. Submit project to IRB for final approval	Three Statistical Analysts and Project Manager	During the three consecutive months following survey closure.
9. Make changes and notify stakeholders of changes	Three Statistical Analysts and Project Manager	Prior to a new survey administration

References

• Arthur, M.W., Hawkins, J.D, Pollard, J. Catalano, R. F and Baglioni Jr., A.J. 2002. Measuring risk and protective factors for substance use, delinquency, and other adolescent problem behaviors: The Communities That Care Youth Survey. Evaluation Review 26; 575  
• Hawkins, J. D., M. W. Arthur, and R. F. Catalano. 1995. Preventing substance abuse. In Crime and justice: Vol. 19. Building a safer society: Strategic approaches to crime prevention, edited by M. Tonry and D. Farrington, 343-427. Chicago: University of Chicago Press.  
• Hawkins, J. D., Catalano, R. F., and Miller, J.Y. 1992. Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance abuse prevention. Psychological Bulletin 112 (1): 64-105.  
• Johnston, L. D., P. M. O’Malley, and J. G. Bachman. 1995. National survey results on drug use from the Monitoring the Future Study, 1975-1994: Volume I. Secondary students. Rockville, MD: National Institute on Drug Abuse.  

More Info

If you have questions about the survey, contact the KCTC Team at Greenbush at [email protected], or call 620-724-6281 ext. 366.