Methodology

The Kansas Communities That Care (KCTC) Student Survey has been administered annually at no charge to school districts throughout the state since 1994 to students in the 6th, 8th, 10th, and 12th grades. Funded by Kansas Department for Aging and Disability Services (KDADS), the survey tracks teen use of harmful substances such as alcohol, tobacco, and other drugs. In addition, the survey provides a measure of teen participation in, perception of, and attitudes toward both prosocial and antisocial behavior at the peer, school, family, and community levels. It provides a measurable level of student risk and protective factors that influence the behavior, attitudes, and opinions of Kansas teens.  

Questionnaire

Technical Assistance

Technical assistance and support for the survey is provided by Greenbush – The Education Service Center, which has been a KDADS contractor since the KCTC Student Survey began. Greenbush has a full-time KCTC Survey Coordinator who has expertise and experience in conducting surveys. Throughout the annual survey cycle, the coordinator manages:  

Additionally, as part of the Kansas Prevention Collaborative, coalitions and school districts have trained Community Support Specialists located throughout the state to promote and assist with KCTC survey protocols for guiding the use of data in local decision making.  

Administration protocols are sent to local KCTC contacts at least one month prior to administration and include survey preparation and administration instructions.  

Data Collection Procedures and Response Rates   

Survey data are collected and reported at the state, regional, county, district, building, and grade levels. Students typically complete the survey in approximately 27 minutes with trained school personnel. Standardized survey instructions are embedded by video at the beginning of the survey. School personnel also have access to a written copy of the instructions. Students hear standardized survey instructions viewed in an embedded video or read as transcribed text. Proctors may play the video or read the instructions to the entire group before starting the survey. Students may also independently view or read the instructions embedded in the survey in both English and Spanish. Additionally, other accessibility options are offered (e.g., access to screen reader technology, low vision).   

All KCTC procedures are designed to protect guardian and student rights and privacy, allowing for self-administered, anonymous, and voluntary participation. The KCTC Student Survey and related documents go through an annual Internal Review Board (IRB) process and approval. In addition, a standardized Annual Review Process is followed prior to submission to the IRB.  

The survey is completely anonymous   

Participation is entirely voluntary

Data are confidential and protected  

The KCTC Student Survey is administered from September to January each school year. Schools identify the exact dates for administration based on their schedules. It is recommended that schools choose the same time each year to strengthen longitudinal comparisons. District registration requires the superintendent’s signature and assignment of a local KCTC contact(s) to receive communications and administration protocols. The KCTC contact works with teachers, counselors, or other survey facilitators to coordinate survey proctoring.   

Schools are responsible for obtaining informed guardian consent. An IRB-approved informed consent form, which includes a link to the survey questions, is sent to all districts and is also available on the website. Best practice has shown the most effective way to obtain informed consent is with the materials in the district enrollment packet.   

The KCTC is a census sample open to all public and private schools in Kansas. It provides local data to school districts to compare data at the building, district, county, and state levels.   

Response rate is calculated by dividing the number of surveys that pass the validity checks by the number of eligible students in the unit (grade, building, district, county, state). Responses are not weighted. Response rates vary from year to year. These variations are noted where applicable on reports at kctcdata.org.  

Data Processing and Analytic Methods  

Data are collected online through a secured server. Once the survey administration is closed, a data download process and standardized cleaning protocol is applied using SPSS to ensure data validity and reliability. Data are aggregated and reported at the building, district, county, regional and state level to display on the kctcdata.org website if inclusion criterion are met. District and building level data are password protected. County, regional and state data are available to the public. For each of these levels, subcategory aggregations are calculated for each grade surveyed (6th, 8th, 10th, 12th) middle school (6th and 8th grade) and high school (10th and 12th). Further demographic data are not currently available on the website but can be requested. SQL queries are used to populate various stakeholder reports.   

Data are reported as percentages to allow for comparisons across levels (e.g., district, county, state). Trends are calculated and reported on the website for evaluating changes over time. Statistical Analysts from the Learning Tree Institute at Greenbush use advanced statistics (e.g., correlations, risk ratios, chi-square) to help identify relationships between variables and to determine significant changes.   

Limitations

The KCTC Student Survey has several limitations. Some limitations occur only within an administration year and/or only for specific school districts or buildings. Those are noted where applicable on reports and on the KCTC Student Survey website. General limitations that are monitored and addressed annually include: 

Survey Review Process

The KCTC Student Survey is reviewed annually, undergoing survey revisions as needed for language, cultural relevance, stakeholder needs, and community needs. The revision process starts with a review of pilot items and a Needs Assessment to determine areas of concern. Statistical Analysts from the Learning Tree Institute at Greenbush determine the next steps based on current best practices in survey research. Changes to survey question and response items can be accessed at the KCTC Question Response Wording Changes.  

Data Availability and Dissemination  

KCTC Student Survey data (1995–present) are available at kctcdata.org, as are additional resources. Availability of data depends on the level of survey participation, high data quality, and data-sharing policies. Variables are standardized to facilitate trend analyses and for combining data. School district and building data are only available through the district’s superintendent. The KCTC Team also performs statistical tests by topic and filters, and sorts the data by race/ethnicity, sex, grade, gender, and sexual orientation. Data requests and other KCTC Student Survey-related questions can be sent to [email protected].

Annual Survey Review Process

A KCTC Annual Survey Review Process provides the KCTC Team at Greenbush (Statistical Analysts, Database Developers, Project Managers, and support staff) an opportunity to investigate if the survey and its administration are current and relevant for the stakeholders of Kansans. Once the survey administration window closes each year, the team:  

  1. Reviews all data and provides data for publication   
  2. Reviews pilot items and provides data for publication   
  3. Revises pilot items if applicable   
  4. Conducts a Needs Assessment including:  
    • Internal queue checklist   
    • Client Satisfaction Survey   
    • Interviews if applicable, as determined by legislative changes, survey changes, stakeholder changes, etc.   
    • Summary of informal stakeholder (guardians, students, school administrators, community organizations, etc.) feedback submitted to the KCTC Team inbox and customer service phone line over the past survey cycle.   
  5. Reviews Needs Assessment. This includes reviewing questions/comments/recommendations submitted by external sources.   
  6. Recommends further analysis if needed.  
    • E.g., focus groups, roundtable, literature review  
    • Use predetermined selected criteria for accepting or declining the addition or deletion of questions  
  7. Presents Needs Assessment results to the Kansas Department for Aging and Disability Services, Behavioral Health Services Commission, Prevention Team for review and determination of further action if needed.   
  8. Submits all documents, including the survey, to the IRB (Institutional Review Board) for review and approval. The IRB is a review committee of at least five diverse people who read and analyze research materials to ensure the protection of participants. This also serves as final quality assurance.  

The KCTC Annual Review Process is a dynamic process that may change as the needs of stakeholders change. The following table outlines the review process:

KCTC Annual Review Process

ActionPerson(s) ResponsibleGeneral Timeframe
1. Data and reports are reviewed, revised, tested, and publishedThe KCTC Team consists of Statistical Analysts, Database Developers, Project Managers, and support staff.   During the two consecutive months following survey closure.  
2. Review pilot items and provide data for publication  Two Statistical Analysts  During the two consecutive months following survey closure.
3. Revise pilot items if applicable   

  
  
Two Statistical Analysts  During the two consecutive months following survey closure.  
4. Conduct Needs Assessment , including internal queue checklist, Client Satisfaction Survey, and interviews (if applicable, as determined by legislative changes, survey changes, stakeholder changes, etc. )Project Manager; One Statistical Analyst  manages interviewsDuring the first two weeks following survey closure.
5. Review Needs Assessment and other external feedback  Two Statistical Analysts and Project Manager   During the first month following survey closure.  
6. Recommend further analysis if needed based on selected criteria  (focus groups or roundtable, for example ) Two Statistical Analysts and Project Manager ; One to Two Statistical Analysts conduct focus groups or surveys, depending on projectDuring the two consecutive months following survey closure.  
7. Submit to KDADS  Greenbush Liaison or other KDADS contactDuring the two consecutive months following survey closure.
8. Submit project to IRB for final approvalThree Statistical Analysts and Project Manager  During the three consecutive months following survey closure.
9. Make changes and notify stakeholders of changes  Three Statistical Analysts and Project Manager  Prior to a new survey administration  

References

More Info

If you have questions about the survey, contact the KCTC Team at Greenbush at [email protected], or call 620-724-6281 ext. 366.