Research and Evaluation Methods
Analytic Insight (AI) designs and conducts studies that inform policy and strategic planning decisions. Many of our projects employ a mixed methods approach with some combination of:
Development of summative and formative evaluation plans, including logic model and evaluation question development;
Design of survey questions and other data collection tools;
Key informant interviews;
Surveys of consumers, residents or businesses;
Advanced data analysis, GIS mapping or network analysis;
Data visualization and reporting;
Formal presentation of results.
Evaluation Planning and Logic Models
AI has extensive experience in planning program evaluations and needs assessment research using a variety of administration methods including mail, telephone, online, and intercept. We have evaluated grants from agencies including CDC, NIH, FDA and SAMHSA, as well as grants emanating from non-profits such as AARP.
Our staff has received extensive training in logic model development. Our work evaluating a CDC Community Transformation Grant, for example, gave us the opportunity to interact with esteemed colleagues from CDC, ICF (the national evaluator of the grant) and numerous public health departments around the country.
Questionnaires and Data Collection Instruments
Questionnaire development is not simply a matter of writing a series of questions. AI puts a great deal of thought into introductory statements, transitions, response categories and anticipating questions respondents are likely to ask.
These introductions, responses and transitions are clearly worded, comprehensive and unbiased. Our concentrated efforts in the design of the questionnaire pay off. Our carefully designed survey instruments produce higher response rates, valid data and ultimately result in valuable information that directly addresses your research questions.
Key Informant Interviews
AI often begins the research process with stakeholder interviews. This approach provides both quantitative and qualitative data to be applied to future questionnaire development, focus group discussion guides and analysis and interpretation of other research results.
In a recent needs assessment of families and children for example, we developed surveys of parents and families regarding their needs, barriers to accessing available services and resources, and challenges they face, beginning with a conversation with the service providers and community-based organizations who work with youth and families in the community.
Additionally, AI began a recent project developing strategic planning for a City's homelessness response by conducting key informant interviews with service providers and stakeholders who provide services or resources to homeless individuals and families in the community. These interviews guided our strategic planning efforts and in the development of feasible actions the City, service providers and other relevant stakeholders can take to prevent and combat homelessness in their community.
AI has conducted focus groups among many specialized groups, including teens, young adults, people who are blind or visually impaired, child care providers and many others.
Focus groups are generally comprised of a small diverse group of people belonging to the community, and are used to gauge reactions, through guided or open discussions, about the topic at hand. Typically, focus groups are a great way to obtain qualitative data; however, AI can use exercises or surveys of focus group participants to obtain quantitative data as well.
Our focus group moderators are trained and certified by the RIVA Institute, the gold standard in focus group training. We proudly point to our experience conducting focus groups, as well as qualitative coding for questionnaire development, for the Population Assessment of Tobacco and Health (PATH) Study, the nation's largest study of tobacco and health, conducted by the National Institutes of Health and the Food and Drug Administration.
AI uses SNAP survey software to format attractive, easy to read survey booklets for mail survey administration. Our SNAP software also enables us to post the survey, with compatible formatting, online and seamlessly combine the mail and online responses. Additionally, AI utilizes Survey Systems CATI (Computer Assisted Telephone Interviewing) software for telephone interviewing.
We combine our mail, online and telephone survey abilities to enhance response rates using a modified Dillman method. First, we coordinate a press release, social media and public announcements of the research. Next, we mail the survey package, with survey booklet, introductory letter and a self-addressed return envelope enclosed in a colorful, client-branded envelope. Surveys are followed by a combination reminder and thank-you postcard. Lastly, we call those who have not responded to previous efforts by telephone. This methodology is cost-effective and can significantly raise response rates beyond what any single method is able to produce.
As a data collection method for hard-to-reach populations such as homeless individuals, AI conducts intercept interviews. We also use intercepts to supplement a survey approach when response rates are low as a way of checking the validity of results on a few key questions.
AI used this approach with great success in our work with the State of Maine, interviewing parents of school-age children regarding their knowledge and experience with the State’s school breakfast program. The timing of the project was during summer, when parents were more difficult to reach regarding a school-related survey. AI conducted intercept interviews at local grocery stores throughout the State, offering a $5.00 gift card to the grocery store for each parent that completed an interview. The process enhanced the stores' relationship with its customers by offering a gift card applicable to their purchase as it enhanced our client's relationship with the public by presenting a professional effort to gather the opinions of their constituents.
AI has extensive and varied analytical abilities that include qualitative analysis, traditional quantitative methods that include tests of significance and association, as well as sophisticated techniques such as raking (complex sample weighting), GIS mapping and social network analysis.
Through data visualization, AI communicates our insights and findings using information graphics, statistical charts, plots and other visual tools. Our reports are comprehensive and visually appealing, combining professional graphics with clear and detailed explanations of our findings.
Presentation of Results
At the conclusion of a project, AI presents the results in-person to an audience of our clients' choosing. This presentation may be to a public audience or a select group of stakeholders or policy makers. This presentation and the discussion that often follows, enhances your ability to ensure the results are useful and provides an opportunity to share ideas for applying the findings and sharing of lessons learned.