Description

Guided by the American Evaluation Association (AEA) Guiding Principles and Evaluator Competencies, the Assistant Director, Research Evaluation collaborates closely with CTSC leadership to advance rigorous, stakeholder-engaged, and utilization-focused evaluation activities. These activities are designed to assess the reach, implementation, effectiveness, sustainability, and translational impact of CTSC programs and initiatives, with a particular focus on the Clinical and Translational Science Award (CTSA) program. Key responsibilities include facilitating data capture and electronic tracking; managing progress and outcomes reporting for funders and stakeholders; conducting in-depth analyses to generate actionable findings for strategic planning and program improvement; implementing innovative approaches and best practices for evaluating and visualizing outcomes and impact; and disseminating scholarly and applied evaluation products.

The Assistant Director acts as a strategic evaluation partner to CTSC leadership, investigators, faculty, trainees, community stakeholders, and national CTSA consortium collaborators. This role requires expertise in evaluation theory and practice, implementation science, translational science evaluation, quantitative and qualitative methodologies, and data visualization within complex, multi-institutional, federally funded research environments.

ESSENTIAL FUNCTIONS

1. Strategic Evaluation, Performance Measurement, and Continuous Quality Improvement: Collaborate with the CTSC Principal Investigator, Executive Director, and Evaluation team to implement evaluation systems for CTSC programs. Ensure alignment with NIH CTSA priorities, translational science principles, and CTSC strategic goals. Develop and administer data collection, tracking, analysis, and reporting strategies across research, education, workforce development, and community engagement initiatives to assess program outcomes and impact. Design and operationalize evaluation frameworks, logic models, outcome measures, dashboards, and key performance indicators (KPIs) to support evidence-informed decision-making, organizational learning, accountability, and continuous quality improvement (CQI). Conduct in-depth analyses and synthesize evaluation findings into actionable recommendations, technical reports, executive summaries, presentations, dashboards, and data visualizations for leadership, investigators, faculty, funders, community stakeholders, and national partners as requested. Interact with the CTSC Executive Director and program leadership to obtain and track investigators who have benefited from CTSC resources and data. Monitor evolving stakeholder and community needs and contribute evaluation findings to support responsive programmatic and translational science initiatives. Apply translational science evaluation frameworks, including the Translational Science Benefits Model (TSBM), to assess scientific, workforce, clinical, community, policy, and public health impacts across CTSC initiatives. Participate in the Association of Clinical and Translational Science (ACTS) Evaluation Special Interest Group meetings and national CTSA evaluation activities to monitor emerging best practices, metrics, and evaluation priorities and disseminate relevant findings to CTSC leadership. (30%)

2. Evaluation Consultation and Methodological Support: Provide evaluation consultation and methodological guidance to program leaders and teams in the design, implementation, and dissemination of rigorous evaluation strategies for educational programs, workforce development initiatives, community engagement activities, and translational research programs. Advise principal investigators in designing the evaluation of grant proposals, which may involve assisting in the design, implementation, and reporting of accomplishments and achievement of outcomes. Develop program theories, measurable outcomes, survey instruments, data collection protocols, performance-monitoring approaches, and dissemination plans, and collaborate with the CTSC Dissemination & Implementation team. Foster collaboration and shared learning among investigators, trainees, faculty, and CTSC programs to support an integrated and cohesive translational science enterprise. (20%)

3. Evaluation Operations, Data Stewardship, and Federal Reporting: Manage Evaluation Unit databases, electronic tracking systems, longitudinal monitoring processes, and reporting platforms to support accurate and timely collection, validation, analysis, and submission of NIH-required metrics, progress reports, and evaluation deliverables. Coordinate with investigators, trainees, faculty, and program personnel to ensure accurate and timely completion of reporting requirements. Support the development, dissemination, tracking, and analysis of evaluation instruments while maintaining high standards of data integrity, confidentiality, professionalism, ethical evaluation practice, and responsible conduct of research. Assist the CTSC Principal Investigator, Executive Director, and Evaluation team with strategic initiatives, evaluation studies, special projects, and other evaluation-related responsibilities as required. (15%)

4. Strategic Planning and Organizational Effectiveness: Contribute to the development and implementation of evaluation-informed strategic plans that align program goals, objectives, performance measures, and accountability structures with translational science priorities, organizational objectives, and funding requirements. Conduct benchmarking analyses, evaluability assessments, environmental scans, and evidence syntheses to support adaptive management, organizational effectiveness, sustainability, and continuous improvement. Identify and assess evaluation-related and operational risks, and provide recommendations to strengthen program effectiveness, impact, and long-term sustainability. (8%)

5. Stakeholder Engagement, Knowledge Translation, and Dissemination: Keep stakeholders informed of CTSC activities, accomplishments, evaluation findings, and translational impacts through clear, strategic, and audience-centered communication. Serve as a bridge among researchers, practitioners, institutional leadership, community stakeholders, and national CTSA collaborators to disseminate evaluation findings, implementation lessons, research accomplishments, and translational science outcomes at local, regional, and national levels. Promote stakeholder-engaged and utilization-focused evaluation approaches that strengthen collaboration, transparency, organizational learning, and the use of evaluation findings. (8%)

6. Scholarly Evaluation and Applied Research: Contribute to scholarly evaluation and translational science research activities by designing, analyzing, interpreting, and disseminating rigorous evaluation studies and applied research initiatives. Assemble and analyze quantitative and qualitative data. Prepare publication-quality figures, visualizations, manuscripts, conference presentations, and grant-related materials in collaboration with principal investigators and interdisciplinary research teams. Serve as first author, co-author, and methodological collaborator on peer-reviewed publications, evaluation reports, presentations, and grant proposals. Present findings at scientific meetings, seminars, professional conferences, and national CTSA evaluation forums. Maintain the highest standards of integrity, confidentiality, ethical evaluation practice, and responsible conduct of research. (7%)

7. Evaluation Capacity Building, Facilitation, and Technical Assistance: Support evaluation capacity building across CTSC programs and partner institutions through training, mentoring, consultation, facilitation, technical assistance, and collaborative learning activities designed to strengthen evaluation knowledge, skills, and utilization. Facilitate stakeholder meetings, evaluation planning activities, workshops, and interdisciplinary learning initiatives focused on evaluation methods, TSBM, and continuous quality improvement (CQI) practices. (5%)

NON-ESSENTIAL FUNCTIONS

1. Support CTSC-wide meetings, retreats, and collaborative initiatives as needed. (3%)

2. Participate in institutional committees, planning groups, and working teams. (2%)

3. Perform additional duties and strategic projects as assigned. (2%)

CONTACTS

Department: Regular contact with the CTSC PI, Executive Director, Deputy Director, Evaluation team, and other staff and faculty members within the CTSC organization.

University: Regular contact with CTSC module leads and directors of CTSC companion training programs and offices for service provision within the CTSC across all of northern Ohio. Contact with UTech (IT services) and senior leadership and faculty to provide advice on planning and securing resources, evaluating and implementing activities.

External: Contact with members of the CTSC's partner institutions, including CWRU, University Hospitals Cleveland Medical Center, Cleveland Clinic, MetroHealth Medical Center, VA Northeast Ohio Health System, University of Toledo and Northeast Ohio Medical University. Interact with CTSA Evaluators and CTSA hubs across the USA (the national "CTSA Consortium") and with national organizations at national meetings.

SUPERVISORY RESPONSIBILITY

May supervise students. This position has no direct supervision of staff employees. Performance feedback will be requested as it relates to individuals serving on committees this position directs.

ESSENTIAL SKILLS

1. Strong track record of experience with leading evaluation efforts in a complex, multi-institutional, and matrixed organization.

2. Advanced proficiency in mixed-methods research and evaluation methodologies, including survey design, qualitative inquiry, focus groups, interviews, case studies, and longitudinal analysis. Expertise in statistical and qualitative analytic software and data management systems (e.g., SPSS, SAS, R, NVivo, ATLAS.ti, REDCap, Tableau, ARCGIS, or equivalent).

3. Excellent analytical skills with a demonstrated ability to translate complex data into actionable recommendations.

4. Strong communication and presentation skills, both oral and written, with demonstrated expertise in technical report writing and grant proposal development.

5. Demonstrated ability to work as part of a team to collaborate effectively with several stakeholders, including administrators, academic units, faculty, graduate students, and staff and external partners.

6. Proven organizational, administrative, and interpersonal skills including the ability to interact on a professional level with faculty, staff, and representatives from the community, government, and industry.

7. Ability to independently plan, write, and submit grant proposals.

8. Strong methodological, analytical, and project management skills.

9. Knowledge of Institutional Review Board (IRB) process and regulations.

10. Demonstrated ability to maintain a high level of confidentiality, professionalism, and sensitivity and to use discretion in interacting with various constituencies. Exhibits good judgment, flexibility, and initiative.

11. Knowledge of and familiarity with university constituencies, organizational structure, policies and procedures.

12. Working knowledge of REDCap (Research Electronic Data Capture) or similar web application for building and managing online surveys and databases.

13. Working knowledge of evaluation practices and proficient in evaluator competencies as defined by the American Evaluation Association (AEA).

14. Demonstrates expertise in Microsoft Office, Google Suite (Drive, Gmail, Docs, Sheets, Forms, Slides, and Calendar), database management software.

WORKING CONDITION

General office work environment with hybrid work option available. This position may require occasional and often unexpected longer than 8-hour days during the week and/or on weekends. Travel as needed for the Clinical and Translational Science Award program meeting once per year, as well as other Evaluation meetings, workshops, conferences, or other university business.

Qualifications

Education and Experience: Doctoral degree (PhD or equivalent) in Evaluation, Measurement, Research Methodology, Public Health, Health Services Research, Social Sciences, Implementation Science or a closely related field, required. Minimum of three years of progressively responsible experience leading program evaluation, performance measurement, applied research within complex research, healthcare, academic, nonprofit, or federally-funded environments.

Preferred Qualifications

• Advanced training in evaluation theory and practice.
• Experience in biomedical, clinical, translational, public health, or health sciences research environments.
• Experience with NIH-, NSF-, CDC-, PCORI-, HRSA-, or other federally funded initiatives.
• Experience in academic medicine or university-based evaluation systems.
• Experience with implementation science, dissemination science, and translational research evaluation.

Application Instructions

All applications must include: Cover Letter, CV, and list of 3 references with contact information.