Director, Psychometrics - REMOTE

This position works remotely requiring a telecommuting agreement and adherence to the UCSC Telecommuting/Remote Work Guidelines. The employee shall be available in a full-time capacity during assigned telecommuting work hours (specific hours to be discussed with supervisor). While telecommuting/remote working, the employee is responsible for ensuring a worksite environment suitable for accomplishing their regular job duties during scheduled hours of work and, if applicable, arranging for dependent care just as they would if they were working at the regular UCSC worksite.

NO VISA SPONSORSHIP IS AVAILABLE FOR THIS POSITION.

STILL ACCEPTING APPLICATIONS: Position open until filled.

NOTIFICATIONS

UC Vaccination Policy: With limited exceptions, COVID-19 vaccinations are required for Covered Individuals under the Policy. Covered Individuals include all employees, students, or trainees who physically access a University facility or program in connection with their employment, appointment, or education/training.

Covered Individuals do NOT include employees who work 100% remotely with NO expectation that they will physically access any University location or program at ANY time. If 100% remote work ends, the employee is subject to policy.

INITIAL REVIEW DATE (IRD)

UC Santa Cruz staff jobs are posted until filled. Application materials submitted by 11:59 pm on the IRD will be routed to the hiring unit for consideration. NOTE: Materials submitted after the IRD will be forwarded only at the request of the hiring unit. Submit your materials before the IRD to ensure consideration by the hiring unit.

The IRD for this job is: 01-05-2023

STILL ACCEPTING APPLICATIONS: Position open until filled.

ABOUT UC SANTA CRUZ

UC Santa Cruz is a public university like no other in California, combining the experience of a small, liberal arts college with the depth and rigor of a major research university. It's known as an unconventional place where innovation and experimentation is part of the campus's DNA. That playful, bold spirit still thrives today, all on a campus renowned as among the most beautiful in the world.

DEPARTMENT OVERVIEW

The Smarter Balanced Assessment Consortium (SBAC) is a public agency supported by 15 members (13 states, one territory and the Bureau of Indian Education). Through the work of thousands of educators, Smarter Balanced created an online assessment system aligned to the Common Core State Standards (CCSS), as well as tools for educators to improve teaching and learning. Smarter Balanced is housed at the University of California Santa Cruz (UCSC) Silicon Valley Extension.

Our work is guided by the belief that a high-quality assessment system can provide information and tools for teachers and schools to improve instruction and help students succeed - regardless of disability, language or subgroup. We involve experienced educators, researchers, state and local policymakers and community groups working together in a transparent and consensus-driven process.

Smarter Balanced staff have a profound commitment to UCSC's Principles of Community including embracing diversity, equity and inclusion.

UCSC Extension provides professional training that reflects the academic rigor of the University of California and the hands-on, roll-up-your-shirtsleeves practicality of Silicon Valley culture. Developed with guidance from key industry leaders and academic experts, our certificate programs match the economic demand of your industry, from engineering to business, education to bioscience. The University of California reviews and approves UCSC Extension courses and programs for academic content, merit and instructor qualifications. Many of our courses may be applied toward degree programs, or may be eligible for professional certificates and licenses.

More information can be found here: https://smarterbalanced.org

JOB SUMMARY

Receives assignments in the form of objectives with goals and the process by which to meet goals. Provides direction to staff according to established policies and management guidance. Administers policies that directly affect subordinate staff. Recommends changes to department policies and practices. Identifies risks and responds accordingly. Provides priority setting and work flow analysis.

The Director, Psychometrics supervises the psychometricians and psychometric work for the Smarter Balanced Assessment Consortium, and reports to the Deputy Executive Program Officer (DEPO), Psychometrics and Research. The Director, Psychometrics will address complex issues and risks with novel solutions consistent with high stakes issues that are important to the Smarter Balanced Assessment Consortium. The Director manages and directs analyses and provides technical guidance regarding the technical measurement aspects of assessments that are part of the Smarter Balanced Assessment Consortium. The Director applies extensive knowledge as an educational measurement professional with an in-depth understanding of psychometrics and/or other statistical analysis techniques designed to support educational assessments and research projects of broad scope and complexity. This role requires a combination of responsibilities for managing people and individual projects. The candidate is expected to have strong management skills combined with advanced training, expertise and proven skills in test development, psychometrics, research, writing and speaking to technical and non-technical audiences, and project management for large scale assessment programs.

The Psychometrics and Research unit identifies, prioritizes, and manages changes and improvements to the Smarter Balanced assessment system, while helping to ensure that the formative, interim and summative components of the system are aligned. The Psychometrics and Research unit develops a strategic vision and leads psychometric analyses, research and data analyses in support of the technical quality of the assessment system.

APPOINTMENT INFORMATION

Budgeted Salary: $93,200 - $135,000 / Annually. Salary commensurate with skills, qualifications and experience.
Under California law, the University of California, Santa Cruz is required to provide a reasonable estimate of the compensation range for this role and should not offer a salary above the posted range. The salary shown above is the budgeted range the University reasonably expects to pay.

No. of Positions: 1

Benefits Level Eligibility: Full benefits

Schedule Information:

• Full-time, Fixed
• Percentage of Time: 100%, 40 Hours per Week
• Days of the Week: Mon-Fri
• Shift Includes: Day

Employee Classification: This is a Career appointment

Job End Date: None

Work Location: UC Santa Cruz Main Campus

Union Representation: Non-Represented

Job Code Classification: 006629 (DATA ANALYSIS SUPV 2)

Travel: Up to 25% of the time

JOB DUTIES

60% - Operations

• Directs and conducts psychometric analyses, such as item exposure, scale score distribution, measurement precision, and test characteristics, including but not limited to, item and scale stability, item bias and population invariance, generalizability, reliability, and validity.
• Serves as the primary contact for quality control of internal psychometric activities, as well as for services provided by service providers that are associated with measurement issues.
• Collaborates with and makes recommendations to the Deputy Executive Program Officer (DEPO), Psychometrics and Research, to develop scopes of work and support the selection of a service provider as necessary. Manages contracts. Escalates issues to the DEPO, Psychometrics and Research based on reviews of service providers work products, as necessary.
• Collaborates with Data Analysis team to develop processes and tools for tracking psychometric and research data and assists in transmission and presentation of data.
• Leads the formulation and prioritization of interventions and solutions based on data.
• Leads the development of written policies and procedures. Manages annual updates to the Smarter Balanced technical manuals.
• Represents the Smarter Balanced Assessment Consortium, by presenting at the Smarter Balanced technical advisory committee (TAC) meetings, other professional psychometric conferences, and client meetings.

20% - Research

• Provides consultation to research team to develop and evaluate instruments, tools and training.
• Directs, designs, coordinates, and conducts research studies of broad complexity and impact.
• Collaborates with Data Analysis team to provide technical assistance on data management and use of research data to analyze and interpret findings.
• Presents research findings to diverse audiences at internal meetings, technical advisory committee meetings, and professional conferences.

20% - Leadership and Supervision

• Manages the team of psychometricians, while staying actively involved in operational and innovative psychometric work.
• Trains, guides, and mentors team members.
• Partners with direct reports and engages with other staff to foster exceptional and productive internal and external relationships.
• Provides team leadership in evaluating and refining investigation methods. Leads investigations/studies for local or external constituents.
• Maintains professional expertise and conducts training on scientific research methods.
• Participates in developing and monitoring operational and budget processes, staff FTE, finance, and human resources.
• Recommends hiring of new employees, salary actions, and performance ratings.

REQUIRED QUALIFICATIONS

• Doctoral degree in Educational Measurement, Psychometrics, or related area
• 5+ years' experience in using psychometrics in educational measurement
• Has solid interpersonal and management skills, including at least one year with demonstrated experience in managing psychometric staff (one or more direct reports) who conduct time sensitive psychometric analyses, while maintaining a positive environment
• Demonstrated outcomes based on managing complex projects related to psychometrics as part of the operation of a large-scale assessment program
• Professional experience in a range of psychometric areas (e.g., field testing, item calibration, scaling, standard setting)
• A strong foundation in applying item response theory (IRT) and classical test theory (CTT) to products in a digital environment
• Track record in psychometric work within computerized adaptive testing (CAT)
• Experience conducting computerized adaptive testing (CAT) simulations
• Experience evaluating computerized adaptive testing (CAT) item pools
• Advanced ability to think creatively to recommend action steps or strategize solutions relative to operational psychometric issues and research
• Ability to plan and interpret psychometric and other advanced statistical analyses
• Experience designing and conducting empirical research studies, with advanced research skills at a level to evaluate alternate solutions and develop recommendations
• Demonstrated ability to communicate complex and highly technical psychometric information in a clear and concise manner both verbally and in writing to both technical and nontechnical personnel
• Knowledge of statistical analysis and related software programs, such as R and Python, as well as knowledge of MS Office software (Excel, Word, PowerPoint)
• Advanced project management skills to effectively manage multiple project assignments, timelines, deliverables, and milestones. Ability to lead and influence internally and externally to project teams
• Ability to lead and influence internally, as well as represent Smarter Balanced externally to project teams
• Experience ensuring that organizational practices, services and behaviors support and promote diversity among staff

SPECIAL CONDITIONS OF EMPLOYMENT

• Selected candidate will be required to pass a pre-employment criminal history background check.
• Ability to work long periods of time at a computer with or without accommodation.
• Must be able to work remotely from a home office and be able to fulfill requirements of the UCSC telecommuting agreement.
• Ability to work overtime including evenings, nights, weekends and holidays to meet operational needs, and work an alternate work schedule shift as required.
• Ability to travel nationally.
• Ability to maintain appearance and conduct suitable for working in a professional setting.
• The University of California has implemented a COVID-19 Vaccination Policy covering all employees. To be compliant with the policy, covered employees must submit proof of vaccination or have an approved exception or deferral, eight (8) weeks from date of hire.

SAFETY STATEMENT

All UCSC employees must understand and follow job safety procedures, attend required health and safety training, proactively promote safety at work, and promptly report actual and potential accidents and injuries.

HOW TO APPLY

Attach your resume and cover letter when applying for this job opening. Do not attach any documents to 'My Activities'. Visit our How to Apply tutorial for detailed instructions on our applicant process.

EEO/AA

The University of California is an Equal Employment Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, age, or protected veteran status. UC Santa Cruz is committed to excellence through diversity and strives to establish a climate that welcomes, celebrates, and promotes respect for the contributions of all students and employees.