Location: Remote

Job Summary:

We require highly experienced business experts to help us push the boundaries in AI capabilities. You will be using your expert knowledge to find questions or scenarios that AI cannot solve, and then creating evaluation rubrics for what a good answer should contain based on your expertise. This is a unique opportunity to gain insight to the future of AI and to do highly rewarding work.

Key Responsibilities:

• Design nuanced evaluation questions that challenge AI models, focusing on complex, real-world business scenarios that require advanced reasoning.
• Identify and validate ideal answers for evaluation prompts, ensuring they are grounded in authoritative sources or synthesized from multiple credible references (e.g., case studies, academic journals).
• Analyze AI model responses through multiple iterations, identifying areas of failure and refining prompts for deeper insights.
• Develop detailed scoring rubrics that outline specific criteria for evaluating AI performance, enabling objective assessment of future model outputs.
• Collaborate with top researchers at the top AI Lab, AI engineers, and business strategists to align evaluation methodologies with project goals.

Required Skills and Qualifications:

• Professor or PhD in Business Administration (or related discipline).
• Expert knowledge in areas such as Business Strategy, Corporate Finance, Organizational Behavior, Strategic Management, and others.
• Strong analytical skills with the ability to craft complex, multi-layered business scenarios.
• Proficiency in sourcing and validating information from credible references to support evaluations.
• Experience in remote, asynchronous collaboration with cross-functional teams.

Preferred Qualifications:

• Familiarity with AI model evaluation and training processes.
• Experience designing and implementing evaluation metrics or rubrics.
• Knowledge of emerging trends in AI and its applications in business.