Remote | AI Domain Expert — $140–$200/hour

We are sharing a specialised part-time consulting opportunity for experienced professionals across fields such as Software Engineering, Finance, Data Science, Legal, and related expert domains to contribute to an advanced AI training and evaluation project focused on reasoning quality, prompt design, data annotation, written feedback, and model-output assessment.

Selected professionals will review AI-generated outputs, design challenging domain-specific prompts, annotate data, conduct quality reviews, and provide structured feedback that helps improve model accuracy, reasoning, and reliability. The role is particularly suited to subject-matter experts with strong analytical judgement and exceptional professional writing skills. No prior experience in AI is required.

Key Responsibilities

AI Output Evaluation

• Review AI-generated responses for accuracy, logic, and relevance
• Assess whether outputs align with professional best practices in the relevant domain
• Identify factual errors, weak reasoning, unsupported conclusions, or missing context
• Evaluate the clarity and completeness of model responses
• Provide structured feedback supporting measurable improvements in output quality

Prompt Design & Refinement

• Design prompts that challenge AI reasoning and domain understanding
• Create realistic scenarios reflecting professional use cases
• Refine prompts to improve clarity, difficulty, and evaluation value
• Test whether prompts meaningfully distinguish strong from weak model performance
• Contribute to development of robust domain-specific evaluation tasks

Data Annotation & Evaluation

• Review and annotate data according to project guidelines
• Apply domain expertise to complex or ambiguous examples
• Maintain consistency across repeated annotation tasks
• Identify edge cases requiring additional interpretation
• Support development of reliable datasets for training and validation

Quality Review

• Conduct quality reviews of AI outputs and evaluation materials
• Review feedback or annotations produced by other experts
• Identify inconsistencies in application of project standards
• Support calibration across contributors
• Help maintain strong quality assurance across project workflows

Professional Feedback & Writing

• Deliver clear, concise, and constructive written feedback
• Explain complex analytical conclusions in accessible language
• Produce professional-quality written materials
• Edit responses for precision, structure, and clarity
• Adapt communication appropriately across technical and non-technical contexts

Critical Thinking & Domain Analysis

• Apply strong analytical reasoning to complex scenarios
• Distinguish substantive errors from acceptable alternative interpretations
• Evaluate arguments, evidence, and assumptions carefully
• Identify hidden weaknesses or inconsistencies in model reasoning
• Apply professional judgement grounded in subject-matter expertise

Ethical AI Evaluation

• Apply thoughtful judgement to ethical considerations in AI development
• Identify problematic or poorly reasoned model behaviour where relevant
• Evaluate whether outputs appropriately reflect professional standards
• Contribute to responsible and careful model-evaluation processes
• Maintain objectivity and consistency across sensitive evaluation tasks

Remote Collaboration

• Collaborate asynchronously with a global community of professionals
• Share insights and evaluation best practices
• Incorporate project feedback into evolving workflows
• Adapt efficiently to changing task requirements
• Communicate clearly across distributed project teams

Ideal Profile

• 3+ years of professional experience in Software Engineering, Finance, Data Science, Legal, or another relevant expert domain
• Strong subject-matter expertise within the applicant's professional field
• Demonstrated critical-thinking and analytical skills
• Exceptional written English
• Strong professional-writing and editing ability
• Proven experience producing high-quality professional materials such as technical documentation, investment memos, legal memoranda, research papers, or strategy presentations
• Experience reviewing, critiquing, or editing complex or high-stakes work
• Strong attention to detail
• Ability to articulate nuanced feedback clearly
• Comfortable working independently in remote environments
• Strong collaboration skills in distributed teams
• Ability to adapt to evolving project requirements
• Experience with data annotation, prompt engineering, or AI-output evaluation is advantageous
• Experience from leading technology companies, consulting firms, financial institutions, law firms, research organisations, or rigorous academic environments is valuable but not required
• No prior AI-training experience is required

Engagement Details

• Part-time independent contractor engagement
• Fully remote
• Compensation: $140–$200/hour
• Work will involve AI-output evaluation, prompt design, data annotation, quality assurance, professional writing, domain analysis, and structured feedback
• Strong subject-matter expertise, analytical judgement, and written communication are central to this engagement
• Assignments may vary depending on the contributor's professional domain
• Collaboration will occur remotely with project leads and other subject-matter experts
• Prior experience with prompt engineering, annotation, or AI evaluation is helpful but not required
• Project scope, workload, domain focus, and evaluation standards may evolve depending on project requirements
• Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party

About the Platform

This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.

By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy