Job Description
Note: The job is a remote job and is open to candidates in USA. Crossing Hurdles is seeking a LLM – AI Quality Analyst to work on a short-term contract basis. The role involves evaluating AI responses and ensuring data quality through various analytical tasks.
- Responsibilities
- Design multi-turn conversational prompts based on personal context
- Evaluate personalized AI responses for relevance, grounding, and helpfulness
- Assess correct and incorrect use of personal data in model outputs
- Perform side-by-side (SxS) evaluation and ranking of AI responses
- Identify grounding errors, poor inferences, and forced personalization
- Write clear, structured rationales referencing specific conversation turns
- Extract and verify model debug information and data source usage
- Maintain strict data hygiene by deleting evaluation conversations
- Skills
- English fluency (reading and writing)
- Strong experience in data annotation, AI quality evaluation, content moderation, or related roles
- Strong analytical thinking and attention to detail
- Ability to evaluate nuanced and ambiguous AI responses
- Comfortable using a primary personal Google account with enabled data sources
- BS/BA degree or equivalent experience in a relevant analytical field
- Strong written communication and structured feedback skills
- Self-motivated and able to work independently in a remote setting
- Reliable desktop/laptop with stable internet connection
- Company Overview
- At Crossing Hurdles, we specialise in customised recruitment and staffing solutions designed to drive success for businesses and professionals. It was founded in 2022, and is headquartered in , with a workforce of 11-50 employees. Its website is https://www.crossinghurdles.com/.
Apply tot his job
Apply To this Job