Data Scientist (NLP and Generative AI Evaluation)

Role Overview

This position involves partnering with various professionals including policy makers, product owners, engineers, and domain experts to translate user needs into clearly defined machine learning and data analytics challenges. The candidate will be responsible for constructing and managing rigorous evaluation frameworks and data collections specific to natural language processing, generative AI, and other related machine learning technologies.

Key Responsibilities

• Collaborate with cross-functional teams to capture user requirements and transform them into analytical problems suited for machine learning solutions.
• Create and sustain strong and reliable evaluation datasets and mechanisms for NLP and generative AI models.
• Design and execute experiments aimed at improving model performance indicators such as quality, accuracy, consistency, reliability, latency, and cost-effectiveness.
• Identify and recommend appropriate models and technologies based on empirical evidence, operational constraints, and user expectations.
• Systematically analyze errors and detect performance shortfalls across different scenarios, guiding prioritization for improvements.
• Develop both automated and manual evaluation techniques while addressing metric limitations and risks inherent in AI-assisted evaluation processes.
• Collaborate with engineering teams to integrate validated enhancements, set up quality assurance checks, and monitor production performance.
• Ensure all data, experimental procedures, and model-related decisions are reproducible, well documented, and comply with responsible AI, privacy, and security standards.
• Effectively communicate findings, trade-offs, and suggestions to various technical and non-technical stakeholders.

Qualifications and Experience

• A bachelor’s degree in Computer Science, Data Science, Statistics, AI, Computational Linguistics, or a related quantitative discipline, or equivalent hands-on experience.
• Demonstrated experience applying data science or machine learning techniques to practical real-world issues, ideally in NLP, generative AI, search engines, or information retrieval.
• Proficiency in Python programming alongside familiarity with SQL, data processing workflows, version control, and software development methodologies.
• Strong foundation in statistical principles, experimental design, model evaluation, sampling strategies, error analysis, and validation of results.
• Experience handling unstructured text and complex data forms, using multiple metrics to evaluate machine learning or generative AI models.
• Knowledge of current AI and NLP concepts including embeddings, language modeling, prompt engineering, and evaluation methods.
• Capability to develop maintainable code and work collaboratively with engineers to transition data science solutions into deployed production systems.
• Excellent analytical, problem-solving, and communication skills tailored to audiences of varying technical backgrounds.

Preferred Skills

• Background in multilingual NLP, evaluation of translation quality, or collaboration with linguists and language reviewers.
• Experience with cloud AI services, vector search technologies, MLOps, production monitoring, or best practices in responsible AI.
• History of building AI-powered products for governmental, communication, regulated, or other high-assurance sectors.
• Understanding of Singapore’s public communications environment and familiarity with public-sector stakeholder needs.