Permanent

Position: Permanent
Role: Senior Prompt engineer
Experience: 6 – 8 years
Work Location: Hyderabad / Bangalore
Notice Period: Immediate to 30 days
Job Summary:
We are seeking a Senior Prompt Engineer with experience in designing and evaluating prompts for large language
models (LLMs) in complex, multi-turn scenarios. You will play a strategic role in crafting high-quality prompt
structures, generating diverse evaluation datasets, and guiding prompt evaluation for dialog systems and
language-to-SQL models. This is a high-impact role for someone who blends linguistic intuition, technical rigor,
and a deep understanding of LLM behaviour.
Key Responsibilities
• Design and refine prompts for multi-turn evaluations, including disambiguation, iterative query construction,
and clarification handling.
• Lead the creation of evaluation datasets to assess model reasoning, output accuracy, and user interaction
quality.
• Analyze model outputs for correctness, coherence, and alignment with user intent.
• Build and validate synthetic or semi-synthetic data using LLMs, with human-in-the-loop review where
necessary.
• Define error classification schemes and support LLM raters in automated and semi-automated evaluation
workflows.
• Collaborate with ML researchers, data engineers, and product teams to scale and improve prompt strategies
across evaluation pipelines.
• Develop structured prompting frameworks (Chain-of-Thought, ReAct, DSPy) to improve reasoning and
accuracy.
• Build prompt versioning systems and A/B testing pipelines.
• Create self-service prompt libraries for internal teams (e.g., Marketing, Product).
• Automate prompt generation/evaluation with Python, LangChain, or LlamaIndex.
• Mentor junior prompt engineers and contribute to documentation, guidelines, and tooling.
• Research emerging techniques (e.g., multimodal prompting, AI agents).
Qualifications
Must-Have
• 6+ years of experience in NLP, data labelling, prompt engineering, or ML evaluation
• Proven expertise in crafting prompts for LLMs (ChatGPT, Claude, Gemini, etc.)
• Strong grasp of multi-turn language tasks, dialog evaluation, and language-to-SQL or semantic parsing
• Proficiency with Python, data annotation tools, and model interfaces (e.g., OpenAI, Hugging Face)
• Attention to linguistic nuance, ambiguity resolution, and conversational context
• Experience with NL2SQL tasks or datasets (e.g., Spider, BIRD, BAT)
• Experience with database platforms (SQL Server, Oracle, PostgreSQL, MySQL)