Technical Solutions Architect, Evals & Fine-Tuning

Innodata Inc. • Remote (United States) • 10h ago

Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.

Scope of the Role:

Innodata partners with leading foundation model labs, hyperscalers, and enterprise AI teams to build the data, evaluation, and post-training systems that make modern LLMs trustworthy and production-ready.

As a Technical Solutions Architect for Evals & Fine-Tuning, you are the technical face of Innodata to our most demanding customers. You sit at the intersection of client AI/ML teams, our research scientists and ML engineers, our subject-matter expert workforce, and our platform teams. You translate ambiguous customer goals — “improve factuality on long-context legal QA,” “build a safety eval suite for our next model release,” “design a DPO pipeline for our coding assistant” — into concrete, scoped, deliverable engagements.

This is a senior individual-contributor role for someone who has done the work: built fine-tuning pipelines, designed eval harnesses, argued with stakeholders about benchmark validity, and earned credibility with sophisticated ML buyers.

What You’ll Own:

Lead technical discovery with prospective and existing customers — foundation model labs, frontier AI teams, and large enterprises — to understand model objectives, gaps, and constraints.
Design end-to-end solutions across the post-training stack: SFT data curation, preference data collection for RLHF/DPO, golden datasets, custom benchmarks, LLM-as-judge pipelines, human-in-the-loop evaluation, red teaming, and multimodal eval (text, image, audio, video, long-context).
Architect engagements that combine Innodata’s platforms (GenAI Test & Evaluation Platform, Annotation Platform, GenAI Workbench) with our global SME workforce across 85+ languages and domains.
Author technical proposals, SOWs, solution diagrams, and pricing models in partnership with sales, delivery, and finance.
Run technical workshops, POCs, and pilot designs that de-risk larger programs and prove value quickly.
Serve as the ongoing technical advisor during delivery, partnering with applied research scientists, AI/ML research engineers, language data scientists, and program managers to keep solutions aligned with the original intent.
Feed customer signal back into Innodata’s R&D and product roadmap — what benchmarks customers actually want, where eval methodology is breaking, what new fine-tuning paradigms are gaining traction.
Stay current on the state of the art in evals (e.g., dynamic and agentic benchmarks, capability vs. safety evals, long-context and tool-use evaluation) and post-training (SFT, RLHF, DPO, RLAIF, rejection sampling, distillation).
Represent Innodata externally — at customer reviews, conferences, and in technical content.

You’ll Thrive in This Role If You Have:

7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles, with at least 2+ years focused specifically on LLM evaluation and/or post-training.
Hands-on experience fine-tuning LLMs (SFT at minimum; preference optimization methods like RLHF, DPO, or KTO strongly preferred) and designing the data pipelines that feed them.
Deep familiarity with LLM evaluation methodology: public benchmarks and their limitations, custom benchmark construction, LLM-as-judge design and its failure modes, inter-annotator agreement, and human eval workflow design.
Strong fluency in Python and the modern LLM toolchain (Hugging Face, PyTorch, vLLM, evaluation frameworks such as lm-evaluation-harness, lighteval, or equivalents).
Excellent technical communication. You can hold your own in a room with research scientists at a frontier lab and, an hour later, brief a non-technical executive on the same engagement.
A consultative mindset: you ask sharp questions, you push back when a customer’s stated request won’t actually solve their problem, and you are comfortable owning a recommendation.
Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or related field — or equivalent demonstrated experience.

The expected salary range for this position is $140,000 – $160,000 USD per year, based on experience, skills, and qualifications.

Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for payment, banking details, or sensitive personal information during the application process. To learn more on how to recognize job scams, please visit the Federal Trade Commission’s guide at https://consumer.ftc.gov/articles/job-scams.

If you believe you’ve been targeted by a recruitment scam, please report it to Innodata at verifyjoboffer@innodata.com and consider reporting it to the FTC at ReportFraud.ftc.gov.