Description Senior Life Sciences Knowledge Engineer Job Description
About the role: As a Senior Life Sciences Knowledge Engineer at Norstella, you will sit at the intersection of deep scientific domain expertise and applied AI development. This role will be embedded within a group of life science thought leaders, but will interface across cross-functional teams of data scientists, machine learning engineers and data engineers. Your work centers on curating high-quality fine-tuned datasets which speak to the desired end-to-end behavior we want a model to internalize. The datasets and annotation guidelines/frameworks that govern it will play a critical role in our efforts to deliver predictive analytics and insights across clients.
Responsibilities: - Translate complex clinical, regulatory, and life sciences subject matter expertise/requirements into repeatable patterns that can be taught to a model through gold standard examples, working closely with data scientists and machine learning engineers to shape the model's schema, vocabulary, and target behavior.
- Through close collaboration between SME and technical colleagues, develop novel methods and parameters of model behavior, based on interpretation of requirements and quick iteration cycles.
- Design, build, and continuously refine fine-tuning datasets consisting of input/output pairs that demonstrate desired end-to-end behavior across the target task surface area, edge cases, and known failure modes.
- Author and maintain the annotation and labeling guidelines that govern dataset construction, ensuring the schema, vocabulary, and definition of "what good output looks like" remain consistent across contributors.
- Define the task taxonomy and output schema in close partnership with data scientists, ensuring data architecture aligns with downstream evaluation metrics and production requirements across NPD.
- Train and enable subject matter expert graders running eval rounds, including translating feedback to how data scientists implement improvements at the tool call layer.
- Run iterative dataset experiments: identify where the model is failing, design targeted example slices to close those gaps, and partner with the human-in-loop SMEs to measure the impact of each dataset change.
- Maintain provenance, licensing, and compliance documentation for every dataset, ensuring all training data meets GxP, regulatory, and intellectual property standards expected in life sciences and clinical settings.
- Conduct new proofs of concept for novel domain capabilities.
- Contribute to Norstella's knowledge base and taxonomy work and help design new agentic workflows based on domain-grounded language models.
Qualifications:
The skills you bring to the table:
- Graduate degree in life sciences, medical sciences, computer science or equivalent professional experience.
- At least 3 years of professional experience in production-grade life science datasets, including with AI-enabled applications.
- Experience working with structured publishing platforms and data tools; comfort with automation concepts
- Experience working with and statistically analysing large and complex data sets, including data cleaning and preprocessing.
- Experience working with Generative AI, especially LLMs, including agents, throughout the entire software development lifecycle (SDLC).
- Experience creating MCPs and consuming them into Agentic workflows.
- Excellent problem-solving skills and the ability to work independently.
- Excellent communication skills, especially between technical and non-technical teams.
Bonus points if you have experience in:
- Experience in developing, evaluating, deploying, and monitoring algorithms and models from proof-of-concept, experimental stages through production, in a reproducible, auditable, GxP-compliant manner.
- Experience with the AWS ecosystem, specifically with services like S3, ECS, API Gateway, SageMaker, and Bedrock.
- Familiarity with CI/CD processes, especially as applied to ML operations (MLOps), preferably with Azure DevOps.
- Experience in fast-paced novel development cycles.
Benefits:- Medical and prescription drug benefits
- Health savings accounts or flexible spending accounts
- Dental plans and vision benefits
- Basic life and AD&D Benefits
- 401k retirement plan
- Short- and Long-Term Disability
- Education benefits
- Paid parental leave
- Paid time off
