About the Role The Omni team at Hark is building the next generation of AI experiences beyond text, enabling models to understand and generate content across multiple modalities, including text, audio, and beyond. Our goal is to create seamless, real-time multimodal intelligence that powers intuitive and immersive user experiences.
As part of the Omni team, you will build the multimodal foundation models at the core of this effort. You'll develop the training stages that adapt pretrained models before post-training. This work includes high-quality and synthetic data mixtures, learning-rate schedules, context extension, and distillation. You'll run experiments to improve model capabilities and work with evaluation and post-training researchers to assess checkpoint quality.
Responsibilities- Develop mid-training recipes using high-quality and synthetic data, including learning-rate decay schedules and data-mixture changes.
- Build synthetic data pipelines with code execution tests, math-answer checks, and filters for reasoning traces.
- Distill capabilities from larger models into smaller models through synthetic data generation and rejection sampling.
- Develop long-context data recipes, including document packing and sequence construction, and evaluate their effect on both long- and short-context performance.
- Evaluate mid-training checkpoints with the post-training team, measure how well results predict subsequent model quality, and select checkpoints for further training.
- Run controlled experiments on learning-rate schedules, mixture weights, and data ordering across text, code, and multimodal training data.
Requirements- Experience with continued pretraining or late-stage training, such as learning-rate decay, high-quality data mixtures, or context extension.
- Experience building synthetic data pipelines, including generation, verification, and filtering, and evaluating their effects on model quality and data diversity.
- Strong experimental judgment, including isolating recipe changes in controlled ablations and assessing whether improvements transfer to larger runs.
- Experience implementing and debugging distributed training workloads across GPU clusters.
- Experience designing evaluations and identifying confounds such as benchmark contamination, prompt sensitivity, and formatting effects.
- Strong engineering skills, including implementing training methods and building reliable data and experimentation pipelines.
- A record of improving model capabilities or training methods through research, datasets, systems, or production work.
Bonus Qualifications- Experience with distillation, rejection sampling, or model-based data generation for reasoning, code, or math.
- Experience with long-context training, including rotary position embedding (RoPE) scaling, context parallelism, or long-document datasets.
- Experience with post-training and evaluating how pretrained checkpoints respond to subsequent adaptation.
- Experience training or distilling smaller models for constrained inference budgets.
- Experience with audio, speech, image, video, or interleaved multimodal datasets.
- Experience detecting benchmark contamination and removing overlapping examples from training data.
CompensationThe US base salary range for this full-time position is between $180,000 - $450,000 annually.
The pay offered for this position may vary based on several individual factors, including job-related knowledge, skills, and experience. The total compensation package may also include additional components/benefits depending on the specific role. This information will be shared if an employment offer is extended.
