About the Role The Omni team at Hark is building the next generation of AI experiences beyond text, enabling models to understand and generate content across multiple modalities, including text, audio, and beyond. Our goal is to create seamless, real-time multimodal intelligence that powers intuitive and immersive user experiences.
As part of the Omni team, you'll develop the text, code, image, and video datasets used to pretrain Hark's models, with a focus on language and visual understanding. You'll identify useful sources, build filtering and synthetic supervision methods, and design training mixtures. You'll measure the effect of data choices through controlled experiments and work with infrastructure engineers to scale successful methods.
Responsibilities- Develop methods to select, filter, deduplicate, and prepare text, code, image, and video datasets at scale.
- Build paired and interleaved datasets, including image-text documents and video-text data. Preserve document structure and the alignment between visual content and text.
- Build quality classifiers and sampling methods, and measure their impact through controlled training experiments.
- Develop synthetic captioning and other supervision methods for image and video data, and assess their accuracy, coverage, and biases.
- Design mixtures and curricula across sources, domains, and modalities. Evaluate how sampling, sequence construction, and media resolution affect model quality and training efficiency.
- Measure corpus quality and coverage, check for benchmark contamination with evaluation researchers, and track source provenance and dataset versions.
- Work with infrastructure engineers to scale data processing and deliver datasets that training systems can load efficiently and reproducibly.
Requirements- Experience developing training data for language or vision-language models, with depth in one or more areas such as text and code curation, image-text datasets, or video data.
- Experience designing controlled data experiments, measuring model improvements, and assessing whether results transfer across training scales.
- Strong Python engineering skills and familiarity with distributed processing frameworks such as Spark, Ray, Beam, or similar systems.
- Understanding of data quality issues such as duplication, noisy supervision, source imbalance, and benchmark contamination.
- A record of improving training data or model performance through research, datasets, systems, open-source contributions, or production work.
Bonus Qualifications- Experience with image-text alignment, interleaved document construction, video frame sampling, or temporal grounding.
- Experience with synthetic captioning, model-based filtering, or embedding-based data selection.
- Experience with data mixtures, curricula, or scaling laws for data.
- Familiarity with tokenizer design, visual representations, or training language and vision-language models.
- Experience with provenance, privacy filtering, licensing metadata, or multimodal benchmark decontamination.
CompensationThe US base salary range for this full-time position is between $180,000 - $450,000 annually.
The pay offered for this position may vary based on several individual factors, including job-related knowledge, skills, and experience. The total compensation package may also include additional components/benefits depending on the specific role. This information will be shared if an employment offer is extended.
