Machine Learning Engineer — AI Architecture Research

Job description

About the Role

We’re looking for a Machine Learning Engineer focused on AI architecture research to help design, prototype, and validate next-generation model architectures. You’ll work at the intersection of research and production — turning new ideas into scalable, real-world systems.

This role is ideal for someone who enjoys questioning architectural assumptions, experimenting with novel model designs, and pushing beyond standard Transformer-style approaches.

What You’ll Work On

- Research and develop new neural network architectures (e.g. alternatives or extensions to Transformers, recurrent / hybrid models, long-context systems)

- Design and run architecture-level experiments (scaling laws, memory mechanisms, compute trade-offs)

- Prototype models end-to-end — from research code to training-ready implementations

- Collaborate with inference and systems engineers to ensure architectures are deployable and efficient

- Analyze model behavior, failure modes, and inductive biases

- Read, reproduce, and extend cutting-edge research papers

- Contribute to internal research notes, benchmarks, and open-source efforts (where applicable)

What We’re Looking For

- Strong background in machine learning fundamentals and deep learning

- Hands-on experience implementing model architectures from scratch

-

Solid understanding of:

- Attention mechanisms, RNNs, state-space models, or hybrid architectures

- Training dynamics, scaling behavior, and optimization

- Memory, latency, and compute constraints at the model level

- Comfortable working in PyTorch or JAX

- Ability to move fluidly between theory, experimentation, and engineering

- Clear communicator who can explain architectural trade-offs

Nice to Have

- Experience with non-Transformer architectures (RNN variants, SSMs, long-context models)

- Background in research-driven startups or open-source ML projects

- Experience with large-scale training or custom training loops

- Publications, preprints, or notable research contributions

- Familiarity with inference optimization and deployment constraints

Why Join

- Work on core model architecture, not just fine-tuning

- Direct influence on the technical direction of a Series-A company

- Small, high-caliber team with fast feedback loops

- Opportunity to ship research into production

- Competitive compensation + meaningful equity

Originally posted on Himalayas