Realistic roles for ML-strong / systems-weak:

1. Pre-training research — your strongest match. Algorithms, architectures, data, scaling. You can pick up the systems side gradually.

2. Alignment research — also strong match. Mostly ML + judgment. Less infrastructure-heavy.

3. Research engineering — bridges ML and systems. You’d implement research ideas, build evaluation frameworks. Decent transition path; the systems work is well-bounded.

Less realistic:

4. Inference engineering — heavy systems focus. CUDA, distributed systems, latency optimisation. Would need substantial systems study first.

5. Pre-training engineering — heaviest systems focus. Distributed training at scale. Year+ of systems work before being effective.

Typical interview process (5-8 rounds):

1. Initial chat (30 min): high-level fit, what you’re working on, what excites you.
2. Background interview (45 min): deep dive on your most recent work. Be prepared to defend technical choices.
3. Technical interview (60 min): ML knowledge. Could be paper discussion, system design, or coding.
4. Coding (60 min): implement something non-trivial. Often a numpy implementation of a known algorithm (LayerNorm, attention, etc.). Tests fluency, not creativity.
5. Research interview (60 min): present research direction or critique a paper. Tests taste and depth.
6. Manager interview (45 min): cultural fit, what kind of work environment you want.
7. Final loop (3-5 hours): meeting with potential team members. Often pair-programming or whiteboard discussions.

What they’re actually evaluating:

• Technical depth: can you reason about systems / models / math correctly?
• Speed of thought: how quickly do you get to the right framing?
• Judgment / taste: what would you research; what would you NOT research?
• Collaboration: can you debate without being dogmatic?
• Communication: can you explain complex ideas clearly?

Common failure modes:

• Strong on math, weak on engineering practicality.
• Strong on engineering, weak on motivating research directions.
• Strong on knowing existing techniques, weak on extending them.
• Knowing every paper but not having an opinion about which are important.

Realistic timelines:

• From “deciding to apply” to “offer in hand”: 2-6 months.
• From “joining” to “shipping first thing”: 6-12 months.
• From “joining” to “having opinions that influence direction”: 1-2 years.

The labs hire slowly and selectively. Be patient.

First 3 months: realistic expectations:

1. You won’t ship anything significant. Frontier labs have enormous existing tooling, codebases, and conventions. Learning these takes weeks to months.

2. You’ll be assigned to a team. Could be pretraining, alignment, inference, etc. Your work scope is initially narrow.

3. You’ll work on small contributions first. Implementing a small experiment, writing an eval, fixing a bug, contributing to a meeting note. The “small first” is by design — you need to demonstrate competence at scale before getting bigger projects.

4. You’ll attend many meetings. Frontier labs are heavily coordinated; daily standups, weekly team meetings, monthly all-hands. Initially, much of this is over your head; that’s normal.

5. You’ll feel slow. Compared to academic work where you could iterate freely, lab work has more constraints (CUDA cluster scheduling, code review, alignment with team direction). This is frustrating initially.

How to ramp up effectively:

1. Learn the codebase, not the science. The papers you’ve read aren’t directly applicable; the existing code IS. Spend the first month reading code, running existing scripts, understanding how data flows.
2. Find a “small but meaningful” first project. Ideally something where you can demonstrate competence in 2-4 weeks. Avoid biting off too much.
3. Build relationships with 3-5 people. Senior people you can ask questions, peers you can collaborate with, junior people whose work you can boost. Network in the building.
4. Attend the weekly group meetings of adjacent teams. You’ll learn what’s happening across the lab. This builds the implicit knowledge of “what’s working, what’s not, what’s coming.”
5. Don’t try to publish a paper in your first 6 months. Counterintuitive: focus on internal contributions, learn the systems, build reputation. Papers come naturally from solid contributions, not from rushed first-author attempts.
6. Read the internal documentation. Frontier labs have extensive internal wikis, design docs, decision logs. Reading these is far more efficient than learning by trial.

Common mistakes:

• Trying to “fix” the existing system before understanding why it’s that way.
• Proposing too-ambitious research directions without buy-in.
• Working in isolation instead of pair-programming with team members.
• Saying “I can do that in a weekend” — usually it takes 3 weeks.
• Underestimating the importance of evaluation and reproducibility.

Where the value comes from in 6-18 months:

• You contribute to a real shipped model improvement.
• You become the “owner” of some system or experiment series.
• You influence the team’s research direction.
• You write a paper (internal or external) on your work.
• You mentor newer hires through their first 3 months.

The trajectory is slower than academic research but the impact per unit of effort is much higher because of the scale of resources.

What papers don’t tell you:

1. The hyperparameter search budget. Did they try 100 configs or 1000? You probably can’t afford the same exploration.

2. Negative results. Their first attempt may have been worse than baseline; the published result is the polished version.

3. Infrastructure assumptions. The paper assumes you have access to certain training infrastructure (kernels, distributed primitives) that you may not have.

4. Data quality dependencies. The improvement may only show up at certain data scales / qualities / distributions.

5. Interaction with other techniques. The improvement might combine with other techniques (RoPE, MoE, etc.) in non-obvious ways.

6. Failure modes at smaller scale. Many techniques work at 70B but not at 7B (or vice versa). Often unstated.

7. What “the baseline” actually was. Was it tuned aggressively? Or just standard hyperparameters?

8. The compute budget. “We trained for 100K steps” — but on how many GPUs? You may not have the same compute.

How to extract more information:

1. Read released code carefully. Often more detailed than the paper. Look at hyperparameters, schedules, and special handling.

2. Check GitHub issues on the paper’s repository. Other replicators often discuss what they had to change.

3. Reach out to authors. Be specific: “I’m trying to apply technique X to model Y, but seeing Z. Any thoughts on what might be different?“

4. Check follow-up papers. Other groups that have built on this work usually report what they had to do differently.

5. Look for community resources. Twitter/X discussions, blog posts, podcast interviews — sometimes important details surface there.

6. Run a small-scale replication. Apply the technique at a much smaller scale first. If it doesn’t work even there, something’s wrong with your understanding.

What to do if you can’t replicate:

1. Adapt the principle, not the recipe. The paper’s specific recipe may not generalise; the underlying idea probably does.

2. Find a similar simpler technique. Often the paper builds on prior work; the prior work may be more accessible.

3. Be skeptical of small improvements. A 1-2% improvement in a paper is often within replication variance. Don’t over-engineer to chase it.

4. Build your own baselines. Compare against YOUR strongest implementation, not the paper’s reported number. This is often more meaningful.

The bigger picture:

Frontier labs have huge advantages: compute, data, tooling, talent, internal lore. As a smaller player, you can’t replicate their full stack. But you CAN apply their high-level insights to your scale.

The papers are a contribution to the field; they don’t have to be exactly applicable to be valuable. The principle is what travels; the recipe is local to the lab that produced it.