Modal | Member of Technical Staff - ML Training System (Mid Level) | Training Concept & Pytorch Interview | Onsite
noah2001·Interview Date:May 7, 2026Region:NA (North America)Hiring Team/Org:training infra
Given a tiny gpt training loop and fix 5 bugs to make the loop running correctly. five bugs are: 1. does not zero grad 2. grad does not step 3. attention scores does not devision by sort(d) 4. mixed used of k and v 5. mask incorrect follow up question: 1. why attention need qkv instead of qk 2. why we need Devision for attn scores 3. why we need mask 4. why we need residual 5 why metrics matter during training
Sign in to view the full interview experience
Create or use your InterviewDB account to read the full Warren post and all shared details.
Sign in to continue0