Member of Technical Staff - ML Training System
Given a tiny gpt training loop and fix 5 bugs to make the loop running correctly. five bugs are: 1. does not zero grad 2. grad does not step 3. attention scores does not devision by sort(d) 4. mixed used of k and v 5. ma...
