NVIDIA | Deep Learning Algorithm Engineer (Junior/New Grad) | Coding | Phone Screen

1. What are important transformer layers, and what is time complexity of each of them in terms of lenght of seuqeunce and batch size. (The interviewer basically wanted to know that i understand that Attention is the bottleneck for long sequences since it is quadratic in seq len, whereas MLP is linear is seqeunce lenght but is quadratic in the hidden dim, The interviewer was helpful and we slowly worked thru this prob...