Nvidia

Deep Learning Algorithm Engineer

30 minute HM screen round. Below were the questions. 1. if you have a GPU at 100% util given a batch size and some throughput/latency numbers, how are these affected when you increase batch size further 2. How to identif...

Deep Learning Algorithm Engineer

1. What are important transformer layers, and what is time complexity of each of them in terms of lenght of seuqeunce and batch size. (The interviewer basically wanted to know that i understand that Attention is the bott...

Deep Learning Algorithm Engineer

1. Explain activation checkpointing 2. Between tensor parallel and data parallel, which would you use for latency sensitive applications 3. given an ML model in the wild, what steps would you take to profile it and optim...

Full stack

Design rate limiter. Input is an array of domain names. We need to output whether or not this domain is allowed. Input: ["abc.com","xyz.com","abc.com"] Conditions 1. Allow 2 requests per domain within 5second interval 2....

System Software Engineer, Distributed Systems

find the number of good subsequence all element in subsequence are unique min(sub)=a, max(sub)=b, all element between [a,b] should be present in subsequence return numbers of all good subsquences example: 13 11 4 12 5 4...

Performance Software Engineering Intern, TensorRT

This one was coding questions with trivia questions in between. Write code to transpose a matrix. Assume a square matrix and a row major layout. Where does this implementation spend most of its time? How does memory acce...

Performance Software Engineering Intern, TensorRT

This was a coding question about graphs and neural networks: You are given a neural network represented as a directed acyclic graph (DAG). Each node is an operator, and edges represent data dependencies. Design and imple...

Deep Learning Software Engineering Intern

This one had one coding question and some trivia questions Coding: Multi Head Attention in PyTorch Implement multi head self attention in PyTorch using basic tensor operations. The implementation should: Project inputs i...

Software Engineering Intern, CUDA Core Libraries

This interview was a combination of trivia and short coding questions in C++. Write a variadic template function that accepts an arbitrary number of arguments and applies a user provided functor to each argument, returni...

Software Engineering Intern, CUDA Core Libraries

The interview was a bunch of technical, non coding questions as well as a deep dive on my resume. Questions asked were on a number of topics: Performant Heterogeneous Systems What do we mean by a heterogeneous system, an...

Senior Software Engineer AI

1. Given a 1 D array of positive integers of size N, and given size of a window w such that 1<w<N , write an algorithm to calculate the mean of the window starting from Earliest starting position and sliding this window...