We look for candidates who have excellent all-around problem-solving and coding capabilities, including data engineering. You will not get an offer if you lack that basic coding aptitude, and can only code against a standard spark API. We process terabytes of data and write or redesign data science algorithms on top of the data. If someone can't articulate why or why not you can process a 50GB of data with Python and your laptop, I don't think that person is qualified to solve the data engineering challenges we have, which deal with multiple billions of rows of transactional data over many years. Plus if anyone is asking for a 100% increment on their current salary, we expect some really big data experience from them, not just that "I processed 50GB of data with Spark".