As more big data platforms begin to support both Spark and SQL, you might wonder which one to choose. This article aims to offer some guidance from a data engineering perspective. We'll focus on how each language supports the development of scalable data pipelines for data transformation, setting aside performance considerations for now. This topic would deserve its own separate discussion.
