Data Engineering

379 readers

2 users here now

A community for discussion about data engineering

Icon base by Delapouite under CC BY 3.0 with modifications to add a gradient

founded 1 year ago

MODERATORS

ericjmorey@programming.dev

Spark vs Presto: A Comprehensive Comparison (www.analyticsvidhya.com)

submitted 9 months ago by ericjmorey@programming.dev to c/data_engineering@programming.dev

6 comments fedilink hide all child comments

December 28 2023 Pankaj Singh writes:

In big data processing and analytics, choosing the right tool is paramount for efficiently extracting meaningful insights from vast datasets. Two popular frameworks that have gained significant traction in the industry are Apache Spark and Presto. Both are designed to handle large-scale data processing efficiently, yet they have distinct features and use cases. As organizations grapple with the complexities of handling massive volumes of data, a comprehensive understanding of Spark and Presto’s nuances and distinctive features becomes essential. In this article, we will compare Spark vs Presto, exploring their performance and scalability, data processing capabilities, ecosystem, integration, and use cases and applications.

Read Spark vs Presto: A Comprehensive Comparison

you are viewing a single comment's thread
view the rest of the comments

[–] sem@lemmy.ml 2 points 9 months ago* (last edited 9 months ago) (1 children)

Thank you! The conclusion is quite good, like use spark as ETL and Presto (Trino) for analytical queries but the article looks very outdated.

Spark is not about RDDs. Today the most usage of Spark is via DataFrame API. And it is not just syntax. The Catalyst itslef provide a lot of performance optimizations, like predicate pushdown on the level of orc/parquet reading, automatic skew joins detection, prunning, etc.

Also Presto in this case should be called as Trino because there was a rebranding in 2020

[–] ericjmorey@programming.dev 1 points 9 months ago

I was a questioning the quality of the source, thanks for confirming that it's not a top quality article.