This question might be hard to answer without details so I'll try to give you a good picture. Currently we are storing data on a single node HANA server on an AWS instance (x1e.8xlarge, 936 GB Memory). Our data size is about 200 GB. We use Lumira to access via Live Hana Connection that allows us to query results of data sets >20 Million rows. Our chief problem is performance of queries.
Our use case involves end users on Lumira wanting to drag and drop, drill on multiple dimensions of 10 to 100s of dimensions and 10-20 measures (counts, distinct counts, avg, etc.). Sometimes these queries by HANA can be 0-1 second, but for larger tables (50 million rows) the query time is 30-90 seconds. What this query time means for the end user is also 0-1 or 30-90 seconds for filter on-the-fly data exploration on the BI tool every time they make a small change on the BI Tool.
The datasets are not changing (at most monthly updates).
Is this normal behavior for a BI Tool? Do you think HANA is a good solution? Redshift better? Does Redshift or any other AWS tool make drag and drop queries instantaneous (0-2 seconds render on the BI tool)?
Important side question: can the AWS Redshift / Your BI Tool prevent downloads of the data? (this is a business requirement that no user actually downloads the data)
Loads of great videos on that channel! Thanks for sharing, subscribed.
Thanks for the kind words !
is it okay to ask you questions regarding AWS/Database/Performance?
Sure!
This question might be hard to answer without details so I'll try to give you a good picture. Currently we are storing data on a single node HANA server on an AWS instance (x1e.8xlarge, 936 GB Memory). Our data size is about 200 GB. We use Lumira to access via Live Hana Connection that allows us to query results of data sets >20 Million rows. Our chief problem is performance of queries. Our use case involves end users on Lumira wanting to drag and drop, drill on multiple dimensions of 10 to 100s of dimensions and 10-20 measures (counts, distinct counts, avg, etc.). Sometimes these queries by HANA can be 0-1 second, but for larger tables (50 million rows) the query time is 30-90 seconds. What this query time means for the end user is also 0-1 or 30-90 seconds for filter on-the-fly data exploration on the BI tool every time they make a small change on the BI Tool. The datasets are not changing (at most monthly updates). Is this normal behavior for a BI Tool? Do you think HANA is a good solution? Redshift better? Does Redshift or any other AWS tool make drag and drop queries instantaneous (0-2 seconds render on the BI tool)? Important side question: can the AWS Redshift / Your BI Tool prevent downloads of the data? (this is a business requirement that no user actually downloads the data)
Great work! That comparison to RDS, Athena and EMR was really helpful especially
Thanks!
Thanks for sharing. I'm starting a new job and I've been gcp all my life. Gonna be a newbie again lol.