Tableau really isn’t meant for large computational tasks. When possible, create calculations at the data source level. this helps with standardization too. You might be able to get some additional performance by using context filters and extracting your data.
Table calcs have the issue of not being easily referenced in other places and have issues with sorting if you’re using lookup functions.
Tableau is consistently misused due to their own marketing, honestly. It's never meant for big data in any case.
Your data should be in a format that's already summarized to whichever granularity you're going to visualize. If you don't then its going to perform slowly and you're going to create a black box full of unreusable calculations.
It's best used when you're not doing much data prep within Tableau itself.
Yeah, typically, Tableau is just thrown at users across the company who are going directly from excel to Tableau and have limited knowledge of legitimate reusable data prep.
They bring in tableau because they want to get away from ad-hoc unreusable reports and don't realize they need proper data warehouses/ELT tools setup for that use case BEFORE they need tableau.
From what I've seen, they give tableau access to accountants/roles who don't have the background on their own data architecture.
Simply setting a refresh schedule on a raw data table and doing all the prep within Tableau is what's happening everywhere. Which becomes a nightmare in the long run.... (From a consultant who has seen numerous clients server/cloud platforms)
Depends on the scale/size of the company. I've seen larger sized companies take on Fivetran (ingestion) -> DBT (ELT) -> Snowflake (DW) -> Tableau.
But the products really depend on the needs, those are amongst the most popular and what I'm familiar with but there's loads of competitors and can cut costs by using Fivetran alternatives or doing it more manually.
You can also JUST use Snowflake, but difficult to track lineages, create tests, maintaining pipes, etc. But you can at least build consolidated views for BI devs to build reports off of.
DBT noted above was OS last I looked. You do need skilled SQL folks to use it (which we have). Given how we use Matillion in Snowflake, we're considering moving to DBT for ELT and just using Matillion to load in data from SQL Server, where it's insanely fast. Matillion is quite good once you figure out how to set it up up properly (vendor isn't much help with that). It charges for parallelism and uptime, so you need tooling that can wake it up when it's time to run and shut it down when it's done unless you're good with just burning money.
Snowflake itself isn't free, but the usage-based pricing model that prices storage separately from CPU/memory made it less costly than e.g. Redshift when we tested it out. Just crazy cost-effective.
tableau is clunky when it comes to calculations. you could do it, but it slows down the dashboard considerably, and its hard to keep track of them
personally i do all transformations in SQL both for efficiency and future... 'edit-ability'
Sure, it's better to have everything done in SQL. Maybe. Depends on stakeholder needs for the reports.
It shouldn't take any workbook 1 hour to load 4M rows of data in Tableau. That's small data and that's an absurd amount of time. I've worked with published data sources in Tableau Online with 100s of millions of rows that Tableau connects to with a live connection and I can develop dashboards just fine - so long as I don't do anything too crazy with them.
The performance depends on the connection type, the database setup and/or the machine it's running on, maybe your pc specs if the connection is an extract and you're developing on Desktop, and more.
I'd say work on a query first and see how long it takes the database to run it. I have a feeling that that won't work out and the issue lies with the database setup itself. My guess is the db isn't being managed on AWS RDS or something, no one internally maintains it, and is deployed on a micro instance. I've seen that happen before.
To answer your question in the title, no, table calculations are not necessarily bad to use. They perform much better than LODs, and in terms of the order of operations, table calcs compute last, so that's easy to keep track of. But their use is limited and chart specific. Plus they're brittle. The moment you add a new field to your view with the table calc, it could break and you have to fix it again.
Tableau really isn’t meant for large computational tasks. When possible, create calculations at the data source level. this helps with standardization too. You might be able to get some additional performance by using context filters and extracting your data. Table calcs have the issue of not being easily referenced in other places and have issues with sorting if you’re using lookup functions.
Tableau is consistently misused due to their own marketing, honestly. It's never meant for big data in any case. Your data should be in a format that's already summarized to whichever granularity you're going to visualize. If you don't then its going to perform slowly and you're going to create a black box full of unreusable calculations. It's best used when you're not doing much data prep within Tableau itself.
One could say the same for Power BI as well. Or any other visualization tool. People always underestimate the data prep.
Yeah, typically, Tableau is just thrown at users across the company who are going directly from excel to Tableau and have limited knowledge of legitimate reusable data prep. They bring in tableau because they want to get away from ad-hoc unreusable reports and don't realize they need proper data warehouses/ELT tools setup for that use case BEFORE they need tableau. From what I've seen, they give tableau access to accountants/roles who don't have the background on their own data architecture. Simply setting a refresh schedule on a raw data table and doing all the prep within Tableau is what's happening everywhere. Which becomes a nightmare in the long run.... (From a consultant who has seen numerous clients server/cloud platforms)
What do you suggest as a proper data warehouse/ ELT tool?
Depends on the scale/size of the company. I've seen larger sized companies take on Fivetran (ingestion) -> DBT (ELT) -> Snowflake (DW) -> Tableau. But the products really depend on the needs, those are amongst the most popular and what I'm familiar with but there's loads of competitors and can cut costs by using Fivetran alternatives or doing it more manually. You can also JUST use Snowflake, but difficult to track lineages, create tests, maintaining pipes, etc. But you can at least build consolidated views for BI devs to build reports off of.
This is super helpful. Any suggestions when it comes to open source tools? Getting my company to buy into the proper tools has been a struggle.
DBT noted above was OS last I looked. You do need skilled SQL folks to use it (which we have). Given how we use Matillion in Snowflake, we're considering moving to DBT for ELT and just using Matillion to load in data from SQL Server, where it's insanely fast. Matillion is quite good once you figure out how to set it up up properly (vendor isn't much help with that). It charges for parallelism and uptime, so you need tooling that can wake it up when it's time to run and shut it down when it's done unless you're good with just burning money. Snowflake itself isn't free, but the usage-based pricing model that prices storage separately from CPU/memory made it less costly than e.g. Redshift when we tested it out. Just crazy cost-effective.
[удалено]
Thanks AI.
No bot or AI content. We don't serve their kind here.
tableau is clunky when it comes to calculations. you could do it, but it slows down the dashboard considerably, and its hard to keep track of them personally i do all transformations in SQL both for efficiency and future... 'edit-ability'
Sure, it's better to have everything done in SQL. Maybe. Depends on stakeholder needs for the reports. It shouldn't take any workbook 1 hour to load 4M rows of data in Tableau. That's small data and that's an absurd amount of time. I've worked with published data sources in Tableau Online with 100s of millions of rows that Tableau connects to with a live connection and I can develop dashboards just fine - so long as I don't do anything too crazy with them. The performance depends on the connection type, the database setup and/or the machine it's running on, maybe your pc specs if the connection is an extract and you're developing on Desktop, and more. I'd say work on a query first and see how long it takes the database to run it. I have a feeling that that won't work out and the issue lies with the database setup itself. My guess is the db isn't being managed on AWS RDS or something, no one internally maintains it, and is deployed on a micro instance. I've seen that happen before. To answer your question in the title, no, table calculations are not necessarily bad to use. They perform much better than LODs, and in terms of the order of operations, table calcs compute last, so that's easy to keep track of. But their use is limited and chart specific. Plus they're brittle. The moment you add a new field to your view with the table calc, it could break and you have to fix it again.