Power Query to tackle consolidating large database to reports or other approach?
I currently manage getting some financial reports out to multiple manufacturing facilities from a large corporate snowflake database. I've struggled with multiple challenges in our current process and I'm rethinking this from the beginning, I have an idea but before spending tons of time trying to implement it and possibly hitting a brick wall I wanted to throw it out here to see if you can save me some time/pain.
So currently I have data connections from snowflake to a big Master Excel file that selects the fiscal year and period and imports large amounts of data. This data is then cross references with other tables in the Master Excel file to group the transactions to certain categories and business units the plants want to see. This grouping data does not exist in the snowflake database. We then filter this data to each of the specific plants and copy/paste (which has a couple issues/problems I just cannot seem to solve) into workbooks that then present it the way they want to see it, sorted by categories and departments but still with all the individual transaction details for their plant to investigate curious issues. This is prepared and sent out weekly for the fiscal months (shows last week info and month to date info).
I had previously set up an access database that I could then link to from excel to do something similar so I was considering building the links and filters into each plant and using power query to access/import the categorized/filtered data into each plants file eliminating the middle step and any copy/pasting. This is option 1.
Option 2 and my real question, can I set a similar query up in an excel file to link to the snowflake databases, with the categorizing tables and such in it and then use power query to import from this Excel query master file to do something without using Access? Benefit being maintenance by other people in the org that are not familiar with access. My big concern being the snowflake databases are huge and without the current year/period filter would easily surpass the number of rows excel will handle. The month/period and plant filters would be in the individual plant files importing only current/relevant info, but the Master Excel Query would be open, set to import the whole history, but that file would never be opened/used except to possibly update a manual lookup table for a new account or business unit. Would this work over the access approach?
[link] [comments]
Want to read more?
Check out the full article on the original site