Let’s do a hill sprint on Power BI Dataflows: The What, How, When and Why of Power BI Dataflows!
I am having a series called hill sprints (since we are climbing mountains – hehe) that will provide a to the point introduction on a topic covering the What, How, When and Why.
Why hill sprints?
Hill sprints are essentially a form of interval training – probably one of the more intense (but engaging) options. They are quick, brutal and to the point. Let me know if you have another fun analogy towards climbing mountains that would make sense for a series name! (Having way to much fun with this)
First Hill Sprint Series will be on Power BI Service. In this series we will go through some of the main components in Power BI Service, explaining what is it, how can you set it up, when should you use it, and why should you use it.
Hopefully, this can provide some quick insights and knowledge on the components and help decide if this is the tool for you with your current setup or challenge.
Power BI Dataflows are essentially Power Query Online that provide a self-servie data integration tool.
This way you can create reusable transformation logic and build tables that multiple reports can take advantage of.
How can you set up Power BI Dataflows?
You set up a Power BI Dataflow in Power BI Service. This is done through the workspace you want to hold the dataflow and by clicking “New”.
Here you can choose if you want to create a new dataflow or build on top of an existing one.
For more information on how to set this up you can follow the Microsoft documentation here.
When should you use Power BI Dataflows?
Dataflows are particularly great if you are dealing with tables that you know will be reused a lot in your organization, e.g. dimension tables, master data tables or reference tables.
If you want to take advantage of Azure Machine Learning and Azure Cognitive Services in Power BI this is available to you through Power BI Dataflows. Power BI Dataflows integrates with these services and offers an easy self-service drag-and-drop solution for non-technical users. You do not need an Azure subscription to use this but requires a Premium license. Read more about ML and Cognitive Services in Power BI Dataflows here.
In addition, Power BI Dataflows provides the possibility to incrementally refresh your data based on parameters to specify a date range. This is great if you are working with large datasets that are consuming all your memory – but you need a premium licence to use this feature.
Why should you use Power BI Dataflows?
Power BI Dataflows can help us solve a range of challenges with self-service BI.
Improved access control
One source of truth for business logic and definitions
Provides a tool for standardization on the ETL process
In my recent projects, Data Governance and Master and Reference data were one of the main deliverables. That means the need to dig into the domain of Data Governance! It is a challenging and somewhat daunting task, but also very motivating as the more I read on the topic, the more I see this value.
In this post, I explain how I prepared for the Data Management Fundamentals Exam to gain the CDMP Associate Certification. My next goal is to get the CDMP Practitioner certification.
So, why should you take the Certified Data Management Professional (CDMP) certification?
With the ever-growing need for insights from data in combination with the exponential growth of data itself, the need for controlling this data has never been more relevant. Still, people either tend to frown or fall asleep when the topic of Data Governance is raised. My experience is that it often is associated with less freedom, more bureaucracy and slower progress.
But that is not true!
In order to serve the insights, consistency, quality, availability and security of the data within an organization, data governance is fundamental. As data is growing, self-service solutions are becoming more available, organizations are more at risk of security breaches as data is not where it should be, ambiguity and distrust in reports and databases as KPIs and master and reference data deviate – to mention a few challenges.
So, how can we build the necessary trust needed in our data?
When preparing for the certification, I learned that governance really is about much more than just making rules and policies – it is about understanding the bigger picture. We cannot only care about the roles, policies, and rules. We also need to consider the architectural approach, data modelling, storage solution, quality processes and security.
Data Governance is relevant to anyone who works with the bigger picture as Architects, Business Managers, IT Managers
Therefore I think the DMDBOK2 is relevant for anyone in a role that works to understand the bigger picture, whether it is technical or business. Not only the data governance-specific roles.
What are the DAMA certifications?
There are 4 certification levels you can take. In this blog I explain how I prepared for the Data Management Fundamentals Exam to gain the CDMP Associate Certification.
CDMP Associate
60 % pass on the Data Management Fundamentals Exam
CDMP Practitioner
70 % pass on the Data Management Fundamentals Exam
70 % pass on two of the Specialist Exams
2-10 years of industry experience
CDMP Master
80 % pass on the Data Management Fundamentals Exam
80 % pass on two of the Specialist Exams
10+ years of industry experience
CDMP Fellow
25+ years of industry experience
Globally recognised & respected thought leadership
A significant contribution to the Data Management profession
CDMP Master
Contribution to CDMP & DMBOK
By nomination
How did I prepare for the Data Management Fundamentals Exam?
I did the 4 following:
Give yourself a deadline
Purchase the Exam
Read the DMBOK2
Do the practice exam over and over (and over) again
1. Give yourself a deadline
Start by making a master plan.
What week are you planning to take the exam? Block your calendar right away. You need approximately 2-3 hours to finish the exam.
DAMA recommends 3 weeks (with some level of focus and time set aside) to prepare for the exam. I personally used 5 weeks from I made the commitment until I took the exam. I think this was a bit too long as I was not stressed enough to prioritize time to prepare for the first couple of weeks.
Set a deadline and block an exam slot in your calendar from the start
tip #1
Anyways, we are all different! The most important thing is that you make a plan and set a deadline for yourself.
2. Purchase the exam
In order to stay true to my deadline I purchased the exam. The exam does not have an expiration date or scheduled time, and you can take it whenever it suits you. Still, I would recommend purchasing the exam when you have given yourself a deadline as the commitment is stronger, and you get access to a training exam that I found super useful!
Purchase the exam right away – this way you have commited from start
tip #2
The cost of the exam is USD$311.
See more on how to purchase the exam here. Choose “Purchase” under the Data Management Fundamentals Exam box.
The DMBOK2 consists of 17 chapters and the first 15 chapters are the ones you will be tested on for the Data Management Fundamentals Exam.
I started by reading the book cover to cover but then realised that I did not have time to do this for all chapters to stay true to my deadline. The chapters are weighted differently for the exam. Therefore, I started prioritizing the chapters with the highest representation in the exam. I skimmed the rest of the chapters, focusing on headlines, sections, figures and definitions.
Focus on the chapters with the highest weight
Tip #3
No
Chapter
Percentage
1
Data Management Process
2 %
2
Data Ethics
2 %
3
Data Governance
11 %
4
Data Architecture
6 %
5
Data Modelling and Design
11 %
6
Data Storage and Operations
6 %
7
Data Security
6 %
8
Data Integration and Interoperability
6 %
9
Document and Content Management
6 %
10
Master and Reference Data Management
10 %
11
Data Warehousing and Business Intelligence
10 %
12
Metadata Management
11 %
13
Data Quality
11 %
14
Big Data
2 %
4. Do the practice exam over and over (and over) again
When you purchase the exam you also get a practice exam.
The practice exam is in the same visual format as the actual exam. You get 40 questions and 30 minutes to answer them. This is the same relative time you will have to answer all the 100 questions on the actual exam. Therefore, you also get to practice answering the questions in the given time.
Personally, I find it demotivating to just read an entire book without a “task” at hand. Therefore, I used the practice exam questions to help me work with the different chapters and sections.
Whenever there was a question I answered incorrectly, I looked it up in the book and made sure I understood the topic for the relevant question. This was a great way to consume the book.
Use the practice exam actively to look up relevant topics in the book
Must admit setting up this blog was not the easiest thing to do. Not because I found it technically difficult, but because I had a hard time convincing myself that my thoughts on the big domain of data could be valuable to someone else.
For the last two years, I have been fortunate enough to join the SQLcommunity speaking at conferences in Europe and joining webinars. Still, I only started working with data three years ago. I have a Master of Science within Marin Technology – so it was not necessarily in the cards for me to switch it all up and change to the world of data!
Having a different educational background and being fairly new to the community comes with the price of a huge imposter syndrome for me. Always. When I am having the best time at the conferences with this amazing community that welcomed me with open arms, I always look around and think “Wow, these people are legends. Experts. MVPs! Why am I here?”.
So, to conquer those thoughts I have decided to make a blog. Because, maybe, some of my thoughts, opinions and learnings will be interesting or even valuable for some of you.
Why DataAscend?
I spend most of my spare time exploring the outdoors and I am currently collecting the highest peaks above 2000 mamsl in Norway that requires climbing. I wanted to relate this blog and the name of the blog to my personality. Also, I think there is a nice analogy with the exponential growth of the mountain of data as well as the development in technology, where we need to work toward continuous learning to keep up with this domain.
When you ascend a mountain you are rewarded with this incredible view and sense of achievement. I think it is the same with learning new things in tech. We need to climb!
Ascend means go up or climb.
Google
What can you expect?
Being new to the domain, I find myself needing to relate everything technical to business value and the “real-life world”. Therefore, I hope my take on technology can be valuable to non-technical people, and people on the business side, as well as for more hands-on technical resources.
So, I will share. My thoughts. Learnings. Opinions. Best practices. Data. Power BI. Data Governance. Girl Power. Hopefully, I can inspire some of you.
So join me as we climb this ever-growing mountain of data!
Marthe Moengen 