Streaming Real Time Data Analytics in Power BI
Senior Consultant - Data Solutions
August 4th, 2017
Are you faced with an issue of needing real-time analytics to monitor your machine learning or IoT? Maybe you need to monitor your call-centre activity or stock market trading?
Maybe you’re still to fully embrace the move to the cloud? A common misconception is an Azure Environment is required to access real-time analytics. Recently, I faced this issue with a client, in this blog I’ll outline possible approaches you can take.
Power BI can address this commonly faced issue in three diverse ways. Obviously, a Full Cloud Implementation is preferential.
Pushing our data from on-premise to the cloud not only makes use of Microsoft’s secure environment but allows for real-time transactions, pushed from your Azure DB, making use of the Event Hub it easily converts those blob files before utilising stream analytics to turn your raw data into the decision-making data you desire.
Okay, I accept there are costs attached to cloud storage, maybe your data must be stored on premise, but if these aren’t a concern, this approach has a rapid development time, excellent scalability and is the desired method to stream.
Don’t want to store your data in the cloud? Have you considered the Custom Event Hub and Stream Analytics approach?
By-passing the Azure DB isn’t possible for all, but if your data storage product allows, you can apply logic to allow the Azure based Event Hub to convert our data before streaming as we did in our full cloud implementation.
By-passing Azure DB has its benefits, we reduce storage costs, we reduce that time-lag overhead from the push to cloud. So why wouldn’t you take this approach; custom logic to push to the hub can come at a cost and is likely to have scalability impacts.
So, what if you’re not cloud ready?
Why not by-pass Azure DB, Event Hub and Stream Analytics all together, making use of the Power BI REST API?
The Power BI REST API allows the ability to push data to Power BI, from which you can apply visualisations to answer those real-time business critical questions. This approach requires the registration of the API within Azure AD to authenticate before pushing as well as data manipulation to JSON prior push to enable Power BI to consume. A bit of up-front work, but this avoids those cloud costs and can be your gateway to real-time analytics.