Articles

Stay informed with our latest Insights and News

Cloud Analytics with Birst

Once more a couple of young USG Professionals enthusiasts wrote an interesting blog about the Hands-on Cloud Analytics that we organized on September 26th.

As newbies in the BI world they did not know exactly what Cloud Analytics solutions are capable of doing. In this hands-on session they got to build a data model, perform some ETL and visualize the data by creating dashboards for meaningful insights. All within a few hours and in the cloud! To read the full article click here.

Enriched customers through data strategy

On July 4th a couple of USG Professionals youngsters attended the Ins & Outs of a Modern Data Architecture in Brussels by Mike Ferguson.

He guided them through the complexities that relate to this matter in order to optimize all kinds of business operations. A bit overwhelmed by the course content and inspired by all the current options and possibilities, they wrote an interesting blog about their experiences and insights. Read the article here and share their entusiasm.

 

Van BI naar BA4ALL

Hoewel ik me momenteel voornamelijk bezig hou met werkzaamheden die vallen onder het kopje “Traditioneel” BI, ben ik van mening dat ik meer moet kennen van onderwerpen als Big Data, Analytics en Data Science. Onderwerpen die hard aan het groeien zijn en een echte hype kunnen worden genoemd.


Echter merk ik dat we vanuit de dagelijkse praktijk nog vrij weinig met deze onderwerpen te maken hebben. Terwijl we ingezet zijn bij een klant, zijn we meer bezig met de sleur van de dag, dan met de nieuwe technologie en onderwerpen. Vandaar dat ik tijdens het laatste BA4All event op zoek ging naar de aansluiting tussen BI en Big Data & Analytics in de praktijk.

Het Big Data & Analytics Insights 2016, op 7 Juni, had als titel: “Finding your Way in the World of Big Data and Analytics”. Het vinden van de weg in deze snel veranderende wereld heeft niet alleen betrekking op de consultants, maar geldt ook voor de bedrijven zelf. Vandaar dat er een diversiteit aan deelnemers deze dag te vinden was. Gedurende de dag presenteerde verschillende bedrijven klantcases en oplossingen. Ook werden er een aantal actieve brainstormsessies gehouden.

Zo is het zeer interessant om je eens af te vragen wat nu de grootste problemen zijn van een Big Data Project? Hierbij kan worden gedacht aan facetten als “Governance”, “Technology”, en “Mindset of the Business leadership” en “Capabilities”. Het slecht onderhouden of het niet borgen van bepaalde facetten zorgt dan voor een slechte adoptie van het Big Data project. Zo kan het in eerste instantie ook al lastig zijn om een Big Data case te beschrijven. Een voorbeeld kan zijn: Het verkleinen van het aantal online aankopen dat wordt teruggestuurd. Dit kan voor veel webshops zeer waardevol zijn, omdat dit een erg hoge kostenpost is. Dit werd bevestigd door één van de aanwezige webshops.

Zo beschrijft een onderzoek van Capgemini, dat Operationele Analytics zeer waardevol kan zijn. Eén van de redenen is dat het relatief simpel is, omdat het te maken heeft met bestaande bedrijfsprocessen, zoals Inventory Management en Capacity Planning. Zodoende is er hier veel waarde te halen voor een groot aantal bedrijven.

Als een klant een Analytics project wil uitvoeren, dient het hierbij gebruik te maken van een Data Science team. Maar aan welk profiel moet een consultant uit dit team voldoen? Welke skills heeft hij of zij nodig? Dit werd in één van de brainstormsessies uitgewerkt. In het plaatje hieronder worden de Data Scientist (DS), Business Analist (BA) en Big Data Engineer (BDE) ingedeeld in de matrix naar de verschillende expertises. Hoewel dit lijstje nog lang niet compleet is, blijkt het al dat je een schaap met 5 poten nodig hebt, om een project te doen. Is dat wel realistisch om te vragen van 1 persoon? Zo wordt vanuit Hortonworks ook al gesproken over een Data Science team.

In de toekomst zal “Traditionele” BI steeds meer verweven worden met Big Data, Data Science en Analytics. Hierbij wordt bij projecten gebruik gemaakt van data oplossingen die deels in de cloud en deels on-premise staan. Daarnaast worden Analytics componenten toegevoegd om nog meer waarde uit deze data te halen. Zoals het optimaal aanbevelen van producten bij webwinkels of het voorspellen van de files.

De verbondenheid tussen deze onderdelen komt mede doordat er veel raakvlakken zijn tussen BI & Big Data. Echter is er ook een verschil tussen deze twee, wat vaak een valkuil is voor de Big Data projecten. Zo kan de grote hoeveelheid Big Data ervoor zorgen dat je verzandt in de onoverzichtelijke bak, die niet geordend is, waardoor je verdrinkt.

Ongeacht Big Data of Business Intelligence, uiteindelijk is het zeer belangrijk om ervoor te zorgen dat het project breed gedragen wordt door de business. Zonder commitment is de kans groot dat een project zal stranden. Het soort project, BI, Big Data of Analytics, is dan van ondergeschikt belang.


De auteur

Benito van Breugel 

Senior Business Intelligence & Analytics Consultant at Capgemini


 

Dit artikel is eerder gepubliceerd als een blog op de website van Capgemini Nederland.

Business Analytics Insight 2016, Brussels - Report

If this fall you had only one day to free yourself from the office routine, it would have been the October 18th for the Business Analytics Insight 2016.

After a warming-up table discussion on the future of the BI landscape for vendors and service providers, the conference was packed with new topics and a world class keynote speaker: Stephen Brobst, elected among the top 5 CTOs in the US and notorious TDWI fellow. His keynote on data warehousing in a virtual world explored the trade-offs between different architectures, cloud economics and best practices in cloud computing strategy.

But there were more reasons to mark this day. With topics like data virtualisation, use cases on Mobile BI and data as a service this conference challenged our audience’s ideas on analytics! The summary of the peer exchange on the dangers of shadow BI fortunately identified enough opportunities for organizations to turn this phenomenon to their benefit and moreover concrete actions were defined as well to overcome shadow BI issues.

The 18th of October was also the action scene for a hands-on session on Business Process Mining & Analytics. After a foundational presentation, it was time to dig in the My-Invenio solution, that definitely lived up to its Gartner col vendor label, and learn how to analyse the past, monitor the present and predict the future trends of organizational processes. Based on industry sample data sets the workshop team poured in process insights thick and fast.

All and all another succesfull edition of the Business Analytics Insight.

 Click here for a full description and presentation of their talks


Big Data & Analytics Insight 2016, Bunnik - Report

The 7th June the Big Data & Analytics Insight 2016 took place in Bunnik and there’s no doubt that it was one of our most interactive conferences so far. The ice breaking session made sure everyone was on his toes and alert.


The result of the brainstorm provided enough material for two peer exchanges. The first peer exchange was at the heart of the data science skills and capabilities discussion and the summary conclusions were firm. The data scientist job, as well as the job title, is a mash up of all kinds of existing expertise that is generally a few steps removed from the everyday realities of their client, being the businesses that want to employ them. The conclusion of the second peer exchange showed that is perfectly possible to lay a foundation stone for a high level Big Data business case in no more than thirty minutes. Talking about group dynamics!

The opening session made the case for deriving value from data but the agenda focused mainly on demystifying the integration challenges that most organisations face. That provided enough cause for some technical sessions on practical jumpstart for Hadoop in the cloud and how Data Vault and Big Data can mix into a success formula.

Our audience was kept on the edge of their seat during the Hands-on Predictive Text Analytics discovering the possibilities of linguistic computing. The live text analytics poll demonstrated convincingly the power of the text mining algorithm.

The conference closed with an animated buzz at the networking reception; it was clear that this day provided sufficient inspiration for the audience.

Click here for a full description and presentation of the talks.

Big Data & Analytics Insight 2016, Brussel - Report

On May 24th the Big Data & Analytics Insight 2016 took place in Brussels and there’s no doubt that it will be remembered as one of our most interactive conferences so far.

After a warming-up with a Hadoop pre-conference tutorial our audience was already prompted to put their brains at work during the ice breaking session. The outcome of this blitz brainstorm fueled the agenda of no less than two peer exchanges. The first one was at the heart of the skills and capabilities discussion and the summary conclusions were firm. The data scientist job, as well as the job title, is a mashup of all kinds of existing expertise that is generally a few steps removed from the everyday realities of the businesses that want to employ them. The summary of the second showed that is perfectly possible to lay a foundation stone for a high level Big Data business case in thirty minutes. Talking about group dynamism!

The conference sessions kicked off with two concrete use cases from Graydon and Orange focusing on the business promise of Big Data & Analytics. Where Graydon emphasized the importance of a good governance model to foster a culture of analytics, Orange showed how they could successfully monetize mobile data for city marketing purposes to name only one of the project benefits. 

The Big Data analytics adoption proceeds gradually but there is still a lot to learn especially about integration. Reason enough for us to also include some more technical oriented sessions zooming in on, among other things, a practical jumpstart for Hadoop in the cloud, the value of ELT over ETL and a demystification of Apache Spark, the currently fastest clustered computing engine. Equally important have become the privacy implications of data sharing and our audience learned to “keep calm but remain suspicious”.

The excellent conference atmosphere was continued during the lively networking reception that concluded this sunny afternoon in Brussels.

 Click here for a full description and presentation of their talks

Textgain Analysing Social Media to Expose Hate Messages

Twitter does not want its platform to be used to promote terrorism. But how can you check millions of tweets per day automatically?

 

Textgain, a new spin-off from the University of Antwerp (Belgium), has developed language technology that can automatically identify hate messages posted by IS sympathisers. Over the last few years, the University of Antwerp has built up significant expertise in the automatic analysis of massive amounts of text. In 2014, for example, our scientists screened all Twitter messages posted in Dutch for the name of famous politicians. This resulted in a `political barometer’ that indicated the particular sentiments being tweeted about a politician or political party. During a popular talent show on Belgian television, the researchers analysed an overwhelming number of tweets in real time, allowing them to predict the winners before the results were announced.

With that experience under their belts, researchers Guy De Pauw, Tom De Smedt and Professor Walter Daelemans recently established the Textgain spin-off. Language technology is central to this new project: “We want to use this spin-off to commercialise the technology developed within the CLiPS (Computational Linguistics and Psycholinguistics), research group”, says De Pauw. “This technology allows us to extract facts, opinions and demographic information automatically from social media data, newspaper articles, emails and so on in a wide range of languages. That type of information has invaluable applications in big data and e-marketing.”

“It is important for companies to know what is being said about them on social media”, explains De Smedt. “However, so much is posted and tweeted that it is impossible for them to screen this data by hand. This is where Textgain can help. It goes further than you might think: it seems obvious that the statement ‘I love it!’ is a positive one. But the technology can also conclude that this comment is more likely to have been written by a woman than by a man. Age and even personality traits can be identified. That type of information is very useful for marketers.”

Security agencies
It’s not only in marketing that Textgain has a role to play. The spin-off also puts its language technology to use in tracing hate messages on Twitter. De Smedt adds: “In February, Twitter announced that they do not want to see their platform being used to glorify terrorism. Around 125,000 accounts have already been closed, mostly those linked to IS and its sympathisers.”

But the fight against hate messages is a hard one. Textgain has now developed software able to detect hate speech and related combinations of words automatically. “In addition, our software continuously adapts itself to the evolving rhetoric. It goes without saying that this technology must be used cautiously, but in the long run we do see opportunities for collaboration with other parties, such as security and intelligence agencies."

For more information contact: Guy De Pauw (guy@textgain.com) or Tom De Smedt (tom@textgain.com)



Analytics from a HR Perspective, Brussels - Report

On December 1st we concluded our ba4all.be working year with a Business Perspective with a focus on HR Analytics. With 2 end user companies on stage, 3 real life cases and an exciting team exercise this promised to become an instructive afternoon.

On December 1st we concluded our ba4all.be working year with a Business Perspective with a focus on HR Analytics. 

SD Worx and KU Leuven represented the end user community with very practical use cases, best practices and recommendations when applying analytics. Both pragmatic but driven by evidence based Human Resources using scientific models. And while both speakers emphasized the large effort that goes to data-capturing and preparation, a common conclusion was that simplicity rules in front of the end user. KU Leuven concluded before the break with a riveting story about process mining, data mining, and text mining, all coming together.

The peer exchange, with its seditious title, prompted the audience to put their brains at work and reflect on the struggle between business and IT. For virtually all teams it was clear that crucial success criteria going forward will be agility and the willingness to experiment. Next to that all groups endorsed an increased level of integration between the two, but supported by a proper level of management buy-in. Some teams, interestingly enough, promoted the need for communities with common goals and with an equal engagement from business and IT and preferably often meeting physically. It reminded us of … right … ba4all. All teams rewarded the necessity of good communication skills, for sure, but it is clear that business and IT should constantly outbrave each other’s creativity.

SAP represented the vendor community and offered us 3 crisp clear take aways. Digitize your Human Workforce for a higher employee engagement, how to predict your employees in 4 ‘clicks’ and dare to make decisions based on data. SAP was firm and stated that organizations that embrace Human Capital Analytics outperform those that don’t. The following live demo, based on a real life use case, showed the ease of applying predictive analytics on employee data with the proper tools.

In the lap of the starting winter, the lively networking reception concluded our ba4all 2015 program. See you all next year!

 Click here for a full description and presentation of their talks 

Analytics Strategy From a Business Perspective, Utrecht - Report

On October 1st we had our second Business Perspective of 2015 in Utrecht! Graydon represented the end-user community with a session that highlighted the do’s and don’ts for fostering a culture in analytics since data is a key component in their value proposition.


For both the customer as well as the buyer journey they are able to make information actionable. From alerts to employees to a direct communication with their customers. With a data driven marketing model based on strategy maps, Graydon is able to respond quickly to required changes and their priorities. They could not emphasize more the importance of good governance when summarizing their Integrity & Governance Reference Model. Identify garbage as fast as you can!

When it came down to selecting the right technology for Analytics, DAMA International was firm. We have too much technology! Knowing where you are and where you want to go are basic questions that need answering first when you want to jumpstart any analytics initiative. The emphasis was on very practical examples from a checklist to a clear statement of work.

The peer exchange, with its seditious title, prompted the audience to put their brains at work and reflect on the struggle between business and IT. For practically all teams it was clear that business departments need to see the value of their data growing, that strengths could best be centralized and that info sharing between business and IT will remain crucial going forward. They should constantly outbrave each other’s creativity.

Business & Decision and Capgemini represented the consultancy community with a refresher about self-service BI and data science being key but only successful when done as a team. It quickly boiled down to their observations that organizations have so much data that the business demands developed much faster than the BI delivery organizations can unlock data. Business & Decision comforted the audience in the room that organizations in many cases do not need a complete BI technology stack to integrate data from different sources. Capgemini illustrated four out of seven of their insights and data principles stating that, at present, mainly the digital transformation compels for IT and business to work together. Data science is the interplay of data, business processes, technology and statistics but it all comes down to balancing between profit, planet and people. An inspiring and mildly philosophical interlude.

A lively networking reception, of course also intentionally bridging the gap between business and IT, concluded the sunny afternoon.

SAP Forum: Business Analytics for All’s Technology Field Trip

Now and then, Business Analytics for All puts on its outdoors gear and attends events of the major players in analytics. Remember the Google event on Big Data Analytics in November 2014? This year, we will be present at the SAP Forum in Tour and Taxis the 9th September. And for next year, discussions are ongoing.

Business Analytics for All is one of SAP’s community partners. What this means is that Business Analytics for All will be exchanging ideas with the members of the SAP community. Also, Business Analytics for All contributes content with a round table (topic lunch) on a vendor neutral topic: the Big Data Maturity Assessment. Mr. Lathouwers, BI Director Europe at Nike will guide you through the evolutionary model and indicate roadmaps for CIO’s, program managers and architects.

Business Analytics for All ’s handpicked sessions on the SAP Forum

This is a virtual track of presentations that will appeal to a broader audience. Enjoy these sessions with one of our Advisory Board members so they can get your feedback for future initiatives.

  
08.15Welcome (Breakfast)
09.15Introduction: The Future of the Digital Economy and SAP’s role
09.35Opening keynote: Adapt or Die
10.30Keynote: The Future of Finance
11.00Case study: VELUX – Opening the window of opportunity to improve results & performance
11.30Coffee Break 
12.00Explore what the latest BI innovations mean to your business
12.30Case study: Process Mining – HANA Platform and Startup at Celonis
13.00Business Analytics for All Topic Lunch: Big Data Maturity Assessment
14.00Evangelism & the Future of Digital Marketing
14.30Plan Simpler with New Generation Planning & Analysis in the Cloud
15.00Hybris @ HUBO
15.30Coffee Break 
16.00Case study: KU Leuven, Sharing experiences of introducing HANA in your organization
16.30Case study: STIB-MIVB Sales Cockpit
17.00Closing
17.30Networking Cocktail till 20:00

 
Business Analytics for All Topic Lunch: Big Data Maturity Assessment

Big Data is this decade’s buzzword. Many organisations are looking into it, but few are embedding Big Data in an overall business and IT strategy. In this round table, a practical approach from a seasoned BI professional will guide you through the Big Data Maturity Assessment: what are the options? What roadmaps are of value and first and foremost: how do you position your organisation on this evolutionary scale? CIOs, program managers and information architects will benefit from this session.

Mr. Peter Lathouwers, BI Director Nike, member of Business Analytics for All’s Advisory Board

Business Analytics Insight 2015, Brussels - Report

The Business Analytics Insight 2015 started off with a slight delay due to Belgian traffic issues again but had lots of good stuff for every attendant. With fourteen presentations on various topics, three actual customer use cases and two interactive workshops, our audience was presented lots of insights or at least food for the mind.

Key-note speaker Mike Ferguson presented two sessions with a particularly padded agenda. He took our audience along to the omni-channel front office and soon arrived at the major areas of importance being data management, an analytical architecture and advanced analytics. Similar to what we learned during a Business Perspective in The Netherlands the week before, it seems that we are producing more information than we can digest. Hence Mike’s valid question “How good is your filter?” As the business becomes more and more autonomous (BYOD), governance will become the worry of the mind the coming years. When Mike explained the dos and don’ts of a data reservoir, he firmly stated that business alignment of information being produced will be critical to success. Finally Mike summed it up with no less than 9 applicable conclusions.   

The peer exchange prompted the audience to put their brains at work again and to reflect on their readiness towards Cloud Analytics. When summarizing the outcome of this brainstorming, it became clear that the main advantages such as availability, centralization and a cost perspective where also listed as … disadvantages. Flip a coin!

The parallel sessions and workshops touched all kinds of topics in the peripheral of Analytics. Most of the speakers did their utmost best to bring practical content from their daily experience and the evaluations showed that the audience highly appreciated that.

During the afternoon key note, Mobile Vikings reported in detail about their City of Things project being the largest living lab ever offered to the technology business. No less than two hundred thousand citizens were monitored by means of thirty four thousand smart devices. The learnings of this exercise were unrivalled.

As usual a lively networking reception concluded the chilly day as autumn clearly arrived that day as well.

 Click here for a full description and presentation of their talks

Big Data & Analytics Insight 2015, Bussum - Report

This Insight Conference had something for every attendant and all of it for organisations rethinking their analytics strategy in the light of Big Data

 
There were talks on tech topics like:

  • The Evolution of Hadoop in Organisations from Mark Vervuurt,
  • Self Service BI Big Data Applications Using Apache Drill by MapR’s Martijn Kieboom and Mats Uddenfeldt,
  • Data Science with R by Bas Minkes

Cloudera’s talk on the Enterprise Data Hub and Cisco’s session on virtualisation completed this tech track.  Customer cases and practical aspects of Big Data ware covered in these sessions:

  • Big Data in Consumer Facing Industries, by Steven Noels from NG Data
  • Business Analysis for Big Data, By Bert Brijs from Lingua Franca

The session “Sentiment and Impact Analysis Use Case By Comparing the Hadoop Distribution vendors’ buzz” gave a practical insight in how far unstructured data can be analysed today. Big Data’s impact on the overall information architecture was also well represented by Bob Becker’s sessions on the Enterprise Data Warehouse and Birst’s Cloud Analytics session. Finally the human and organisational aspects were also covered:

  • Martin Haagoort’s talk on the data scientist’s profile,
  • HP’s Ewout van Opstzal’s Driving Business Innovation with Big Data,
  • Big Data Maturity: the Photo and the Movie, by Jorg Heizenberg
  • Beyond Shipments and Share by Bob Becker

The attendants were also put to work in the peer exchange “Guiding Principles for Realizing Big Data Value” and came up with some interesting conclusions.

All in all, this day was packed with inspiration and ideal primer for the next Insight Conference which is due the first of October: “Developing an Analytics Strategy from a Business Perspective”. We’ll see you there!

Arhs Cerebro Real-Time Engine

Arhs has proven for more than 10 years now to deliver “classic” BI solutions. Arηs introduced BI 3.0 in 2013, which demonstrated a paradigm shift in the following areas : Mobile BI, Cloud Computing, Social Media Analytics, Self-service BI and Big Data. Our goal is to improve society.

Business Objective

Demonstrate our capacity to create a Lambda Architecture, where the best of two worlds can be combined. This is the batch mode processing with Hadoop together with the real-time layer engine of Apache Storm. These will be used to handle massive amounts of data and to produce new insights to our customers. The project was initiated to analyze via the different social media the sentiment of the European jobseekers, and the impact of their regional mobility due to the different conferences and activities on the European Job Days. With the large potential discovered during this initial phase, the project was enlarged globally to diverse actionable and searchable content.

Context

According to a recent Gartner survey, 64% of IT companies are already investing in big data or have it in their plans over the next 12 to 24 months.

Doing Big Data can mean several things and the technologies or platforms to use, depend on the type of Big Data challenges you are trying to face. Basically, Big Data can be broken up into 3 dimensions all tackling different problems, to create value out of the mass of information available:

  • Volume --> Scaling of data
  • Variety --> Different forms of data
  • Velocity --> Analysis of real-time data

To demonstrate the capacity of the velocity dimension, it has been decided to realize a tool for real-time analysis of data. Classic Big Data examples can be found along the web, so the project was initiated not to do yet another map-reduce application counting hash tags in tweets. 

Set up

The goal of the project is to visualize the feeling of social media regarding a particular topic, which is being achieved via sentiment analysis. Twitter communicated that mid-2014, on average, around 20.000 public tweets per second are posted, making it a good reference for the input data stream.

The main characteristics of the project are:

  • Read tweets from Twitter;
  • Filter tweets;
  • Calculate sentiment of tweets;
  • Store raw tweets & sentiment;
  • Visualize sentiment on web;
  • Make the system scalable;
  • Make the system fault-tolerant.

Evaluation of the traditional approach

Instead of jumping right into the buzzed Big Data technologies, the reflection was made on how to solve this using proven technologies and our current experiences. A classic solution would be based on queues and workers. Our strong background in java would make it possible implementing this using JEE, meaning having a set of Message Driven Beans wired together with JMS. However this solutions has one major drawback, it doesn’t scale well horizontally, which was one of the objectives. 

Use case: A system with queues and workers is configured to evenly balance the workload over different workers. No problems so far.

Imagine scaling to create more throughput. Deployment of a new worker is creating an additional (logical) queue. This implies to reconfigure and redeploy the first set of workers in order to enable the system to rebalance the workload over the new system. These cascading dependencies make it a tedious task to scale these types of systems.

Another drawback of a system with queues is that it makes your system slow. The queues are a a way of making the system fault-tolerant. This is achieved by taking a message of the queue only if the worker has acknowledged that it successfully processed the message, so when a worker fails the message isn’t lost. The use of queues however, requests that every time you push a message, the system will need to persist that message on the disk, serializing it and deserializing it when your worker wants to consume the message, making your system slow.

Additionally coding this type of solutions will require a lot of code for routing and message serialization instead of coding the actual business logic.

Sentiment Analysis with Storm and Kafka

These drawbacks make it a good opportunity to start with recent technologies such as Storm and Kafka and build a horizontally scalable, fault tolerant, real-time processing engine with guaranteed data processing.

Twitter Streaming API

To get input data into our system, it has been decided to source from Twitter, which has a Firehose API, a streaming service allowing to listen to all public tweeting in (near) real-time. Because only a handful of Certified Product Partners have access to the Twitter Firehose API, our approach was to use Twitter’s Streaming API. The Streaming API allows you to listen to the public tweets, just like the Firehose API, but it comes with limitations: Twitter only returns a small percentage of all tweets matching your search query in real-time.

Kafka instead of Queues

Kafka is a persistent, distributed, replicated pub/sub messaging system originally developed at LinkedIn and designed to overcome some of the problems caused by the traditional Queues – Worker pattern. Kafka has the following three design principles:

  • Very simple API for both producers and consumers;
  • Low overhead in network transfer as well as on-disk storage;
  • A scaled out architecture from the beginning.

Storm instead of workers and intermediate message brokers

Apache Storm is a free and open source distributed real-time computation system. Storm will replace the workers and intermediate brokers from our original setup. It has three types of components:

  • A spout is a source of streams in a computation, in our case the Kafka Spout;
  • A bolt processes any number of input streams and produces any number of new output streams. The business logic of the application can be found in the bolts;
  • A topology is a graph of spouts and bolts, where each edge of the graph represents a bolt subscribing to the output of some other spout or bolt.

The project

Let’s go over the different steps for the set up of the application

  • Use the Twitter Streaming API to get all the tweets, present in a set of keywords, and push it to a Kafka Topic.
  • Now that the messages are arriving in the Kafka topic we need to create a Spout to subscribe to the Kafka Topic to get the related tweets for processing. This is where Storm comes into the picture.

  • Use Storm to do the real-time processing of the messages
  • Setup the topology with the necessary groupings.
 
Before going into the topologies itself, let’s first address the bolts. The twitter sentiment analyses grouped the different actions that had to be done in a set of bolts. Wiring the bolts and spout together is done by a topology. In the topology you define how spouts and bolts are logically connected and you hint on the parallelism.

 

 

 

Conclusion

In the exciting world of big data today, Arηs has been able to prove its capability to work successfully on new stacks of information and tools, providing new knowledge and insights to the business, thus improving their capabilities for decision making.

Contact Info

Patrick Adler
BI Manager Arηs Developments Group
e-mail: Patrick.Adler@arhs-developments.com

 

Big Data & Analytics Insight 2015, Brussels - Report

With an almost full house the Big Data & Analytics Insight set the record! In short, the conference had lots of good stuff for every attendant. With fifteen presentations on various topics, one can’t guarantee that every presentation has hit the spot but the opening speech from dr. Carsten Bange surely made a splash in our audience. It was dr. Bange as usual: to the point, and very clear.

In his first presentation, dr. Bange illustrated that the marginal cost of accessing and transferring information has dropped to “zero” with the consequence that we generate and use an exponentially growing amount of data and that we have transitioned from Hardware to Software … to Data. He pointed out unmistakably that new data sources will enable new explorative analytics. Dr. Bange complemented this session later on with BARC’s, sometimes surprising, research results on the current state of Big Data in enterprises.

Timo Elliott and Laurent Fayet brought two interesting user cases at Kaeser and Euroclear and they clarified the business value that Big Data Analytics can bring whether it is from corrective, preventive and predictive maintenance to risk mitigation in a resilient capital market infrastructure.

The peer exchange prompted the audience to put their brains at work again and reflect on the Data Warehouse and its role in Big Data. When summarizing the outcome of this brainstorming, it became clear that besides cost, of course, the main drivers for Big Data are increased flexibility and the growing need for data platforms that enable experimental analysis.

The parallel sessions and workshops touched all kinds of topics in the peripheral of Big Data. Most of the speakers did their utmost best to bring practical content from their daily experience but it is clear that Big Data and moreover Big Data Analytics have only reached puberty yet. There is still a steep learning curve on the path to value creation as Laurent Fayet stated rightfully when explaining his Big Data Analytics Maturity Model earlier.

All and all there was a lot of interaction during almost all presentations and workshops and most certainly during the lively networking reception that concluded the busy day.

Click here for a full description and presentation of their talks

The BI Survey 15 ... Wanted: Your Opinion!

Take part in the world's largest survey of business intelligence users. In close collaboration with BARC, we would like to invite you to participate in The BI Survey 15, the world's largest annual survey of business intelligence (BI) users.


BARC's annual survey gathers input from thousands of organizations to analyze their buying decisions, implementation cycles and the benefits they achieve from using BI software.

As a participant, you will:

  • Receive a summary of the results from the survey when it is published
  • Be entered into a draw to win one of ten 35 EUR Amazon vouchers
  • Ensure that your experiences are included in the final analyses

Click here to take part

Business and technical users, as well as vendors and consultants, are all welcome to participate. You will be able to answer questions on your usage of a BI product from any vendor and your experience with your service provider.

The BI Survey 15 is strictly vendor-independent: It is not sponsored by any vendor and the results are analyzed and published independently. Your answers will be used anonymously and your personal details will not be passed on to software vendors or other third parties. The BI Survey 15 should take about 20 minutes to complete. For further information, please contact Adrian Wyszogrodzki at BARC (awyszogrodzki@barc.de). 

Dr. Carsten Bange, founder and managing director of BARC, will present the results from the survey - at the Business Analytics Insight 2015 conference on 5 November - to get an insight what really drives projects and how BI products are used. The large international sample of the BI Survey also allows for a benchmarking of Swedish BI users with their peers around the globe.

Thank you in advance for taking part.

Big & Fast Data: The Rise of Insight-Driven Business

In the Netherlands, the big data agenda appears to be driven by both IT and business issues. The country is increasingly a preferred destination for European businesses seeking to establish a big data hub: A combination of legal protection, infrastructure and technical skills makes the Netherlands a prime contender here.


However, the picture is different when we look at companies based in the Netherlands, and our survey results paint an overall picture of relatively slow adoption. The reality varies according to industry, in our experience.

There are key innovators in heavy manufacturing and consumer products/ retail who are accelerating quickly, and embracing bold adoption strategies to explore new use cases. On the other hand, other sectors are being more cautious and conservative.

Where adoption is slow, there is cause for concern, because the barriers to competition are low in the Netherlands, as they are in many European countries. Businesses need to review their strategies at pace.

Perception of big data as a disruptor

There should be concern in the boardroom in the Netherlands, as the perception of big data is significantly different from that in countries that are leading the field. Respondents were less likely than average to report big data related disruption from start-ups, new competitors moving into their industry from other industries (10% vs 20%), or existing competitors launching new products (8% vs global 24%) or services (18% vs global 33%). Nor did the majority of Dutch respondents anticipate these forms of disruption over the next few years.

Awareness of big data opportunities

Like other European respondents, those from the Netherlands were less likely to agree strongly with statements about the importance of big data to organizations than those in the BRIC countries (Brazil and China) or the US. Of interest, they didn't generally see big data as likely to enable new revenue streams, or as a revenue driver in its own right, for example. However, many respondents agreed that decision-makers increasingly require data in real-time; it may be that businesses are unwilling to self-disrupt, and prefer to use existing business information but access it more quickly.

Implementation approach Around

56% of respondents said they had implemented or were in the process of implementing big data technology or would do so in the next 12 months – the lowest figure for any country or region in our survey apart from the Nordics. The average was 71%. Just 4% of respondents in the Netherlands said they had already implemented this technology – again the lowest of any country or region. The big data agenda is usually driven by IT, with the second driver, business strategy, a long way behind.

Netherlands organizations are more likely than most other countries to have put in place additional data security to protect customer data and (particularly) additional measures relating to data privacy – a finding that suggests some of their reticence in adopting big data may be due to security and privacy concerns.

As noted above, there are sectors that run counter to the trend of slow implementation, and are adopting big data much more rapidly. The remainder would be well advised to re-evaluate their market and strategy and see if there are threats or opportunities that they need to address.


Capgemini


With almost 145,000 people in over 40 countries, Capgemini is one of the world’s foremost providers of consulting, technology and outsourcing services. The Group reported 2014 global revenues of EUR 10.573 billion. Together with its clients, Capgemini creates and delivers business and technology solutions that fit their needs and drive the results they want. A deeply multicultural organization, Capgemini has developed its own way of working, the Collaborative Business ExperienceTM, and draws on Rightshore®, its worldwide delivery model. Learn more about us at www.capgemini.com/insights-data or contact us at insights@capgemini.com.

Text Analytics from a Marketing Perspective, Brussels - Report

On 31 March we had our first Business Perspective in Brussels! Belga News Agency and The Coca-Cola Company represented the end-user community explaining practical cases of Text Analytics in their organization.


The objective of Belga News Agency was to evolve from problem to solution and offer journalists all channel searches and in-depth content whereas The Coca-Cola Company aimed for a complete brand & reputation dashboard. The challenges to overcome varied from differing metrics to disparate sources which are issues that are not new to many organizations of course. Also filtering out British slang in view of accurate sentiment analysis was a hard nut to crack. We learned for instance that if a Brit says “Coke is sick”, it doesn’t quite mean what we would understand.

The peer exchange prompted the audience to put their brains at work and reflect on the value or hype of Text Analytics and how to build the business case for Sentiment Analysis. When we got to the summaries, each group’s spokesperson had 2 minutes of airtime to conclude. A good warming up for our ignite sessions later in the program.

Deloitte and SAP represented the consultancy community and they provided insights on how technology can support Text Analytics. Deloitte was firm and stated that clients want you to solve a problem, not be a smart ass. The demo that followed showed how to identify the customers that require your attention. SAP then explained their strategy to provide agile marketing insights with SAP Business Intelligence and they demoed a time and subject oriented analysis on social media indicators. Key for them was to do this in a self-serviced and visualized approach.

The value of Text Analytics in the new world of Big Data was endorsed multiple times during the lively networking reception that concluded the afternoon.

The presentations can be downloaded here:

Setting the Text Analytics Strategy at Belga News Agency
Marketing Meets Analytics to Optimise Customer Insight (*)
Customer Management Solutions enabled by High-Impact Analytics
Improving Customer Understanding through Sentiment Analysis Using Agile Visualizations

(*) For reasons of confidentiality, the presentation of Coca-Cola cannot be published nor distributed.

Taking a Deeper Dive in Text Analytics, Stockholm - Report

Thursday afternoon, the 19th March, in a nice corner room of the Sheraton Stockholm Hotel. About forty people absorbed the ins and outs of text analytics by Mårten Lindblad from Accando and Bert Brijs from Lingua Franca.


But before that, Martin Haagoort, showed them the way to keep what worked in the old days and elaborate on it in  the new era of Big Data.  And Per Englund went through a step by step approach on how to deal with the massive amount of patient data that is generated from wearables and predictive systems.

The peer discussion on building a business case for text analytics came up with at least six relevant and interesting ideas which can be explored further.

All in all the attendants were satisfied, giving the event an overall score of 3.9 on a five point scale.

Building the Business Case for Text Analytics, Stockholm - Report

Six teams had the choice between developing a use case or a benefit map for text analytics. Discussions went on for a while before the groups reached a consensus on what to tackle and how to present the results.


Maybe some were a bit hesitant because the deep dives on text analytics were planned after the peer discussion but after a few prodding and stimuli from the discussion leader, ideas popped up in all teams. This is what they came up with:
 

  • Check e-mails in call centres for urgent topics

  • Analyse customer reviews to respond faster to complaining customers

  • Check social media on how your organization is perceived

  • Prioritise service requests

  • Improve customer service center performance

And the winner, as far as ba4all is concerned, is… analyse TV commercials and their actors.  A well known retailer has a a series of commercials on Swedish television in the format of a soap. Via social media and text analytics they could analyse the impact of the actors that contribute to the image of the retailer and enhance the power of the message and those that don’t.

This exercise proves: innovation is about immersion in ideas and technologies and sharing. When everybody contributes, all participants win. That is the power of a Business Perspective from BA4All: dare to share and get back to the office with fresh ideas.

The Future of the Data Warehouse, Bussum - Report

Business Analytics for All went to the root cause of powerful analytics: Data. In his opening speech, Ronald Damhof made a compelling case to love your data and presented a logical framework to position the various ways of data exploitation.


Nico Klaassen closed the afternoon with some practical insights in master data management and what happened in between was also worth listening to: agile data warehousing by Dirk Remacle scored high in the members’ evaluation sheets and Eric Schaap reminded us that Data Quality is not just a technical issue. The members also got to work in group discussion son the future of the data warehouse in light of the emerging Big Data technologies.

  More Information

The Data Quality Game: how Experience Educates

This game was played in the morning before the start of the actual Insight Session, the 9th December.

Imagine you’re part of one of six or seven teams that have to construct roads, tunnels and bridges in a region connecting imaginary cities. Imagine you need to register carefully all data that can help you achieving your goal. Then you experience that your job is part of a process and where others rely on your contribution and you do the same with the input from other teams. That’s when it hits you: data quality is no trivial matter in any organisation and the cost of bad data can be prohibitive. This game is what the data quality doctor orders for any data quality project kick off.

Business Analytics Insight 2014, Stockholm - Report

Our first Insight Conference in Stockholm proved to be an intense working day, stuffed with content, knowledge exchange and the experience of sharing thoughts and insights with peers.


All this under the professional guidance of Magnus Penker who’s contagious enthusiasm lit up the audience.

Per Englund from Karolinska University Hospital set the records straight on Big Data and Mikael Ekström from eBuilder made a compelling case for effective self-service BI to improve customer care in a Cloud Business Process as a Service. Lingua Franca’s Bert Brijs sketched the foundation of data visualisation and led the speed geeking session on self-service BI.  The results of this speed geeking session are here on the website Speed Geeking Session on Self Service BI in Stockholm.

The audience was inspired by four blitz sessions in 45 minutes on to-the-point topics like innovation through information networks, predictive analytics and data exploration, the mobile phone as an improvement to shopping experience and Big Data, a new data warehouse fashion (question mark)… Daniel Adler, Lars Sveding, Andreas Broman and Magnus Björk delivered their insights in a rapid fire of bullet points.

Johan Petersson from Acando made a point when he illustrated how Agile Analytics drive new business. Finally, Margy Ross from Kimball University closed the day with her insights on project management for data warehouse and BI avoiding scope creep and yet remaining as flexible as possible within the constraints of time, budget and quality.

The overall appreciation of this day was a walloping 5.1 on a six point scale! So, don’t miss the next appointment with the latest insights in analytics!

Big Data and the Data Warehouse: Opponents or Close Friends?

It is the innovation hype all over again. History has shown that any innovation claims to make its predecessors obsolete: cinema would replace theatre, television would replace cinema and Internet would dwindle television to nothing. The same observations can be made in Business Analytics where some Big Data pundits are claiming the demise of the data warehouse due to this new data architecture. So when Business Analytics for All challenged the audience to answer the question: “The current technology evolution will make our Data Warehouse obsolete! Fact or fiction?” they showed their common sense.


The main arguments for keeping the data warehouse where the monitored quality and audit trail of the data, the need for structured and governed reporting, the integration with existing OLTP systems providing functionality appreciated by the users, keeping history intact, the compliancy with privacy regulation, etc…

The hardest part of this discussion was to find arguments for dumping the good old data warehouse and replacing it with the Hadoop stack. Time-to-market and cost reduction were the major arguments but enthusiasm was low in the groups looking for answers and arguments.

A highly surprising observation was that many contributors mooted that the Data Warehouse is for unstructured data and that Big Data will cover the needs for unstructured data. Narrowing down Big Data to this key difference is probably doing dishonor to the concept but it indicates that many of us are still in the learning curve. More about this in our program of 2015!

The second question for discussion provoked more output:

“How to organize for the future of BI/DW? What are the top 3 challenges?”

It was logical that the counterarguments and the challenges for a Big Data architecture would be the DWH pros raised to the -1 power. So let’s skip the quality, governance, privacy and other challenges derived from the previous question and focus on some interesting insights “a priori”. In other words, they will need to gather empirical evidence.

Here’s what the groups came up with:
 

  • Education: can we educate the users and decision makers fast enough to cope with this new approach?
  • Managing volatility: what part of the data is persistent? (I refer also to my article on volatility here: http://bit.ly/15kgg0U )
  • Reduced time to evaluate the information value
  • Missing semantics: bringing facts and context together will prove a lot harder than in the DWH
  • Establishing the business case: how to evaluate something that hasn’t materialised and has very few documented proxies?
  • Integrate with the data warehouse

All in all, the participants were satisfied with the process and the results of this discussion. Let’s see what happens in practice.

Bert Brijs, Lingua Franca

Speed Geeking on Self Service BI in Stockholm

Business Analytics for All’s speed geeking is a way of developing your analytical skills and using your creativity in a solution oriented way. In Stockholm our first speed geek session ever was a success as the evaluation forms showed.


These were the three discussion items:

  • Why do you use Self Service BI (SSBI)?
  • What are the major problems encountered?
  • Will IT become obsolete ?

The major reason for using SSBI was time to market, flexibility and faster decision making. The major problems were governing all these users and their data as well as delivering usable information and user friendly interfaces. Training the users was also seen as an issue to take in to the equation. But the overall mood towards SSBI was very positive and of course, nobody in the room, neither business nor ICT professionals deemed ICT becoming obsolete because of SSBI. ICT would rather evolve into an enabler, taking care of data quality, the infrastructure and manage analytics as a strategic asset.

These are -in a random order- the answers to the first question: “Why do you use SSBI?”

 

Empower the staff, self-care, transparency and trust, time to market agility, quality, freedom, better service level, flexibility, innovation – unleash creativity, exploration & adapt to learning process were the shortest and most powerful answers.

But some groups formulated concepts like:

  • Distributed fact-based decision making
  • New analysis and problems detected
  • Better requirements for the data warehouse
  • The right information at the right moment to the right person
  • Bring in third party info to test one’s hypothesis
  • Ad hoc queries on limited data

As you can see, a lot of inspiration and business use cases for SSBI were put forward by the groups.

The second question (“What are the major problems encountered?”) showed a lot of consensus between the groups on the real issues as this table shows:

 

Finally the somewhat challenging question about IT’s presumed obsolence provoked clever and moderate comments like:

  • A new role for IT, that will never be obsolete
  • A new ecosystem to build
  • Infrastructure to maintain and optimise
  • More interest in the analytical side of data
  • New skills will evolve:
    • Data science
    • Integrators of quality data
    • Business logic from internal and external systems are crucial
    • Control of SSBI
    • Proactive IT function

There was just one group who come up with a disappointed footnote: “CIO as board member far away from the business perspective”. Something we hope will remain an exception in these days were IT is becoming a strategic asset.

Bert Brijs from Lingua Franca led this speed geeking session. If you are interested in his perspective on SSBI and the speed geeking session, read his blog via this link.

The Future of Customer Analytics, Bussum - Report

"The Future of Customer Analytics” scored high. With an overall score of 4.09 on a five point scale, this community event can be called a success. Intensive interaction, exchange of ideas in a creative and open atmosphere… What more can you expect from an analytics community?


Inspiring speakers? Sure! Frederik de Bleser on data visualisation, Ernesto Ongaro on noSQL and Niko Brouwer on roadblocks toward performance management scored exceptionally high and Gerard Struijf managed to invigorate his audience during the Customer Analytics workshop.

Customer Analytics is a critical discipline within the Business Analytics domain and evolves rapidly, boosted by new technologies and the Big Data phenomenon. ba4all will keep you posted when new developments materialise.

And last but not least: the presentations are available here.

Welke prijs betalen we voor slechte data? (Dutch)

De wereld draait door en alle zaken waar ik afhankelijk van ben draaien goed door. Mijn salaris wordt elke maand betaald, van de overheid komt er kinderbijslag en kinder opvang toeslag binnen. Mijn hypotheek en gas, water en elektra worden elke maand netjes van mijn rekening afgeschreven. Elke maand vindt dit proces weer plaats en eigenlijk is het best wel bijzonder dat dit foutloos gaat.


Hoewel, misschien is het wel helemaal niet zo bijzonder, maar juist super bijzonder! Data ligt ten grondslag aan deze processen en zorgt ervoor dat het allemaal blijft draaien, maar wat kost het als er ergens een kink in de kabel zit en deze processen niet meer goed gaan?

In vorige artikels heb ik het data kwaliteitsproces in fasen besproken. Echter er moet een aanleiding zijn om dit proces te gaan initiëren. Als dat in projectvorm dient te geschieden, dan heb je meestal een business case nodig en daar zit vaak de bottle neck, want wat levert een verbeterde data kwaliteit op? Hoeveel geld kost het een organisatie dat een adres van een klant onvolledig is? Hoeveel geld kost het een organisatie dat een e-mailadres invalide is? Hoeveel uur is een medewerker kwijt met het handmatig oplossen van problemen naar aanleiding van slechte data kwaliteit?

Wat zijn de kosten?

Het is lastig om aan dit soort zaken een prijskaartje te hangen. Bijvoorbeeld als een e-mailadres van een klant invalide is, kan je hierdoor een potentiële klant niet die mooie aanbieding sturen die helemaal past bij de levensfase van de klant. Echter dit is wellicht niet het enige probleem. Het kan ook tot oorzaak hebben dat je te veel foutieve e-mails naar bijvoorbeeld het domein @hotmail.com stuurt, je op de blacklist van Hotmail terecht komt. Hierdoor ben je als organisatie überhaupt niet meer in staat om e-mails naar alle Hotmail adressen te sturen. De kosten van een invalide e-mailadres zijn dan niet alleen de mogelijke opbrengsten van de mooie aanbieding, maar daarbij zouden ook de eventuele kosten van op een blacklist terecht komen moeten worden meegenomen.

Wat is de toegevoegde waarde?

Maar wat levert een goede data kwaliteit allemaal op? In bovenstaand voorbeeld heb je wel de eventuele conversie op een aanbieding. Het kan ook zijn dat je personeel zich met andere zaken kan bezig houden en niet schaarse tijd verknoeit aan allerlei neveneffecten van een slechte kwaliteit op te lossen. Maar het is niet alleen het bedrijfsproces wat last kan hebben van een slechte data kwaliteit, het heeft ook impact op rapportages of integrale klantbeelden die worden samengesteld. De beslissingen die als organisatie worden genomen op basis van rapportages kunnen gebaseerd worden op correcte dan wel volledige gegevens bij een correcte data kwaliteit.

Misschien nog wel het belangrijkste een goede data kwaliteit draagt bij aan een goed imago! Als organisatie wil je liever niet een overleden persoon aanschrijven en je wilt al helemaal niet met naam en toenaam in de wakkerste krant van Nederland staan.

Wat kan je eraan doen?

Als je bovenstaande leest, zou je bijna kunnen denken, waarom doen organisaties dan niets aan goede data kwaliteit! Zolang alles goed draait, waarom zou je dan iets doen?! Echter data en een goede kwaliteit zijn ook gewoon zaken die onderhouden moeten worden. Thuis voer je ook preventief onderhoud aan een CV-ketel uit, juist om te zorgen dat de ketel een langere levensduur heeft. Dit moet je ook doen met je data. Data kwaliteit verslechtert vaak ongemerkt, door veranderingen die niet goed worden doorgevoerd, hierdoor gaat het van kwaad naar erger zonder dat een organisatie zich dat (tijdig) realiseert.

Om inzicht te krijgen in de data kwaliteit van een organisatie kun een soort van data APK laten uitvoeren. Laat eens door een externe partij je gegevens doorlichten en tegen de verschillende business rules aanhouden en objectief bepalen hoe goed het gesteld is met de gegevens binnen je organisatie. Naar aanleiding van de resultaten kan je als organisatie dan een compleet data kwaliteitsproces inrichten of natuurlijk deze periode controle uitvoeren. Capgemini biedt sinds kort deze dienst ook aan, Data Quality as a Service.

Het niets doen door organisaties wordt veroorzaakt door iets anders, data kwaliteit is niet sexy. Een organisatie beseft niet goed genoeg wat de gevolgen zijn van een slechte data kwaliteit en wat de opbrengsten kunnen zijn van goede data. Deze trend moeten we gaan keren, ik voorspel een nieuwe trend: elke onderneming checkt regelmatig zijn data!


De auteur

Elja Knol 

Senior Consultant Data Mangement at Capgemini


 


Op de rommelzolder van Big Data (Dutch)

“Sometimes in order to clean up, it is necessary to make a mess.” Ik weet niet of je wel eens een huis hebt leeggeruimd. Of gewoon een kamer of zolder hebt opgeruimd. Je komt allerlei spullen tegen waarover je een beslissing moet nemen: bewaren, verkopen, weggeven, weggooien.


Bij het opruimen moet je iedere keer beslissen hoe veel waarde het ding heeft wat je wilt opruimen. Is het waard om op Marktplaats te zetten, moet ik het bewaren voor later gebruik of is het gewoon troep dat in de afvalbak thuishoort?

Misschien heb je wel eens televisie­programma’s gezien waarin vondsten van zolder plotseling veel waard blijken te zijn. Het lijkt wel of iedereen waardevolle spullen heeft, maar dat is natuurlijk niet zo. Meestal komt er een externe expert aan te pas om de juiste waarde te schatten. Heel soms valt het mee, maar meestal valt het tegen.

Maar uit eigen ervaring weet ik dat ik opruimen niet leuk vind. De meeste anderen doen het ook met tegenzin. Want waarom zou ik tijd besteden aan iets waar ik het nut niet van inzie?

Opruimen

Thuis ruimen de meeste mensen pas spullen op wanneer het moet. Er moet verhuisd worden, je gaat een volgende fase in je leven in. Maar in de meeste gevallen ga je pas opruimen wanneer de kast te vol raakt en er geen nieuwe spullen meer in kan stoppen. Maar je kan ook naar IKEA gaan om nieuwe kasten te kopen. Ook een oplossing. Of een extra schuurtje plaatsen. Of een ruimte huren bij Shurgard, City Box of hoe die firma’s ook heten.

In de wereld van Big Data is het hetzelfde. Er wordt heel wat data verzameld en opgeslagen. En omdat opslag zo goedkoop schijnt, raakt de kast nooit vol. En zonder een volle kast is er geen reden tot opruimen. Hoeveel organisaties ken je die bij een data-opslagprobleem maar gewoon extra schijfruimte kopen in plaats van op te ruimen?

Het nadeel van steeds maar meer opslaan is, zoals je kunt raden, dat je op den duur niet meer weet wat je eigenlijk allemaal in huis hebt. Je kan de gegevens die je zoekt niet meer vinden, je weet niet meer wat het betekent en eigenlijk is het een zooitje aan het worden. Maar niemand komt op de rommelzolder van Big Data, dus wat geeft het.

Wat van waarde is…

Om goed te kunnen opruimen moet je weten wat de waarde van de spullen is. En in tegenstelling tot de spullen op zolder, is er meestal geen externe expert die de waarde voor je kan taxeren. Maar is dat een reden om het maar niet te doen?

Wanneer je niet weet wat de waarde van de gegevens is die je het opgeslagen, is er een uitdaging. Want aan het opslaan en beheren van gegevens zijn toch kosten verbonden en die kunnen toch wel oplopen. Waarom iedere jaar veel geld uitgeven aan de opslag van gegevens die niets waard zijn? Niets waard zijn in de zin van dat ze nooit gebruikt worden of eigenlijk geen “business value” hebben. Waarom bijvoorbeeld klantgegevens bewaren van klanten die nooit meer iets bij je kopen? Met de kans dat de gegevens verouderd en niet meer bruikbaar zijn? 

Het bepalen van de waarde van gegevens kan lastig zijn, maar brengt wel wat op. Het zoeken naar ROT*-te gegevens, deze verwijderen of goedkoper opslaan, kan veel geld besparen. Klinkt theoretisch goed, maar kan het ook praktisch?

Moderne ECM en andere Big Data-applicaties maken het mogelijk de waarde van gegevens te bepalen. Dan kan bijvoorbeeld heel simpel op basis van het gebruik van gegevens: data die nooit wordt benaderd en dus eigenlijk onnodig is, kan weg worden gegooid. Maar ook geavanceerd: door gebruik te maken van content- en data-analyse kan, aan de hand van de betekenis van de gegevens, de waarde worden bepaald.

Zoals je vroeger eigenlijk nooit een geldige reden had je kamer niet op te ruimen, heb je nu geen reden meer om je gegevens niet op te ruimen. De gereedschappen om snel en eenvoudig de ergste rommel op te ruimen zijn er. Je bespaart een hoop geld aan de opslag, het beheren en het zoeken door onnutte gegevens. Opruimen zorgt ervoor dat er weer ruimte ontstaat om nieuwe dingen te doen.

*)  ROT = “Redundant, Obsolete & Trivial”


De auteur

Reinoud Kaasschieter 

Expert in Business Information Management at Capgemini 


 

Advantages of Implementing a Data Warehouse During an ERP Upgrade

Upgrading an ERP system represents a number of challenges to many organizations. However, many of these challenges can be alleviated by integrating a Business Intelligence (BI) solution during the upgrade process. This is especially true if the Business Intelligence solution is based on a data warehouse.


The key activities involved in upgrading an ERP system are data and report conversion and creation of new reports. If the upgrade process is viewed as an opportunity to resolve data quality issues, it will be beneficial to include a data warehouse and a BI solution as part of the upgrade. This document discusses the benefits of incorporating a data warehouse when an organization’s ERP system is upgraded.

Data Conversion

When a company has to determine the scope of data conversion from the legacy ERP system to the ERP new system, several issues have to be considered:

  • The conversion of master data
  • The conversion of transactional data
  • The amount of data in the new system from a performance perspective

Because the data model in the new ERP version may be different from the old data model, it is often an advantage only to convert relevant master data to the new system.

Transaction Data

When an ERP upgrade includes a BI solution and a data warehouse, the legacy transaction data are stored in the data warehouse. From this point, the data warehouse will be able to support the organization’s reporting requirements. If the conversion has been performed properly, it will be possible to close down the old ERP system as soon as the new ERP system is up and running because all relevant information is accessible from the data warehouse. Not having to maintain the old system represents considerable cost savings.

When estimating the costs of conversion, it is advisable to view master data and transaction data as separate entities. This way, it is possible to determine the exact costs savings of establishing a data warehouse during the upgrade process. Often, the costs of converting transaction data from one ERP version to another may actually cover the entire costs of establishing the data warehouse.

Furthermore, when incorporating a data warehouse, it is possible to create an additional database on the data warehouse server. This database can then be used for storing an exact copy of the entire data moel as well as actual data from the old system. As a result, the structure of the entire BI solution does not have to be in place right away because the extra database can later be used as a data source in the finished BI solution.

Data Quality

The trustworthiness of the reports that will be created on the new system relies on the quality of the data after conversion. Therefore, evaluation of the master data quality before conversion is of the essence. When the master data cleansing is completed before the conversion, it ensures a smoother, faster, and less costly conversion process.

Report Conversion

When the conversion process has been carried out by establishing a data warehouse and a number of multidimensional cubes, the task of converting reports from the legacy system still remains.

Converting Reports

Typically, when companies use the same ERP system for many years, reports are created on an ad hoc basis. Some of these reports may therefore be obsolete, some may display the exact same data only in different ways, and some no longer provide the required amount of information. Furthermore, conversion of historic transformations may be carried out only to maintain reporting over time. Consequently, it is recommended to review the reporting needs carefully instead of simply converting all existing reports.

There are a number of ways of converting the reports, but before doing so, the list of reports on the employees’ wish list should be evaluated based on the following three criteria.

  • Reports that have to be obtained from the ERP system
  • Reports that are beneficial to obtain from the BI solutions
  • Reports that are obsolete

Experience shows that 20% of the reports are obsolete, 40 % have to be obtained from the ERP system because they concern external documents such as production papers, picking lists, and more; and the remaining 40% may with advantage be obtained from the BI solution and based on data in the data warehouse. ERP reports often require expensive custom programming, and replacing 40% of the reports with data warhouse driven reports will therefore result in substantial savings as well as better reports because the data quality has been improved.

Data Warehouse Driven Reporting

ERP upgrades are disruptive, and the system or some of the underlying infrastructure may change during the upgrade and affect the stability of the reporting. However, if the ERP system is integrated with a data warehouse, the data warehouse will typically handle more than 40% of the required reports – reports that will remain stable and trustworthy during the upgrade process. As a result, employees will continue to have a central point of information, and the negative effects of the ERP upgrade will be minimized.

Contrary to standard ERP reports, reports designed in a data warehouse can include information from a variety of data sources such as Excel budgets, CRM data, manufacturing solutions, and many more. Automated reporting with a data warehouse can therefore replace the time-consuming, manual process of collecting data from the ERP system and various other sources, and processing them in Excel before the final reports are ready to be distributed within the organization.

Common Reporting

Once the new ERP system is up and running, data from the new system can be loaded into the data warehouse, which also contains the legacy data. The result is a common data set – including historical data – that can be used in any given report without having to take into consideration the origins of the data. The user is able to view data on sales, prices, debtors, and so on from any period of time.

Furthermore, with a data warehouse it is possible to include data from a variety of other data sources such as Excel, and thereby increase the value of the reports even more. Moreover, applying data warehouse driven reports also means a move away from static printouts to more dynamic reporting options. With a data warehouse and a BI solution, users are empowered to view and analyze their data from a variety of angles and in a variety of advanced graphs and charts in their favorite reporting tool.

Conclusion

The tasks involved in the ERP upgrade process become more straightforward and less time consuming when a BI solution is part of the upgrade, and more people are able to take active part in the process. Knowledge of the data sources and the business is central to creating a reliable BI solution – not programming skills.

 


timeXtender

timeXtender’s data warehousing automation platform is all about simplifying the data warehouse process and minimizing the time spent on turning complex data into valuable information. Our unique, agile software TX2014 brings you results five times faster than other business intelligence solutions on the market. In an easily comprehensible design, customized to accommodate the specific requirements in your company.

For more information visit www.timextender.com

IM groeit sterker, het is tijd voor innovatie! (Dutch)

Een recente IM-survey laat zien dat Business Informatiemanagement (BIM) sterk in de belangstelling staat en dat de rol van Informatiemanager langzaam verschuift. De belangrijkste bevindingen zijn dat Informatiemanagement in volwassenheid groeit, dat de IM-er vaker opereert op middenmanagement niveau, dat IM een adviesrol heeft aan de business (Directie, Raad van Bestuur, boards, etc) op het gebied van IT-mogelijkheden en van projectportfolio.


Volgens het rapport zijn de teams van de IM-er worden kleiner, waarbij werkzaamheden bij andere rollen komen te liggen. Het is daarbij opvallend dat de druk wederom vooral vanuit de business komt en de roep (vanuit IM) om duidelijkheid over prioritering van de werkzaamheden en de positionering van Informatiemanagement sterker is geworden. Ook komt het duidelijk naar voren dat een IM-er niet meer “nergens wakker van ligt”. Er is dus werk aan de winkel voor de IM-ers om de regie op de informatie en de informatievoorziening te begeleiden vanuit de business op tactisch niveau. Beleidsbepaling alleen is niet meer voldoende, nu moeten de plannen worden omgezet in resultaten. Onderwerpen waar de IM-er zich vooral mee bezighoudt zijn mobile(er) werken, (door) zelf services voor medewerkers en samenwerking in de keten. Daarnaast, op hoger niveau gaat de aandacht van IM-er naar de inhoud van de informatievoorziening, bijvoorbeeld door het bewaken van het portfolio en het opstellen van de business architectuur. In de survey werd ook geconstateerd dat het meeste budget naar beheer gaat, terwijl organisaties meer moeten innoveren en wendbaarder willen zijn.

Innovatie - moet het nou?

De laatste bevinding is voor ons heel interessant en vraagt om verdere uitwerking. Innovatie is tegenwoordig vereist onder andere doordat organisaties als leveranciers van producten en diensten sneller worden geconfronteerd met de verwachtingen van de afnemer. Relevantie van diensten en producten voor de markt is ook beter zichtbaar voor concurrentie en andere (potentiële) klanten door dat de feedback openbaar woord gegeven (consumenten fora). Het is door onderzoekers vastgesteld dat er meerdere soorten en maten van innovatie waarneembaar zijn – van incrementeel (verbetering van iets wat al bekent en zeker is) tot radicaal (ontwikkelen van een geheel nieuwe aanbod voor onbekend markt). Dat laatste is voor veel organisaties niet haalbaar en misschien niet per se noodzakelijk om in de huidige dynamische en veeleisende omgeving succesvol te kunnen zijn. Volgens onze visie zal de volwassenheid van Informatiemanagement voornamelijk gezocht moeten worden in het bijdragen aan de mogelijkheden om innovatie binnen organisaties te incorporeren in de bestaande (beheer-) situatie.

Informatiemanager als katalysator voor innovatie

Een eerste stap, het zogenaamde “laaghangend fruit”, zou kunnen zijn om de kosten van beheerbudget te verlagen ten gunste van kosten voor innovatie, of zoals wij steeds vaker zien, juist de innovatiekracht vanuit beheer te laten komen. Kostenverlaging van beheerbudget en ontzorging op het bestaande landschap wordt vaak gerealiseerd door verregaande applicatierationalisatie, uitbesteding van het applicatiebeheer of door uitbesteding van bedrijfsondersteunende processen, zoals Finance, Facility, etc. Dit vraagt om enerzijds een stevige visie op de (toekomstige) eisen aan de bestaande systemen en functionaliteiten, maar anderzijds vooral ook om lef om de uitvoering hiervan uit handen te geven vanuit het vertrouwen dat de regie hierop stevig staat. Echter, met “stevig” bedoelen wij hier niet “vast”. Een belangrijke factoor om succesvol te kunnen innoveren is volgens ons vooral dat vertrouwen opbouwen door het op een structurele wijze business ideeën in een vroeg stadium met IT uitwisselen. Zo kunnen de door de business verwachte benefits meteen afgewogen worden tegen haalbaarheid van de oplossing. Informatiemanagement kan zorgen voor een open klimaat en vroege betrokkenheid van de IT bij het opstellen en veranderen van doelen en zo kan IM meerwaarde opleveren door het beter managen van het programma  portfolio. We merken bij organisaties dat er nu steeds vaker wordt gevraagd naar een toepassing van agile op strategisch niveau.

Het “Think big, act small” principe in innovatie

Op moment dat we met een probleem zitten valt tegenwoordig automatisch een vraag: is er al een app voor? Zo bij organisaties, als er behoefte aan verandering wordt vastgesteld, valt er een vraag of er tools bestaan om een probleem op te lossen en de benodigde verandering te bevorderen. Die zijn er wel, echter zijn we van mening dat innovatie binnen de organisatie kan al eerder gebeuren wanneer een nieuw idee mensen aansteekt om hun gedrag te veranderen.

Een voorbeeld van innovatie die vrijwel in elke sector te zien is, is het toepassen van social media. Adviseurs als Jeroen Bertrams en Sheryl Brown waarschuwen dat je er beter niet aan moet beginnen tot dat je er intern klaar voor bent. Dat wil niet zeggen dat je moet wachten, maar juist meteen acteren – intern! Als eerste stap wordt geadviseerd om via de social media naar de klanten te luisteren en al op basis daarvan als organisatie kijken in hoever je in staat bent om op feedback in te spelen. Dat geeft inzicht in wat de omgeving verwacht en wat is haalbaar voor de organisatie om op korte termijn aan te bieden. Vergelijkbare introvisie en voornamelijk zonder “big bang” implementaties direct acteren, lijkt een mogelijke stap bij innovaties op gebied van informatie uitwisseling. Wij zijn overtuigd dat het beïnvloeden en stimuleren van steeds meer open discussies bij het vaststellen van visies en doelen een cruciale rol van de Informatiemanager is. Van hieruit werkt de Informatiemanager proactief samen met de business om innovaties te introduceren en vanuit de bestaande (beheer-) situatie in te richten, met meer focus en snelheid. Zo is de Informatiemanager een katalysator en facilitator voor het invoeren van veranderingen.

Download the survey here.


De auteur

Paulina Cierniak 

Change & Communication | Information Management | Stakeholder Management at Capgemini 


 

Books That Bring Business and BI Specialists Together

We do this by organisation Insight sessions and conferences but also by suggesting books that bring business and BI specialists together. Three books make a serious effort to do so: “Data modelling for the Business” by Steve Hoberman and “Business Analysis for Business Intelligence” by Bert Brijs and the third and updated edition of “the Data Warehouse Toolkit” by Ralph Kimball and Margy Ross.

 

Data Modeling for the Business

Data Modeling for the Business

This book is a fine plea for the use of a high level data model (HDM) as an discussion platform to exchange ideas , concepts, definitions and business rules to produce clarity and consistency in the organisation’s information management. But the book is more than a plea; it is packed with methods, tips, decision matrices, and case studies to apply in your organisation. The authors even take an effort in illustrating the importance of marketing the HDM to get everyone on board and keep the model and the exchange platform alive. This is a must-read for staff and management of any information savvy organisation.

Data Modeling for the Business, Steve Hoberman, Donna Burbank, Chris Bradley , Technics Publications, New Jersey, ISBN 978-0-9771400-7-7


Business Analysis for Business Intelligence

Business Analytics for Business Intelligence

Also a plea for better understanding between the business and the Business Intelligence designers and developers. Brijs makes the transition from high level, strategic concepts to down to earth interview guides, decision rules and practical checklists seamlessly. The book also deals definitely with the fallacy that business analysis for BI is a carbon copy of analysis for application development.

The book is full of practical examples and approaches for finance, marketing, operations, HRM and provides a step by step method for BI project start-up with the necessary templates a BI project manager will appreciate.

Business Analysis for Business Intelligence, Bert Brijs, CRC Press, Taylor & Francis Group, New York, ISBN 978-1-4398-5834-9

 

The Data Warehouse Toolkit, Third Edition

The Data Warehouse Toolkit

The bible of Dimensional modelling has been updated and completed with lots of extra ETL design and architecture information. Just mentioning the topics discussed illustrates the completeness of this seminal work on data warehousing.

  • Fundamental concepts
  • Basic fact table techniques
  • Basic dimension table techniques
  • Integration via conformed dimensions
  • Dealing with slowly changing dimension (SCD) attributes
  • Dealing with dimension hierarchies
  • Advanced fact table techniques
  • Special purpose schemas

No further comment is needed.

The Data Warehouse Toolkit, Ralph Kimball and Margy Ross, Wiley Computer Publishing, ISBN 978-0471200246

Information Management Insight 2014, Brussels - Report

What a shame, due to mega traffic jams, many of our expected guests didn’t make it on time and some probably turned back instead of enjoying an inspiring Information Management Insight 2014 conference.

 

In short, the conference had lots of good stuff for every attendant. With eleven presentations on various topics, one can’t guarantee that every presentation will hit the spot but the opening speech from Dr. Carsten Bange surely made a splash in our audience: to the point, and very clear. His presentation was useful illustrating trends in analytics in the Low Lands and compared to the global average. This created new insights for all of us. During the keynote of Mr. Boute from Google, one could hear a pin drop! With a lot of enthusiasm and passion, he provided us with some surprising, sometimes instigating insights in the Googley way of thinking.

Most of the speakers did their utmost best to bring practical – and therefore sometimes controversial – content from their daily experience. There was interaction during almost all presentations and, most certainly during the networking breaks.

 For a full description and presentation of their talks, click here

Have Your Voice Heard in The BI Survey 14

In close collaboration with BARC, we would like to invite you to participate in The BI Survey 14, the world's largest survey of business intelligence and performance management users with over 3.000 respondents.

 

This survey is strictly vendor-independent. BA4ALL and our partners only assist in this process inviting users to participate in the survey. BARC does not accept vendor sponsorship of the survey, and the results are analysed and published without any vendor involvement.

The Survey examines BI product selection and usage among users in areas including business benefits, costs, proportion of employees using the product, competitiveness, recommendation, innovation, performance, customer satisfaction and agility.

You will be able to answer questions on your usage of a BI product from any vendor. Your answers will be used anonymously and your personal details will never be passed on to vendors or other third parties.

Both business and technical users, as well as vendors and consultants, are welcome to participate. If you are answering as a consultant, please answer the questions (including the demographic questions) from your client’s perspective; the survey will ask you separately about your own firm.

What you will get out of it

You receive a summary of the results from the full survey.

The BI Survey should take about 10 minutes to complete. Click the link below to take part in the BI Survey 14.

Insight Session on Data Visualisation

Exploring New Frontiers in Data Visualisation, Brussels - Report

The session started with a short positioning presentation where the Advisory Board members Walter Vanherle and Bert Brijs indicated the do’s and don’ts in visualisation and drew the audience’s attention to organisational and strategic aspects of visualisation


Eneco’s presentation on the selection process of a visualisation and analytical tool give us an insight in the dynamics behind the selection process.

The second session from St. Lucas University College of Art & Design showed innovative visualisation software and the results from data analysis from social media in the telecom market. The presenters delivered an inspired presentation, inspiring the audience to many questions about the software architecture and the potential applications.

The 75 attendants gave an overall score of 4.3 on a five point scale.

The presentations can be downloaded here:

 The Science of Data Visualization: Eneco Uses Analytics to Deliver a Better Customer Experience
 The Beauty of Social Media: When Art Meets Data

BARC on Self-Service Business Intelligence in the Benelux

Business Analytics for All cooperates with the Business Application Research Center (BARC) from Dr. Carsten Bange. One of BARC’s premium deliverables is the annual BI Survey which has been consistently going on for over a decade.


This allows BARC to detect clear trends in the market, based on user data. That makes BARC a unique source to position your organisation’s analytical strategy against the market averages. In this article we address BARC’s findings on Self Service BI.

Based on 96 surveys from mainly The Netherlands and Belgium, compared to 2284 respondents from 69 countries over the globe, BARC concludes that we are lagging in the self-service segment of analytics. There is a lower than average adoption for self-service BI tools and issues like poor data governance are compounding the adoption problem.

Slow query performance and lack of interest from the business users add to the slower pick up rate.

More on self-service BI, mobile BI and BARC’s findings on Big Data can be found here.

Visualization Tools: Ignite Self Service for Business Departments

When control shifts from IT to business. The rise of self-service data visualization tools in the past decade has reached a new phase. Until now focussed visualization tools like Tableau, Qlikview, Board, Spotfire or others where suppressed by the Enterprise BI Suites chosen by IT-departments. Despite of that, the power of these visualization tools has been demonstrated in business departments that managed to keep a low profile to prevent IT from intervening.


Now the next phase approaches: the phase where business no longer accepts IT constraints like long delivery times, poor graphics and limited analytic capabilities of the enterprise BI suites. A phase where IT needs to acknowledge and facilitate these tools to actually be the business partner they’ve been calling themselves. The past decade business departments would pick a below-the-radar-strategy working with these tools to prevent IT discussions, but now slowly a confrontation arises.

This new era of self-service, customization and BYOD (bring your own device) also comes with a new generation of business people eager to select their own software, not willing to abide by IT deciding what’s best for them. Their software is mainly selected by personal preferences and mainly driven by usability and fast results. Within the mobile world we’ve been seeing this trend for some years now; successful apps commonly have a limited functional focus, combined with great user experience.

And also within the BI analytics world we recognize this trend of “smaller” applications  in the visualization tools. No longer the limited functionality of the enterprise BI Suites is taken for granted, and this new generation of business people is demanding from IT to enable the use of these tools.

Combined with trends like Big Data all major vendors in the BI field have picked up. IBM came up with Cognos Insight, SAS launched SAS Visual Analytics and Microsoft is also moving fast into this direction with the latest versions of Excel and Sharepoint. And all of these solutions are designed and build to look great, come with inspiring visuals, offer focussed functionality, have a very user-friendly and intuitive interface and provide extensive self-service capabilities.

With all excitement for end-users causing huge differentiated demand, IT faces a new challenge. The problem is that most of these tools don’t meet the requirements for enterprise usage (YET!). Basic enterprise functions like centralized management of licenses, corporate design standards (templates) and security are non-existing or provide very limited amount of control. Although the tools are far easier to work with than most enterprise BI suites, and even show faster and better looking results, the missing Enterprise capabilities directly cause resistance and despair within IT organizations. There’s no clarity any more on responsibilities, traditional centralized IT controls have now  suddenly become business driven. IT organizations are struggling to find the right mode to support these tools.

And, from a IT perspective, not anticipating these tools can be a career-limiting move, as the trend is showing exponential growth over the next years. The larger tool vendors are investing massive amounts of money and time in these tools, and where data volumes keep on growing, reality becomes more complex every day and storytelling and visualization become essential tools to build your case, there’s no holding back. So IT needs to come up with facilities to embed these tools into the corporate landscape, without compromising compliancy and risks. And that’s a hard nut to crack!

Waiting for the smaller tools to obtain enterprise capabilities, or waiting for the vendors of enterprise BI suites is not an option. The future is now, and it’s here!

When IT keeps a traditional look at these tools with centralized control and development teams and expect business users to keep on building requirements, the whole company will be bound for disaster. But also C-level managers need to change their mind-set. Centralized risk management and controls are keeping the organization from gaining competitive advantage, and don’t match with these new types of software.

These visualization tools need a decentralized, hybrid approach. Still things like risk identification, guidelines and quality standards can be set centrally, but applying these rules is moving from centralized IT teams towards business units, departments and even to an individual level. And this requires a very, very clear governance on software level. Just like when you download apps from a store, and you have to accept the conditions, organizations need to build governance structures that have personalized SLA’s for software and end-users.

With the growing responsibility for end-users a lot of work is needed on defining governance and growing awareness. As a quote from one of the Spiderman movies: “With great power, comes great responsibility”; and end-users need to be made aware of that. Only putting up guidelines and a governance model will not get you very far, involving and engaging with the end users is the true key to success of these hybrid operating models.

Visualization tools and other end-user centric tools with specific focus areas are a on a roll. It’s a trend that cannot be stopped, and if IT wants to really drive innovation, really wants to add value to their business, they find a way to cope with the challenges. Building hybrid governance models, where walls between business and IT disappear, where individuals get empowered and where tomorrow’s business models are invented. Visualization and analytics are an indispensable power in your organizations quest for finding the sweet spots in your ever growing data, so start TODAY! 
 


About the author

Paul Delgman

Over the past 10 years, Paul Delgman has fulfilled many roles at different organizations. With a bachelor’s degree in Information & Communication Technology (BICT) and a master’s degree in Business Information (MBI) his talents are widely developed. This allows him to fit in easily throughout an organization, on both the technical (IT) and the business sides. This also means that in his various positions, Paul has ultimately been involved in bridging the gap between the business and IT. Moreover, he has developed a passion and talent for stimulating people from different backgrounds to work together better. His technical background combined with his eye for the human side means that Paul is able to close the gap between IT and the business. Although the gap between “technology” and “business” in business processes is disappearing, he regards the change process that people and society still have to undergo as the greatest challenge of the decades ahead. Paul has knowledge of and experience in (agile) software development, IT management, information management and Business Intelligence and draws on his enthusiasm when called on to repair derailed business processes.

Making More Sense Out of Data, Brussels - Report

Data visualization makes massive data streams from measurement systems, social media and other BigData sources ready for insightful analysis and decision making. How? That is the central theme of BA4All’s Insight of March 13.


Hear firsthand how
Eneco uses visualization & analytics to deliver a better customer experience. And don’t miss this opportunity to discover the state of the art of data visualization in the Belgian telco market.
 

Register now as there are only limited seats available. This session is free for BA4ALL members (use promo code MEMBER).

P.S. Have you already contributed to The Belgian BI Survey? After completion and analysis, you will receive a valuable report to position your organization in the broader Belgian BI landscape. Click for a ten minutes’ exercise.


This Insight session is organized by Business Analytics for All, with the support of SAS and USG ICT Professionals.

       

Visualization – Faster Insights for Everyone

The power of visualization plays an important role in decision making. This book answers the questions of why visualization is so important in the interpretation of large amounts of data, and how different companies use data to make decisions.


A simple, static report isn’t enough to reveal the answers locked inside the data. On the other hand, analyses that only econometrists can understand are useless for decision makers who don’t have that expertise. Business users need easy, straightforward analyses in order to be smarter, quicker and more direct when anticipating change. Making adjustments on time is crucial. Visual analytics bridges this gap and brings the magic of the analysts’ tools within reach of the business user.

We often hear, “there is no time for analyses, because all of our time goes into making reports.” Or, “every report we make brings new questions.” Those are exactly the problems that are addressed with visual analytics. It helps interdepartmental teams come up with solid analyses and forecasts, without the need for lengthy preparations. By using mobile technologies (such as tablets), users even have direct access to visually presented insights generated on the fly. This means decisions can be made quickly by the user. 

Read full story

EURES Big Data Proof of Concept

By using advanced analytics, hadoop and a truck load of Twitter data, ARΗS helps EURES (European Job Mobility Portal) to understand what free movement of persons in Europe truly looks like.


Business Intelligence 3.0

 

Business Intelligence (BI) is about getting the right information, to the right decision makers, at the right time. Traditionally, BI platforms have provided entities with the ability to capture historical operational data in reports or dashboards. This data is cleansed, modelled and manipulated in order to be compared to specific key performance indicators (KPIs), allowing decision makers to take action whenever required. During the last decade, there was a move towards more interactive content and Business Analytics including lower data latency.

The focus in Business Intelligence has moved to BI 3.0, adding Big Data to the equation. BI 3.0 includes the integration of unstructured and multi-structured data, e.g. from social networks, sensors, smart meters and machine data, with traditionally structured data, and applies data warehouse automation techniques, real-time reporting and predictive analysis to create valuable insight with business value, anywhere and anytime via Mobile solutions.

EURES (European Job Mobility Portal)

The purpose of EURES is to provide information, advice and recruitment/placement (job-matching) services for the benefit of workers and employers as well as any citizen wishing to benefit from the principle of the free movement of persons.

EURES has a human network of more than 850 EURES advisers that are in daily contact with jobseekers and employers across Europe. In European cross-border regions, EURES has an important role to play in providing information about and helping to solve all sorts of problems related to cross-border commuting that workers and employers may experience.

Set up in 1993, EURES is a co-operation network between the European Commission and the Public Employment Services of the EEA Member States (The EU countries plus Norway, Iceland and Liechtenstein) and other partner organizations. Switzerland also takes part in EURES co-operation. The joint resources of the EURES member and partner organizations provide a solid basis for the EURES network to offer high quality services for both workers and employers.

Big Data Proof of Concept

EURES is present on all major social networks today, such as Facebook, Twitter, LinkedIn… EURES was interested to see if Arηs could capture new insights and spot new trends from those social networks. Most of those social networks already can provide basic statistical information about its usage such as the number of likes, content, photos, videos, visits, duration, …. Our approach was different as we were looking for deeper insights.

Our POC is based on Twitter, but it could have been done on any other social media network as well.

 
 

   

We connect to the twitter account of EURES, load all the tweets with Hashtag @EURESjob, and put this in a hadoop ecosystem (technologies involved are Flume, Sqoop, HBase, MapReduce, Pig, Hive & HCatalog). And as a result we obtained a huge volume of unstructured flat files. We then performed some text analysis and datamining (via SAS Text Miner) on these unstructured files. For example we investigated the geographical localisation of the persons sending the tweets. We finally executed some trending analysis to retrieve some new trends and insights. For the visualization part, we have used Microsoft Excel 2013, but could have done it on any other visualization tool on the market too.

 These new insights help EURES to better develop and orient its ambitious program by deeply understanding the actual behavior of their participants.



Patrick Adler

Patrick Adler has more than 13 years of experience in large and small Business Intelligence projects for the banking, insurance, government and telecommunication sectors. He started his professional career as Analyst Programmer immediately after his master degree. Through various projects, he gradually evolved to a technical expert in Business Intelligence solutions, acting as a point of contact for infrastructure and technical implementations. Besides the technical aspects, he was able to elaborate the functional aspects of the projects, acting as a Business Analyst and able to understand the business requirements. Together with his strong technical background, this enabled him to implement solutions on advanced BI architectures. His current role within Arηs Developments since 2011 is BI Manager for Luxemburg.

Arηs Group

Arηs is a fully independent group of companies, created in 2003, specialized in IT services for large organizations, focusing on state-of-the-art software development, business intelligence and infrastructure services. Our values are to leverage excellence, to be result oriented, to ensure commitment and to ensure adaptability.

Arηs is ISO 9001:2008 certified. Arηs Developments and Arηs Developments Belgium, for their Software Development and Maintenance activities, have been appraised at Level 2 of the CMMI Institute’s Capability Maturity Model Integration. Arηs Cube had the PSF approval as "Operator of Communication Networks and Secondary Computer Systems in the Financial Sector".

The origin of the Arηs [aris] name lies in the Greek mythology. The H is a Greek eta, pronounced [i] in Modern Greek. Arηs is the son of Zeus and the God of War and correctly represents in our eyes the intelligence, strategy, leadership and vision, essential in business. That's why the logo is the combination of the eta Greek letter and a Greek soldier helmet.

Thoughts on Predictive and Customer Analytics Sessions, Brussels - Report

Imagine you’re… No you don’t need to imagine anything. You are a customer, everyone is a customer. And what customer tracking technology today enables makes Edward Snowdens revelations old news. That was the Business Analytics for All insight session of Jan. 30 in a nutshell.

 

But let us focus on the business aspects and let sociologists and ethics professors deal with the societal  issues this new technology brings along, just as they did when bronze replaced flint, iron replaced bronze, horsepower was replaced by steam engines etc,… etc…

 

Andrew Pease

Andrew Pease from SAS opened the session with “Scripting the Customer Conversation”, positioning analytics as a filter on massive amounts of data or in my humble opinion the equivalent of modeling prejudice. But the alternative to that approach is even worse: it is modeling white noise…

Takeaways from this session where:

  • Sensitise your users
    My two cents: because BI without users lacks the intelligence part of Business Intelligence.
  • Match your strategic focus with your operations
    I would go further and try to permeate all processes with the strategic perspective and vice versa.
  • Balance your business instinct with data driven insights
    I was very happy to hear this from a guy with a degree in psychology.  The value of pre-scientific knowledge and less quantifiable competences like intuition, gut feeling and outright arrogance and stubbornness have given us companies like Virgin, Apple, McDonalds, The Bodyshop , Ben & Jerry’s etc…
     

Van Ossel

Vlerick Professor Gino Van Ossel developed the argument even further when he stressed the need for a culture of failure. In other words: allow trial and error and don’t try to corner every uncertainty with extensive marketing research because by the time you reach consensus over the course of action the environment has changed and more agile competitors are outrunning and outfoxing you.

Other takeaways from this session:

  • Retail should evolve towards omnichannel marketing:
    creating one interaction environment for the customer whether he’s in a virtual or a physical environment. The hilarious counterexample from a well known retail chain showed there is still a lot of ground to cover in the retail world: imagine you order garments online but can’t pick them up nor return them in the shop…
  • Technology can enhance our knowledge of the sales funnel upwards
    Something I have been evangelizing for a few years: big data from social media and tracking devices like wearables, PDAs and other gizmos will track the entire response logistics: from eyeballs on a website, an advertisement, a display and the product right to the checkout. Russell Colley’s DAGMAR dream will finally come true.
     

Elliott

SAP Technology evangelist Timo Elliott blew the audience away with a rapid fire of slides illustrating the immense potential of the newest technologies in analytics. Just go to the NBA site or to the City of Boston and see for yourself.

My general conclusion

An example of a sales funnel for durable goods: extending the customer insight through Big Data
 

Sales Funnel of durable goods


Technology allows very old dreams of marketers to come true:

  • To know and understand the response logistics from product design or customization via the attention – interest – desire – action chain to the sales counter,
  • To decompose aggregate marketing information to the lowest grain: the individual consumer,
  • To get deep understanding of switching behavior while the consumer is shopping,
  • To monitor online the impact of all marketing variables

The more technology evolves and delivers value, the more need for human relationship and context management becomes a necessity. These are two sides of the same coin in business intelligence: user acceptance and user integration of data driven decision making will be the bottleneck in any marketing technology push.

And finally, all this new stuff still needs to pass the test of a well balanced business case. The balance between the added value of information in hard currency and a softer evaluation of improving the strategic position with new technology needs careful study.
 


 

About the author

Bert Brijs

Bert Brijs is a member of our advisory board He is Senior BI Consultant, owner of Lingua Franca bvba & Lingua Franca B.V. Bert works as a Business Analyst and Enterprise Information Architect for large organisations in Europe. He is a regular speaker at conferences and publishes regularly for KCEP and IT Performance Improvement. Bert is author of a book, Business Analysis for Business Intelligence (BA4BI), with a dedicated website http://www.ba4bi.com He has an education in linguistics, management and ICT and is a strong advocate of permanent and just-in-time learning.

 

 

Business Analytics Insight 2013, Brussels - Report

Data warehousing and business intelligence guru Ralph Kimball presented in Belgium for the first time since 2000. Four speakers between Ralph Kimball’s sessions, four topics and one unifying thought: BI is getting to big to allow failure.

 
The first Business Analytics for All Insight session which took place in Brussels the 12th November gathered over 270 attendees to hear Ralph Kimball’s insights on the data warehouse design principles and how the Big Data phenomenon fits in this architecture.

But between Ralph’s talks in the morning and in the afternoon, four other topics were discussed which all lead to the same conclusion: BI has become too big, too much of a strategic commitment to allow for sloppy business analysis and project management.

Annelies Aquarius, European BI Project manager from the Coca-Cola Company illustrated the anytime- anything-anywhere aspects of mobile BI. Jelle Defraye from Laco made a case for self service BI.  Jos Van Dongen from SAS taught us the basics from data visualisation and Guy Van der Sande from USG ICT Professionals explained why a well organised BI Competence Center (BICC) is essential to manage technology trends and changing business requirements.

Agile Scrum BI: The Product Owner’s confusion

As a speaker at a conference on Agile BI in mid-December 2012, I provided a session dedicated to the role of the Product Owner. The content of that session was based upon a mix of real-life experience as a Product Owner, and on theoretical reflections.

I concluded that the role of Product Owner is hard to fill in the ‘right’ way, which makes Agile Scrum hard to kick-start and makes the process feel either flawed or incompletely designed and/or described. This observation remained true when I applied Agile Scrum BI in practice as a member of a Product Owner Community. This article zooms. in – again – on a couple of important elements of the implementation of Agile Scrum in general, and more particularly focuses on certain points for attention in the execution of the role of a Proxy Product Owner in a Business Intelligence environment.

The Proxy Product Owner

A Product Owner role as described in the Agile Scrum theory is a business profile, and therefore he/she is a member of one or more business departments, not a member of the IT department. Not having a Product Owner in the business is usually considered to be an adverse organizational setup. A Proxy Product Owner is a person acting as a placeholder for the actual Product Owner. In essence, the Product Owner fills two major roles: proxy client when dealing with the Scrum team, and product management coach when dealing with the customer.

Fig.: a schematic representation of the Proxy Product Owner role

 

During an earlier Agile Scrum start-up assignment, a valid reason for kick starting an Agile Scrum BI organization with Proxy Product Owners was the absolute faith and belief in its advantages, and being ahead of the business departments in terms of knowledge of the subject. A partial or full buy-in was projected as a future goal based on the expected success. Drawbacks of a Proxy Product Owner are, however, that:

  • the distance to the business is too great despite all efforts to bridge the gap;

  • there will probably always be a lack of empowerment in decision-making (mandate) about product backlog prioritization, release planning, and whether to accept or reject work results.

In general, a Proxy Product Owner will need to put in quite a lot more effort not to end up in conflicts and miscommunication, a slowdown in decision-making, and a decrease in productivity and morale.

Briefly put, Agile Scrum theory describes a User Story as one or more sentences in the everyday or business language of the end user or user of a system which captures what a user does or needs to do as part of his or her job function. It captures the ‘who’, ‘what’ and ‘why’ of a requirement in a simple, concise way, often limited in detail by what can be handwritten on a small paper note card. It is my belief that a Product Owner must master a mature level of business narrative in order to really capture the essence of what is needed. The sharing of knowledge is one of the cornerstones.  User Stories are crafted to generate understanding, not necessarily only action. Why things happened/occurred as they did needs to be translated into a ‘well-told’ User Story. This does not mean narrowing it down to simply transmitting bulleted tasks, actions and values through narrative. All these communicated abstractions are typically ‘dead on arrival’. In this sense, to me, a good User Story is much more than a so-called ‘reminder to have a conversation with the business’. And I believe that mastering the craftsmanship of the business narrative is an undoubted skill of the Product Owner.

Skills and commitment necessary for a good Product Owner

A couple of years back, I was part of a BI development team that decided to make the strategic shift towards an Agile Scrum BI organization. Despite all the time we had to prepare this – guided and supported by an external Agile coach – and despite a relatively high number of dedicated strategic and tactical workshops, the true sense, the content and the responsibilities of the Product Owner were still being frequently debated many months after ‘go live’. This was within a community that needed to start acting as Proxy Product Owners while still being part of an IT department.

In any case, figuring out what to build (and what to build first!) is the core job of the Product Owner. It is difficult. It is probably a full-time job and it demands close collaboration with the development team on an ongoing basis. The team requires guidance and direction (e.g. actively managing the product backlog, answering questions when they arise, providing feedback, signing off work results, etc.). In simple terms, the Product Owner sits in the driver’s seat, deciding what should be done and when the software should be shipped. To deliver value, Agile Scrum requires an efficient, accurate mechanism for determining the vision for a solution and a feasible pipeline for translating that vision into concrete, deliverable backlog items that can demonstrably deliver tangible benefits. It is for this reason that the performance of the Product Owner becomes the prime limiter of the team’s success. Something that we learned at my current customer very soon after ‘go live’ with Agile Scrum BI.

Since Business Intelligence does not equal application development, and we cannot formally define it as a ‘product’ since it is more of a process, the skills required to do a good job are somewhat more demanding. In general, the Product Owner skill set includes knowledge of the following:

  1. business analysis (thorough understanding of the customer needs);
  2. product lifecycle management;
  3. configuration management;
  4. organizational marketing;
  5. active stakeholder communication and management;
  6. consensus building;
  7. IT program management;
  8. basic knowledge of how software is developed and deployed.

Agile Scrum requires some proficiency in all the above – and the awareness to solicit additional support when necessary. Key for a Product Owner is:

  • understanding the customer’s needs and how a solution/product will satisfy those needs (vision!);
  • to be able to manage solution development and to actually participate in high-level solution design decisions;
  • possessing a degree of innovation and an entrepreneurial spirit.

This broad skill set implies that ideally, the Product Owner would be a hybrid: someone who is able to look outward, understanding the end customer’s needs, and someone who looks inward, managing the value stream that transforms the customer’s needs into a solution ready to be used by that customer.

Agile Scrum BI - Body of Knowledge

Knowledge is theory. We should be thankful if action of management is based on theory. Knowledge has temporal spread. Information is not knowledge. The world is drowning in information but is slow in acquisition of knowledge. There is no substitute for knowledge. - W. Edwards Deming

Early in any development or project process, uncertainty and risk are the Product Owner’s enemies. Knowledge can be regarded as the opposite of risk. When uncertainty is high, the focus should be on knowledge acquisition. This can be done in many ways, for instance using technical spikes, prototypes or R&D experiments to explore emerging technologies. This cannot immediately be translated into value but it is still very important because it reduces risk, an important Product Owner responsibility. The main objective is to stabilize and preferably boost the customer value curve as much as possible.

Any noticeable lack of the knowledge and skills required by a Product Owner as described above, make an Agile Scrum structure fragile. I have learned that Product Owner guiding principles, methods, tools and techniques are not very well documented in Agile Scrum practices (also there are few useful books). You often need to look elsewhere for this information. I see a great opportunity to contribute to Agile Scrum theory by means of a more detailed ‘Body of Knowledge’. 

Disclaimer

This presentation is the property of the solution provider/speaker and strictly confidential. With viewing and/or downloading this information, the recipient acknowledges and agrees that:

  1. This document is not intended to be distributed, and if distributed inadvertently, will be returned to the solution provider/speaker as soon as possible
  2. The recipient will not copy, fax, reproduce, divulge, commercially use or distribute this confidential information, in whole or in part, without the express written consent of the solution provider/speaker
  3. All of the information herein will be treated as confidential material. This presentation is for informational purposes only