Data Quality From The Ground Up

My Blog has moved!

2010-10-20T10:39:00.000-07:00

As of June, 2010, I've moved! http://sheezaredhead.wordpress.com/

Thank you all for your support, comments, collaboration and encouragement. I hope to see you on the new site!

Sincerely,

Jill Wanless

Don't! Lie to me.

2010-06-02T04:12:00.000-07:00

Lie to Me is one of the few shows on TV I actually watch. I LOVE it! It tells the story of Dr. Cal Lightman (played by another favorite: Tim Roth) and his team who use applied psychology to read facial and body language clues to expose the truth and lies in criminal investigations. The show is so popular that in November of last year it was renewed for another 9 episodes the first of which is scheduled to play on June 7th. I can't wait!

What does this have to do with Data Quality? Nothing specifically, but it has everything to do with working with others to achieve common outcomes.

If you have been following my blog or tweets, or have read some of the comments I have posted on the blogs of others, you may have noticed that I sometimes make reference to behaviors I have observed that don't enable trusting, collaborative partnerships; in fact they do quite the opposite! And these behaviors, displayed through facial and body language and, although quite subtle, can be very detrimental to the successful outcome of any project or program.

So why am I writing about this? I read body language. Steady. All the time. In every single encounter and meeting. I know pretty much what you're thinking, who you like or dislike, what pushes your buttons, and when you are lying (OK, that's pretty harsh, how about 'when you are not telling the truth'). Yes, those times when you verbally agree to something while shaking your head in a back and forth manner tell me that actually you DON'T agree at all, and if you do it with your arms folded across your chest or without eye contact it also tells me that you STRONGLY disagree.

And when you ask for my feedback but focus your eyes elsewhere it tells me that actually, you have no intention of listening to my feedback and in fact, you are probably thinking about what you need to pick up at the grocery store on the way home :p

These types of behaviors do not show trust. They do not suggest partnership, 'we are in this together' or 'I care about your opinion or insight'. They tell me that some form of approach, unilateral decision or direction decision has been made and that regardless of what my, or the insights of others may provide, it will not be taken into consideration.

This is an issue. If you want your project, program, event or activity to be successful, you need to be aware of the motivations and feelings of others, and especially if you are working together as a team. Team: –noun - a number of persons associated in some joint action..

So if you have a common goal, and you want to achieve a successful outcome, you need to work together as a team. And in order to work together as a team, you need to trust each other. And in order to establish trust, you need to be honest. Just tell the truth. If you are making a unilateral decision you might have a perfectly good reason for doing so. Explain the reason. Be open about it. Say: "I need to do X because of Y", or "I appreciate your feedback but cannot act upon because of this reason".

You will be surprised at what will happen. The team will appreciate the honesty. They will, in turn, be more open and engaging. You will establish credibility. You will gain supporters. You will have increased chances for a successful outcome. And guess what, your body and facial language will look more like someone who is relaxed, trusting, open and engaging: direct eye contact, informal stature, relaxed arms. You get the picture.

So why do I love the show Lie to Me so much? It takes the reading of body language to the next level. It shows examples of minute facial expressions and body movements that you might not have noticed before and provides some real life examples as to what they probably mean. And I'm a good student! So watch out! I know what you're thinking. Don't lie to me!

"If you tell the truth you don't have to remember anything" ~Mark Twain~

Attributes of a Data Rock Star

2010-05-12T06:47:00.000-07:00

Earlier this week, Jill Dyché (@JillDyche), successful author, blogger, BI, MDM and Data Governance consultant and all around information guru, created a flurry of creactivity (I just made that word up – it’s a combination of the words creative and activity), when she tweeted a simple response that suggested a couple of data rock star types, in response to an excellent on-line article Are You a Data Rock Star? by Elizabeth Glagowski.

Elizabeth’s article has some excellent descriptions and examples of the attributes of what makes a great data rock star and Jill’s vigorous (and often humorous) take on business and IT alignment always identifies the rock star behaviors of being able to communicate the linkage between a company's information and its business value.

The results of the flurried creactivity, was Jean-Michel Franco (@jmichel_franco) coming up with a name that was quickly adopted; “The Rolling Forecasts”, and Jim Harris (@ocdqblog), in his latest Obsessive Compulsive Data Quality blog post, coined the brilliantly perfect and perfectly brilliant lyrics to the band’s first song: “You can’t always get the data you want”.

So what I decided to do here was attempt to compile all these rock start attributes and behaviors in a simple format, so that they can be easily re-used and referred to. I plan on adding these to our internal wiki and identifying them as behaviors of successful data stewards. I a) hope they get read and b) hope that they get people thinking, behaving, changing…

· Excellent communicator of business and IT concepts using common language

· Ability to link information to business value

· Effective at communicating concepts and new ideas at early stages in order to reduce change management efforts

· Has excellent self awareness and understands the link between trust and partnership

· Is able to express thoughts and opinions in various ways in order to be able to provide feedback when others may not be interested in hearing it

· And seeks out and is receptive to feedback and continuously provides the opportunity for others to provide it

· Actually listens to the feedback and changes behavior/process/approach for continuous improvement (don’t get me started on people who ask for feedback but couldn’t give a rat’s a**..)

· Understands the link between clarifying expectations and how that will lead to success

· Ability to know how to engage and enthuse others – must understand the body language, communication preferences, motivations and needs of others

· Must be able to spot opportunities and take advantage of them – and especially do it in a way that others are unaware of it

· Must be comfortable pushing the boundaries in order to change things and do so in a way that others are unaware the boundary is being pushed

· Must be comfortable exerting authority and using it appropriately – all the while smiling and engaging others

· Is able to identify key success measures from both business and IT perspective and communicate effectively – at the beginning to confirm what is expected and throughout to continue to re-iterate value

· Is well liked and respected – this will ensure access to resources, tools, other stakeholders, hidden information (you KNOW that happens), and will help pave the way through political and cultural roadblocks

· Be able to articulate solutions as practical and logical and tie them directly to group/organizational goals

What do you think?

Interview with Data Quality Pro

2010-04-13T04:29:00.000-07:00

Dylan Jones, founder of the hugely successful Data Quality site: Data Quality Pro, has graciously interviewed me on my approach to achieving some data quality success without the benefit of a business sponsor. Read the full interview here: Interview with Jill Wanless

It was a great experience as it really got me thinking about what it was that really helped our team achieve what we were able to. Dylan does a great job of summarizing the key points and I'd also like to add that achieving data quality success is not a one man job. It requires a team of dedicated and like minded individuals with a good mix of technical savvy, schmoozy sales skills and some serious creativity genes. What I'd really LOVE to do is get a picture up of all of them so you could meet them and see why they are so special. Yeah right..naturally whenever I mention even the slightest suggestion of showcasing their specialness they quickly hide the look of horror on their faces and change the subject. I'm still working on that ;)

So thank you Dylan for being such a supportive champion in my attempts to share our story, and thanks to the best team anyone could ever have!

There I go, getting all Verklempt again..

Data Governance From The Ground Up?

2010-04-07T06:38:00.000-07:00

Not really but we can’t sit around forever waiting for the governing roles and responsibilities to be finalized so…

Let me back up a bit. I was the manager of a Data Quality Team. We used creative (aka guerilla) tactics to raise awareness, get buy in, and improve the quality of our customer information. 3 years later I am now a Senior Advisor on the business side, participating in the development of formalized Data Governance. How awesome is that? So what do I mean when I say: “Data Governance from the ground up”? Well due to the fact that Data Governance means a profound change in how we manage our data for both people and processes, lots of discussions and consultations are taking place at the executive level to understand the impacts and begin the process. So in the meantime, a few business stakeholders (yes THEY did it..not me!) have initiated the development of a working group, the goal of which is identify and understand some of the priority data issues that require resolution in order be somewhat organized and prepared for the time when a formal council is established.

The strategy will be to focus on data that is required to support upcoming business objectives, and work together to develop recommendations (and supporting documentation), so that when the formalization occurs, the group is well prepared to begin obtaining approval for and implementing the recommendations. The approach will be to work iteratively starting with the highest priority and pain points, and communicate regularly on the results and successes. The iterative approach will allow the participants and processes to be adjusted and fine tuned as necessary, and will increase the likelihood of buy-in and success.

What happened in Meeting 1?

We identified potential stakeholders and agreed that others will be identified as we move forward.
We agreed that the scope of the data that we need to focus on will be the basic Company and Contact data: name, address, phone, etc..
We agreed upon the initial approach: identify and define company and contact lifecycles, identify and define companies and contacts, identify and define any known risks and issues with companies and contacts.
And oh yeah, they made me the chair so I get to facilitate, encourage discussion and communicate results...must be my big mouth :)

I wanted to share this for a couple of reasons: first, it’s another step up in the Data Quality journey and it will be interesting and hopefully beneficial to others to share the ride, and second, I’m hoping if we get stuck on something I’ll write about it and obtain the feedback and opinions of other subject matter experts. So if you have any questions, comments or feedback, it would be most appreciated.

Thanks!

One small step for IT, one giant step for Data Quality.

2010-03-26T06:58:00.000-07:00

I had a vision. My vision was to get buy-in from IT resources to start data management activities at the start of every IT project. Tough thing to achieve when performance is strictly related to delivery, not quality. I could speak [rant..whatever they wanted to call it :P] to them about benefits until I was blue in the face, but until I could describe what the benefits were in terms of project delivery risk management, they quickly went back to managing projects to the way they are used to so they could quickly start on the next one.

I knew that to really get them to listen I needed to put it in their perspective. What are the project risks they encounter every day? What goals are they trying to achieve and how did this tie into the satisfaction of their clients?

Here is what I know:

They are busy all the time
They have a lot of projects to deliver and limited resources - so no time to even think about new processes
They have some satisfied and some not-so satisfied clients
They are measured on project delivery (dates) not project quality

So how did I get their attention and show them that pro-active data management activities could help them achieve some of their goals?

I made a diagram. Everyone loves a diagram and no one has time to read text anymore. Plus, I made the diagram in such a way that it has a look similar to a Gantt chart. The goal of the diagram was to show that by not integrating data management activities into their project processes they were actually increasing the data related after project churn: Change requests, data clean-up, data reconciliation etc. These post project activities also used more not less resources (including business resources), which in turn cost money and time and led to less than satisfied clients.

When put in that perspective, most of them nodded their heads in agreement and even provided their own examples of what that after project churn looked like. I’d like to say that after a few of these presentations the corporation saw the light and made a broad announcement that from this day forward Data Quality is a top priority! I’m kidding here…! What did happen though is that the requests to provide Data Quality direction in pre-project analysis increased significantly thereafter. One small step for IT, one giant step for DQ :)

Here is the diagram. Click on it to see a larger view.

I'm not dead yet!

2010-03-16T07:25:00.000-07:00

No, I was not unwell, just laying low for a bit and trying to get my bearings. I am part of a new team and just wanted to make sure I wasn’t rocking any boats with my social participation. It turned out to be a good move. The organization I work for (including my new boss), just finished drafting up some proposed guidelines for employee’s use of social media. I thought I’d just sit tight and wait until they came out so I could have a look and make sure I wasn’t breaking any corporate policies. As it happens I’m good to go, my new boss is cool with whatever I do as long as:

1/ I don’t get him fired

2/ I don’t get his boss fired

I like his nice broad guidelines, I should be able to follow them easy peasy. I suggested to him they should be our corporate policy for employee’s use of social media. Nice and short and ‘to the point’! Alas, some people have a need for more complete and specific information … details schmetails....

We still have an MDM initiative underway and Data Governance will play a big part of the solution. What is happening right now is a lot of process stuff to get a project of this size approved and organized; funding, governance, approvals, stakeholder participation, budgeting etc.. Things should start to ramp up in the next month or so and I will be sure to post some stories on what the goals, challenges, successes and fun 2010 brings!

Thanks Goodness! I felt like I was missing all the fun!

And yes, in addition to being a Star Trek fan I am also a fan of Monty Python! Brilliance is Brilliance…say no more…say no more..

Guerrilla tactics lead to....change!

2010-02-09T04:55:00.000-08:00

Guerrilla tactics lead to... change.

So there I was, a Data Quality manager in IT without a business sponsor. My team and I were applying Guerrilla communivative tactics to raise awareness and get stakeholder engagement when all of the sudden...WHAM!! The ship came in, the planets aligned, the hard work turns to luck, we beat The Kobayashi Maru!!

For those of you unfamiliar with Star Trek, the Kobayashi Maru describes a training exercise for Star Trek cadets and it was designed to be a no-win scenario. And let’s face it, if you are running a DQ program and you’re sitting in IT you can only get so far before you absolutely must have a business sponsor.

Our Data Quality team started out 3 years ago and made a lot of good progress. We have an executive (I.T. mind you) approved mandate, we have resources and a budget, we maintain a wiki of over 1000 corporate terms and the related metadata, and we have a small group of business clients who we support and (of course) love dearly. But NOW…now we have a business sponsored Master Data Management (MDM) initiative underway and guess who gets to be the Business Lead for MDM and be ‘The One’ responsible for establishing (and getting business buy-in for) the Data Governance! That’s right, you heard me, I said BUSINESS Lead! I am now officially in ‘the business’ and am reporting directly to ‘the sponsor’! It almost makes me verklempt!

So how did it happen to me? I suppose if you were to watch the movie you would say it was a logical progression. I prefer to think it was the communivative, guerrilla, in your face stalking…nothing says ‘I care’ like persistent determination. So forgive me if I’ve been absent too long, the last few weeks have been interesting to say the least. I am very lucky to be part of this sure to be exciting ride, and I promise there will be more interesting stories ahead.

My New Year Data Quality resolutions for 2010

2009-12-30T05:28:00.000-08:00

Because we all know that if we write it down and publish it for all to see we will have a much better chance of success! So here they are:

‘Communivate’ quality issues to a much broader audience – no more suppressed results – ever!
Establish employee Data Quality performance targets – for ALL employees. (This could be a multiyear task however since it’s a big challenge it will be lots of fun to do)!
No more Mr. (Ms.) nice guy – those of you who know better yet keep causing Data Quality issues beware! I have stats on you! Fear me!! ...or maybe I’ll just nicely advise them that I can help them get ready for the soon to be established performance targets – see item #2.
Establish measures and track success of how well our IT project processes are following the supposedly integrated information management protocols – I know it’s not my job to do but sheesh, nobody else is doing it dammit!
Communivate results to the broad audience (see item #1)
Obtain agreement from ‘informal’ data stewards (those that I have established relationships with after stalking for months on end...) to partner with us DQ folks and develop some agreed upon roles and responsibilities. (These will become part of the employee performance targets.)
Take ‘quasi’ business sponsors to task for not doing what they said they would do (also part of the employee performance targets).
And last but not least, I also hereby promise to swear less often, or perhaps more quietly, when hitting the roadblocks that stand in the way.

I can’t wait to reward myself...

‘We will open the book. Its pages are blank. We are going to put words on them ourselves. The book is called Opportunity and its first chapter is New Year's Day’. ~Edith Lovejoy Pierce~

All I want for Christmas is..

2009-12-23T07:52:00.000-08:00

Dear Santa:

I hope you are well. I have been very good this year (don’t listen to the rumours) and would appreciate the following for Christmas:
1. A business sponsor – so when I find a problem that requires a business decision or communication I don’t waste 3 weeks (and I don’t come cheap!) trying to find someone who cares.
2. Data Stewards – who are measured on how they manage the data they are accountable for.
3. Integration of our Data Management Software – so we can match and enhance at the source of data input rather than performing it after the fact.
4. IT best practices – where every IT project includes data profiling.
5. Complete Business Requirements – these would include:
• The identification of the Stewards for the related data and processes
• Data Definitions
• Business Rules
• Governance processes
• Stakeholders
• Business Purpose of data
• Quality requirements for data
• Training requirements related to quality of data
6. Comprehensive Benefits Realization – where costs associated to managing data re-actively are identified
7. Corporate Training for new employees that includes Data Quality best practices
8. Accountability for those who know better yet still create bad data
9. Employee Performance Measures that include at least one objective related to quality of data
10. Effectively communicated and measured corporate policies and procedures related to quality of data

I know I’m asking for a lot, but if you could do this for me I promise it will improve the satisfaction of our customers, reduce our operating costs and make me one seriously happy camper.

Thanks very much,
Sincerely,
Jill

Ways to 'Communivate' your Data Issues

2009-12-09T10:15:00.000-08:00

Part 1 of:
The Purple Cow approach to Data Quality
Or: How to have fun while trying trying to jump data quality sponsorship hurdles
Or: how to use innovative communication tactics to reach your Data Quality objectives.

‘Communivate’ is a combination of the words communicate and innovate, and it means to communicate in an innovative way. Our team uses it a lot to describe how we get our message across. We are one of those insane teams (aka Sneezers ) who constantly push the boundaries of ‘appropriate’ tactics to get the job done, and are always coming up with new terms to describe our approaches. (Makes me wonder if coining new terms is a DQ thing?)

Background
I am responsible for implementing a Data Quality (DQ) program and I have no business sponsor. As a result, my team and I put an enormous amount of effort into achieving the following:
· Raising awareness
· Communicating poor DQ issues
· Stalking (did I say stalking?) I meant to say identifying and engaging business stakeholders
· Developing business cases to educate business and IT on best practices
· Getting buy-in

Essentially, we collect a lot of data and share it with whoever will listen. And because we don’t have that essential business sponsor, we need to communicate over and over (and over) the same messages to various stakeholders. It can get tiresome [insert shot of Dracula sucking enthusiasm out of lifeless body here] after a while…

So, here is one example of how we communivate,
Goal 1: Raise Awareness
The Strategy? Find a Captive Audience.
Since we don’t have a business sponsor we don’t have the same corporate tools to spread the word. Internal intranets, team portals and corporate newsletters are all off limits, so instead we took the message to the people. Because we’re sneezers, we wanted to push the boundaries and have fun. The team printed off screenshots of seriously bad data and posted them (under covert secrecy – more sneezer fun) on the doors of washroom stalls. You could not get a more captive audience than that.
The results? We ran this campaign 4 times over a 1 year period and by the end of the year our communivative strategy AND the message we were trying to achieve was mentioned by a Senior VP in a corporate communication, we received 57 positive (and I tree hugging negative) comments and another Senior VP asked us when our new campaign was going to start. (Ok, they still are not sponsoring us but they do like us, so one thing at a time..).

Awareness raised, goal achieved.

Data Quality, My Story

2009-11-15T15:00:00.000-08:00

I wish I could write more frequently. I have a lot to say and I'm always writing stories in my head. Every week our Data Quality program moves a little bit farther up that maturity ladder and I'd like to share what worked with the world! The problem is I struggle with how to take what happened, what is real, and write about it in a way that does not cross any of my organisations confidential boundaries. Which is why I struggle. Do I say this or that..will that get me in trouble..what ARE the rules in these days of social media?

Anyway, I've been inspired lately by some great bloggers on the subject of Data Quality: OCDQ Blog, Jill Dyche, Dylan Jones, Steve Sarsfield...
Their stories are relevant, informative and personal, which is why I like to read them. And I do push the boundaries at work every single day. It's the only way to make change happen. And it has worked so far....So I will tell my story, and hope that maybe someone else will benefit just like I have. As Spock would say: "The needs of the many, outweigh the needs of the few, or the one".

Yep, bit of a Trekkie fan!

Data Quality Team Resources

2009-09-01T04:25:00.000-07:00

First, let me apologize for the delay in getting this post out. The team was busy working on a big data clean up project that took most of the summer and a lot of resources. The good news is that the effort provided enough statistics to make a business case for process and system changes to improve the data going forward. One small step at a time!

In order to share what types of resources I believe are important for a successful data quality program, I'll need to provide a brief overview of what makes up our Data Quality Program.

Our program is based on recommendations that came out of an initial assessment of the maturity level of our data quality and include the following:

Data Profiling
For more information check out Wikipedia : Data Profiling

This is the most important part of a data quality program. It allows you to see where your data problems are AND track the progress of any improvements. Once you start measuring the quality of your data you should continue to measure it on an on-going basis.

Resource Requirements
You'll need an analyst with a strong analytical mind. I use the term analyst only because data quality resource profiles are so new, so analyst can also refer to business analyst or data analyst or information analyst. They need to be someone who likes to get the bottom of things and who sees every problem as a challenge to be overcome. They also need to be good at developing and writing business cases, as the results of your data profiling generally lead to a case for making changes to a process or system. They should be an excellent communicator (ok, everyone should be an excellent communicator!) and have good influencing skills...they need this to get the data extracts that some groups don't like to give up :)

Data Definition
Data definition is a simple idea: define your data so that creators, users and stakeholders understand the meaning and purpose of the data. Remember the definition of data quality: "the data is in a state fit for it's intended purpose"? And although the logic behind defining you data is simple, the effort to do this is sometimes the most challenging and time consuming. Have you ever tried to get everyone to agree to a definiton of a 'customer'? Not so easy. And yet without a definition, you have nothing to work towords in a data quality program. You simply cannot understand, improve, consolidate or convert the data without understanding the meaning.

How did we do this? We started with the data that most concerns everyone and that is the basic customer information: Name, address, segmentation, etc. We obtained the definitions from the data warehouse and we put them all together in a word document, posted it on a shared directory and sent links to everyone who we thought would be interested in this information. The results were as follows:
-People contacted us to advise us that a definition was out of date - good! We ensured the definition was updated.
-Business Analysts began to refer to the definitions as part of their requirements documentation - good!
-Business users began to refer to the definitons when discussing potential changes - good!
-New employees used the information as part of their training - good!

We then purchased a simply wiki tool, added the definitons and published this information on the corporate web site. That was in late 2007. Today we have a corporate wiki which contains over 1000 terms (definitions, business rules and purpose) and over 60 articles and help guides, it gets on average over 800 hits per month and has 16 contributers. The contributers are business users of the information who care about the accuracy and have agreed to particpate in the upkeep of the information - they are our future Data Stewards.

Our goal - to have every corporate term published in the wiki and have the information managed by stewards.

Resource Requirements
We have 2 types of resources to manage this function:
One is an excellent writer and editor - they should be excellent communicators and should write from a business perspective. Why? Because you need to use simple, everyday language that everyone can understand. For example, if your term is called 'blue sky', your definition or business rule information should say "why the sky is blue" rather than "diffuse sky radiation and its impact on colour perception". Their main responsibility is to review the information provided or contributed by others and ensure it is clear, easy to understand and formatted correctly. Kind of like a wiki editor.
The second is a business analyst with really good (and I mean REALLY good) sales skills. It's their job to extract the information out of people's heads, emails, documents, user guides, folders and systems. They also need to get users of the information to become contributers - a big change for non-social media types.

Those are the main functions of our program - and the most important. We also implement data quality improvement projects (see my apology at the top of the page), provide data quality related support, develop help guides and tips and tricks and perform manual and automated cleansing and enrichment using a data management software tool. For the next post, and I promise not to leave it so long, I'll identify some of our biggest challenges with not having executive sponsorship and how we overcame them using some purple cow techniques :)

Data Profiling - Basic Measures

2009-03-16T09:30:00.000-07:00

Last week I talked about how data profiling is an important first step in getting your data quality program off the ground. Data profiling is just another term used to describe the process of examining the data available in an existing data source (e.g. a database or a file) and collecting statistics and information about that data.

There are a lot of robust data profiling tools available but you should be able to begin performing analysis using a common tool such as excel or access.

Here are the basic measures that we started with:

Company (or Account) Name
· Contains the word(s) “duplicate” or “out of business”
· Contains invalid characters

Company (or Account) Address
· Is blank
· Contains invalid characters
· Contains the words Address Not Known” or “unknown”

City
· Contains invalid characters and or 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
· Contains the word “various” or “unknown” or 'city Not Known'

Company (or Account) Annual Sales (or income)
· Is Blank
· Contains a numeric value less than 100$

Primary Contact First Name
· First Name is blank
· Contains the word(s) ‘Unknown’ or ‘Not Known’
· Contains the words Address Not Known” or “unknown”

Primary Contact Second Name
· Second Name is blank
· Contains the word(s) ‘Unknown’ or ‘Not Known’
· Contains the words Address Not Known” or “unknown”

Primary Contact Phone
· Is Blank

It’s important to keep in mind that you should be measuring against the business rules. For example, if a phone number is mandatory then you should not need to check if there are any blanks.

It is also important to understand the purpose for the data. For example, if the purpose of collecting the Company (or Account) address is to mail documentation then you should probably be checking to see if some of your address data contains Physical address information (such as 123 Main Street) vs. Mailing address information (such as P.O. Box 123).

The Initial Results

It doesn’t matter if you are a data geek or not, the results are always very interesting. Some of the statistics we identified as a result of our initial analysis were as follows:
47% of our address information was blank
10% of our address information was ‘Not Known’ or ‘unknown’
8% of our Company Names had ‘Duplicate’ in the name

What do your results tell you?

Your results tell you where you need to focus efforts to improve or even remove the data. If you are finding a lot of blanks (or null values) for your data then your company probably is not using the information collected and you may want to stop collecting it.

The Purple Cow (aka ‘how to stand out from the rest of the crowd’)

Since we didn’t have a sponsor and we had no idea who would be interested in the results we made a splashy ‘Did you know’ flyer that listed the results and posted them in the washrooms – on the doors of the stalls. Needless to say we got people’s attention...

Next Week
Next week I’ll talk about the types of resources we have on the team and how one of the most important attributes your team members can have is their ability to market the message.

From the Ground Up

2009-03-09T04:31:00.000-07:00

I've been thinking about writing about Data Quality for a while now. I'm responsible for a Data Quality Program at a Crown Corporation and since the program started just 2 1/2 years ago a lot of progress has been made. What's the big deal you say? Well the program has no executive business sponsor (bad!), our team has been floating around different groups trying to find a home (bad!), there is no formal mandate (bad!) and yet despite all those no-No's we have made a lot of great progress! We are a team of enthusiastic creative thinkers who have broken the mold and achieved success due to 2 key strategies:
1/ Our program is based on Industry Best Practices
2/ Our methods for communicating and engaging others are very 'Purple Cow' - "In his book Purple Cow: Transform Your Business by Being Remarkable, Seth Godin says that the key to success is to find a way to stand out--to be the purple cow in a field of monochrome Holsteins.

The title: 'Data Quality - From the Ground Up', is just that. Implementing a successful Data Quality Program without an executive sponsor or a mandate starting from the ground up CAN work and the goal of writing this is to share these strategies in the hopes that what has worked for us will help others achieve the same success.

Getting Started - The Basics
This week I'll start with the basics; those industry best practices that are logical and do-able.
1/ Identify the important data
For us it was Customer type data and we started with the basics; name, address, city, province/state, country, phone, fax, email, website.
2/ Profile the important data
Data profiling is just another word for data analysis. Get an extract of the data and start with the most basic analysis. Is it complete? How much of it is blank?
3/ Find someone who cares
For us it meant someone in IT, as IT has known about data quality issues for a long time. Better, would be someone on the business side, but take what you can get.
4/ Communicate your results
Find a way, any way to communicate what you've found. Post the results on your intranet, send them in an email (interesting profiling results tend to get forwarded), or post them by the printers or water cooler. More information coming later on some of the 'purple cow' methods that worked for us.
5/ Define the data
Start gathering and documenting the basic definitions and business rules for the important data and share it. You would not believe how people will thank you.

This was our approach and we only used some basic Microsoft tools like excel for the profiling and word for the definitions. Today we have an enterprise data management tool for the profiling and a wiki with over 900 corporate definitions. Not bad for 2 1/2 years.

There are a lot of other things that can and need to be done as well, but the methods described above is a good start and you can't go wrong.

In the coming blogs I'll talk about the results of each of the above and explain how we logically progressed to where we are today. I'll also share some fun 'purple cow' stuff that we've done with amazing results.

Next Week: What did we find when we profiled our important data and what did we do about it?

Thoughts to ponder: Be a yardstick of quality. Some people aren't used to an environment where excellence is expected.
~Steve Jobs~