How do you ensure that the data is 100% clean apart from manual review?

Hi!

So I am working on cleaning up our customer data quality to arrive at a customer masterdata. I tried to check for duplicates, nulls, invalid email formats and phone numbers, etc. I also tried to review with business some logic, like an inactive customer cannot have an active subscription etc.

However, my problem is when just skimming the data, I still see some weird data quality issues-- like a full name and last name combined (i.e., last name is made redundant and entered in both full name and last name), some company names have zzzz or are named customer, some first names have Mr and Mrs, etc. Is this the part where AI will be useful? Or is there a more deterministic and appropriate approach for this?

What are your thoughts?

reddit.com
u/Arethereason26 — 3 days ago

How do you ensure that the data is 100% clean apart from manual review?

Hi!

So I am working on cleaning up our customer data quality to arrive at a customer masterdata. I tried to check for duplicates, nulls, invalid email formats and phone numbers, etc. I also tried to review with business some logic, like an inactive customer cannot have an active subscription etc.

However, my problem is when just skimming the data, I still see some weird data quality issues-- like a full name and last name combined (i.e., last name is made redundant and entered in both full name and last name), some company names have zzzz or are named customer, some first names have Mr and Mrs, etc. Is this the part where AI will be useful? Or is there a more deterministic and appropriate approach for this?

What are your thoughts?

reddit.com
u/Arethereason26 — 3 days ago

How do you ensure that the data is 100% clean apart from manual review?

Hi!

So I am working on cleaning up our customer data quality to arrive at a customer masterdata. I tried to check for duplicates, nulls, invalid email formats and phone numbers, etc. I also tried to review with business some logic, like an inactive customer cannot have an active subscription etc.

However, my problem is when just skimming the data, I still see some weird data quality issues-- like a full name and last name combined (i.e., last name is made redundant and entered in both full name and last name), some company names have zzzz or are named customer, some first names have Mr and Mrs, etc. Is this the part where AI will be useful? Or is there a more deterministic and appropriate approach for this?

What are your thoughts?

reddit.com
u/Arethereason26 — 3 days ago

What does your day-to-day look like as data managers? What are the things you wish you knew before?

Hi! I have been asked by my current boss to become a data manager and lead our team. I will be handling a mix of analysts, engineers, architects and even developers.

​

I understand that it is very different for each role and company, but I just wanted to get some perspective on what your day-to-day looks like as a data manager (or even chief data officer, or VP of Data).

​

​

What are the things you wish you knew before when starting in the role?

reddit.com
u/Arethereason26 — 20 days ago

What does your day-to-day look like as data managers? What are the things you wish you knew before?

Hi! I have been asked by my current boss to become a data manager and lead our team. I will be handling a mix of analysts, engineers, architects and even developers.

​

I understand that it is very different for each role and company, but I just wanted to get some perspective on what your day-to-day looks like as a data manager (or even chief data officer, or VP of Data).

​

What are the things you wish you knew before when starting in the role?

reddit.com
u/Arethereason26 — 20 days ago

Analytics Center of Excellence? Thoughts & Experience?

In our strategy discussion with CIO, the thought of establishing an analytics center of excellence has been raised. The goal is to have a single point of contact and a well-defined org structure under analytics. It also helps raising visibility

reddit.com
u/Arethereason26 — 1 month ago

Analytics Center of Excellence? Thoughts & Experience?

In our strategy discussion with CIO, the thought of establishing an analytics center of excellence has been raised. The goal is to have a single point of contact and a well-defined org structure under analytics. It also helps raising visibility internally. What are your thoughts and experience on this?

reddit.com
u/Arethereason26 — 1 month ago

What are the most important things to remember when responsible for platform migration?

We are migrating our CRM, and have been assigned to take the lead in data quality & reporting. I am feeling a bit overwhelmed and excited. For those who have such experience, what can you advise?

reddit.com
u/Arethereason26 — 2 months ago

What frameworks you are using to assess data maturity? What do you think are the strong signs that an organization has high data maturity?

Hi! My CTO and I, a data analyst, wanted to plan for a high-level data strategy to improve the data culture within the organization. As you know, it begins with assessing the current data maturity level of one's organization and narrowing the gap.

I am searching for different frameworks, but I do not see a common one. In addition, I also wanted to get your thoughts about what makes an organization be considered data-mature.

reddit.com
u/Arethereason26 — 2 months ago

What frameworks you are using to assess data maturity? What do you think are the strong signs that an organization has high data maturity?

Hi! My CTO and I, a data analyst, wanted to plan for a high-level data strategy to improve the data culture within the organization. As you know, it begins with assessing the current data maturity level of one's organization and narrowing the gap.

I am searching for different frameworks, but I do not see a common one. In addition, I also wanted to get your thoughts about what makes an organization be considered data-mature.

reddit.com
u/Arethereason26 — 2 months ago

I am torn in my career whether to keep being a generalist or start working into specialization. I am a senior data analyst already but I have only worked as a generalist as either I am the sole data analyst at the company, or I was hired for a specific project.

reddit.com
u/Arethereason26 — 2 months ago