How do you ensure that the data is 100% clean apart from manual review?
Hi!
So I am working on cleaning up our customer data quality to arrive at a customer masterdata. I tried to check for duplicates, nulls, invalid email formats and phone numbers, etc. I also tried to review with business some logic, like an inactive customer cannot have an active subscription etc.
However, my problem is when just skimming the data, I still see some weird data quality issues-- like a full name and last name combined (i.e., last name is made redundant and entered in both full name and last name), some company names have zzzz or are named customer, some first names have Mr and Mrs, etc. Is this the part where AI will be useful? Or is there a more deterministic and appropriate approach for this?
What are your thoughts?