How much data engineering can someone learn by themselves?

I recently joined company as their only dedicated data person. they have other SWEs and a couple other people that pull data and use it, and they’ve only recently built their data lakehouse last year using databricks, but I’d be the only dedicated data person. so any data problem I’d probably deal with eventually. right now I’m dealing with the crm and cleaning it, but eventually I suspect that my duties will increase and I’ll be doing some form of data engineering, so I was wondering if I can essentially learn data engineering myself? or at least well enough?

reddit.com
u/SoggySand297 — 2 days ago

What types of roles did you all come from? Did anyone start in one similar to mine?

I recently started in a crm data position and my role is very data entry and data cleaning heavy. While automation isnt one of the requirements it is something I’m trying to do, but it is a bit harder than expected especially with how messy the data is ie missing data and multiple entries in the crm for the same projects. Im also the only dedicated “data” person and there are a couple of SWEs that only last year build the data lakehouse, so maybe I can get involved in that. I was told that because the role is also new to the company the duties can expand.

reddit.com
u/SoggySand297 — 10 days ago

Am I wasting my time in this role?

Hi, I’ve only recently got a job as a CRM Data coordinator, the duties are pretty simple - clean the data and enter it into the CRM. While I am working on trying to automate some of the processes and the rest can’t completely be automated, I don’t think, there’s lots of very messy data. the role itself didn’t ask for technical knowledge in the jd, only really excel, no SQL or python. the role itself is a new one for the company and while the company has some other SWEs working with the data in the background and they’ve build their data lakehouse only last year, im the only dedicated “data” person. while i know i should probably be more grateful to even have a job right now, I can’t help get this feeling that im wasting my time.

reddit.com
u/SoggySand297 — 11 days ago

When doing entity resolution, is a lot of it spent chasing down people?

Hi, I’m new to all these and this is my first job out of college. Recently I got a job and some of my first tasks are to fix some data in the CRM, some of the data is okay but there’s a lot that is very messy. It’s in excel and I need to enter the data in the excel files into the CRM. The thing is though the data in excel usually contains incomplete information, so I’ll need to look in the CRM and try to figure out in which project the data goes into using dates, supervisors or some other things. Sometimes this works but lots of times there’s multiple projects in the CRM with the same names, dates, and supervisors only slight variations of eachother. If at that point I can’t figure it out I usually message the person in charge of said area if they know, for example if the project belongs to healthcare I’ll message the person that oversees it. Is this bad? Should I be doing something else to not bother them?

reddit.com
u/SoggySand297 — 14 days ago
▲ 6 r/excel

How would you use Excel to match messy project names to CRM records?

Repost with better title**.** Hi, I recently began a job as a data coordinator, my first tasks are basically data cleaning and data entering into a CRM. The problem is that the data isnt very clean. I'll give an example, I am given an excel file with name of a project, date, title, awards - my job then is to to figure out where in the CRM is this specific project and enter the data. The problem is that the excel data doesnt contain the projects ID, and when I try searching the name of the project in the CRM I'll get back something like Fairview Elementary School, Fairview ES, KUSD Fairview, Fairview ES Building A, etc. So essentially I'll have to go into each one of these projects and try to find the right one using other given data from the excel sheet, like dates. Is there a way to speed this process up or am I just going to have to do it manually? Right now what Im doing is going row by row and searching each project in the CRM, looking through the multiple projects given back by the CRM and comparing and contrasting. The entire CRM database contains around 40k, and I think I am able to export it into CSV and Excel, if that helps out. Any advice would be helpful. Thanks.

reddit.com
u/SoggySand297 — 14 days ago
▲ 7 r/excel

Can this task be made easier or automated?

Hi, I recently began a job as a data coordinator, my first tasks are basically data cleaning and data entering into a CRM. The problem is that the data isnt very clean. I'll give an example, I am given an excel file with name of a project, date, title, awards - my job then is to to figure out where in the CRM is this specific project and enter the data. The problem is that the excel data doesnt contain the projects ID, and when I try searching the name of the project in the CRM I'll get back something like Fairview Elementary School, Fairview ES, KUSD Fairview, Fairview ES Building A, etc. So essentially I'll have to go into each one of these projects and try to find the right one using other given data from the excel sheet, like dates. Is there a way to speed this process up or am I just going to have to do it manually? Right now what Im doing is going row by row and searching each project in the CRM, looking through the multiple projects given back by the CRM and comparing and contrasting. The entire CRM database contains around 40k, and I think I am able to export it into CSV and Excel, if that helps out. Any advice would be helpful. Thanks

reddit.com
u/SoggySand297 — 14 days ago

Can this task be made easier or automated?

Hi, I recently began a job as a data coordinator, my first tasks are basically data cleaning and data entering into a CRM. The problem is that the data isnt very clean. I'll give an example, I am given an excel file with name of a project, date, title, awards - my job then is to to figure out where in the CRM is this specific project and enter the data. The problem is that the excel data doesnt contain the projects ID, and when I try searching the name of the project in the CRM I'll get back something like Fairview Elementary School, Fairview ES, KUSD Fairview, Fairview ES Building A, etc. So essentially I'll have to go into each one of these projects and try to find the right one using other given data from the excel sheet, like dates. Is there a way to speed this process up or am I just going to have to do it manually? Right now what Im doing is going row by row and searching each project in the CRM, looking through the multiple projects given back by the CRM and comparing and contrasting. The entire CRM database contains around 40k, and I think I am able to export it into CSV and Excel, if that helps out. Any advice would be helpful. Thanks

reddit.com
u/SoggySand297 — 15 days ago