Facebook
Twitter
LinkedIn
Whatsapp
Will data engineering be automated in future?
By - Kaustubh Katdare • 7 months ago • 13k views
No, data engineering will not be 100% automated in future. However, several aspects of data engineering will be aided by AI; but still require human intervention.
The AI has seen unbelievable advances in recent times and people are worrying if AI robots will replace the data engineering in future. The data engineering roles are specialists roles.
Replaced by robots - a website keeps track of the jobs that are being replaced, and will be replaced in future suggests that the chances of automation in data engineering are very slim: about 2.9%. This clearly indicates that data engineering won't be fully automated at least in the next 10-15 years.
Data engineering involves collection, validation, conditioning and maintenance. Each of these steps requires a specific skill-set that's better applied by the humans. The amount of subjective decision making involved in the process is an indicator that AI won't come after data engineers easily.
Let me know your thoughts. Do you think data engineers need to worry about getting their jobs replaced by AI?
Replies
-
Kaustubh KatdareWill AI replace data engineers?
AI will not fully replace data engineers. As discussed, only certain components of the entire data engineering workflow can be automated. Look at the typical functions of data engineers -
- Prepare data, verifying it, cleaning it etc.
- Build data pipelines to migrate data from source to destination; which could be a cloud on on-prem server.
- Deploying pipelines at client sites and/or on the cloud.
AI can take care of some parts - like verification, cleaning of data. But for mission critical projects, human intervention will be needed.
Data engineers will adopt AI to automate certain tasks under their observation. DE will see a good inclusion of AI in several of the workflows; and engineers need to ensure that they are aware of latest trends and tools in this domain.
Note: Only logged-in members of CrazyEngineers can add replies.
Other questions
Can an engineer become a doctor? If you got forced into studying engineering and want to go back to your first love - medicine (MBBS), what are your options? Let's discuss.
Yes, an engineer can become a doctor. But it won't be easy. Engineering degrees worldwide are mostly 4-year degree courses and medicine too requires an average of 4-5 years followed by mandatory internship. Not everyone has the patience, money and time to dedicate a decade only to learning.
If you are an engineer or engineering student who wants to...
I was recently asked if a data engineer can become a data science. How difficult is it for data engineers to switch career to data scientist and what type of training is involved. Looking for responses from data engineers and data scientists.
Elon Musk has completed his takeover of Twitter and one of the new changes he's proposed is $8/mo for the Twitter blue tick.
Do you think this move is justified? Let me know!
When it comes to building a successful business - does age matter? Well, the data shows that it does. The average age of tech entrepreneurs at the time of IPO is about 43.3 years!
Now that's totally contrasting to the news that we read daily. The Internet is full of success stories of entrepreneurs who became successful business owners and millionaires in their early 20s or 30s. But the data shows the truth - the older you are, the more are your chances of starting a successful business.
What do...
The IT and technology industry is going through massive layoffs. The latest news being Salesforce laying off 2500 employees and Meta planning to show pink slip to, if I'm not wrong, 11000 employees worldwide.
How long do you think these layoff cycles will last? Someone said this will continue till January 2023. What are your thoughts?