CrazyEngineers
  • No, data engineering will not be 100% automated in future. However, several aspects of data engineering will be aided by AI; but still require human intervention.

    The AI has seen unbelievable advances in recent times and people are worrying if AI robots will replace the data engineering in future. The data engineering roles are specialists roles.

    Replaced by robots - a website keeps track of the jobs that are being replaced, and will be replaced in future suggests that the chances of automation in data engineering are very slim: about 2.9%. This clearly indicates that data engineering won't be fully automated at least in the next 10-15 years.

    Data engineering involves collection, validation, conditioning and maintenance. Each of these steps requires a specific skill-set that's better applied by the humans. The amount of subjective decision making involved in the process is an indicator that AI won't come after data engineers easily.

    Let me know your thoughts. Do you think data engineers need to worry about getting their jobs replaced by AI?

    Replies
Howdy guest!
Dear guest, you must be logged-in to participate on CrazyEngineers. We would love to have you as a member of our community. Consider creating an account or login.
Replies
  • Kaustubh Katdare

    Administrator1yr

    Will AI replace data engineers?

    AI will not fully replace data engineers. As discussed, only certain components of the entire data engineering workflow can be automated. Look at the typical functions of data engineers -

    1. Prepare data, verifying it, cleaning it etc.
    2. Build data pipelines to migrate data from source to destination; which could be a cloud on on-prem server.
    3. Deploying pipelines at client sites and/or on the cloud.

    AI can take care of some parts - like verification, cleaning of data. But for mission critical projects, human intervention will be needed.

    Data engineers will adopt AI to automate certain tasks under their observation. DE will see a good inclusion of AI in several of the workflows; and engineers need to ensure that they are aware of latest trends and tools in this domain.

    Are you sure? This action cannot be undone.
    Cancel
Home Channels Search Login Register