Top five developing issues in data science and AI revealed by surveys of data executives

Surveys of data executives have revealed the top five developing issues in the field of data science and AI, shedding light on the current thoughts and actions of those closely involved in these areas.

Firstly, while there is a high level of excitement surrounding generative AI, it is not yet delivering economic value to the organisations that adopt it. Although many believe generative AI has transformational potential, only a small percentage of companies have fully implemented it in production at scale.

Secondly, data science is transitioning from an artisanal activity to an industrialized process. Companies are investing in platforms, processes, and methodologies to increase productivity and deployment rates. This shift towards industrialization is aided by automation and the reuse of existing data sets, features, and models. Furthermore, Machine Learning Operations (MLOps) systems are utilised to monitor the accuracy of machine learning models.

The third issue revolves around data products, with two versions dominating discussions. While some organisations consider data products to include analytics and AI capabilities, others view them as separate components, focusing solely on reusable data assets. Clarity in the definition and discussion of data products is crucial to ensure effective product development.

The fourth issue highlights a diminishing prominence of data scientists. As related roles emerge, such as data engineers, machine learning engineers, and data product managers, the demand for specialised data scientists is reducing. The rise of citizen data science is allowing non-experts to leverage automated machine learning tools, thus further reducing the need for professional data scientists. However, certain aspects of data science still require the expertise of professionals.

Lastly, there is a trend towards integration and consolidation in data, analytics, and AI leadership roles. Rather than having separate chief data and analytics officers, organisations are now combining these functions within broader technology and digital transformation roles. The aim is to provide clearer leadership and collaboration in data and technology-oriented services. However, there is a lack of collaboration within organisations, as C-level executives reported relatively low levels of collaboration with other tech-oriented leaders.

Source: Sloan Review – MIT