The emergence of technologies such as AI and machine learning, along with sophisticated analytics, offers opportunities for smart manufacturers to transform their businesses radically to create new product and service offerings while maximising the efficiency of supply chains and processes.

Contemporary computing models such as Cloud and, increasingly, Edge computing release huge amounts of sensor- and device-related data, to help with decision-making.

To succeed, however, companies need to be able to leverage this vast trove of data. In far too many cases, much of the data that is produced at the Edge is discarded, rather than transferred to a core environment for long-term storage.

For now, at least, the sector is lagging many other industries when it comes to capturing this opportunity.

That is a key finding in a report written by Seagate based on IDCs surveys, called ReThink Data: Put More of your Business Data to Work from Edge to Cloud. That study, which surveyed 1500 global enterprises, delved into the performance of several sectors including manufacturing and revealed that the vast majority of data available to organisations simply goes unused.

Manufacturing, despite having one of the highest enterprise data centre footprints and high levels of instrument connectivity, is actually recording one of the lowest levels of data growth compared to other sectors. In part, this is because it is difficult to expand the capacity of all that on-premises infrastructure especially when compared to scalable Cloud infrastructure more common in other sectors.

The irony is that the methods, strategies, and processes that typically make manufacturers successful seem to stop on the shop floor. For an industry steeped in the heritage of automation and integration, evidence suggests that these lessons are lost when it comes to managing data.

The authors of ReThink Data found that somewhat counterintuitively, manufacturing had the lowest level of task automation in data management and the lowest rate for full integration (single platform) of data management functions.

Among the key findings of the report:

The disconnect between assets and data management is not only inhibiting manufacturers from maximising the return from their current plant and equipment, it is also stifling innovation and opening up opportunities for more nimble and aggressive competitors to grab lucrative niches in the market.

So why does this disconnect exist?

A study in 2018 by industry analyst IDC, looking into the integration of Information technology and operational technology, said that many assets on the factory floor remain disconnected.

IDC says that this speaks to a strategic weakness around core enterprise architecture and infrastructure, with the researchers explaining that much current legacy infrastructure simply wont be able to cope with the inevitable growth of connected assets entering the plant.

The temptation for many companies will be to try to implement ad hoc processes to connect and manage assets, leaving them unable to rely on the underlying infrastructure for comprehensive management.

IDC also found that, even for companies that want to solve these issues, there is another problem: their aging workforces were trained for a different era of manufacturing. Too many lack the hard and soft IT skills that are prerequisites for the shop floor of tomorrow.

The Importance of DataOps

As plant and equipment become more digitised, generating vastly more data, the successful manufacturers will be those who can connect the data creators (which can include people and machines) with the data consumers (such as C-Suite and general executives).

Its a discipline in IT referred to as DataOps.

Seagate argues that DataOps should be part of every data-management strategy, which should also include data orchestration from endpoints to the core, data architecture, and data security.

The mission of DataOps is to provide managers with a holistic view of data and to enable users to access and derive the most value from data whether it is in motion or at rest.

This is another area, however, where many manufacturers are lagging. The ReThink Data report reveals that less than a third of manufacturers indicated that they have fully or even partially implemented a DataOps capacity.

The development and deployment of this skillset need to accelerate if manufacturers are to remain at the crest of the competitive curve. DataOps capabilities are essential to leveraging emerging technologies such as AI and machine learning, and to correlate data from core, Cloud, and Edge data sources.

The work of DataOps teams allows data science and data analytics professionals to use technologies like AI to transform data into the information needed by decision-makers.

And as the ReThink Data Report explains, Being able to correlate data from disparate sources is a capability not easily available through other means. Because it is difficult, those organisations able to master it can expect to have an edge over the competition.

More than Technology

The authors of ReThink Data also make clear that DataOps is an important part of implementing the necessary cultural change required to implement new ways of working by facilitating the sharing of data and breaking down organisational silos.

To succeed, however, leaders still need to drive a strategy that implements global standards, global data architecture, and global data management and delivers access to the same analytical tools by global teams.

Seagate says that rolling back reporting functions to IT can provide global tools, capabilities, and solutions that every group can leverage.

The various groups within the enterprise should get out of siloed management of their own data, and allow the IT-instituted tools to do that globally. In doing so, the teams will be freed to make decisions based on insights from reliable, global, accessible pools of data.

The article was written by Andrew Birmingham for the Which-50 Digital Intelligence Unit for Seagate. Members of the DIU provide their insights and analysis for the benefit of our readers. Membership fees apply.

Image source: Photo by Kiefer Likens on Unsplash

Here is the original post:
Manufacturers need to maximise the competitive opportunity of data - Which-50

In the slow process of developing machine learning models, data scientists and data engineers need to work together, yet they often work at cross purposes. As ludicrous as it sounds, Ive seen models take months to get to production because the data scientists were waiting for data engineers to build production systems to suit the model, while the data engineers were waiting for the data scientists to build a model that worked with the production systems.

A previous article by VentureBeat reported that 87% of machine learning projects dont make it into production, and a combination of data concerns and lack of collaboration were primary factors. On the collaboration side, the tension between data engineers and data scientists and how they work together can lead to unnecessary frustration and delays. While team alignment and empathy building can alleviate these tensions, adopting some developing MLOps technologies can help mitigate issues at the root cause.

Before we dive into solutions, lets lay out the problem in more detail. Scientists and engineers (data and otherwise) have always been like cats and dogs, oil and water. A simple web search of scientists vs engineers will lead you to a lengthy debate about which group is more prestigious. Engineers are tasked with construction, operation and maintenance, so they focus on the simplest, most efficient and reliable systems possible. On the other hand, scientists are tasked with doing whatever it takes to build the most accurate models, so they want access to all the data, and they want to manipulate it in unique, sophisticated ways.

Instead of fixating on the differences, I find its much more productive to acknowledge theyre both immensely valuable and to think about how we can use each of their talents to the fullest capacity. By focusing on the things that unify data scientists and data engineers a dedication to timely, quality information and well-designed systems the two sides can foster a more collaborative environment. And by understanding each others pain points, the two teams can build empathy and understanding to make working together easier. There are also emerging tools and systems that can help bridge the gap between these two camps and help them meet more readily in the middle.

MLOps is an emerging area that applies the ideas and principles of DevOps practices to the data science and machine learning ecosystem. It lifts the burden of building and maintenance off of data engineers, while providing flexibility and freedom for data scientists. This is a win-win solution. Lets take a look at some common problems, and the tools that are emerging to more effectively solve them.

Model orchestration. The first major hurdle when trying to put a model into production is deployment: where to build it, how to host it, and how to manage it. This is largely an engineering problem, so when you have a team of data scientists and data engineers, it typically falls to the data engineers.

Building this system takes weeks, if not months time that the data or ML engineers could have spent improving data flows or improving models. Model orchestration platforms standardize model deployment frameworks and help make this step significantly easier. While companies like Facebook can invest resources in platforms like FBLearner to handle model orchestration, this is less feasible for smaller or emerging companies. Thankfully, open source systems have started to emerge to handle the process, namely MLFlow and KubeFlow. Both of these systems use containerization to help manage the infrastructure side of model deployment.

Feature stores. The second major hurdle to taking a model from the lab to production lies with the data. Oftentimes, models are trained using historical data housed in a data warehouse but queried with data from a production database. Discrepancies between these systems cause models to perform poorly or not at all and often require significant data engineering work to re-implement things in the production database.

Ive personally spent weeks building out and prototyping impactful features that never made it to production because the data engineers didnt have the bandwidth to productionize them. Feature stores, or data stores built specifically to support the training and productionization of machine learning models, are working to alleviate this issue by ensuring that data and features built in the lab are immediately production-ready. Data scientists have the peace of mind that their models are getting built, and data engineers dont have to worry about keeping two different systems perfectly in line. Larger corporations like Uber and Airbnb have built their own feature stores (Michelangelo and ZipLine respectively), but vendors that sell pre-built solutions have emerged. Logical Clocks, for example, offers a feature store for its Hopsworks platform. And my team at Kaskada is building a feature store for event-based data.

DataOps. Theres no experience quite like getting paged late at night because your model is behaving strangely. After briefly checking the model service, you come to the inevitable conclusion: something has changed with the data.

Ive had variations on the following conversation more times than I like to admit:

Finding the issue is like finding a needle in a haystack. Fortunately, new frameworks and tools are coming into place that set up monitoring and testing for data and data sources and can save valuable time. Great Expectations is one of these emerging tools to improve how databases are built, documented, and monitored. Databand.ai is another company entering the data pipeline monitoring space; in fact the company published a great blog post herethat explores in greater detail why traditional pipeline monitoring solutions dont work for data engineering and data science.

By using tools to reduce the complexity of asks and by growing empathy and trust between data scientists and data engineers, data scientists can be empowered to deliver without overly burdening data engineers. Both teams can focus on what they do best and what they enjoy about their jobs, instead of fighting with each other. These tools can help turn a combative relationship into a collaborative one where everyone ends up happy.

Max Boyd is a Data Science Lead at Kaskada. He has built and deployed models as a Data Scientist and Machine Learning Engineer at several Seattle-area tech startups in HR, finance and real estate.

Visit link:
How to get your data scientists and data engineers rowing in the same direction - VentureBeat

New Jersey, United States,- Market Research Intellect aggregates the latest research on Data Science and Machine-Learning Platforms Market to provide a concise overview of market valuation, industry size, SWOT analysis, revenue approximation, and regional outlook for this business vertical. The report accurately addresses the major opportunities and challenges faced by competitors in this industry and presents the existing competitive landscape and corporate strategies implemented by the Data Science and Machine-Learning Platforms market players.

The Data Science and Machine-Learning Platforms market report gathers together the key trends influencing the growth of the industry with respect to competitive scenarios and regions in which the business has been successful. In addition, the study analyzes the various limitations of the industry and uncovers opportunities to establish a growth process. In addition, the report also includes a comprehensive research on industry changes caused by the COVID-19 pandemic, helping investors and other stakeholders make informed decisions.

Key highlights from COVID-19 impact analysis:

Unveiling a brief about the Data Science and Machine-Learning Platforms market competitive scope:

The report includes pivotal details about the manufactured products, and in-depth company profile, remuneration, and other production patterns.

The research study encompasses information pertaining to the market share that every company holds, in tandem with the price pattern graph and the gross margins.

Data Science and Machine-Learning Platforms Market, By Type

Data Science and Machine-Learning Platforms Market, By Application

Other important inclusions in the Data Science and Machine-Learning Platforms market report:

A brief overview of the regional landscape:

Reasons To Buy:

About Us:

Market Research Intellect provides syndicated and customized research reports to clients from various industries and organizations with the aim of delivering functional expertise. We provide reports for all industries including Energy, Technology, Manufacturing and Construction, Chemicals and Materials, Food and Beverage, and more. These reports deliver an in-depth study of the market with industry analysis, the market value for regions and countries, and trends that are pertinent to the industry.

Contact Us:

Mr. Steven Fernandes

Market Research Intellect

New Jersey ( USA )

Tel: +1-650-781-4080

Our Trending Reports

Process Analytics Market Size, Growth Analysis, Opportunities, Business Outlook and Forecast to 2026

3D Printing Filament Market Size, Growth Analysis, Opportunities, Business Outlook and Forecast to 2026

Air Brake System Market Size, Growth Analysis, Opportunities, Business Outlook and Forecast to 2026

Anti-Graffiti Coatings Market Size, Growth Analysis, Opportunities, Business Outlook and Forecast to 2026

Astaxanthin Market Size, Growth Analysis, Opportunities, Business Outlook and Forecast to 2026

See the original post:
Data Science and Machine-Learning Platforms Market 2020 Size by Product Analysis, Application, End-Users, Regional Outlook, Competitive Strategies and...