Collaborating through open source to advance AI – TechRadar

Artificial intelligence is one of those hype phrases that comes with its fair share of baggage: will its potential ever be realized; will it enhance humans, or make them obsolete; is it really that revolutionary?

One area of the debate that is often overlooked - one of the more positive aspects of modern innovation in fact - is the way big tech companies like Google, Amazon, Facebook and Microsoft are working together to help progress AI. These companies have been the focus of much criticism over the last few years, consolidating their influence and dominating specific parts of our lives but when it comes to AI, something is different. That something is open source.

The sheer number of open source tools available to developers - from libraries to frameworks, IDEs, data lakes, streaming, model serving and inference solutions, and even the recent end-to-end tool aggregator, Kubeflow, means businesses can now harness all the knowledge they have accumulated over the years. In other words, open source has become that common ground for companies that would traditionally be seen as competitors. They are already working closer than many suspect, and it is the unique combination of AI and open source that will foster better relationships moving forward.

The benefits of open source technology to IT development have been well documented over the years. The first open source program, the launch of Netscape Navigator in 1998, is said to be one of the initial starting points for this trend. The strategy Netscape chose was to emphasize the business potential of sharing the softwares source code. As with science, if all researchers kept their methods secret, progress and innovation would take place much more slowly. With developers racing to deliver the next big thing, secure and easy-to-deploy software frameworks are essential to supporting this.

However, there are numerous barriers for businesses looking to develop successful AI-based technologies. It is no secret that AI and machine learning development can be an expensive process. Not only that, but development requires significant computing power and data sets in place to build and train an advanced model. The open source community offers potential solutions to these challenges by encouraging collaboration as well as expertise and resource sharing. For example, open source software allows IT management teams to access frameworks, data sets, workflows, and software models in the public domain and as such reduces training costs. At the same time, the open source community is always monitoring the code for flaws and vulnerabilities - adding an extra layer of security and also making such concerns a common responsibility.

In another front, the rise of ops methodologies has greatly increased the efficiency of developers to bring solutions into production. As an example, Kubernetes - the open source platform which automates the deployment and management of containerized applications - has become a mainstream technology for enterprises to get into DevOps, and is now being extended to Machine Learning ops (MLops), allowing complicated AI workloads to be kept up to date.

It will come as little surprise that big tech companies have traditionally been private with their source codes, libraries and methodologies. Which poses the question as to what makes AI the differentiator for these giants to begin revealing methods from the core of their businesses and unleashing their own open source APIs?

Essentially, the progression of AI remains paramount, and big tech companies have made leaps and bounds in developing the technology over the past few years. Open source allows any developer or IT team to facilitate cheaper, faster, more flexible and secure deployment. Development through open source helps accelerate the adoption of numerous frameworks and software solutions through support from a large community of contributors. So, with big tech embracing open source, their work can then be further developed, explored, adapted and improved. Looking to the future where AI is expected to be ingrained into everyday life, open sourcing AI will foster innovation and reach maturity even sooner.

Google is one company leading the way, having made its popular machine learning framework, TensorFlow, open to the public. This subsequently led to the creation of TensorFlow Extended (TFX) which matured into Kubeflow, an open source project designed to enable using machine learning pipelines to orchestrate complicated workflows running on Kubernetes, all based on Googles internal method.

Meanwhile, Facebook has open-sourced DeepFocus, its AI-powered framework for rendering natural, realistic focus effects in virtual reality (VR), and the Microsoft Cognitive Toolkit has also embraced open source with the ultimate aim of training deep learning algorithms to function like the human brain.

Its also important to remember that all big tech companies are made up of people. In particular, the machine learning field has a long history of being very open and cooperative. Those running the ML labs inside of Google, Facebook and Microsoft have been pioneering the field for decades and have always worked fairly transparently and cooperatively. A key reason we have seen such major advancement is precisely that effective cooperation.

Ultimately, trust is the key factor here. Big tech companies have seen the trust in them dwindle because of the consolidation of personal data they hold, and the perceived notion of the unnecessary power and influence it allows them to hold. But now, these companies are becoming completely open as it relates to the development of this technology, which will form the backbone of our future.

Even over the past six months which have been typified by massive world change as a result of COVID-19, tech collaborations have emerged, such as Apple and Google jointly working on a contact tracing solution.

Going forward, these ongoing relationships will likely spark a new era whereby big tech and even public sector organizations work together in order to foster innovation and help overcome crises. And with such established tech companies betting so heavily on the openness of AI, it is clear that its development will continue to transform and flourish in the near future.

Link:
Collaborating through open source to advance AI - TechRadar