Determined AI makes its machine learning infrastructure free and open source – TechCrunch

Machine learning has quickly gone from niche field to crucial component of innumerable software stacks, but that doesnt mean its easy. The tools needed to create and manage it are enterprise-grade and often enterprise-only but Determined AI aims to make them more accessible than ever by open-sourcing its entire AI infrastructure product.

The company created its Determined Training Platform for developing AI in an organized, reliable way the kind of thing that large companies have created (and kept) for themselves, the team explained when they raised an $11 million Series A last year.

Machine learning is going to be a big part of how software is developed going forward. But in order for companies like Google and Amazon to be productive, they had to build all this software infrastructure, said CEO Evan Sparks. One company we worked for had 70 people building their internal tools for AI. There just arent that many companies on the planet that can withstand an effort like that.

At smaller companies, ML is being experimented with by small teams using tools intended for academic work and individual research. To scale that up to dozens of engineers developing a real product there arent a lot of options.

Theyre using things like TensorFlow and PyTorch, said Chief Scientist Ameet Talwalkar. A lot of the way that work is done is just conventions: How do the models get trained? Where do I write down the data on which is best? How do I transform data to a good format? All these are bread and butter tasks. Theres tech to do it, but its really the Wild West. And the amount of work you have to do to get it set up theres a reason big tech companies build out these internal infrastructures.

Determined AI, whose founders started out at UC Berkeleys AmpLab (home of Apache Spark), has been developing its platform for a few years, with feedback and validation from some paying customers. Now, they say, its ready for its open source debut with an Apache 2.0 license, of course.

We have confidence people can pick it up and use it on their own without a lot of hand-holding, said Sparks.

You can spin up your own self-hosted installation of the platform using local or cloud hardware, but the easiest way to go about it is probably the cloud-managed version that automatically provisions resources from AWS or wherever you prefer and tears them down when theyre no longer needed.

The hope is that the Determined AI platform becomes something of a base layer that lots of small companies can agree on, providing portability to results and standards so youre not starting from scratch at every company or project.

With machine learning development expected to expand by orders of magnitude in the coming years, even a small piece of the pie is worth claiming, but with luck, Determined AI may grow to be the new de facto standard for AI development in small and medium businesses.

You can check out the platform on GitHub or at Determined AIs developer site.

More here:

Determined AI makes its machine learning infrastructure free and open source - TechCrunch