Were used to hearing that AI and machine learning is hopelessly complex, impossible to implement quickly, and that if you want to get on board the machine learning bandwagon youll need to invest heavily in PhDs, specialists and expensive experts.
This way of thinking is simplistic and behind the times: machine learning is a broad set of technologies, and over the past few months and years there have been huge strides in making machine learnings benefits much more accessible to startups, scale ups and lone developers alike.
Over the past few months Ive spent a great deal of time investigating, learning about and iterating on a number of different machine learning technologies to take advantage of the vast quantities of time series data we have about infrastructure performance from my companys product.
Were collecting billions of metrics every data from hundreds of thousands of systems, all of which can be used to understand patterns and make future predictions. Read on for some easy, actionable advice on how to get started from scratch with machine learning its easier than you think!
Google made headlines in 2015 by open-sourcing TensorFlow, their internal AI and machine learning framework. Released as an open source project, TensorFlow is following the same strategy as Kubernetes provide such a good product that it becomes the industry standard, and offer a hosted, managed cloud version for those who dont want to maintain it themselves.
You can run TensorFlow workloads yourself but Googles Cloud Machine Learning Platform offers a much more optimised version, running on proprietary TensorFlow Processing Unit chipsets. The strategy is all about making Google Cloud the best choice for these jobs.
However, popularity can be deceptive and based on my personal experience TensorFlow is often not the best solution for startups and small companies. TensorFlow is great in that you get a high degree of control over your project but that control comes at a cost. TensorFlow is a framework, and weve found it requires significant data science knowledge and a lot of trial and error in building, iterating and improving your models.
Its not a toolset you should pick up if youre after easy results or plug-and-play functionality. Unless youre a big corporation (which were not) or have the budgets to hire data scientists to get into model development, it might be tricky to secure enough budget to invest in TensorFlow from the start, so youd be much better trying more simplistic managed solutions first.
For companies just starting out, the best place to begin is looking at the managed service solutions from the likes of Amazon, Microsoft and Google. These solutions are much more accessible to generalist teams, and companies that use them get the benefit of vendors updating them and improving service over time. Indeed, your own datasets help to improve the models!
This is because the larger the training data set, the more accurate the models can be. Anyone can play with theoretical models but the truly interesting work comes out of having real data, and this is an advantage the big players have even before they add your data into the mix.
Weve found that Amazon Machine Learning is a great place to start. AML differs from TensorFlow in a number of ways: with TensorFlow, you build your own models and can then execute them against your datasets wherever you like whereas AML requires you upload your dataset to Amazon then use their API to execute queries. The downside is you dont get to control the models and cant see into the workings of the system you rely on Amazon to get it right. This plug and play type approach but is less customised and flexible, so you may end up needing replacing it with something more specialist in the future.
If you need a very particular type of functionality detecting items in a video, speech to text or translation, then there are specialist services from all the cloud providers. These services use machine learning behind the scenes, but you dont need to think about it send over the item for analysis and get the results through an API. These APIs are quite specific and so if they do a good job, you can just leave them to get on with it. Its unlikely youll want to customise them enough to make it worth starting from scratch.
Outside of the big three cloud providers, there are a host of technology startups including Algorthmia, BigML and MLJar aiming to offer machine learning through an API or SaaS application.
Ive seen many companies make the mistake of rushing into machine learning without having a clear use case in mind, and this is a significant error. There are robust ecosystems around each of the above MLaaS platforms, and so youll need to have awareness of the APIs available to you. Tools like Amazon Polly (text to speech) or the Google Cloud Video Intelligence API deliver specialist functionality without requiring a high degree of knowledge as a prerequisite.
Since they are offered as an API, you can mix and match across providers and even test which does a better job where the service is the same. Most people will probably stick with the cloud platform the rest of their infrastructure is hosted on, but thats not always necessary (data transfer cost and latency may become an issue once you hit scale though).
At my company, weve been migrating from IBM Softlayer to Google Cloud and the data transfer fees of (encrypted) traffic across the internet is part of the total cost consideration, and an incentive to complete the move quickly! Once its all within Googles network then the lower (or zero) data fees apply when using their services, and Google is widely considered to have well designed machine learning capabilities.
Ive found the advantage of using machine learning as a service APIs is that any developer can pick them up and start playing. Serious machine learning with TensorFlow requires a lot of time and real data science knowledge, which may be worth investing in over the long term. However, to get something up and running quickly and test the value proposition to your users, there are a variety of options.
Ive had a lot of fun testing out the different machine learning APIs and solutions out there, and this element of fun and discovery makes it much easier to lead a team on a small exploratory project. Ive also found that implementing something like Googles 20 percent time, or even an internal hackathon could also be a good opportunity to get everyone focused on building an initial prototype.
Machine learning is a very over-hyped set of technologies its currently ranked by Gartner as a buzzword, at the very top of their peak of inflated expectations. However theres a vibrant set of technologies under this umbrella term, and you dont necessarily need to have a highly-specialised workforce to take advantage of them. Start small, use the managed services provided by the big tech firms, and youll be surprised by how far you can go.
Read next: 5 Facebook tips and tricks to make your life easier
Follow this link:
You don't need to be an expert to integrate AI in your startup - TNW
- Classic reasoning systems like Loom and PowerLoom vs. more modern systems based on probalistic networks [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Using Amazon's cloud service for computationally expensive calculations [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Software environments for working on AI projects [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- New version of my NLP toolkit [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Semantic Web: through the back door with HTML and CSS [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Java FastTag part of speech tagger is now released under the LGPL [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Defining AI and Knowledge Engineering [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Great Overview of Knowledge Representation [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Something like Google page rank for semantic web URIs [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- My experiences writing AI software for vehicle control in games and virtual reality systems [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- The URL for this blog has changed [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- I have a new page on Knowledge Management [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- N-GRAM analysis using Ruby [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Good video: Knowledge Representation and the Semantic Web [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Using the PowerLoom reasoning system with JRuby [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Machines Like Us [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- RapidMiner machine learning, data mining, and visualization tool [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- texai.org [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- NLTK: The Natural Language Toolkit [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- My OpenCalais Ruby client library [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Ruby API for accessing Freebase/Metaweb structured data [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Protégé OWL Ontology Editor [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- New version of Numenta software is available [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Very nice: Elsevier IJCAI AI Journal articles now available for free as PDFs [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Verison 2.0 of OpenCyc is available [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- What’s Your Biggest Question about Artificial Intelligence? [Article] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Minimax Search [Knowledge] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Decision Tree [Knowledge] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- More AI Content & Format Preference Poll [Article] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- New Planners Solve Rescue Missions [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Neural Network Learns to Bluff at Poker [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Pushing the Limits of Game AI Technology [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Mining Data for the Netflix Prize [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Interview with Peter Denning on the Principles of Computing [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Decision Making for Medical Support [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Neural Network Creates Music CD [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- jKilavuz - a guide in the polygon soup [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Artificial General Intelligence: Now Is the Time [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Apply AI 2007 Roundtable Report [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- What Would You do With 80 Cores? [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Software Finds Learning Language Child's Play [News] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Artificial Intelligence in Games [Article] [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Artificial Intelligence Resources [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Alan Turing: Mathematical Biologist? [Last Updated On: April 25th, 2012] [Originally Added On: April 25th, 2012]
- BBC Horizon: The Hunt for AI ( Artificial Intelligence ) - Video [Last Updated On: April 30th, 2012] [Originally Added On: April 30th, 2012]
- Can computers have true artificial intelligence" Masonic handshake" 3rd-April-2012 - Video [Last Updated On: April 30th, 2012] [Originally Added On: April 30th, 2012]
- Kevin B. Korb - Interview - Artificial Intelligence and the Singularity p3 - Video [Last Updated On: April 30th, 2012] [Originally Added On: April 30th, 2012]
- Artificial Intelligence - 6 Month Anniversary - Video [Last Updated On: April 30th, 2012] [Originally Added On: April 30th, 2012]
- Science Breakthroughs [Last Updated On: April 30th, 2012] [Originally Added On: April 30th, 2012]
- Hitman: Blood Money - Part 49 - Stupid Artificial Intelligence! - Video [Last Updated On: April 30th, 2012] [Originally Added On: April 30th, 2012]
- Research Members Turned Off By HAARP Artificial Intelligence - Video [Last Updated On: April 30th, 2012] [Originally Added On: April 30th, 2012]
- Artificial Intelligence Lecture No. 5 - Video [Last Updated On: April 30th, 2012] [Originally Added On: April 30th, 2012]
- The Artificial Intelligence Laboratory, 2012 - Video [Last Updated On: April 30th, 2012] [Originally Added On: April 30th, 2012]
- Charlie Rose - Artificial Intelligence - Video [Last Updated On: April 30th, 2012] [Originally Added On: April 30th, 2012]
- Expert on artificial intelligence to speak at EPIIC Nights dinner [Last Updated On: May 4th, 2012] [Originally Added On: May 4th, 2012]
- Filipino software engineers complete and best thousands on Stanford’s Artificial Intelligence Course [Last Updated On: May 4th, 2012] [Originally Added On: May 4th, 2012]
- Vodafone xone™ Hackathon Challenges Developers and Entrepreneurs to Build a New Generation of Artificial Intelligence ... [Last Updated On: May 4th, 2012] [Originally Added On: May 4th, 2012]
- Rocket Fuel Packages Up CPG Booster [Last Updated On: May 4th, 2012] [Originally Added On: May 4th, 2012]
- 2 Filipinos finishes among top in Stanford’s Artificial Intelligence course [Last Updated On: May 5th, 2012] [Originally Added On: May 5th, 2012]
- Why Your Brain Isn't A Computer [Last Updated On: May 5th, 2012] [Originally Added On: May 5th, 2012]
- 2 Pinoy software engineers complete Stanford's AI course [Last Updated On: May 7th, 2012] [Originally Added On: May 7th, 2012]
- Percipio Media, LLC Proudly Accepts Partnership With MIT's Prestigious Computer Science And Artificial Intelligence ... [Last Updated On: May 10th, 2012] [Originally Added On: May 10th, 2012]
- Google Driverless Car Ok'd by Nevada [Last Updated On: May 10th, 2012] [Originally Added On: May 10th, 2012]
- Moving Beyond the Marketing Funnel: Rocket Fuel and Forrester Research Announce Free Webinar [Last Updated On: May 10th, 2012] [Originally Added On: May 10th, 2012]
- Rocket Fuel Wins 2012 San Francisco Business Times Tech & Innovation Award [Last Updated On: May 13th, 2012] [Originally Added On: May 13th, 2012]
- Internet Week 2012: Rocket Fuel to Speak at OMMA RTB [Last Updated On: May 16th, 2012] [Originally Added On: May 16th, 2012]
- How to Get the Most Out of Your Facebook Ads -- Rocket Fuel's VP of Products, Eshwar Belani, to Lead MarketingProfs ... [Last Updated On: May 16th, 2012] [Originally Added On: May 16th, 2012]
- The Digital Disruptor To Banking Has Just Gone International [Last Updated On: May 16th, 2012] [Originally Added On: May 16th, 2012]
- Moving Beyond the Marketing Funnel: Rocket Fuel Announce Free Webinar Featuring an Independent Research Firm [Last Updated On: May 23rd, 2012] [Originally Added On: May 23rd, 2012]
- MASA Showcases Latest Version of MASA SWORD for Homeland Security Markets [Last Updated On: May 23rd, 2012] [Originally Added On: May 23rd, 2012]
- Bluesky Launches Drones for Aerial Surveying [Last Updated On: May 23rd, 2012] [Originally Added On: May 23rd, 2012]
- Artificial Intelligence: What happened to the hunt for thinking machines? [Last Updated On: May 25th, 2012] [Originally Added On: May 25th, 2012]
- Bubble Robots Move Using Lasers [VIDEO] [Last Updated On: May 25th, 2012] [Originally Added On: May 25th, 2012]
- UHV assistant professors receive $10,000 summer research grants [Last Updated On: May 27th, 2012] [Originally Added On: May 27th, 2012]
- Artificial intelligence: science fiction or simply science? [Last Updated On: May 28th, 2012] [Originally Added On: May 28th, 2012]
- Exetel taps artificial intelligence [Last Updated On: May 29th, 2012] [Originally Added On: May 29th, 2012]
- Software offers brain on the rain [Last Updated On: May 29th, 2012] [Originally Added On: May 29th, 2012]
- New Dean of Science has high hopes for his faculty [Last Updated On: May 30th, 2012] [Originally Added On: May 30th, 2012]
- Cognitive Code Announces "Silvia For Android" App [Last Updated On: May 31st, 2012] [Originally Added On: May 31st, 2012]
- A Rat is Smarter Than Google [Last Updated On: June 5th, 2012] [Originally Added On: June 5th, 2012]