When OpenAI first published a paper on their new language generation AI, GPT-3, the hype was slow to build. The paper indicated GPT-3, the biggest natural language AI model yet, was advanced, but it only had a few written examples of its output. Then OpenAI gave select access to a beta version of GPT-3 to see what developers would do with it, and minds were blown.

Developers playing with GPT-3 have taken to Twitter with examples of its capabilities: short stories, press releases, articles about itself, a search engine. Perhaps most surprising was the discovery GPT-3 can write simple computer code. When web developer, Sharif Shameem, modified it to spit out HTML instead of natural language, the program generated code for webpage layouts from prompts like a button that looks like a watermelon.

I used to say that AI research seemed to have an odd blind spot towards automation of programming work, and I suspected a subconscious self-preservation bias, tweeted John Carmack, legendary computer game developer and consulting CTO at Oculus VR. The recent, almost accidental, discovery that GPT-3 can sort of write code does generate a slight shiver.

While the discovery of GPT-3s coding skills may have been somewhat serendipitous, there is, in fact, a whole field dedicated to the development of machine learning algorithms that can code. The research has been making progress, and a new algorithm just recently took another step.

The algorithm, called machine inferred code similarity (MISIM), is the brainchild of researchers from Intel, Georgia Institute of Technology, University of Pennsylvania, and MIT. Trained on the huge amount of code already publicly available on the web, MISIM can figure out what a program is supposed to do. Then, after finding other similar programs and comparing it to them, MISIM can offer ways to make the program faster or more efficient.

It isnt the first machine learning algorithm to make recommendations or compare similarity, but according to the researchers in a new preprint paper on MISIM, it was up to 40 times more accurate at the task when it went head to head with several of its most advanced competitors.

Near term, the AI could be a useful sidekick for todays programmers. Further out, the field could open programming to anyone who can describe what they want to create in everyday language or bring machines that write and maintain their own code.

The pursuit of computers that can code is almost as old as modern computer science itself. While there have been advances in programming automation, the recent explosion in machine learning is accelerating progress in a field called machine programming.

In a 2018 paper on the field, a group of Intel and MIT researchers wrote, The general goal of machine programming is to remove the burden of writing correct and efficient code from a human programmer and to instead place it on a machine.

Researchers are pursuing systems that can automate the steps required to transform a persons intentthat is, what they want a piece of software to dointo a working program. Theyre also aiming to automate the maintenance of software over time, like, for instance, finding and fixing bugs, keeping programs compatible, or updating code to keep up with hardware upgrades.

Thats easier said than done, of course. Writing software is as much art as it is science. It takes a lot of experience and creativity to translate human intent into the language of machines.

But as GPT-3 shows, language is actually a skill machine learning is rapidly mastering, and programming languages are not so different from English, Chinese, or Swahili. Which is why GPT-3 picking up a few coding skills as a byproduct of its natural language training is notable.

While algorithmic advances in machine learning, like GPT-3, are key to machine programmings success, theyd be useless without good training data. Luckily, theres a huge amount of publicly available code on sites like GitHubreplete with revision histories and notesand code snippets and comment threads on sites like Stack Overflow. Even the internet at large, with accessible webpages and code, is an abundant source of learning material for AI to gobble up.

In theory, just as GPT-3 ingests millions of example articles to learn how to write, machine programming AIs could consume millions of programs and learn to code. But how to make this work in practice is an open question. Which is where MISIM comes in.

MISIM advances machine programming a step by being able to accurately identify what a snippet of code is supposed to do. Once its classified the code, it compares it to millions of other snippets in its database, surfaces those that are most similar, and suggests improvements to the code snippet based on those other examples.

Because MISIM classifies the codes purpose at a high level, it can find code snippets that do the same thing but are written differentlytheres more than one way to solve the same problemand even snippets in other programming languages. Simplistically, this is a bit like someone reading a New Yorker article, identifying its topic, and then finding all the other articles on that topicwhether theyre in Der Spiegel or Xinhua.

Another benefit of working at that higher level of classification is the program doesnt need the code to be compiled. That is, it doesnt have to translate it into the machine code thats executed by the computer. Since MISIM doesnt require a compiler, it can analyze code snippets as theyre being written and offer similar bits of code that could be faster or more efficient. Its a little like an email autocomplete feature finishing your sentences.

Intel plans to offer MISIM to internal developers for just this purpose. The hope is itll prove a useful sidekick, making the code-writing process faster, easier, and more effective. But theres potentially more it can do. Translation between computer languages, for example, could also be a valuable application. It could perhaps help coders update government software written in archaic languages to something more modern.

But Justin Gottschlich, director of machine programming at Intel, has an even grander vision: the full democratization of coding.

Combine MISIM (or something like it) with natural language AI, and future programmers could simply write down what they want a piece of software to do, and the computer whips up the code. That would open programming to anyone with a decent command of their native language and a desire to make something cool.

As Gottschlich told MIT Technology Review, I would like to see 8 billion people create software in whatever way is most natural for them.

Image credit: Markus Spiske /Unsplash

More:

This AI Could Bring Us Computers That Can Write Their Own Software - Singularity Hub