Software developers are now using artificial intelligence to write their code. They are finding that AI is as flawed as human beings.
GitHub, a Microsoft subsidiary that offers tools for hosting code and collaborating on it, released a beta version last June that used AI to aid programmers. The program called Copilot will automatically guess what you are trying to do and then write it down.
Alex Naka is a data scientist for a biotech company who signed up to try Copilot. He says that the program can be very useful and has transformed his work. He says that it allows him to spend less time scouring the internet for API docs and examples on Stack Overflow. It feels a bit like my work has changed from being a generator to one that discriminates it.
Naka discovered that there are many ways errors can sneak into his code. He says that there have been instances when I didn't notice a subtle error in a proposal. It can be difficult to find this information, possibly because the errors seem different from mine.
AI may produce faulty code, which could lead to high risks. NYU researchers recently examined the code generated by Copilot. They found that for certain tasks, where security is critical, around 40% of the code contained security flaws.
Brendan Dolan Gavitt, a NYU professor involved in the analysis, said that the figure was a bit higher than I expected. However, Copilot wasn't trained to write code. He was taught to produce text that would fit a given prompt.
Copilot and similar AI-powered tools could be the beginning of a revolution in how software developers code. AI is being used to automate mundane tasks. There is a growing interest. Copilot, however, also points out some of the dangers of current AI techniques.
It makes mistakes that are different from the ones I would make.
Dolan-Gavitt discovered that the Copilot plugin code contained a list restricted phrases. These restrictions were likely put in place to stop the system from copying code that was already known or blurting out offensive words.
Oege de Moor is vice president of research at GitHub, and one of the copilot developers. He says security has been a concern since the beginning. According to Oege de Moor, the NYU researchers only cite a subset code that has security flaws.
De Moor created CodeQL, which is a tool that automatically detects bugs in code. To ensure that their work is secure, GitHub recommends developers use CodeQL and Copilot.
OpenAI, a well-known AI company that is doing groundbreaking work in machine learning, developed the AI model for GitHub. Codex is an artificial neural network that predicts the next character in text and code. To learn how to write code, the algorithm read billions upon billions of lines stored on GitHub.
OpenAI built an AI coding tool using Codex. It can do some amazing coding tricks. It can convert a typed instruction such as "Create an array of random variables between one and 100" and return the largest to working code in multiple programming languages.
GPT-3 is another version of OpenAI. It can produce coherent text on a subject but can also regurgitate biased or offensive language from the dark corners of the internet.
Some developers have wondered if AI could automate their work through Codex and Copilot. As Nakas's experience has shown, it takes a lot of skill to use the program as developers often have to tweak or vet its suggestions.
