CODACYDE, IDE/CLI Plugin

The idea for creating a plugin arose from Codacy's aim to enhance its role in the development process. Beyond quality and coverage reporting, our suite alerted users to errors detected after a commit or PR was pushed, visible in the Git provider's or Codacy's UIs.

However, there was a missed opportunity to directly engage developers within their IDE environments.

Another requirement was to identify market opportunities for the newly introduced AI feature. The product manager and I would need to assess whether investing resources in integrating the AI feature into the initial MVPs was feasible based on my research.

A wrong approach could compromise the plugin's output quality.

Functional Capacity

Before generating ideas, understanding the limitations is crucial.Codacy initially lacked the capability to trigger an analysis automatically or through the 'Save' command in an IDE; it could only be done manually.

However, this limitation led to an innovative solution: two analysis modes could be separately triggered by clicking on respective buttons.

From the UX perspective, it was acceptable, but we faced delays in the analysis stages. Mapping features showed how immediate reporting like linting would interact with each mode. While launching the MVP with both modes was an option, we needed to decide on our development approach

Persona Building

The service's complexity stems from its diverse audience — engineers rarely use Codacy's UI.

Data shows that the primary users of the web app suite are team leads, senior engineers, and product managers. Creating the plugin would expand our reach to daily code writers, providing more touchpoints.

Competitive Analysis & Usability Testing

I always pilot apps myself to carry out heuristic markups and note my behaviour as a regular user. However, I don't write code and can only assume what engineers find useful in an IDE plugin.

To gather workflow feedback and evaluate competitors' plugins, I invited engineers to participate in the research. They performed specific actions, which we recorded and discussed in interviews.

We tested on both VS Code and IntelliJ, as their different UX guidelines and requirements would influence our decisions

With the recordings and engineers' feedback, I could analyse the data to understand the market better. I divided the configuration into six primary functionalities, each to be tackled separately:

Registration, plugin installation process
Configuration set-up options
Active usage experience
Code analysis triggers
Reporting
Special features, if any

Registration was a major consideration. The plugin wasn't a standalone product with its infrastructure but rather a different shell for the web app, augmented with AI and fix suggestions.

AI Opportunities

At the moment of the research, AI was still weak at writing big pieces of code and failed to understand the context. Generative AI, thanks to its capability to learn from the users and replicate their code writing, was successfully used for repetitive tasks.

After analysing our competitors’ practices, I listed the options that we could explore.

AI generates fix suggestions

Suggestions, ranging from snippets to full functions, provided on the fly based on users' commands and existing code

Vulnerabilities: reliability.

Al shows the outline of a project

A competitor's plugin clarifies code purposes for project understanding, activated by right-clicking.

Vulnerabilities: value is not clear.

AI brings links to documentation

Initially implemented without AI, some competitors integrate this feature to search the web for relevant information.

Vulnerabilities: reliability of open-source documentation.

Documentation was already a feature in Codacy's web app and didn't need AI assistance. Integrating it into the IDE would be straightforward; the main decision was where and when to display the information.

Project outlining seemed unnecessary for the initial MVPs.

The most challenging and intriguing option would be AI-generated fixes, a pattern I observed during my research.

There are two types of analysis plugins: those solely focused on generating AI suggestions and those covering all analysis topics except AI suggestions.

Product Strategy

Goal: define product value, user outcome, and output; propose a product roadmap and MVPs.

Value

Provide early feedback and assistance

Outcome

Expand product coverage

Output

AI-supported IDE plugin for VScode and IntelliJ

Basic plugin features include analysis and security, while secondary features like AI-generated fixes, documentation, and quick fixes based on configuration files are also important.

Following trends and introducing untested functionalities could have jeopardized the entire product. The vision proposed by the PM and me focused on launching the plugin with basic features first, with plans to add complexity in subsequent iterations.

The initial iteration would provide our first feedback to enhance basic features, while the team worked on AI.

Quick fixes based on the configuration file could replace AI fix suggestions, offering an affordable and reliable functionality.

This research helped clarify the long-term goals and opportunities for the product. Facing tough competition, the plugin would undergo multiple iterations to gain audience traction.

See other cases

Iress — Wealth Management & Financial Advisory

Founded in 1993, software is provided to the financial services industry in the Asia-Pacific, North America, Africa, and the UK & Europe, with more than 500,000 users globally.

View case

Stenn — Factoring & Invoice Financing

The largest and fastest-growing online platform for financing small and medium-sized businesses engaged in international trade.

View case