Will AI Code Review Replace our existing review/scan tools?

Trying out AI Powered Code Review with the goal of understanding is it a replacement for our existing review/scan tools, complement our existing tools, or is it just a fancy name for regular code review?

Code Review and Scanning

Talking about code review/scanning, top of my mind it will be sonarqube, trivy, snyk, ruff, and others. This tools already became top of mind when I do want to implement code scanning to our CI/CD pipeline.

Now, while planning to adding a tool for our CI/CD pipeline, an emerging tools that we see more and more is “AI Powered” Code Review, which am not sure even what is the difference and how the “AI” part works. Or is it just a fancy name for regular code review?

Benchmarks?

As I am new in this AI Code Review things, am not sure which tools are “good”, so I just take my chance by following the benchmarks that are provided in https://codereview.withmartian.com, and I choose coderabbit as the tools that I will use.

SCA and SAST are not being replaced with AI Code review, but it became an independent tools that I will use alongside AI Code review. It scan and catch vulnerabilities, and other static/deterministic issues.

Use case for AI Code Review

At the first approach, am experimenting code review with the goal of helping me catching all of the bugs, and do “AI Review” to catch other issues, but even in my mind, am curious what result that it will spite: “Will it show catching bugs early? will it do dependency scanning? linting? or what?”

And ya, I found that it by default scanning the changes in the diff codebase, which is understandable. If they do always code scanning in all of the code, then the token and time consumption will be wild. But now, as this repo is already has a code and dependency, so it didn’t get catched then? If in the future there is found new vulnerability from the existing dependencies, will it catch it? The default answer is no.

So this is where our SCA and SAST tools come in. They doing their job as usual, but they are not doing it for every single code change, they are doing it for the codebase as a whole, so they can catch vulnerabilities that are not in the diff codebase.

Now, I have question: what do they catch then? It’s a news for me too that they even catching the “not best practices” documentation/decisions. For example, I have a architecture decision where it catch pip audit command that we implemented.

Other than that, we are able to implement coding standards that are adopted in the team. In this case, the example is Clean Code Architecture. I feel that it’s quite nice feature :)

Conclusion

The tools has many feature that I have not yet explored, and I believe there is many possible use cases that I have not yet discovered. But when I am adopting this tools, coding review with standard feature will be the things that I focus first.

For now, I feel that the code review has less impact to me. It’s good tools to have in my toolbox, but I need to take a note: this tools shining where the bottleneck is reviewing code, which are now it’s getting a traction in each day where code generation getting faster, and the bottleneck is code review.