Review assistance tools

[YadavAkhileshh]

Hey @MakisH, I have been looking at the open PRs and the GSoC contributions have really built up quickly. Keeping up with all of them manually is getting tough. I wanted to suggest integrating CodeRabbit, it is basically an AI reviewer that sits inside GitHub and automatically goes through every new PR. It writes a short summary of what changed and leaves comments on the actual lines of code if something looks off, things like security issues, broken syntax, or messy JavaScript. This all happens before any maintainer even opens the PR. It is free for open source projects and setup is super quick. For PRs that are already open you can just drop @coderabbitai review in the comments and it will pick those up too. It is not going to replace real review obviously but it takes care of the repetitive stuff so maintainers can spend time on things that actually need their attention.

[hanzalahwaheed]

We already use Github copilot for occasional AI reviews in this repository. Not sure if adding one more AI code reviewer would be okay.

[NishantSinghhhhh]

Hey, I'd partially push back on this. I've previously contributed to the Palisadoes Foundation which used CodeRabbit, and the reviews were mostly stale and generic — not driving meaningful changes. It got noisy enough that their deployment was breaking weekly.

Not confident it'd be net positive here. Happy to explore other tooling options though.

[MakisH]

Hi there! Thank you all for the inputs.

In our repositories, two changes are happening at the same time: (1) significantly more attention due to GSoC, (2) higher rate of AI-assisted contributions (which, independent of quality, are typically more verbosely described), due to the year we are in.

As a reviewer, I need to (quickly) understand what each PR does and how good it is. Previously, I could partly rely on the fact that the person that submitted the PR already had a very good understanding of what they are doing while developing the respective contribution, because there was no other way than investing a lot of time on it. This is no longer a safe assumption by default, especially if there is no previously established trust with the contributor. I have to look at more contributions (orders of magnitude more than I expected), more verbose contributions, and of more contributors that I have not yet established any trust with.

I have used the Copilot reviewing feature of GitHub for preliminary or additional reviews. So far, it did help me notice some details, and it partially helped me understand what was going on, especially with technologies I am not very familiar with. But it generally did not make comments on the bigger picture (which is too much to ask from such a tool), and I had the feeling it generally echoed what was already in the PR description, sometimes suggesting going further into the wrong direction.

I would be interested in exploring what any other reviewing-assistance tools can do, but I also think that there are still a few of the "traditional" checks that we could still use to check the quality of a contribution, and some measures to restore some of the self-reviewing assurance of the previous years.

I leave this issue open to collect concrete suggestions, and I would revisit this after the GSoC proposals review is through. Any links to other PRs where you have interacted with such tools would be useful. I know there are many such tools, I would be interested in specific experiences and examples.

[AyushBurde]

hey @MakisH is there any open issue in which i can contribute?

[MuhammadAashirAslam]

Hi @AyushBurde , you can explore the codebase but currently no PRs are accepted . More information at https://precice.discourse.group/t/next-steps-in-gsoc/2827