This two-person research project investigates whether reinforcement learning can encourage useful code qualities beyond passing tests. It compares correctness-focused training with added Ruff style rewards, an AI judge, and their combination.
My contribution
- Implemented reward experiments, including Ruff-based style checks, combined rewards, runtime experiments, and an AI judge.
- Added structured logging and evaluation tooling to inspect training behaviour and generated outputs.
- Helped consolidate the experiments into a public archive with analysis scripts, provenance records, and checks that recompute saved results.
- Built the Code / Choice application used for human comparison.
Research design, experiments, and conclusions were collaborative. These contributions are my part of that shared work.
What the evidence supports
The public archive contains 12 formal runs across four conditions and three seeds, with 2,400 held-out outputs. In those saved runs, style rewards increased Ruff compliance, while correctness did not improve uniformly.
These are descriptive output-level rates, not independent task counts or pass@k. Training timeouts affected one style run, and the small human comparison involved the two research authors. Neither establishes a broad claim about developer preference or judge accuracy.
Public materials
The evidence map connects the reported results to their saved records and limitations. The repository includes scripts for checking those records without retraining. Exact reproduction of historical GPU training has not been verified.