cm0002@suppo.fi to Programmer Humor@programming.dev · 2 months agoTech bros having their production environments nuked by AI is never gonna get oldsuppo.fiimagemessage-square160fedilinkarrow-up1947
arrow-up1947imageTech bros having their production environments nuked by AI is never gonna get oldsuppo.ficm0002@suppo.fi to Programmer Humor@programming.dev · 2 months agomessage-square160fedilink
minus-squareegregiousRac@piefed.sociallinkfedilinkEnglisharrow-up15·2 months agoWhat evaluation setup lets the thing being tested see the test results and gives access to the test source code?! Slop evaluating slop.
minus-squaremyotheraccount@lemmy.worldlinkfedilinkarrow-up7·2 months agoThe model is so smart, we let it write it’s own evaluation setup /s
minus-squareAwesomeLowlander@sh.itjust.workslinkfedilinkarrow-up1·2 months agoGiven that they mark it as cheating, isn’t that essentially part of the test in itself?
What evaluation setup lets the thing being tested see the test results and gives access to the test source code?!
Slop evaluating slop.
The model is so smart, we let it write it’s own evaluation setup /s
Given that they mark it as cheating, isn’t that essentially part of the test in itself?