• LaughingLion [any, any]@hexbear.net
    link
    fedilink
    English
    arrow-up
    4
    ·
    15 hours ago

    All the big models are trained on vast datasets that include copyrighted material and so on.

    But localized finetunes almost all are trained on curated data from other models. IE, you plug in a fancy prompt into another, smarter model to generate a ton of content you then use to train the LORA for the smaller model. Then you merge that LORA into the smaller model. In a way, they are stealing from thieves to get their finetunes, in addition to abliteration (decensoring).