lseif@sopuli.xyz to Programmer Humor@lemmy.ml · 5 months agoIf AI can now speak Italian, it can certainly replace us...sopuli.xyzimagemessage-square35fedilinkarrow-up113arrow-down10cross-posted to: notmineworld@lemmy.world
arrow-up113arrow-down1imageIf AI can now speak Italian, it can certainly replace us...sopuli.xyzlseif@sopuli.xyz to Programmer Humor@lemmy.ml · 5 months agomessage-square35fedilinkcross-posted to: notmineworld@lemmy.world
minus-squarestingpie@lemmy.worldlinkfedilinkarrow-up1·5 months agoCan’t find the exact source–I’m on mobile right now–but the code for the gpt-2 encoder uses a utf-8 to unicode look up table to shrink the vocab size. https://github.com/openai/gpt-2/blob/master/src/encoder.py
Can’t find the exact source–I’m on mobile right now–but the code for the gpt-2 encoder uses a utf-8 to unicode look up table to shrink the vocab size. https://github.com/openai/gpt-2/blob/master/src/encoder.py