lseif@sopuli.xyz to Programmer Humor@lemmy.ml · 5 months agoIf AI can now speak Italian, it can certainly replace us...sopuli.xyzimagemessage-square35fedilinkarrow-up19arrow-down10
arrow-up19arrow-down1imageIf AI can now speak Italian, it can certainly replace us...sopuli.xyzlseif@sopuli.xyz to Programmer Humor@lemmy.ml · 5 months agomessage-square35fedilink
minus-squarestingpie@lemmy.worldlinkfedilinkarrow-up1·5 months agoCan’t find the exact source–I’m on mobile right now–but the code for the gpt-2 encoder uses a utf-8 to unicode look up table to shrink the vocab size. https://github.com/openai/gpt-2/blob/master/src/encoder.py
Can’t find the exact source–I’m on mobile right now–but the code for the gpt-2 encoder uses a utf-8 to unicode look up table to shrink the vocab size. https://github.com/openai/gpt-2/blob/master/src/encoder.py