•1 min read•from Machine Learning
Interactive map of GPT-2's token embedding space - tap any token and explore [P]
32,070 alphabetic tokens from GPT-2-small's WTE, no forward pass and no context.
Works on mobile. Pinch to zoom, tap a token to see its nearest connections, tap a neighbour to walk the graph. Search box to jump anywhere.
Layout is t-SNE over a compressed representation of the embedding table; edges are a minimum spanning tree in that space, so every line is a real nearest-kin relationship,
[link] [comments]
Want to read more?
Check out the full article on the original site
Tagged with
#GPT-2
#token embedding
#WTE
#t-SNE
#embedding space
#minimum spanning tree
#alphabetic tokens
#nearest connections
#machine learning
#graph
#visualization
#interactive map
#token atlas
#embedding table
#forward pass
#context
#mobile
#representation
#algorithm
#data