gpjt/8xa100m40-lore-1-baseline-disabled
Text Generation • 0.2B • Updated • 199
LoRE
Models using low-rank bottlenecks for the embeddings or the output head. See https://www.gilesthomas.com/2026/10/low-rank-vocab-matrices
-
-
gpjt/8xa100m40-lore-10-both-corrected-smart-init
Text Generation • 0.1B • Updated • 190 -
gpjt/8xa100m40-lore-11-output-only-corrected-smart-init
Text Generation • 0.1B • Updated • 211 -
gpjt/8xa100m40-lore-2-both-ordinary-init
Text Generation • 0.1B • Updated • 225 -
gpjt/8xa100m40-lore-3-both-smart-init
Text Generation • 0.1B • Updated • 242 -
gpjt/8xa100m40-lore-4-input-only
Text Generation • 0.1B • Updated • 242 -
gpjt/8xa100m40-lore-5-output-only
Text Generation • 0.1B • Updated • 238 -
gpjt/8xa100m40-lore-6-new-seed-baseline-disabled
Text Generation • 0.2B • Updated • 226 -
gpjt/8xa100m40-lore-7-new-seed-input-only
Text Generation • 0.1B • Updated • 233 -
gpjt/8xa100m40-lore-8-new-seed-both-smart-init
Text Generation • 0.1B • Updated • 242 -
gpjt/8xa100m40-lore-9-new-seed-output-only
Text Generation • 0.1B • Updated • 264