Datdanboi25 (Dan P)

reacted to appvoid's post with 🔥 about 5 hours ago

Post

1,159

You know, I can't believe we are actually doing it, literally advancing the frontier at the edge! We've come a long way since the first models on open slm leaderboard, and, as many things that change, performance is one of the main things that has exploded.

7 years ago, the best we had was GPT-2. Today, we got 90m models beating smollm2! I can't barely see what a future with increasingly powerful small models will look like.

All I can say is that the best is yet to come. And will be agentic.

8 replies

·

reacted to DedeProGames's post with 🔥 10 days ago

Post

3,449

Meet Supra2.5-Ultra-Preview-001. 230M parameters, ~40.4B training tokens.

SupraLabs has released the first preview of its Supra2.5-Ultra family: a compact, Llama-style base model for text completion and small-model experimentation.

What's inside?
• 230M parameters
• 32 transformer layers
• 2,048-token context window
• Grouped Query Attention: 12 attention heads, 4 KV heads
• 49,152-token vocabulary
• SwiGLU activation, RoPE and tied embeddings
• BF16 weights and Hugging Face Transformers support
• Apache 2.0 license

Training
The model was trained on approximately 40.4B tokens from FineWeb-Edu, DCLM-Baseline and FineMath using a single A100 SXM.

Reported zero-shot results
• HellaSwag: 37.18% normalized accuracy
• PIQA: 65.78% accuracy
• ARC-Easy: 57.11% accuracy
• ARC-Challenge: 26.28% normalized accuracy

Want to experiment?
Code on REPO.

Model, weights and usage example:
SupraLabs/Supra2.5-Ultra-Preview-001

4 replies

·

replied to Banaxi-Tech's post 11 days ago

Head up bro, hyped for BM3!!!

reacted to Banaxi-Tech's post with 👍 11 days ago

Post

4,221

Should I even continue making models?

27 replies

·

replied to Hoglet-33's post 12 days ago

Haha I wanna see them sooner

replied to Hoglet-33's post 14 days ago

1:Individually Definitely!

3:a 150m!!!

replied to Banaxi-Tech's post 14 days ago

Bro... I am sorry to break it down to you, but this is mathematically impossible, bcs your context itself eats up all the parameters. you end up with negative parameters (which is absolutely NOT a thing)😭
Your model is your embedding space.

Anyways... good luck with whatever you are doing, man. research is research at the end of the day.

What is bro talm bout

replied to Banaxi-Tech's post 15 days ago

Hyped for BananaMind3!!!

reacted to Banaxi-Tech's post with 🔥 15 days ago

Post

4,697

Well GPT X3 takes the lead. As of right now.

We are now announcing BananaMind 3 🍌! (not ai for those emoji guys)

All models will use BGA (which is almost just NSA) and our BM3X architecture.
Its sizes will be:
BananaMind 3 Flash Lite, 3M parameters at a context of 8K context.
BananaMind 3 Lite, 10M parameters with 16K context.
BananaMind 3 Flash, 25M Parameters with 16K context.
BananaMind 3 Pro, 50M parameters with 24K context.
BananaMind 3 Ultra, 100M parameters with 32K context.
And lastly, BananaMind 3 Max with 150M parameters and 64K CONTEXT.

I can assure you BananaMind 3 Max WILL beat GPT X3 or match it, we won't release it otherwise. We hope for a 40+ INTELLIGENCE INDEX!

BananaMind 3 may also be partnered with dot labs.

We will cancel BananaMind 2.1 and BananaMind 2 Ultra.

As of the BETU SLM Leaderboard we may need to release it after October 11, im very busy right now (even though we said We will release BETU leaderboard before Oct 11 😟)

12 replies

·

reacted to raincandy-u's post with 🔥 17 days ago

Post

3,825

20K parameters can tell a story. 🚀

🤗 We trained a ~20k-parameter Transformer that can actually write stories!

raincandy-u/MacroStories

→ ~50× smaller than the 1M-parameter TinyStories model
→ ~3,000× smaller than AlexNet
→ 81 KB in FP32

yayyy the whole model. ૮ ˶ᵔ ᵕ ᵔ˶ ა

She has a 32-dimensional hidden state, a 378-token vocabulary, and just one decoder block — recurrently applied 4 times with shared weights.

Despite having only 19,969 parameters, she can maintain a simple narrative across 100–300 words: establish a goal, encounter a problem, take relevant actions, and reach an outcome.

She runs extremely fast on CPU — no GPU required. The entire model is tiny enough to load almost instantly! ☺️

3 replies

·

reacted to Hoglet-33's post with 🔥 26 days ago

Post

6,253

Hey everyone! I got sidetracked from my main projects and decided to test out the BananaAll app and see if I could make a small model not regress too much during SFT. Here is what happened:

The base model I chose was BananaMind/BananaMind-2.1-Pico-Preview, and the dataset I used was SupraLabs/SupraThink-Dataset-500x

I trained for 5 whole steps using a LoRA adapter.

Results:
A model that scores better on some benchmarks and worse on others, and still lacks most general capabilities.

You can find the model here: Hoglet-33/Hogleto

Credits:

- Thank you to @Banaxi-Tech for the BananaAll app (works perfectly on Windows and CPU)
- GPT-6 Sol for knowing how to merge some confusing files created by the app
- Myself for the idea
- Someone else somewhere who might have contributed to some of my ideas and might in the future
- And readers like you!

6 replies

·

reacted to danielhanchen's post with 🔥🔥 28 days ago

Post

9,166

Unsloth has surpassed 500M model downloads on Hugging Face! 🦥🤗

Qwen3.8-27B GGUF is already Unsloth’s #1 most-downloaded model ever.

Thanks for all your support! Follow us:

unsloth

1 reply

·

reacted to their post with 🔥 29 days ago

Post

3,066

ForgePlex-M1-6M first model trained on AxiomicLabs TrainWork

ForgeWorks/ForgePlex-M1-6M just dropped from

ForgeWorks , and is the first model to ever be trained on our TrainWork training framework.

Achieving an Intelligence Index of 6.87 and taking #22 in the <10m category on the AxiomicLabs/Open_SLM_Leaderboard, very impressive work for a first model.

Give it some love!

7 replies

·

reacted to Banaxi-Tech's post with ❤️ 29 days ago

Post

4,878

We're excited to release BananaAll, our SLM Super App.

It allows you to do EVERYTHING you need to do to trains SLMs in a single app, no terminal, no 30 chrome tabs.

The train tab allows you to train models, select datasets from presets, and use other ones with auto mapping, model size slider, it automatically generates a training script for you.

Then after you've trained the model or want to compare it to competitors, the evaluation tab, run ARC EASY, ARC Challenge, Hellaswag, PIQA, Arithmark 3, BananaMind Base Bench and more! Simple Results screen.

And lastly the inference tab, run your trained models or others.

Normally you would need seperate apps or scripts for that, but the BananaAll Super App lets you do all of that in a single app.

We also trained a small 2.5M parameter model on 200M tokens of Fineweb edu, The results: BananaMind Base Bench 854 and 53% on PIQA. On only 200M tokens.

Check it out at https://github.com/BananaMind/BananaAll.

25 replies

·

replied to Banaxi-Tech's post 29 days ago

Ooh cool stuff!

reacted to ereniko's post with ❤️ 29 days ago

Post

320

NEW CLAUDE HAIKU MODEL COMING

FINALLY AFTER A YEAR

ANTHROPIC ANNOUNCED HAIKU 5.5

reacted to Enderchef's post with ❤️ 29 days ago

Post

3,051

AxiomicLabs released new benchmark, Tiny Theory of Mind, to test your SLM models' Theory of Mind Intuition!
Check it out and like it!

AxiomicLabs/Tiny_Theory_of_Mind

reacted to Enderchef's post with 🔥 about 1 month ago

Post

3,051

AxiomicLabs released new benchmark, Tiny Theory of Mind, to test your SLM models' Theory of Mind Intuition!
Check it out and like it!

AxiomicLabs/Tiny_Theory_of_Mind

replied to Banaxi-Tech's post about 1 month ago

Id like to see it but not sure if I want to give out my full legal name haha

Dan P

AI & ML interests

Recent Activity

Organizations

Dan P

AI & ML interests

Recent Activity

Organizations

Datdanboi25's activity