How to Build Your First SLM
Train a small language model to sound like you — without breaking the bank.
Large language models (LLMs) are great, but they’re not personal. They don’t know your tone. They don’t match your style. And they certainly don’t speak like your grandfather, your best friend, or your brand.
I mean, let’s be honest — right now, LLMs still feel like novelties. They’re impressive, but impersonal. Useful, but heartless. When you ask an LLM to write a story or a message, it often comes back sounding like a generic assistant. It lacks voice. It lacks warmth. It lacks you.
That’s where Small Language Models (SLMs) come in — compact, local, customizable models that you can train on your own data. Think of them as chatbots that sound like you, not like a Silicon Valley cheerleader.
This article walks you through how to build your first one.
You don’t need a PhD. You don’t need a data center. You just need the right ingredients, the right tools — and a voice worth preserving.
Step 1: Choose a Base Model
Start with a pre-trained, open-source model. These models already “know” language — your job is to adapt them.
Popular options include:
- Phi-2 (Microsoft): Small and powerful, especially for instructional tuning
- Mistral: Versatile and supports fine-tuning well
- TinyLlama: Lightweight and fast
You can find these models on Hugging Face or use them locally via tools like Ollama or LM Studio.
Where This Thing Lives
You have two options:
1. Local — Run the model on your own computer with tools like:
2. Cloud — Use platforms like:
Some tools have built-in UIs. Others are command-line only. Pick what you’re comfortable with.
You don’t need a fancy front-end — but you can build one yourself later on.
Local vs. Cloud: Pros & Cons
Local:
- Free after initial setup
- Full privacy — nothing leaves your machine
- Limited by your own hardware
- Requires some installation and space
- Instant response (no network delay)
- Not easily scalable
Cloud:
- Easy to start from anywhere
- Access to powerful GPUs
- Data privacy depends on provider
- Costs scale with use (can get expensive)
- May introduce response delays
- Easy to scale for teams or heavy use
Running in the cloud gives you access to stronger hardware — but you’ll trade privacy for power and pay by the hour.
Common Cloud Costs (as of 2025)
- Google Colab Free: Free, but limited RAM and GPU timeouts
- Colab Pro+: ~$49/month with access to A100 GPUs (great for LoRA fine-tuning)
- Hugging Face Inference Endpoints: Starting at ~$0.06/hr depending on model size
- AWS SageMaker: Pay per compute instance — flexible but pricey if unmanaged
Pro Tip: What Happens After Download?
After choosing a model (like Phi-2), you’ll typically download it into a folder on your computer — Documents, Desktop, or an external hard drive are all fine. You’ll open an app (like LM Studio) or a terminal window and “point” the software to that model file. If you’re using command-line tools, that means typing a few simple commands — like telling it where to find the model and what interface to load.
You don’t need to build your own UI — most tools include one. If it’s command-line only, think of it like an old-school text adventure: you type in what you want, and it replies.
Step 2: Build Your Dataset
The model needs to learn your voice. That means feeding it examples.
Great source material includes:
- Personal blog posts
- Emails and letters
- Text threads
- Journal entries
- Screenplays or scripts you’ve written
- Interviews, podcasts, or voicemails (transcribed)
Keep formatting clean. Use plain text or JSONL.
Pro Tip:
Training doesn’t make you that person. It learns from their phrasing and structure — not their genius.
You can train on Batman Begins and sound a bit like Christopher Nolan. But you won’t be Nolan. More importantly, using copyrighted scripts without permission is a legal gray area. Be smart about what you feed your model.
If you’re building a model for someone else (like a parent or client), use their actual voice — not a celebrity’s.
Step 3: Fine-Tune the Model
You’re not retraining from scratch. You’re doing LoRA fine-tuning — adjusting a few parameters to adapt the model to your style.
Tools like PEFT, QLoRA, and Axolotl make this easier.
Think of LoRA as adding a new “accent” or perspective to the model — fast and reversible.
You’ll need a GPU (or rent one via Google Colab) and about 50–500 samples to start seeing results.
Step 4: Run Inference Locally
Now test it.
Use your fine-tuned model in a chatbot or prompt environment. Tools like LM Studio or GPT4All let you load the model and talk to it immediately.
Ask questions. Give it scenarios. See how well it matches your voice.
Want to share it? You can export your model and share with collaborators or run it in a local app.
Pro Tip:
“Inference” is the process of using a trained model to generate outputs — like text replies — based on new input. Once you’ve trained your SLM to sound like you, inference is how you use it.
Running inference locally gives you full control, no latency from internet calls, and no usage fees. It means every prompt stays on your machine — which is huge for privacy and security. And because local inference is often faster, it creates a smoother, more responsive creative loop.
Final Thoughts
This isn’t just for software engineers. It’s for:
- Writers who want a creative partner
- Filmmakers who want a script doctor
- Designers building assistants for their brand
- Family members preserving a loved one’s voice
You don’t need to invent the model. You just need to guide it.
Starter Resources
Model Hosting & Inference:
Fine-Tuning:
Data Prep Tips:
- Clean your text.
- Keep formatting consistent.
- Group by speaker if conversational.
Legal Note: Don’t train on copyrighted content you don’t own. Use original work or get permission.
Now What? Owning Your Model Means Owning the Future
You just trained a model to sound like you. So what’s next?
- Creative Confidence: Even if you never share it, you’ve built a tool that mirrors your voice. That means you should trust AI more — or at least understand how AI itself is created — because it feels like you.
- Collaboration: Working with others? Your model becomes a co-writer, a first-draft generator, or even a creative sparring partner. Bring it into the room.
- Monetization (Soon): The market isn’t mature yet but imagine this — a $5/month subscription for fans of your writing. Or embedding your model into a tool others can use. The moment personal AI goes mainstream, your voice will already be trained, tuned, and ready.
And if you want to go further? This model could become the actual brain of your very own AI agent — tuned to your tone, your style, and your goals (We’ll talk about agents in future articles.)
Next Up: Creative Prompting 201 — how to shape tone, voice, and nuance.