Using LM Studio to Run LLMs Easily, Locally and Privately

LM Studio makes it easier to find and install LLMs locally. You can also interact with them in the same neat graphical user interface.
Warp Terminal

Break free from limited cloud-based AI models that are too pricey and do not respect your privacy by installing LLMs directly on your computer. This empowers you with greater control over your data, all without relying on big tech companies.

My previous articles explored building your own Private GPT or running Ollama, empowering you with advanced open source LLMs. However, these solutions often require running lengthy commands in the terminal.

LM Studio streamlines the process entirely. Imagine a user-friendly application akin to the familiar interface of ChatGPT, where you can effortlessly download and utilize various Large Language Models.

Installing LM Studio on Linux

βœ‹
LM Studio Linux app is currently in the beta stage.

LM Studio provides an AppImage for Linux users. If it's your first time installing an AppImage program, you can read our AppImage guide.

You can download the AppImage from the LM Studio website's homepage.

LM Studio website
LM Studio's official website

Exploring LM Studio

Now let's explore the LM Studio and try running some AI models for fun:

LM Studio running in Linux
LM Studio first look

As you can see, on the top we have a search bar to look for any AI models, you can even paste the direct URLs of repositories from Hugging Face.

πŸ’‘
Hugging Face is an AI company renowned for its open-source libraries, like Transformers, that facilitate easy access and development of advanced machine learning models for natural language processing and other tasks.

Downloading the first AI Model

It is quite easy to download AI models in LM Studio. Just search for the model, look for the relevant specs and requirements, and download it.

For example, I am running this on my laptop with 16GB RAM on a Ryzen 5 5500U with no dedicated GPU; thus, I would look for some small models with 2B to 7B instruct parameters.

Search for LLMs in LM Studio
Searching for Llama 2

Okay all this information might intimidate you; let me explain what is happening here:

  • I searched for the Llama 2 model in the search bar at the top
  • The left pane shows you the different models relevant to your search
  • Once a model is selected, you can see its specs and download options on the right pane

If you look closely, you will notice that there are four options that says "FULL GPU OFFLOAD POSSIBLE". I will choose a model from those options only because LM Studio already analyzed my system resources and suggested the most suitable options.

Downloading LLMs in LM Studio
LLaMA 2 downloading pane

Running LLaMA 2 model

Once an AI model has been downloaded, let's start chatting with the LLM.

Load the model first and then all you have to do is go to the "Chat" section (chat bubble emoji in left), at the top, press the drop down menu that says "Select a model to load" and from below you can select any model you have downloaded.

Loading LLaMA 2 model to chat in LM Studi
Loading LLaMA 2 model to chat

Now that you have selected your model, it's time to chat with your personal, totally private, offline AI assistant.

Chatting with AI model in LM Studio
Chatting with AI Model (Click to Expand)

I have changed the default system prompt to a personalized one i.e. "You are a French language assistant AI" and I must say, the response was quite accurate.

I know that was a prompt you weren't expecting, but I'm learning French, so bear with me.

Let's ask our model to write a docker-compose file for WordPress with MySQL database:

Writing docker-compose file in action

It's already been a minute and it's still going:

Same docker-compose file after 2 minute

Okay, I don't want you guys to feel the pain of waiting for it to finish. It took 4 minutes and 56 seconds. No wonder AI computing is expensive. You need good hardware (read GPU).

This was the response if you are wondering:

Some AI models take a long time to response
Finally a response after 4 minutes of struggle

Final Thoughts

If you have a powerful system with plenty of RAM and dedicated GPU(s), I highly recommend installing LM Studio.

You have full control over your data since it runs locally on your machine. LM Studio also features a user-friendly interface and allows you to experiment with the latest AI models beyond what companies typically offer. This is truly a win-win situation.

And yes I am well aware the system I use for testing was quite slow but I was targeting the average Joe and yes I will try to run this on a Raspberry Pi because why not, that is the fun of tinkering with technology.

If you have suggestions for more AI tools, feel free to comment.

About the author
Abhishek Kumar

Abhishek Kumar

I'm definitely not a nerd, perhaps a geek who likes to tinker around with whatever tech I get my hands on. Figuring things out on my own gives me joy. BTW, I don'tΒ useΒ Arch.

Become a Better Linux User

With the FOSS Weekly Newsletter, you learn useful Linux tips, discover applications, explore new distros and stay updated with the latest from Linux world

itsfoss happy penguin

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to It's FOSS.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.