Forem: I-dodo

Now it's EASY to do function calling with DeepSeek R1

I-dodo — Sat, 22 Feb 2025 00:06:45 +0000

Function Calling with DeepSeek R1

🚀 Excited to share that node-llama-cpp now includes special optimizations for DeepSeek R1 models, improving function calling performance and stability. Let's dive into the details and see how you can leverage this powerful feature.

Basic Example: Function Calling with DeepSeek R1

Here's a quick example demonstrating function calling in action:

import {getLlama, LlamaChatSession, defineChatSessionFunction, resolveModelFile} from "node-llama-cpp";


const modelPath = await resolveModelFile("hf:mradermacher/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q4_K_M");

const llama = await getLlama();
const model = await llama.loadModel({modelPath});
const context = await model.createContext();
const session = new LlamaChatSession({
    contextSequence: context.getSequence()
});

const functions = {
    getTime: defineChatSessionFunction({
        description: "Get the current time",
        handler() {
            return new Date().toLocaleTimeString("en-US");
        }
    })
};


const q1 = "What's the time?";
console.log("User: " + q1);

const a1 = await session.prompt(q1, {functions});
console.log("AI: " + a1.trim());

Recommended Models for Function Calling

Looking for the best models to try out? Here are some top picks:

Model	Size	URI
DeepSeek R1 Distill Qwen 7B	4.68GB	`hf:mradermacher/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q4_K_M`
DeepSeek R1 Distill Qwen 14B	8.99GB	`hf:mradermacher/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q4_K_M`
DeepSeek R1 Distill Qwen 32B	19.9GB	`hf:mradermacher/DeepSeek-R1-Distill-Qwen-32B-GGUF:Q4_K_M`

Pro Tip: The 7B model works great for the first prompt, but tends to degrade in subsequent queries. For better performance across multiple prompts, consider using a larger model.

Usage TIP

Before downloading, estimate your machine's compatibility with the model using:

npx -y node-llama-cpp inspect estimate <model URI>

Try It with the CLI

You can also try function calling directly from the command line using the chat command with the --ef flag:

npx -y node-llama-cpp chat --ef --prompt "What is the time?" <model URI>

What do you think? Is this useful? What are you going to use it for?

Let me know in the comments :)

Open source file sharing!

I-dodo — Mon, 06 Nov 2023 18:42:13 +0000

If you want to share files fast, without any size limit - welcome!

I wanted to share with you a cool project I was working on:

My-folder-online

Limitless peer-to-peer file sharing

Some of it's cool features:

Peer-to-peer sharing (for privacy) 🫣
High speed (using WebRTC) ⚡️
Directory sharing (share a whole directory) 📁
PWA (installable webapp) 📱
FileSystem API (no zip files needed for directory download) 🗂️

https://my-folder-online.vercel.app/

Leveraging WebRTC technology, MyFolderOnline facilitates lightning-fast file transfers, enabling hassle-free sharing of large files and directories. Say goodbye to tedious upload/download processes and enjoy a seamless sharing experience.

If you think this project is cool, please star it on GitHub :)

https://github.com/ido-pluto/my-folder-online

What next?

Improve leading page
Preview of files
Support for mobile file sharing

What do you think you can do with this?
Do you have any ideas for improvement?

Use ChatGPT locally!

I-dodo — Sat, 01 Apr 2023 17:55:05 +0000

If you ever want a local chatgpt like a chatbot that is not affected by the capacity problem that the chatgpt has or you want to ask him something that he is unwilling to answer, you need to hire about this.

There is an open source project called CatAI, it uses the open source model of meta called llama to create a local AI assistant with cool chat UI.

It can answer everything without a second thought.

Usage

It is very simple, first let's install the model:

npm i -g catai

Then we choose the model to download, for this example let's install the 7B model.

catai install 7B

Then let's start using it with the simple command:

catai serve

Good luck!