Name: Jan AI
Uploaded: 2024-12-17T15:33:32.951Z
Duration: 2 min 33 s
Description: An open-source AI assistant that operates entirely offline, providing privacy-focused AI interactions on your device.

jan-ai

An open-source AI assistant that operates entirely offline, providing privacy-focused AI interactions on your device.

Jan AI

Hello guys. In this video I'm going to discuss about jan. A and what it is all about. Uh with the help of this open source tool you will be able to turn your computer into an AI computer. Some of the features of using jan.

AI are you'll be able to run any AI models like Lama or mistra directly on your device for enhanced privacy. No need for an internet connection. Keep all your data and processing locally. 

Second is that you will be able to connect to amazing remote apis like open AI Gro or Mistral API. Uh using uh this particular apis you will be able to get capabilities without needing Advanced Hardware with all the processing being handled in the cloud. 

Uh this uh models that we are specifically going to download over here once it is downloaded it will be completely offline. So conversation preferences and model uses stay on your computer. It is secure, exportable and can be deleted at any point of time. So complete privacy is basically there. 

Jan is optimized for any device from PCS to multi-gpu clusters. It's fast on Nvidia GPU and apple M Series. Right so I specifically have Nvidia GPU on my uh system on my workstation. So let me just go ahead and show it to you. First of all you have options to

Download it in Mac Linux uh Linux uh various version of Linux Mac right and even in Windows. So I will probably go ahead and download with respect to Windows. 

So once I download it I will go ahead and execute execute this uh you will be able to see over here I've already downloaded it. So I will just go ahead and click double click on this. Once I probably double click on this it will start installing and uh once it is installed I will probably go ahead and show you like how this entire uh app actually looks like when you're Lo running in your local machine. 

So here you'll be able to see that preparing updates and this will probably take hardly couple of minutes for the installation to take place. So here uh now you can see this is the simple desktop app uh right now I have not downloaded any specific models over here. 

Let's see whether I will be able to change this into some black screen okay um Services show all so I don't see any options at such so let me just go over here and share AI Agent Store with colleagues.

Explore the Hub so once I specifically explore the Hub any model that you specifically want to import even from hugging face you will be able to import it just download it in your local and just import it over here. Other than that uh you'll be able to see over here is that this this already provides you many many models like M mral instruct 7B then you have llama 3B Q4. The best thing about all the specific models right Gemma model is also there Cloud cloudy 3 highq is also there cloudy 3 Opus is there.

But if you really want to use all this particular model you also need to have an API keys right uh but uh if I probably consider so see over here grock Lama 38b now I don't even have to probably worry about uh you know downloading the models over here because I can directly use it right for all the other models let's say like Mistral and llama I need to download it here I can directly use it right. So let's see one example so grock Lama 38 billion parameters so it is from Gro Gro.

Already provides you Lama 3. Lama 3 8 billion gamma 7B construct and all. So here you'll be also having mral openi GP. You'll also be able to use it. All you need to probably give API over here, right? It'll specifically ask you for the API over here. Also, with respect to GPT 40, here also you have uh apis uh which you can use it.

Okay, so let me just show you one of the example. Uh one I will probably show you uh with respect to this grock Lama 3. Okay, here I'll just click on use. Okay, and let's see. Uh I will ask a message hi. 

Okay, so here you can see invalid API please check your API key settings. So okay, so once I probably go to click on settings, the grock uses API key for authentication. View API key for to do the referencing over here, right? So I will just go ahead and put my API key. 

Recently from the grock itself, I had downloaded my API key. So this is what I'm going to copy and paste it over there. So let me just go ahead and paste it over here. So once I probably do this and again go.

Back to my chat now. Let's go ahead and see this. Uh, my model is basically selected to Grock Lama 3B 8 billion. So now let me go ahead and write hi. So here you can see hi. How is it going? Okay. And I can basically do or write any mer code like can you provide the python code to play tic tac toe game? Right, so I will just go ahead and execute it. Right, so here you can see how fast it is. 

And obviously we know from Gro how you can easily create this. Already in my previous videos I've shown you how you can create the API from Grock itself and all the code will be visible over here. Okay. And, uh, obviously if you are accessing through APIs, uh, then definitely you require an internet connection. But if I go ahead and see the explore the Hub over here, let's see which is the smallest uh model that I can actually use. 

Uh, so this is Gamma 2B uh 2B Q4. So let's download this model. Uh, I think it is hardly 1.40 GB. So just to show it to you so that you can also apply it for other. So I will just go ahead and

Download this so here you can see 2.49%. 

And the best thing about all these models right the format is GG UF. Right when I say GG UF. Right so in hugging face what this GG UF model is all about. 

So if I go ahead and search right what is a ggf file it is a file format for storing model for inferencing with gml and executor based on gml. Ggf is a binary format that is designed for faster loading and saving of money models. Right so definitely go ahead always with respect to ggos some kind of quation and all is also already applied in this. 

Okay so here you can see how quickly the model is specifically installed. Now I'm going to use this particular model. 

Okay and start my conversation over here. So let me just go ahead and write it hi. Right so I will be able to see all these things information starting G model gamma 3 how are you. 

So let me provide provide me uh python code to perform binary search and here you'll be able to see it is using this open source model that is Gemma 2B to provide.

You the entire code right here. You also have lot of other features where you can probably use inferencing parameters. You can set up the temperature. You can set up the top P value. You can set up the max token. The stop token also you can set up. You have model parameters like what should be your prompt and all engine parameters with respect to the context length and all number of GPU layers and all. 

So here you can see completely even though if I remove the internet connection I should be able to work it out right over here. Um once the model needs to be downloaded only for that internet connection is required. But other than that, uh this will be a very amazing thing for all the developers if they quickly want to check out any kind of uh examples any kind of application that they are specifically developing. If they get an access to this to quickly check it out they can actually go ahead, you know. 

So, uh I hope you like this particular video. This was it from my side. I'll see you in the.

Next video have a great day head and uh please go ahead and check it out. Uh the link will be given in the description. This is not a promotion video but definitely this will be helpful if you're specifically using it. 

So yes I will see you in the next video. Thank you take care.

Note, that there may be slight differences in text and video. Watch video for the original content.

Jan AI Video