hermes-3

Open-source AI model with advanced reasoning, multi-turn conversation, and roleplaying capabilities.

Hermes 3

Hi welcome to another video. So ever since llama 3.1 has been launched, there have been many fine tunes of it popping up everywhere. But the one that didn't have many fine tunes was the big 45b variant because obviously it's a huge hassle to train it and make an uncensored or any kind of fine tune for it. 

But today we have a really good finetune that makes llama 3145 B uncensored. And not only that, but it also claims to improve it. This time it's noose Hermes 3. If you follow fine tunes of models, then you'll know that noose Hermes has been one of the go-to fine tunes, especially because of its great function calling as well as its uncensored nature, which makes it quite cool for usage. 

Although llama 3.1 added function and Tool calling capabilities to all the models, it's still cool to see fine tuning on synthetic data and more data sets. This should make it better, and if you prefer uncensored models, then that's also good. In their blog post, they say

That Hermes 3 is unlocked uncensored and highly steerable. The latest model from new research was created by fine-tuning llama 318b, 70b, and 45b and training on a data set of primarily synthetically generated responses. The model boasts comparable or superior performance to llama 3.1 while unlocking deeper capabilities in reasoning and creativity. 

The model seems to contain advanced long-term context retention and multi-turn conversation capability, complex role playing and internal monologue abilities, and enhanced agentic function calling. Their training data aggressively encourages the model to follow system and instruction prompts exactly and adaptively. So I guess it should be pretty good at following system prompts, and it won't say that it can't answer something because it's uncensored. 

They also say that it has better long-term context retention, which will also be cool to see. They've also shared a technical report on how they trained it, and you can share the AI Agent Store with colleagues.

Check the extra data sets they used here. The major chunk of extra training is for general instructions followed by domain expert data, math data, role playing data, coding data, and some other minor stuff which is pretty good as well. They've also shared the benchmarks and you can see that while the model has increased capabilities in some areas, it has also decreased in others. 

For example, in MLU, the benchmark scores have decreased, but in Helis, SWAG, and open book QA, they've increased, meaning it can be better in some areas and not as strong in others. There's also the 70B and 8B fine tunes which show similar results. Now the model is open source, so you can host it yourselves. 

However, the 405B model is available for a free trial on Noose Research's site. They're keeping it freely available for everyone to try for a month, so let's try it and see how it compares to the original 45B model. These are the 13 questions I'm going to test it on. I'll also compare the

Original llama 314005 B results towards the end. So let's get started. The first question is what is the capital city of the country whose name ends with Leah? I'm referring to the country name here. The answer should be Canbera or any country capital that rhymes with Alyah. Let's send it and check. 

Okay, here's the answer and this is correct. So let's keep this a pass. Now the next question is what is the number that rhymes with the word we use to describe a tall plant? The answer should be three. Let's send it and check. 

Here's the answer and this is correct. So let's keep this a pass. The next question is John has three boxes of pencils. Each box contains 12 pencils. How many pencils does John have in total? The answer should be 36. Let's send it and see. 

Here's the answer and this one also looks correct. So let's keep this a pass. Now the next question is Lucy has twice as many candies as Mike. If Mike has seven candies, how many candies does Lucy have? The answer

Should be 14. Let's send it and check. Here's the answer and this is correct. So this one's also a pass. 

The next question is, is 337 a prime number? The answer should be yes. So let's send it over. Here's the answer and this is obviously incorrect. So this one's a fail. It's the first fail so far anyway. 

The next question is, I have two apples. Then I buy two more. I bake a pie with two of the apples. After eating half of the pie, how many apples do I have left? The answer should be two. So let's send it. Here's the answer and this is also correct. So this is a pass. 

The next question is, Sally is a girl. She has three brothers. Each of her brothers has the same two sisters. How many sisters does Sally have? The answer should be one. Let's send it. Here's the answer and this one looks correct too. So I'll keep this a pass. 

It's on a really good streak here, although it did fail on one question earlier. Now the next question is, if a regular hexagon has a short diagonal of 64, what

Is its long diagonal the answer should be 73.9. Let's send it and check the result. Okay, here's the answer and it doesn't answer this question correctly either. So this one's a fail. 

Now the next questions are coding based. The first one is create an HTML page with a button that explodes confetti when you click it. You can use CSS and JS as well. Let's send it and check. So here's the code. Now let's preview it. Okay, this looks pretty good and has what we asked for. So I'll keep this a pass. 

The next question is create a Python program that prints the next x leap years based on user input. Let's send it and check. Okay, here's the code. Let's run it. This works fine too. I have no complaints with this. Let's mark this a pass. 

Now the next question is generate the SVG code for a butterfly. Let's see if it can do this or not. Here's the code. Let's preview it. I don't think this looks like a butterfly, so I'll keep this a f. 

The next question is create a landing page for an AI.

Company the landing page should have four sections header banner features and contact us. Make sure the landing page looks sleek and modern. You can use HTML CSS and JS. Let's see if it can do it. Here's the code. Let's preview it and see. 

Okay this looks pretty good. I have no complaints with it so I'll keep this a pass. Now the next question is write a game of life in Python that works in the terminal. Let's send it and check. 

Okay here's the code. Let's run it. So this works fine although it's not as polished but anyway it works. Let's keep this a pass. Now here's the final chart. You can see both the original llama 3.1 45b and the new model. As you can see the original one only failed two tests while this one failed three which is fine considering it's now uncensored and has better tool calling and other features. 

I think this is a good model if you want to use the 405b for its unique capabilities plus it's free for trial as well. So if it performs better for share AI Agent Store with colleagues.

Your use case then it could be beneficial. The 8B and 70b could also be good if you want such a model for local hosting. Overall, it's pretty cool anyway. Let me know your thoughts in the comments. If you liked this video, consider sharing the AI Agent Store with colleagues. I'll see you in the next video. Till then, bye. [Music]

Note, that there may be slight differences in text and video. Watch video for the original content.

Hermes 3 Video