smolagents

A lightweight, open-source library by Hugging Face for building AI agents with minimal code, supporting various language models and secure code execution.

Hey everybody. So this is a super cool library, but it's super new and there's not like any documentation on it whatsoever. Even from the smolagents team. So I figured I'd kill a few birds with one stone here and go over this review exactly what smolagents is, how it works, uh and then give you some kind of like documentation and like a resource guide that you can use as you go through smolagents. 

So starting off with exactly what this is, it's um exactly as you would think, right? It's it's a it's an agent platform. Uh and then uh they build it for its simplicity. Um it's kind of um their goal within it. It's very easy to install. It's just pip installing the package and then you define what you want the agents to do. 

Um and then so um keeping in uh line with like everything being open source, it utilizes duck.

Go for its search tool as opposed to Google so you get a free search tool which is really cool. Um, but um, and then it can utilize other tools in a toolbox. And then so it's kind of like if you're familiar with agent platforms, um, it's very straightforward in how that works overall. Right, it's a toolbox and then there's tools in the toolbox. And you can even define custom tools and then um, there's predefined tools and custom tools. And other than that, there's not really a lot more to it. You can call LLM models within the framework. It will default call LLM models. Um, it will default call uh, Bert if you uh, distill Bert if you need to um, make LLM calls. Uh, and you don't actually specify the model. Uh, I tested that out so you have that there. 

Uh, but other than that, it's very straightforward. Um, and then again, there's not a lot of documentation on it. So are any so far? So let's let's like go into um, the agents itself. Right, so first thing I do uh, is based off of uh, what they provide within the GitHub. I install and I like set up small agents and then so install it. I run it uh, and I run it with a simple query uh, which is how many seconds would it take for a leopard at full speed to run across the P de arts in France in Paris. 

And then so uh, first of all, like I would highlight this and I would point this out if it was anyone else. I'll point this out in this instance um, when you pip install the smolagents package, it's small but like uh, with this code here, but like it's installing a lot of packages. Like at the end of this, like and it goes through um, and then like I mean all of this execution takes like this is standard, right? Well, I mean it's it like you see all these installed packages at the end like like uh, all of this right like right here. 

Um, and then so uh, it's just it's a lot of packages that it installs and then most of them are very straightforward and expected. Right, like radio fast API light LLM is a big package in and of itself. Um, e2b code interpreter um, safe https sounds pretty good. I know I need the duck Dugo search so like https so I know I need some of these and a lot of these. But like it's just I mean it's a lot it's doing and installing and again if it was like any if it was a small entity like I'd be like I'd highlight this and be like like I I I would not trust this overall. Right, cuz it's it's doing so much with the.

Package installs um but again it's hugging face and and they're like they reason why they put it like they lay it out very specifically in the GitHub that uh they want this to be compact and lightweight. A mission accomplished there right. 

And so there's a trade off like it's a if it's compact and lightweight on the front end it still has to install packages on the back end. And so it does it installs quite a bit just highlighting that. 

So let's move on from that though uh and let's go through and let's actually run it right. So we have our query uh that we run and then let's see how how this model performs. 

And then so it it performs and it operates this query exactly as you would expect an AI agent too. So it's going to break this query down this uh larger query into smaller subqueries. 

And then so the first thing that it needs to do then the first question it ask itself is um uh how many seconds would it take for Le to run at full speed. So it asks the full question and then it needs to know the length of the P the arts in Paris. 

And then so it does this right so it does a duck.

Go search via Wikipedia and it gets the length which is uh 509 ft um or 155 M. Uh and then it goes through and then it it's uh the it gets the whole Wikipedia article and then from there uh it cross references that with the top speed of a leopard. Uh and then the first result that it gets back is uh 56 to 60 km which is about 35 to 37 mph as the top speed of the leopard. Then it gets other data about the leopard that is insignificant to its query. Uh and then its next step is to uh put this into to code to calculate it out right. So uh it calculates out that the the top speed of a leopard is 58 km per hour. 

Uh so it takes essentially like the average between 56 and 60. Uh and then it um converts that to meters per second so 3.6 meters per second. Um and then it calculates the length of the point the Arts as 155 M. And then so it outputs the answer as 9.62. Uh as the the the response so it's final answer is 9.62. So pretty cool there how it works overall right. 

And then so it works very straightforward and exactly how an agent would operate. And then so if you have like very simplistic tasks like this you you could run your agent all day long. Let's go with a more complex task right. So in this instance for our more complex task uh let's say uh we want to know the average uh building price of a house in San Francisco. We'll we'll we'll do that as our query right. 

And then so again it it's got to go through all this massive package installs and then we go through there. Uh and then it does the query and then so first thing it needs to do is break this down into a few different subcategories right. So it needs to know average cost of C in San Francisco average cost of Steel average cost of bricks and then it searches those right. So that's what it's going to use for its building cost materials. 

It's going to get results back for all of these and it's just utilizing and it's going to average out the results that it gets from this web from these different web pages. And from these different web pages it's going to get different results right. And it doesn't upfront and we run into the first problem in highlighting this when it comes to agents right. Like this agent is going to treat all of these results the the same whether it's from 2024 2023. Uh the uh how good the like website is comparative to each other you can build out and you can make your agent more.

Complex but then you're building out and making your agent more complex by doing that. In this instance, we're not doing that right. So it's just going to average out all of these, treat all these websites the same. Uh, a website that's a website and that's all it is. It gives us data and then so it's going to average out the cost right. 

So uh, for cement it's $3 per uh square foot. Um, for the minimum cost and our maximum cost will be $4. Uh, so it's going to average it out to $3.50 and our still cost same thing right. Uh, 440 cents is the minimum that it got, a125 is the maximum and then so it's going to average it out. 

Um, same thing with bricks. I found that the max was 75, the minimum is 55 and then so it's going to average it all out. It's going to take a house of 2,000 sare ft. It's going to average out our labor costs and assume that our labor costs are based off of our material costs at 2.25x material cost. 

And so uh, upfront and highlighting this, this is a bad calculation for labor cost right. Like um, this going to work in this instance and for what it's utilizing it for in these calculations just because it's how the math works out. So if you just look at the end result of what it calculates here, it's going to calculate 48,000 for the labor cost compared to 21,750 for the material cost which is good right. 

And but like I don't like you would want a better labor calculation than just simply flat out 2.25x your uh mature your labor cost right. It just seems to me uh, you would want that to be more complicated but that's what it's doing in this instance. 

And then so if you don't check it and you just let it run, that's what's going to happen right. And then like uh, that's the next big thing that I want to highlight within this is that like so these agents they're going to run and like without supervision right. Is kind of like the big thing that is missing within here. 

Uh, like there's small agents which is awesome and kind of where we're at. But like where we're at like there's not a small supervisor within this right. Like and that's kind of like the to me the missing element that is making this uh more like not more than novelty for my approaches uh when it comes to actual real world applications right. 

But then so let's go into like before we dive further into that, uh let's dive into like an actual real world use case right. Like uh, I'll

Demonstrate what it will do. So let's let's have it make an API call. And then in this instance I'm going to have it make an API call to Catfax just because I don't want to deal with API keys and all of that. I just want something simple. 

And then I want it to uh visualize the data. Uh and then in this instance I'm going to do something really cool. Okay, cuz I did this before and I did this once. So I'm going to have it visualize the data, but then I'm going to have it go through and logic check itself. So it's going to go through and it's going to create a report based off of this data. It's going to pull the data, summarize it, and then pull a report. 

And it's going to ask itself like okay, is this summary and is this visualization like would a human actually find this useful? If no, it's going to do it again and it's going to try it again. And so uh we run it. Uh no model was just was supplied so it defaults again to Dilbert as I mentioned at the top. If you don't in this instance cuz it needs an LM model so I didn't define it. 

Uh I wanted to see what would happen. It defaults to Bert which was fine. And then so uh it fetches the fact, it analyzes it, it generates the data for a report, and then it uh creates a summary, generates the final report, evaluates the usefulness of the report, finds that the report isn't useful, uh and then regenerates the report. 

And then so here's our final report that it does. And then so it just pulled a cat fact, right? Just a random cat fact. Um and then so which is that cats have been domesticated for half as long as dogs have been. It found that this tends to be uh it found like very highly is a negative sentiment um within this. 

And then it plots the mean, the max, um and the minimum with regards towards the sentiment analysis that it found uh within this particular. It's basing this graph off of analysis here and then it gives you uh its charted analysis. 

And then so that's kind of a full breakdown as to um what all this does, right? So bringing you from smolagents does bringing you from the most simplistic example to more complex use cases and bringing in LLM models. 

And then if you want to, it would be easy to modify this code, bring in your own LLM model, bring in uh like GPT-3 as a supervisor if you want to within this, right? And that's where it could get um more complex and you could add real complexity.

To this and then it becomes a real agent framework. Um that's really robust in that instance that you're paying a lot of money for. But it would be there for you and very useful in that instance, right? Um and but so highlighting overall, uh what tools are in the toolbox. 

So again this entire framework, bringing it back up a high level, this entire framework is essentially just giving, uh agents in this instance or a machine learning agents or algorithms in this instance access to tools within a toolbox. And then so what exact tools do they have? The first that they first tool that they have is local execution tools, uh which allows them to just run python SN code snippets. 

And then they're like pre-built snippets for like mathematical calculations, data analysis, file manipulations and data transformations like very straightforward, right? Like they're um like the adjustable wrench like your go-to, uh tool. 

And then, uh API integration tools. So like they know how to make API calls and they're very well trained, uh within that. They have an agent that specifically calls that's a specific call agent for API integration tools. So this be like your socket wrench. 

Um managed agents, agents that can call um other specialized agents to complete subtasks. Uh these agents they can be configured so, uh agents I can call agents that's basically it, right? It's like your WD40, uh within the toolbox like, uh whatever you need to do. 

Like if you need like, uh you can daisy chain agents together to perform a lot of tasks. Like and then it's, uh if well this task is very complicated and there's no way that a single agent can do it, well three agents daisy chain together could do it. 

Um and then you can pretty much do a lot of things by doing that just by daisy chaining multiple agents together. And then, uh moving on you have multimedia tools very straightforward, right? Like, uh if you have to deal with images or audio you have an agent that's very specifically trained for images and or audio. 

And then, uh all of these are individualized agents again, right? So they're individual tools like I reference them as different tools within the toolbox and that's exactly how they are, uh right? So then if like if the you need to then, uh have more specialized python libraries so you need to, uh access like, uh nltk or or a mat plot lib anything along those lines you have.

A specific agent that's uh trained for more advanced coding. Right, so you have your basic agent for coding and then your more advanced agent and the and an agent that decides what needs to be done within that uh. And then you have custom tools you can create your own tool and define how that works. 

And then you just have to define the tool you give it a name, a description, and how it operates, the logic of it uh. And then you have your advanced planning tools which are like uh logic-based tools. Right, so this is a uh what is coordinating all of this. 

And then if you need to actually do plans and then so um generate tasks on the fly, generate plans on the fly, etc. That's what this tool is doing. And then data visualizations anything that needs to do that for chart visualizations, etc. 

Uh, and then your final answer tool. Um, so the uh taking all of the data that you're getting in and like ingesting all of this huge amounts of data that it often gets. Uh, as we saw with those queries, right, like those queries are taking in a lot of data back uh from like the Wikipedia search for example. 

Um, and then so it's cross-referencing and then it's just is this your final answer. Then it's just uh querying itself and then that like that's very powerful especially when it comes to these agent models. Right, just adding an additional loop on itself very straightforward. 

Uh, and then uh remote execution tools basically just allow it to uh execute code within a sandbox environment so that it doesn't uh like go off the rails and like actually like mess up uh code in a real-world environment. And then so it's very straightforward as to how this operates. 

And then the last one is uh last here this tool integration workflow is just there for you if you want to again create your custom tools. You define the tool, you add it to the toolbox, and then you define how the agent utilizes the tool. Very straightforward, right? 

But so uh all of that combined and all of this combined is your agent toolbox. And then so the toolbox is again like the entire essence of this. And then each individual tool is combined and can be an individual agent uh or it can be a framework. 

Uh, it doesn't have to be um a probabilistic agent. It could be a very um linear function and only a linear function uh and that could be a tool. Right, so you have linear and probabilistic tools just like you would.

When it comes to any other machine learning task but they break down to tools in this instance for the toolbox. 

And then the machine or the uh agent in this instance is just utilizing the tools that you give it. 

And then it goes through and performs those tasks. 

And then so again like the major drawback to this is that this works wonderfully as long as the agent is executing every single step within this accurately. 

But if there's one step within this that's wrong or that you don't like it's going to throw this all off. 

And then so going back the one thing I don't like about this calculations that it's doing within this is this total labor cost. 

Right I hate this calculation that's doing within this. 

It's doing it every single time and all of these outputs. 

And we're getting this like unless I tell the model to not do this it's going to do this a 100 times a thousand times without supervision. 

Right and that's the the the major drawback to this. 

So be a waste of tokens if I decide like I flat out don't want this data or it's not useful because of that like kind of in very investment calculation based all of that. 

Right and then so really highlighting that as the drawback within this that's kind of where we're at. 

Right like uh you 1 million per should never run this without human supervision. 

Like absolutely have human supervision checking over this even if it's the most basic agents that you're running. 

But you do have agents and you can eliminate humans from a lot of the equation within this to simplify it. 

Note, that there may be slight differences in text and video. Watch video for the original content.

smolagents Video