Name: Azure AI Foundry
Uploaded: 2025-02-15T07:35:54.119Z
Duration: 2 min 33 s
Description: A comprehensive platform for building, customizing, and managing AI applications and agents at scale.

azure-ai-foundry

A comprehensive platform for building, customizing, and managing AI applications and agents at scale.

Azure AI Foundry

Hey, everyone. I'm Priyanka Vergadia and today welcome to our new video series Sip and Sync with Azure. I've invited Govind to the show today to talk about Azure AI and how AI has been just a topic that every customer is wanting to talk about. 

Govind. >> Yeah, thanks for having me Priyanka. This is on top of everyone's mind, as you can imagine. Literally every customer conversation I go to. This is all they want to talk about. Absolutely happy to demystify some of this. 

>> Perfect. >> It's a lot of options. >> Here's my question. The first thing that I'm sure as you walk into a customer conversation, the first thing that I'm having to answer is I want to get started quickly and now how do I begin? 

>> That's a good question. The easiest way to get started is with our AI foundry. You can go to ai.azure.com and quickly spin up a few resources and with a few clicks, you can have something up and running. It's that easy. 

>> That's amazing.

Walk me through a little bit of what's happening there in the playground and what are some decisions I'm making there. 

First of all, let me ask you what's your favorite drink? 

Oh. Great. Okay. Well, I love latte. 

I'm actually a tea drinker. What we're going to do here is try to get Priyanka to try a tea that is good for latte lovers. What we're going to do is go to ai.azure.com. As you can see, there's a lot of playgrounds here. We're going to pick the chat playground. In the Chat playground, we're going to upload a document Barista manual document. Then we're going to ground the model to that data as well as the instruction telling it to make some recommendations. 

That's pretty much rag retrieval augmented generation. 

That's correct. Exactly. As you just saw it was easy with a few clicks. Now we're going to ask it recommend some tea options for latte lovers. Holla, here you go. 

Okay. Wow. Well, some options that I don't really like. But then there's matcha, matcha I like.

Then masala chai sometimes I mean, not a chai fan, but I can go for masala chai once in a while. Not bad. We did great. 

Great. As you saw, that was super quick. We were able to deploy a web application grounded in your data and that was super easy. Now, there's also, you might want to evolve this experience even more. You might want to customize it even more. You can still do that in AI Foundry without opening up a whole code sandbox. You can just go to Prompt Flow and you can just take the same experience into Prompt Flow and you can then add more integrations there if you'd like. Maybe making some API calls, etc. 

The cool thing here is over the last decade all of us developers did this test-driven development. Now to really make apps reliable, we got to do Eval-driven development and Prompt Flow lets you do that. You can quickly have your Eval set up. You can test, measure, refine your prompts as you add new tools and stuff.

Now if you want to make your AI truly autonomous and agentic, then we have agent runtime that we just announced. With agent runtime, this takes you from a task-oriented AI to a goal-oriented AI where the models can plan, reason, and reflect on their actions. It comes with managed RAC and managed function calling so your agents can make complex decisions while staying grounded in your data.

That's amazing. We basically saw that Playground can get us quickly started with any idea that I might have. We just literally converted me to a tea drinker which is absolutely fascinating in just a few clicks. I'd love for you to give it a try. The links are in the description. 

But really now it gets me thinking about the second thought that the customers bring up which is I want more control. I want to know exactly which model I'll use. Maybe I'll use multiple models. Maybe I want an application that uses this kind of database. I just want more control for my application.

Where do I go for that? Great question again. This comes up again all the time because customers want more control and you know they have trade-offs that they need to make. 

Again, we have a whole decision tree and we actually have a whole blog the link is below and they'll break down this complex decision tree that you're seeing on the screen. What you have here are essentially a few key choices that you need to make. 

Where to host it. Well, before we go there, actually pick your model first, whether it's multi-modal or if you care about cost or latency and the embeddings that you need. Then you need a database so I can remember the interactions as well as the data that it needs to be grounded on. 

Then you want to host it somewhere. We have choices like app service, container apps, functions, etc. Again, go to our blog. We'll break it down and give you details and the trade-offs that you need to make. 

Awesome.

Really if I summarize that decision tree with a use case and let's say I'm trying to process documents. This is a very common use case with AI. The first thing I would do is I'll make a model choice. Let's say I pick GPT 402 to be my model that I will use. 

Then the second decision would be using maybe Azure AI Search and then Search would help me do the embeddings and store the embeddings in the right place for retrieval. Retrieval augmented generation would be part of that. 

Then the third thing I would have to make a decision on is where do I want to deploy it? Do I want to use AKS or what kind of container-based service I'm going to use and then deploy that app. 

Then obviously you can make a bunch of more decisions along the way for monitoring and other things. But is that a good summary? >> That's fair. >> Great.

Now one of the other questions that I hear coming up is as we're building and moving towards these Agentic AI workflows. We all have made our choices on the frameworks and we feel passionately about our frameworks. Some of us really like LangChain. Some of us are using semantic kernel or autogen. How do you take that choice and use Azure for it? 

That's a good question again. We truly are a platform and as such, we support pretty much any framework. In fact, we make contributions to LangChain, LlamaIndex, etc where we have abstractions for our managed services. You can quickly use them as well. In addition, of course, we open-source Semantic Kernel and AutoGen which power our copilots. They have to be enterprise-grade and really security-minded first. 

Really you can host them on container apps or functions or AKS. Again, the decision tree is going to help you make that choice. 

That is amazing.

Then the next thing I want to mention is as you're making all these choices. If you're ready to get started with your use case, we've got a great host of resources that we will put down in the description and as links and you can get started with those resources right away. 

Perfect. Remember some of the killer GenAI apps such as get a Copilot, Chat GPT is built on the same services that is available for you. You can build those production great resilient reliable applications and agents. 

Awesome. Well, thank you so much, Govind, for walking us through how to make a decision and demystify AI because this is definitely top of mind for everybody. Thank you for joining us today. We hope that you found this information helpful and it helps you get started with AI in some way, shape, or form. 

We would love to know how you felt about the episode. What are you thinking about when it comes to AI, let us know in the comments below, and don't forget to join us in the next episode of sip and sync.

For that, make sure you subscribe and like so you don't miss the notifications.

Note, that there may be slight differences in text and video. Watch video for the original content.

Azure AI Foundry Video