I'm on top of open AI API's. And so we are very committed to that. And what does that mean to developers who have you know, look, my I always think of Microsoft as a platform company, a developer company and a partner company. And so we want to make, for example, we want to make GitHub available get up copilot available as at the enterprise edition available to all the attendees here so they can try it out. That's awesome. Yeah, we're very excited about that. And you can count on us to build the best infrastructure in Azure with your API support, and bring it to all of you and then even things like the Azure marketplace for developers who are building products out here to get to market rapidly. So that's sort of really our intent. Yeah.
Great. And how do you think about the future? Future the partnership or future of AI or whatever? Yeah, there anything you want?
No, no. There are a couple of things for me that I think are going to be very, very key for us, right. One is, I just described how the systems that are needed as you aggressively push forward on your roadmap requires us to be on the top of our game and we intend fully, to commit ourselves deeply to making sure you all as builders of these foundation models have not only the best systems are for training and inference, but the most compute so that you can keep pushing forward on the frontiers because I think that that's the way we're gonna make progress. The second thing I think both of us care about, in fact, quite frankly, the thing that excited both sides to come together is your mission and our mission. Our mission is to empower every person and every organization on the planet to achieve more and to me ultimately, AI is only going to be useful if it truly does. Empower, right? I mean, I saw the video you played earlier. I mean, that was fantastic to see. Those were hear those voices, describe what AI meant for them and what they were able to achieve. So ultimately, it's about being able to get the benefits of AI broadly disseminated to everyone I think is going to be the goal for us and then the last thing is, of course, we're very grounded in the fact that safety matters. And safety is not something that you'd care about later, but it's something we do shift left on and we're very, very focused on that with you all.
Great. Well, I think we have the best partnership in tech. I'm excited for us to build AGI together.
I'm really excited to have a friend thank you very much for coming. Thank you so much.
Okay, so we've shared a lot of great updates for developers already and we got a lot more to come. But even though this is developer conference, we can't resist making some improvements to charge GPT so a small one track GB team now uses GPT for turbo with all the latest improvements, including the latest knowledge cut off, which will continue to update that's all live today. It can now browse the web when it needs to write and run code, analyze data taken generate images and much more. And we heard your feedback that model picker, extremely annoying. That is gone. Starting today. You will not have to click around the drop down menu. All of this will just work together. Chat GBT yeah
chat GBT will just know what to use and when you need it. But that's not the main thing. And neither was price. Actually, the main developer request there was one that was even bigger than that. And I want to talk about where we're headed. And the main thing we're here to talk about today. So we believe that if you give people better tools, they will do amazing things. We know that people want AI that is smarter, more personal, more customizable, can do more on your behalf. Eventually, you'll just ask the computer for what you need. And it'll do all of these tasks for you. These capabilities are often talks in the AI field about as agents. The upsides of this are going to be tremendous. At Open AI. We really believe that gradual iterative deployment is the best way to address the safety issues, the safety challenges with AI. We think it's especially important to move carefully towards this future of agents. It's going to require a lot of technical work, and a lot of thoughtful consideration by society. So today, we're taking our first small step that moves us towards this future. We're thrilled to we're thrilled to introduce GP TS GPT is our tailored versions of chat GPT for a specific purpose. You can build a GPT a customized version of GPT for almost anything with instructions, expanded knowledge and actions, and then you can publish it for others to use. And because they combine instructions, expanded knowledge and actions, they can be more helpful to you. They can work better in many contexts and they can give you better control. They'll make it easier for you to accomplish all sorts of tasks or just have more fun, and you'll be able to use them right within church if you can, in effect program a GPT with language just by talking to it. It's easy to customize the behavior so that it fits what you want. This makes building them very accessible, and it gives agency to everyone. So we're going to show you what GPT is are how to use them, how to build them. And then we're going to talk about how they'll be distributed and discovered. And then after that for developers, we're going to show you how to build these agent like experiences into your own apps. So first, let's look at a few examples. Our partners that code.org are working hard to expand computer science in schools. They've got a curriculum that is used by 10s of millions of students worldwide. code.org crafted lesson planner GPT to help teachers provide a more engaging experience for middle schoolers. If a teacher asks it to explain for loops in a creative way, it does just that. In this case, it will do it in terms of a video game character repeatedly picking up coins super easy to understand for an eighth grader. As you can see, this GPT brings together code.org extensive curriculum and expertise and lets teachers adapt it to their needs quickly and easily. Next candidate has built a GPT that lets you start designing by describing what you want in natural language. If you say make a poster for debit, a debate reception this afternoon this evening, and you give it some details. it'll generate a few options to start with by hitting Canvas API's. Now this concept may be familiar to some of you. We've evolved our plugins to be custom actions for GPS. You can keep chatting with this to see different iterations. And when you see when you like, you can click through to Canva for the full design experience. So now we'd like to show you a GPT live. Zapier has built a GPT that lets you perform actions across 6000 applications to unlock all kinds of integration possibilities. I'd like to introduce Jessica, one of our solutions architects who is going to drive this demo. welcome Jessica.
Thank you all for being here. My name is Jessica Shea. I work with partners and customers to bring their product to life. And today. I can't wait to show you how hard we've been working on this so let's get started. So to start, where your GPT will live is on this upper left corner. I'm going to start with clicking on the Zapier AI actions. And on the right hand side you can see that's my calendar for today. So it's quite a day. I forgot to use this before so it's actually already connected to my calendar. To start, I can ask what's on my schedule for today. We built GPT with security in mind. So before it performs any action, or share data, it will ask for your permission. So right here, I'm going to say allowed so GPT is designed to take in your instructions, make the decision on which capability to call to perform that action and then execute that for you. So you can see right here, it's already connected to my calendar, it pulls into my information. And then I've also prompted it to identify conflicts on my calendar. So you can see right here, I actually was able to identify that so it looks like I have something coming up. So what if I want to let Sam know that I have to leave early. So right here I say lets them know, I gotta go chasing GPUs. So with that, I'm going to swap to my conversation with Sam. And then I'm going to say yes, please run that. Sam did you get that? I did. Awesome. So this is only a glimpse of what is possible and I cannot wait to see what you all will build. Thank you. And back to you, Sam.
Thank you, Jessica. So those are three great examples. In addition to these, there are many more kinds of GPT is that people are creating and many many more that will be created soon. We know that many people who want to build a GPT don't know how to code. We've made it so that you can program the GPT just by having a conversation. We believe that natural language is going to be a big part of how people use computers in the future. And we think this is an interesting early example. So I'd like to show you how to build one.
All right, so I'm going to create a GPT that helps give founders and developers advice when starting new projects. I'm gonna go to create a GPT here and this drops me into the GPT builder. I worked with founders for years at YC and still whenever I meet developers, the questions I get are always about how do I you know think about a business idea. Can you give me some advice? I'm gonna see if I can build a GPT to help with that. So to start GPT builder asks me what I want to make. And I'm going to say I want to help startup founders think through their business ideas and get advice after the founder has gotten some advice. Grill them on why they are not growing faster. Alright, so to start off, I just tell the GPT a little bit about about what I want here, and it's gonna go off and start thinking about that. And it's going to write some detailed instructions for the GPT it's also going to let's see asked me about a name how do I feel about startup mentor? That's fine. That's good. So if I didn't like the name, of course I could call it something else. But it's you know, going to try to have this conversation with me and start there. And you can see here on on the right in the preview mode that it's already starting to fill out the GPT where it says what it does it has some like ideas of additional questions that I could ask. And you know what I actually saw it just generated a candidate. Of course I could regenerate that or change it but I sort of liked that. So I will say that's great. And you see now that the GPT has been built out a little bit more as we go. Now, what I want this to do, how it can interact with us. There's like a talk about style here. But what I'm going to say is I am going to upload transcripts of some lectures about startups I have given please give advice based off of those. Alright, so now it's going to go figure how to do that. And I would like to show you the Configure tab. So you can see some of the things that were built out here as we were going by the builder itself. And you can see that there's capabilities here that I can enable. I could add custom actions. These are all fine to leave. I'm going to upload a file. So here is a lecture that I picked that I used to that I gave with some startup advice and I'm going to add that here. In terms of these questions. This is a dumb one. The rest of those are reasonable, and like very much things founders often ask. I'm going to add one more thing to the instructions here, which is Be concise and constructive with feedback. All right. So again, if I had more time at show you a bunch of other things, but this is this is like a decent start. And now we can try it out over on this preview tab. So I will say what's a common question? What are three things to look? What are three things to look for when hiring employees at an early stage startup. Now that's going to look at that document I uploaded. It also have of course, all of the background knowledge and GPT four. That's pretty good. Those are three things that I definitely have said many times. Now we could go on and it would start following the other instructions and grill me on why I'm not growing faster. But in the interest of time. I'm going to skip that. I'm going to publish this only to me for now. I can work on it later I can add more content, I can add a few actions that I think would be useful and then I can share it publicly. So that's what it looks like to create a GPT with thank you
by the way, I always I always wanted to do that after like all of the YC office hours. I always thought man someday I'll be able to make a bot that will do this and they'll be awesome. So with GPS, we're letting people easily share and discover all the fun ways that they use chat GPT with the world. You can make private GPT it's like I just did. Or you can share your creations publicly with a link for anyone to use. Or if you're on chatting, PT enterprise, you can make GPT just for your company. And later this month we're going to launch the GPT store, take a screenshot. You can list a GP. Thank you. I appreciate that
you can list a GPT there and we'll be able to feature the best and the most popular GPT is of course we'll make sure that GP T's in the store follow our policies before they're accessible. Revenue sharing us we're gonna pay P take a screenshot of the most useful and the most used GPS, a portion of our revenue. We're excited to foster a vibrant ecosystem. With the GPT store. Just from what we've been building ourselves over the weekend. We're confident there's gonna be a lot of great stuff. We're excited to share more information soon. So those are TPTs and we can't wait to see what you build. But this is a developer conference and the coolest thing about this is that we're bringing the same concept API.
Take a screenshot
many of you have already been building agent like experiences on the API. For example, Shopify sidekick, which lets you take actions on the platform discords Clyde, lets discord moderators create custom custom personalities for and snaps my AI a customized chat bot that can be added to group chats and make recommendations. These experiences are great, but they have been hard to build. Sometimes taking months teams of dozens of engineers. There's a lot to handle to make this customer assistant experience take a screenshot. So today we're making that a lot easier with our new assistants API.
The assistants API includes persistent threads. So they don't have to figure out how to deal with long conversation history. Built in retrieval, code interpreter Jesus are working in Python interpreter and a sandbox environment. And of course, the improved function calling that we talked about earlier. So we'd like to show you a demo of how this works. And here is Romane. Our head of developer experience Welcome
Thank you. Morning. Wow, it's fantastic to see you all here. It's been so inspiring to see so many of you infusing AI into your apps. Today, we're launching a new one.
I'm so excited for this, but I cannot stand this fucking fake bullshit excitement that these presenters have. Alex
is in the API. But we are also very excited to improve the developer experience for you all to build assistive agents. So let's dive right in. Imagine I'm building wanderlust a travel app for global explorers and this is the landing page. I've actually used GPT for to come up with these destination ideas and for those of you with a keen eye, these illustrations are generated programmatically using the new Dali three API available to all of you today. So it's pretty remarkable. But Let's enhance this app by adding a very simple assistant to it. This is the screen we'll come back to it in a second. First, I'm going to switch over to the new assistants playground. Creating an assistant is easy. You just give it a name. Some initial instructions for model in this case, I'll pick GPT for chewable, and here I'll also go ahead and select some tools. I'll turn on code interpreter and retrieval and save. And that's it our assistant is ready to go. Next I can integrate with two new primitives of this assistance API, threads and messages. Let's take a quick look at the code. The process here is very simple. For each new user, I will create a new thread and as these users engage with their assistance, I will add their messages to the threads. Very simple and then I can simply run the assistant at any time to stream the responses back to the app. So we can return to the app and try that in action. If I say hey, let's go to Paris. All right. That's it with just a few lines of code users can now have a very specialized assistance right inside the app. And I'd like to highlight one of my favorite features here, function cooling. If you have not used it yet, function calling is really powerful. And as Sam mentioned, we're taking it a step further today. In our guarantees the JSON output with new ad latency, and you can invoke multiple functions at once for the first time. So here, if I carry on and say hey, what are the top 10 things to do? We're gonna have the assistant respond to that again and here what's interesting is that the assistant knows about functions, including those to annotate the map that you see on the right. And so now all of these pins are dropping in real time here. Yeah, it's pretty cool. And that integration allows our natural language interface to interact fluidly with components and features of our app. And it truly showcases now the harmony you can build between AI and UI where the assistant is actually taking action. But next, next, let's talk about retrieval. And retrieval is about giving our assistant more knowledge beyond this immediate user messages. In fact, I got inspired and I already booked my tickets to to Paris. So I'm just going to drag and drop here this PDF. When it's uploading, I can just sneak peek at it. Very typical United flight ticket and behind the scene here, what's happening is that retrieval is reading these files, and boom, the information about this PDF appeared on the screen. And this is of course, a very tiny PDF but assistants can parse long form documents from extensive text to intricate product specs depending on what you're building. In fact, I also booked an Airbnb. So I'm just going to drag that over to the conversation as well. And by the way, we've heard from so many of you developers how hard that is to build yourself. You typically need to compute your embeddings you need to set up chunking algorithm. Now all of that is taken care of. And there's more than retrieval. With every API call, you usually need to resend the entire conversation history, which means you know, setting up a key value store that means like handling the context, Windows serializing messages and so forth. That complexity now completely goes away with this new stateful API. But just because of when he is managing the CPI does not mean it's a black box. In fact, you can see the steps that the tools are taking right inside your developer dashboard. So here if I go ahead and click on threads, this is the thread that I believe we're currently working on and see like these are all the steps, including the functions being called with the right parameters and the PDFs I've just uploaded. But let's move on to a new capability that many of you have been requesting for a while. Crude interpreter is now available today in the API as well. Yes, that gives the AI the ability to write and execute code on the fly but even generate files. So let's see that in action. If I say here, hey, we'll be four friends staying at this, Airbnb. What's my share of it, plus my flights? Alright. Now here, what's happening is that could interpret our nudist than it should write some code to answer this query. So now it's computing you know the number of days in Paris number of friends it's also doing some exchange rates calculation behind the scenes to get this answer for us. Not the most complex math, but you get the picture. Imagine you're building a very complex like finance app that's crunching countless numbers, plotting charts, so really any task that you'd normally tackle with good then could intrapreneur will work great for you. All right, I think my trip to Paris is sorted. So to recap here, we've just seen how you can quickly create an assistant that manages state for your user conversations, leverages external tools like knowledge and retrieval and could interpreter and finally invokes your own functions to make things happen. But there's one more thing I wanted to show you to kind of really open up the possibilities using function calling, combined with our new modalities that we're launching today. While working on dev day, I built a small custom assistant that knows everything about this event. But instead of having a chat interface while running around all day today, I thought why not use voice instead? So let's bring my phone up on screen here so you can see it on the right. Awesome. So on the right, you can see a very simple swift app that takes microphone input. And on the left, I'm actually going to bring up my terminal logs so you can see what's happening behind the scenes. So let's give it a shot. Hey there, I'm on the keynote stage right now. Can you greet our attendees here at DEF day?
Hey, everyone, welcome to dev day. It's awesome. To have you all here. Let's make it an incredible day. Oh, shit.
Isn't that impressive? You have six unique and rich voices to choose from MDPI each speaking multiple languages so you can really find the perfect fit for your app. And on my laptop here on the left, you can see the logs of what's happening behind the scenes too. So I'm using whisper to convert the voice inputs into text and assistant with GPG for Chobot. And finally, the new TTS API to make it speak. With things to function calling things get even more interesting when the assistant can connect to the internet and take real actions for users. So let's do something even more exciting here. Together. How about this? P Assistant, can you randomly select five deputy attendees here? And give them $500 In opening credits?
Yes, checking the list of attendees. Done. I picked five day attendees and added $500 of credits to congrats to Christine M Jonathan Steven G. luz que and Suraj s.
Alright, if you recognize yourself, awesome, congrats. And that's it. A quick overview today of the new assistant CPI combined with some of the new tools and modalities that we launched, or starting with the simplicity of a rich text or voice conversation for you and users. We really can't wait to see what you build. And congrats to our lucky winners. Actually, you know what? You're all part of this amazing open AI community here. So I'm just going to talk to my assistant one last time before I step off the stage. Hey Assistant can you actually give everyone here in the audience? $5 No banana credits.
Sounds good. Let me go through
that function we'll keep running but I run out of time. So thank you so much, everyone. Have a great day. Back to you some.
Pretty cool. All right. So that assistants API goes into beta today and we are super excited to see what you all do with it. Anybody can enable it. Over time, GPS and assistants are precursors to agents are going to be able to do much much more they'll gradually be able to plan and to perform more complex actions on your behalf. As I mentioned before, we really believe in the importance of gradual iterative deployment. We believe it's important for people to start building with and using these agents now to get a feel for what the world is going to be like. As they become more capable. And as we've always done, we'll continue to update our systems based off of your feedback. So we're super excited that we got to share all of this with you today. We introduced GPS, custom versions of chat GPT that combined instructions, extended knowledge and actions. We launched the assistance API to make it easier to build assistive experiences with your own house. These are our first steps towards AI agents, and we'll be increasing their capabilities over time. We introduced a new GPT four turbo model that delivers improved function calling knowledge, lowered pricing, new modalities and more. And we're deepening our partnership with Microsoft. In closing, I wanted to take a minute to thank the team that creates all of this. Open now has got remarkable talent density, but still, it takes a huge amount of hard work in coordination to make all this happen. I truly believe that I've got the best colleagues in the world. I feel incredibly grateful to get to work with them. We do all of this, because we believe that AI is going to be a technological and societal revolution. It'll change the world in many ways. And we're happy to get to work on something that will empower all of you to build so much for all of us. We talked about earlier, how if you give people better tools, they can change the world. We believe that AI will be about individual empowerment and agency at a scale that we've never seen before. And that will elevate humanity. To a scale that we've never seen before either. We'll be able to do more, to create more and to have more. As intelligence gets integrated everywhere, we will all have superpowers on demand. We're excited to see what you all will do with this technology and to discover the new future that we're all going to architect together. We hope that you'll come back next year. We launched today is going to look very quaint relative to what we're busy creating for you now. Thank you for all you do. Thanks for coming here. Today.
