Fireside Chat | Unlocks the Power of Knowledge with information discovery and sharing, Denis Yarats (CTO, Perplexity.ai) and Minfa Wang (CTO, oneGen & ex Waymo Tech Lead)

5:05PM May 30, 2024

Speakers:

Armand Ruiz

Keywords:

models

ai

people

perplexity

query

gpt

users

product

publicity

questions

guess

enterprise

work

search

build

answer

years

context

system

hallucination

Is doing ml Research and Production Institute before. So yeah, excited to be here. Thank you very much. Dr Dennis, it's your time.

Well, technically, I'm not doctor, like dropped out from my PhD from the very last days.

I think you need to Oh, yeah.

So technically, not a doctor. I haven't finished. I think I need to defend my PhD. I'll do it later. Okay, yeah,

I think we'll dive into that part. But to begin with, let me give a bit of formal intro. So you know, Dennis is the co founder and CTO of perplexity. Ai. I'm sure that at this point, most of you probably have heard or used perplexity AI already, but in case you do not know, perplexity is an AI Power Search Engine that leverages advanced technologies like natural language processing and machine learning to deliver accurate and comprehensive answers to user queries, and under Denise and the team's leadership, perplexity AI has emerged as a strong contender in the search engine industry. And yeah, Thanks Denise for joining us today.

Thanks for having me.

Yeah. So perplexity is an amazing product and amazing business. And I have lot of questions to dive into about the product details. But to begin with, I want to ask you about the start of the journey. So, as you alluded so in 2021 or 2022 you were at NYU, pursuing PhD degree, working with Yang Lee Kung, working on exciting researching artificial intelligence, I guess. How did you meet your other co founders, Robin and Johnny? And how do you guys decide to start publicity?

Yeah, it's funny story around like, 2022 I was working on reinforcement learning research, and sort of like published the paper. And two days later, Aravind and his group at Berkeley, sort of like published exactly the same paper. So started, like communicating, and then, like later, like, started collaborating a little bit I spent, like, some time at Berkeley, and yeah, and then yeah, and then we've just been communicating and around, like, 2000 early 2022 when I was about to finish PhD, Aravind was at open AI at that time, and we were like, tokens. And it was, like, very clear that GPT is getting better and better. And we decided, okay, so there's going to be, like, an opportunity to do something there. So we decided to start the company. And it so happened that Johnny, the our third co founder, also became available. I used to work with Johnny around, like, 10 years ago at Quora. We actually build, sort of like Quora, ranking, feed and digest emails. I don't know if anybody used Quora before, but all the recommendations, I think that Johnny and I was working on and so, yeah, it was kind of like perfect timing for us, and we decided to start the company and just work on search.

Oh, yeah, very inspirational. So I remember that at the beginning of perplexity, it was now designed to be a general search engine, right? It was designed to be some form of text to SQL tool, right? Could you walk us through, like, how did the product kind of evolve over time, and what were the pivotal moment for the transformations. Yeah,

this was a, kind of a we always wanted to build search, but it was at that time, you know, you go to investors and you and you tell them, I want to build a search company. And they like, look at you, kind of like, you stupid or something, because they tell, like, oh, there's Google already. Like, so, what are you doing? And so, kind of like, we decided to get an early traction. We're like, Okay, so we're gonna, like, masquerade as a text to SQL. We're gonna focus more, maybe, like, more like structured data. But still, you know, on the sort of like, back of our heads, we always been thinking about search, and we actually, like, prototyped, so like, original version of perplexity. We we actually hit it, like, maybe in October 2022 before, you know, chatgpt came out, and we kind of like use it as a Slack bot and as a discord bot. So we'd like would ask different, like questions about, like, how to run the company and stuff like that. So it was, like, very useful. And then, you know, kind of a chat GPT came out, and it was just like a very, kind of, like this moment brought a lot of attention to generative AI, and people came more receptive to this technology. And it was like perfect timing to, you know, find something with search. So because, you know, early, early, sort of, like, early problems with chatgpt, or, like, it was just like, it would, like, hallucinate a lot. It's not clear where the information is coming from. And so we kind of, like address that through perplexity. And it's been, yeah, it's been going since then.

Yeah, awesome, awesome. So we'll dive into the hallucination problem in a bit. But let's talk about kind of the product format today, like, as you mentioned, you know, we in the old days, like when you talk about, you are going to build a general search engine that will compete with Google, with Google, people may, you know, laugh at you or challenge you, right? And this kind of user experience of search has been kind of stable for the last decade, right? People enter a query and then they get a response back, they get a list of pages back. Kind of, what problems do you see with that experience, and how is kind of capacity designed differently to address some of the problems?

Yeah, that's a very good question. I think, you know when, when Google started, I think the metric that they were like, one of the key feature was page rank. So essentially, it's like, how trustable, how reputable is the source, right? It's kind of like, very similar to the academic citations like H index and it that was working well. But over time, people as particular like CEOs, SEO, spam and stuff like that, they would, you know, try to game this system and verify it and and basically, right now, like, became a very challenging to just rely on page rank, because you have to, like track the entire history of the user session and so and that kind of like broke the original idea behind search. Like, instead of, you know, getting the answers to your questions, you sort of like getting answers that are influenced by some other metric, and this metric essentially adds right so kind of like you're trying to, you know, give an answer condition on that this answer is also going to produce you some revenue. So it's not exactly aligned with the what users want. It's more likely aligned what, what, what the investors and like shareholders of the company want and and it was the LLM technologies, I think it's first time became possible to truly understand the sort of like intent of the page, to truly run like content of the page much better. And like give the answers that that users actually want, instead of like search results that kind of like masquerading for the good answers, but still maybe ranked for you with some other intent. So I think that's a key realization. So we kind of like rank the content, not the any other signals,

okay, awesome, by the way, I think you might need to adjust my position a little bit for better sound perception. Okay, yeah, maybe a bit closer and like holding it in this angle. But yeah, yeah, thank you for that answer. And I remember Arvind once described the experience of publicity as like Wikipedia and GPT having a baby. I guess that's a positive perception of it. And there's also kind of skepticism or criticism, criticism saying that publicity might just be a wrapper of chatgpt. And what is your answer to that? I guess in order to provide the smooth and snappy user experiences, you have to kind of develop system that goes way beyond just a wrapper. Could you point out some of the systematic challenges that you encounter,

yeah, I mean, it was, we started, like, intentionally to be a rapper because, I mean, that was the right time to do that, right? Because there's this, like, technology of GPT that you can use through API that was not available before. So, you know, like in the early stages, let's say, like five years ago, if you want to launch AI product or machine learning product, you first need to, you know, collect data, train the model, like, make sure it works, and then only launch the product and then see if the product has market fit, right. So that's that takes some time. And kind of like we didn't have that time. So the better strategy was, okay, let's build a product first, let's see if there is a market fit for it, and then we will start collecting data. I think, like data flywheel is very, sort of like underappreciated, and kind of like a very important aspect. And it can be, like, considered as a mode. So like, we first build a product, we start getting users, we start like collecting data, and then with this data, we can basically train whatever models we want. So and we like slowly been like rolling out things to our own infrastructure, so ranking models, LLM, models that are specifically tuned for our product. So, yes, we started as a wrapper, but that was the right idea. I think, like, in retrospect, I think this was the best idea we've had. I feel like, if we did something else, that wouldn't work, and, yeah, now, like, now we have data, now we have product, and so we can just optimize.

Yeah, I remember another interview once you mentioned that at the very early days, you already realized that the model is not your mode, but the data and the products are right. And one thing I'm curious is that, would you be worried if by just looking at the user queries today, will lead you to a local minimum, like because when the user issues a query, if your system does not handle it well, and then the user may give up, and then maybe your system will only be able to handle kind of the queries that is already working well enough. It's like a chicken and egg problem, kind of,

sure. Yeah, that's true. But I think, like, basically all of the queries that we kind of like, have a way to understand if query was not answered well. And so all of this then goes into the logs, and we, you know, being used as a way to improve the system in the future. So it's kind of like it takes time to set up this data flywheel, but once you get it working, it's, it's very powerful. So essentially, the system, kind of like self, improves over time. And so that's that that's, uh, takes some time to build, but that's a very crucial aspect of what we're doing.

Okay, so creating the data flywheel is kind of the key for iterating on the product side. Yeah, that makes sense. And I want to dive a bit deeper on the quality side. So you mentioned earlier about the hallucination, right? So LLM is kind of known for having hallucination problems, and for a search product, users are really caring about the authenticity of the information, the correctness of the information, right? And I guess precisely solves this by providing references to the pages when it generates the AI summary. But also from yesterday's panel, a lot of the people also discussed that even with retrieval, augmented generation with the references, sometimes the model may still hallucinate. It's not perfect, right? And also, I can also imagine cases where you just, you just do not have the answer, do not have the context to the question that the user is asking, right? How? How does privacy handle this type of situations, and what the user experience look like? Yeah,

that's, that's, that's a very hard problem. I think it's, it's not fully nobody, I think fully solved it. I think that the directions where we're going there is, you know, first of all, you want to like, minimize the amount, the number of time where you don't have like ground in for the LLM. So essentially that means you like optimizing your retrieval system, your search engine, so you basically provide LLM with all the necessary content. In cases where, as you said, there is no such information can be found. You basically have to train the model to kind of like, refuse to answer, instead of, like, hallucinate. And that can be done through like, rlhf and some other techniques where you basically teach the model saying, okay, so if you if you don't have this grounding, then you know, just don't rely on, like, on your internal knowledge, or maybe, like, rely less on it, and then just, like, try to produce an answer, or try to say, like, Okay, so maybe can, I don't know the answer, could you, like, could you rephrase your question, for example, or could you add like, additional information? So, but, yeah, not, not solved, but it's definitely over. Even since we started, it was like, over, like, over, like, a year, I think that this thing, like hallucinations, like dramatically being improved, and now, like, I remember in the first, early versions of perplexity, yeah, that's been, like, pretty common. But like, right now we, like, very rarely see you have to, kind of, like, try your best to trick the model. So it's, I feel like right now, like hallucinations happen where you kind of, like intentionally try to have model to hallucinate if you basically asking, like, normal questions, you know, from like, not distribution. I think it's very rare at this point, but still, it happens. Sometimes I

see, is there an established evaluation mechanism to measure hallucination? Okay, yeah,

yeah, but this is, this was a very critical component of our system. I think we have a we rely on, like humans, we rely on other llms to, sort of, like, understand if this answer has hallucinations or not, and then, yeah, a lot of this, likely, recently, can be done, like, pretty much automatically. You don't even need a lot of humans. You can just, like, train another LLM that's going to do this for you. And so we have, kind of, like, you have this systems that, just like, based on the search results, based on the query, based on the answer, it kind of like indicates, what are the chance of, like hallucination,

okay, okay, great, to know. So another kind of challenge, technical challenge for building a search engine is handling real time information. For example, a lot of the users may be curious about the stock price of Nvidia today, like whether or not it increased another 20% and could you maybe point out some of the unique challenges of like, why is it difficult to handle fresh or real time information, And how does publicity go around this problem.

I mean, difficulty comes from is just like, there is a lot of stuff happening on the internet, and so you just need to, you cannot you cannot crawl everything you cannot crawl. You basically have to, there is a trade off between the freshness of the index and the size of the index. And so you have to, kind of like, do, like 8020 so you decided, okay, so maybe this is, like certain domains, or like certain types of types of queries that we want to recall much faster, so like, have like, much faster update time. And for rest of the indexes, like, not as fast as like, maybe like, every, every, every week or something. And so you have to, like, understand. So we have like, a classifiers that kind of, like, tell per domain, kind of, like, does it need to be frequently updated? Does it not need, does it doesn't need to frequently updated? Or like, even like per page level, and, and, yes, you have the system that kind of like tells you. And then you have a crawling system that get basically, once it get received, it receives a URL, it decides, Okay, do I need to recrawl this URL or not? On the other hand, I think we also specifically for stocks. We're working with the like some API providers, for example, like, recently launched the this, like, like stack, like visualization tool, which called taco, and so you can ask like questions and like pull information from different providers and kind of like build the plots and stuff like that. So yeah, I think it's going to be just about working with, like some partners, but also sort of like prioritize, prioritizing what part of the web do you need to be crawl more frequently.

Okay, got it. So it's not like one single solution that addresses it, but more like a family of sophisticated engineering process that work to address the problem. Okay, now I wanted to zoom out a little bit and talk about kind of startup building in general, right? So you transition from academia to industry, and how do you like this transition so far? Do you enjoy working with a startup. And, yeah, do you recommend other kind of people from academia to try the same thing as well?

Yeah. So actually, I did like this transition twice. First I was in industry. I was, you know, like at being I was at where I was at Facebook, AI research, so it's kind of like academic things, and then I moved to PhD, and then I moved back to industry. I think, like, PhD is a great thing to do. I probably wouldn't recommend it to do it right now. I feel like, especially in AI and like, machine learning, I think, like, it's very challenging even even when I was, like, about to finish, it was kind of and I was lucky I had, like, access to the Facebook cluster so I can run like, as many jobs as I want. But like, you know, in a general school, you probably not going to have, like, a lot of compute. And as we know, like, a lot of interesting stuff right now requires a lot of compute. So it's, it becomes like a little bit tougher to do research. So you kind of like, have to focus on things that maybe require, like, different architecture or stuff like that, so something very risky and so on. Like, you know, obviously one of, one of the things after PhD, you want to get it, you want to get a job. And if you work on some risky stuff, it's just like, maybe you're never going to publish or anything. So it's a little bit like mail engine. But you know, PhD gives you a lot of sort of, like, teaches you to to work hard. Like, especially, like, startup is like, is the same as like, having, like, conference deadlines every week. So you just, like, constantly have to push and like, work hard so that that's that's come from PhD, but also sort of like thinking deeper on, sort of like the problems and trying to find, like the the most practical and the best solution possible. So instead of like engineering, engineering something like, very quickly and like, maybe that's something that's not going to work well, you like, take your time to think about and then build. So I think, yeah, kind of like having this research plus engineering background. I think it's very helpful to building a company.

Okay, okay, so since 2022 and 2023 you know, since the Boeing of GPT 3.5 GPT four, there is a lot of excitement on J AI, right? And we have witnesses, an enormous number of startups are born in these two years. But kind of you look at it, majority of them are enterprise facing. And despite the passion of building consumer facing apps, there are not a lot of consumer facing apps really gained the product market fit like where. But I mean publicity is one of the few, right? And could you kind of explain to people like in your journey, like, what? Why is it so difficult? What are the unique challenges of building a consumer facing app? What are the yeah main factors and how this propensity survive? Yeah, it's

exactly right. It's like way harder, in my opinion, to build a consumer company than an enterprise company. And the reason for this is it's, you basically have product market fit, or, you know, it's like, very binary, because, like, do you have users, or you don't have users? And it's like, you can, you can tell it, like, in a few days with enterprise, you kind of, like, can basically lie to yourself, you know, maybe I'll gonna get this contract in, like, a in a year, or like, six months. So you can, like, very like, prolong your journey for like, a long time, and you, you know, maybe eventually going to, like a big deal or stuff like that, but it's, it's, it's, it's not binary. And so that's why I would, in my opinion, it's easier to build enterprise than customer than user facing company, but it, but I feel like, again, in my opinion, it's, I feel like, building consumer companies way more exciting than building enterprise we, you know, as we started like, we spent like, maybe a month trying to build this, like, text to SQL engine, and that was supposed to be like enterprise offering, And it was just like, you know, we would go, like, and try to show this thing to somebody, and they're like, Oh, it doesn't work. Like, why do I need it? So you basically have to do like, a lot of sales, and that's like, not fun. And when we start doing like consumer facing, the user facing product, it was just like, our excitement. We just, like, changed over there, and it was, like so much fun to work on this stuff, because you constantly, especially like, on Twitter or x, we would like receive a lot of positive feedback. So it's, like, really exciting to see that people actually using your product. And it's very instant feedback loop. So it just help us to move much faster. And yeah,

yeah, awesome. So as you mentioned, in order to build a successful startup, you not only require a good idea, but it also requires kind of relentless execution, and your team has been working hard and diligently towards this goal. And in terms of team building, like, what kind of qualities do you look for when building your team, and how do you foster a culture of innovation and collaboration?

Yeah, I think this is very important, perhaps, like, one of the most important things, I think we early on when we only have, like, only co founders, like three people in the company we are, in order to hire, like, first people, we we didn't have, like a normal interview process, we would, we would like invite people from our network to sort of like for this, like a trial project, beer trial project. So they would like work with us for like couple of days, as as if they were already in the team, and then we see if, you know, if they do well, if, if we can work together, if, do they move fast? Do they want to work hard? Like, are they smart and stuff like that? And it's like, very clear, you cannot get this signal to, like, normal interview. And I think we probably hire like first, like 1015, people like that. Now we obviously cannot do that anymore, because it's just, like, challenging. But that was that helped us to build, like, a very strong core of the company where everybody is very aligned. Everybody is, like, very strong, technically, and sort of like mission driven and and so that was, like, super helpful. And then those those 10 people now like bringing in more people, and they kind of like, can preserve our, like, our culture and mission. So I think that was, that was super helpful in terms of, like, what kind of like people we hire, I think, like somebody who is, you know, very curious, who's like, can work fast, can work smart, and so, like, independence was very important feature where you just, like, you know, tell somebody, okay, this is what we need to do, like, very high level, and then just go on, like, figure out things. So that's maybe, like, the best qualities.

Okay, thanks for sharing. I'm sure a lot of the fellow startup founders will learn a lesson or two from your hiring process. And we kind of covered the past and the current. Now I want to maybe talk a bit about future perspectives. So, you know, publicity kind of benefited a bit from the advancement of Jai lately, right? And kind of the existing tech giants are waking up being is, you know, trying to come back, leveraging technology from open AI, Google is rolling out AI overview. And there's also rumors saying that open AI makes it rolling out a sound search GPT product. How do you see the kind of the landscape evolve in the next few years?

Yeah, it's definitely, I think the, it's a very exciting, I think, like, it basically like, validate, validates our idea, like, knowing that this big company is sort of like pursuing exactly the same direction as we do. It's kind of like, grows the market makes, I guess, the pie, like, even bigger for everybody, including for us. And kind of like, you know, yeah, makes things exciting and validation. Obviously, there is like more competition right now, but I feel like there is just, like so much stuff to do here. And I think there is going to be like room for everybody, you know. And so like, yes, Google or like Microsoft, they're like, trying to go to this direction, but, like, they still haven't solved the fundamental challenge of the, you know, advertisement. Like, because this, the showing answers is kind of like, at odds with advertisement. And so, like, even, even right now, if you go to Google and you will see this, what is it called AI or reviews. You would never see them on queries that actually monetizable, because that breaks immediately the ads right? So you only see on something for like you already have, like a knowledge base on the right. So it's doesn't add much value, in my opinion, but yeah, so it's still, like, they haven't, they haven't solved the fundamental challenge, why we're, like, able to be in the first place as a company. But you know, yeah, it's a, it's going to be exciting next few years for sure. And we just, we have a lot of stuff that we're working on, and I think it's, yeah, we have a, we can stand some competition. Yeah.

So you talked about the monetization model, right? And you mentioned that Google is running on this business and publicity so far is running on this subscription based model, right? And do you see this model to sustain the long term? And have you, would you consider like incorporating ads as well in your ecosystem,

I think, like subscriptions, it's probably the way. I guess it's going to go. It's maybe it's not going to be like a flat fee. I don't know this is a fact, but like in the future, the model is going to be more intelligent. They can do, like, a lot more stuff. And so essentially, what you're going to be paying is, like, for compute. I'm like, okay, see, I have like, more complicated tasks that I want to solve, and that requires, like, more compute, but it's going to save a lot, a lot of time for me. So then maybe I pay, like, a little bit more for that, instead of like, if it's like a simple query, then you don't pay much. So it's kind of like, somehow, like, probably, I'm guessing, going to be, like, proportional to the compute you spend and that kind of, like, completely different, like, monetization system in terms of ads, yeah, I don't know. I guess we haven't thought about like, we're probably going to, you know, explore something for free users. Obviously, I think if you ads in general, can be useful if you just don't overload your home page with ads, right? I think, like, sometimes if, if people searching, they want to buy something, it's actually very useful to show like an ad, but it needs to be very like, high quality, very targeted. And so then it actually helpful, I think, like, and then I guess the data thing is like enterprise, so we have like enterprise offerings as well. So I think we currently, like exploring all of those three things. And, yeah, that's probably, I'm guessing, how it's going to be like in the future, some combination of those.

Yeah, that sounds interesting. So besides monetization, there is also some interesting breakthroughs in multimodal models. And besides text, voice has also become a potential new channel for people to have access to information, right? And people are getting excited about, I mean, since the release of GPT four, oh and the Gemini from Google IO, people are getting excited about the future perspective of using voice to search and access information. Do you feel that, does that impose challenges to publicity, or do you feel like publicity will actually benefit from this?

Yeah, I think we, we benefit. We actually on our like phone app been, been having, like voice input and output for like several months now. I think we also have, like, something that is sort of like aI generated podcast, so where you, you know, every day from, from your like perplexity feed, you can essentially create, like, a five minute briefing that is AI generated. And you can go to the like Spotify or like Apple podcast and like, listen it while you're driving or whatever. So I think it's a link to be useful, especially on the mobile devices. Like, imagine in the future, you can, like, pick up your phone and say, Okay, so like, like, search something for me, or like order Uber, or like stuff like that. So that definitely stuff is going to happen. And I think, yeah, it's very exciting. I think we benefit from it, because it just, like, makes the product better.

Okay, okay, cool. There's one more question, and then I will open the floor for the live audience. So what are the kind of remaining or interesting upcoming projects or new initiatives that you are particularly excited about at perpicity? Could you give us some hints? Yeah,

I think we something that we're working on, and I think we're going to, like, have, like, some versions of it releasing over over time soon, it's, it's sort of like this system where you can answer, like, much more complicated questions, you know, something that requires, like, multi step reasoning that maybe, like, takes, takes a look at many sources and stuff like that. So not, not like your usual, like Google query, where you ask, like, you know, like a navigational like, What is the weather? But like something that is can save you a lot of time. And so I think this is going to be the first step. But like, over time, I see this as a sort of, like, where maybe you're going to give like tasks, or like very complex, like research questions, and the system is going to work, and maybe, like, hour later, it's going to come back with you, was the answer, but the answer is going to be like, so good, so you don't have to do saved a lot of time. So I think like, things like that is going to be important. I think the other thing we are looking into is sort of, like personalization and like trying to see if, you know there is like some stories or like pages that we generate is going to be like relevant for our discovery feed. Yeah, those, those things is going to be very useful,

I see so ability to answer complex questions and complete tasks and personalization sounds like the big areas to focus on. Okay? Thank you. So I think we are now ready to get some questions from the audience. Yeah, I'm here. Can you

hear me? Hear me? Oh, hey, Dennis, thanks a lot for sharing your insights on perplexity. A question I have is context limits are increasing a lot faster than a lot of people expected. I'm curious how you guys think about this at perplexity. What do you how do you guys think you'll need to think change your rag pipeline. How do you think we should be thinking about it differently? You know, both inside and outside of in the industry,

I missed the first part. What was that? So,

context limits are increasing very rapidly. Context, what context? Contexts, context limits. So I'm curious how you guys think about this in terms of rag pipeline perplexity and how we should be thinking about it outside as well.

Yeah, I think, like, right, this is a, I mean, so right now, like the models that are sort of like coming with, like, a bigger, bigger context, I think this is, in some applications. So useful thing, but like, it is all, like, one context doesn't come for free, right? It's still, you have to run the attention over those tokens. It's kind of like trading off, like, computation and and so, like, well, it and besides, it's like, very difficult to train model for like, one context, because there's just, like, like, data distribution is not that long. So you have to be, like, very creative. I think what we do, general, our system tries to optimize how much content we use, context we use. And this is done like, from like ranking. So we kind of like, for each request, we can, like, allocate how much context we want to use, and then we basically just, like, tell the ranking system, in case of like, try to extract as much information, as much relevant information as possible for this query, but still stay within this context. And then here you kind of like, trading up the quality, latency and cost, essentially. So it's kind of this, like, multi dimensional optimization problem where, you know, yeah, there's like, multiple barriers. But I think, like, it's very useful for like, I think like, right now it's probably going to be very hard to do something with, like, context size four or even eight. But I feel like if you have like, 16 or 32k I think you can do a lot of interesting stuff was that going beyond

just as a follow up, do you think that even with context limits increasing, we're still bounded by latency, because for like, a consumer facing product, you know, you need to get the response out quickly?

Yeah, exactly. The main limitation was the was a long context is its latency and cost, right, because it's still attention in them, so it only suitable for like, certain applications. Maybe, if it's like you want to answer something quickly, that's probably not going to be the good idea. But as I mentioned, if you're doing something more complicated, and where, like, latency maybe is not as important as the quality and where you can allow, like, maybe this task run for like 10 minutes or something like that. So then I think that that's gonna be very useful.

Okay, see a hand from the jack jacket? Hey,

Dennis, I'm Nick. I'm a product manager at vectara rag as a service company. I do have a question around your talk about the enterprise focus of publicity, AI, which is one of the new direction. How do you see that differ from enterprise search companies, for example, Glenn and others,

I think it's probably going to be, you know, we're probably not going to focus a lot on things like, you know, like customization for like, specific cast, specific company, but like, we just, you know, going to Tell them, okay, see, like, we receive a lot of requests. People ask him, can you have like, a perplexity for, like, internal documents? And I think that that's that is possible, but like, we don't want to deal with the at least, like, for now, for was the all the access rights and stuff like that, those things, we're just going to tell Okay, so, like, give us documents. We can index it for you, and we can, you know, like, show you answers. So a lot of people okay with that. And I think that's that's working, we kind of, like, still want to maintain the general product. We don't want to, like, customize a lot of stuff, and our key is quality and latency. So we just want to, like, provide the best answers and and the biggest I also think, is if, like, people going to have not only access to their internal documents, but also the web and kind of like this synthesis can, just like, bring a lot of useful stuff.

Maybe we'll from that side, I

thank you, Dr Garris, for your presentation sharing. So I have two questions. And the first one is that I've learned that perplexity use third parties large language model and indexing. So I'm wondering if you're gonna build your own large model and the indexing sometime later? Do you have such plans? And my second question is that, because there are actually a number of AI search engines in the industry, but perplexity performs the best, so I would also like to know your key success factors. Thank you.

The first question was, could you repeat to be used?

So I've learned that perplexity now are using the large language model and indexing provided by third parties. So I'm wondering if you're gonna plan to develop your own large language models and indexing. Yeah,

I see. I mean, yeah, we do use, I mean, we do use, like, Frontier models, like GPT four, like load three, right? I think that's nobody has those models, and we're, like, not in a position to build those models anyway, but we a lot of our traffic actually serve them in house models. So we have, like, our own inference. We have our own like training. So we take, like, a lot of like, open source models, like llama three, for example, is a great model, and we post train it in a way that is suitable for our example. We constrain it in a way that is, like, reduces hallucinations, because it's kind of like needs to be exactly constrained for, like, our system. And then, yeah, those models are, like, pretty good, and like, they serve a lot of traffic and a lot of, like, smaller models for, like, all the orchestration and stuff like that that is always sort of like, been served in house, you know, yeah, Frontier models. I think this is going to be developing. I think, like, we're going to use them. It just like, that's actually one of the advantages we as a platform can offer our users many different frontier models. So if it's like, you know, like, if it's like, you're like, open, AI, you can only like, offer GPT four, but if you want to use like load, for example, you cannot do that. So I think that's that's very useful in terms of index. I think, yes, we are. We have our own index, I think, and like ranking system is specifically designed for the good answers. So like, something that we prioritize over like classical search engines, is not the sort of like links relevance or like documents relevance, but actually like content. So we don't really need links, right? So we just like, want to find like snippets of the information in the pages that can answer that question. So it's like completely quite different, like ranking system and something that we build in house, and I think it's been working. Well,

okay, we'll have time for two last questions.

Hi Dennis. My name is Zen. I'm working on AI agents, and I have question, like, we all know that Devin are pretty good at coding, and I'm curious how much agent is playing a role in perplexity search today, and how are you foresee perplexity AI agent in the future? Yeah,

we were working on it. I think we're gonna release some this or next week that is going to go towards that way. It's not going to be optimized for voting, because that's not what we're doing. But like, it's going to be mostly optimized for like, research, so you'll be able to ask like, very complicated questions and get the answers and yeah, so yeah, we're definitely working on, I think this is a it's going to be future, but like, not only us, but like a lot of other products,

will that be agent behind every single search, or just for a specific slice of the search,

it's going to be for the full web.

Looking forward to it? Okay?

One last question, to this lady, could you help pass down the mic?

Hi, Dennis, so I had you touched upon ensuring that you're able to access the latest information or the latest data as part of search, but another component in the pipeline, which is a generative model component, how do you ensure that you know any piece of information that it may not have seen, either during pre or post training or any new concepts that it may not have an understanding of it's able to interpret, you know, those new pieces of information and generate appropriate responses. Yeah.

I mean, this is this happens through like, that's why, like, large language models are so powerful these days, is just that a lot of like, generalization happens, right? So there this models are very big, very capable. And so, you know, yeah, obviously, maybe, if there's like, something like very new, like physical concepts, or like, maybe like new theorem, it wouldn't understand. But if it's like, you know, your normal news and stuff like that, I think it's gonna be like generalized quite well so, and we definitely like when we post train our models, we make sure that there is a lot of, sort of like, a lot of this stuff already happens in the training data set, and so that this model says, sort of Like, do pretty well.

Okay with that, we will conclude today's affairs chat. Thanks again, Dennis for coming. Please give the speaker a round of applause. And yeah, we're all excited about the future of capacity and thank you. Thank you for the good questions and having me. Thanks applause.

Thank you, mimfa, Dr Jats,

our next presenter is the VP of product at IBM's AI platform presenting how to shape the future of enterprising using Gen AI please welcome to the stage. Armand Ruiz, you

you so people can prepare a question afterward as well.

I Alright. Hi everybody. Thank you for coming here today and very nice meeting you all. I'm from IBM, and I'm going to spend the next few minutes all the work that we've been doing at IBM in the last decade, especially in the last two or three years. I've been at IBM for 13 years, so I joined exactly when we announced Watson. So I'm sure many of you remember IBM Watson, and we've been on the AI space for over a decade. So let me see how this works. All right, so I did some presentations two or three years ago talking about foundation models, and at that time, GPT, I was free. GPT, I was not very popular. But since then, we've been in this journey of crazy innovation. We're getting new foundation models every week of different sizes, of different domains by different companies that are raising a lot of capital to train many of these foundation models. So what I want to spend some time with you today is to challenge, try to change, maybe, the point of view on how we should approach AI and this race to get the best foundation models, just for context as well. I work a lot with enterprise customers, B to B, and the big thing here is how we can help businesses and enterprises and code all that information and data that they have into foundation models. So I think this is, this is the biggest challenge that we have right now. We have a lot of information that public information is already going into foundation models, but there is a lot of data that is behind the firewalls, behind enterprise domain that needs to go into foundation models, and that's actually really hard to do. And just to step back for a second, I want to get a quote from Leibniz, which, over 300 years ago, already found a way to encode for in a different way. With, with binary representation of information. So they took books of that era, into math knowledge, they were able to see.