Yeah, so safety guide rails. Now imagine you are a junior DevOps specialist, or you're, you know, you're on the team there at Royal Bank or something, and they put you in charge of making a chat bot to put on the web page of the company, whatever it's representing the bank. Things like safety guide rails. There's guard rails, there are certain very foundational things that we just won't bend on, right? They're going to be very strong. Nothing about violence. If you ask your AI, how do I make a bomb or something that'll say, you know, for safety reasons, I can't say that, or anything about, you know, racism or any kind of like, sexual thing or whatever. So those are very foundational safety guidelines that are just built into every foundation model. In fact, even if you don't provision for that yourself, if you're building on top of Vicuna or chat, G, P, T, or, you know, any of the other big ones, it's already baked into the foundational embeddings of the language. Yeah, it's based into the foundational embeddings of the language. Now you can additionally choose, and we'll see how to do that in the project. You can apply additional guide rails, guard rails, so if you're on the TD Bank, and then the customer asks, you know, what was should I invest in? You know, investment X or investment y? That's not really like a kind of a safety or violence thing, but you would probably want to train your model to, I'm not going there Go, go call this number and talk to a human expert, because there are so many factors, and it has to do with so much of the individual situation. You would not want your model to give advice. Somebody else might come back and sue you on later. But anyway, this is sort of Yeah. So that's where I was going with all of this, um, talking about how, I guess these are sort of Yeah. I was talking about unified process. I was talking about the emerging saying there's a reason why unified process does not really fit. I mean, you should still learn unified process, because it's quite likely that you might at certain points in your career. Maybe your first job is not AI, right? Maybe it's a it's a developer for a website or something. So this, by itself, would still be worth much more than the cost of what you're paying for this course, and I will teach you unified process. However, what we're developing here for AI is what I call unified model engineering process. And to the best of my knowledge, it's possible there are things I haven't found or whatever. I don't think anybody is formally talking about a software development methodology for AI, because it's a pretty new thing. Still, most AI models are still pretty much being hand crafted. It's not being generated on a large scale basis. However, I know from having been a teacher for many years, it's much easier to get students to learn a methodology than if I just give them and which I think a lot of the times in programming that happens, the instructor just throws a bunch of examples, and the student has to make sense of them, and they get confused. They say, I don't understand the subject, so I will be teaching you unified model engineering process, and what is the first thing we do when we're using unified model engineering process? The lack of an answer tells me you did not do what I asked you to do, which is to read the first three chapters of my book. Who did not read the first three chapters of my book? Hands up. If your hand is up, take your other hand and slap yourself in the punishment, because you got to do what I'm telling you. Thank you. That's better remember how much that hurt. And next week we all this stuff, all right, so having said all of that, I guess I Yeah, so that was the last thing. So emerging technologies, so Cloud DevOps, CICD, software build which sort of fits in with our architecture. And then we have our concepts over here, which is the stoic casting parrot the math. I'm going to be teaching you a little bit of math. So if somebody talks about the gradient descent vector or a language embedding. Now, the difference if you go and study math at university, they make you learn it to the bone, and you have to be able to do all the proofs of it. We're not going to get into that here. I was talking to William about the best way to handle it, and we decided that the best way is, I'm going to teach you how to do the math procedurally. I'm not going to teach you the proofs, because that would require a very in depth background and analysis and so on, but you're at least going to have a intuitive intuition, a feeling about how the math works, right? So having said all of that as a background, now, let's jump in and get one on our lab. I'm.
Oh, by the way, one thing, one more. Closing comment here. So next token generation that is produced by what's called the gradient descent vector. That is a calculus thing. To really understand what that means, we need to understand a little bit about calculus, differentiation, minimum, maximum, but I'll get you through it. So gradient descent vector. And then the other foundational concept, in my opinion, I've been thinking, I think a lot about what's the best way to boil and distill this down to the simple concepts for new learners. The other foundational concept is building and embedding. Building and embedding, building a language, embedding and an embedding now is a matrix. If you remember your, you know, if you studied algebra in high school or something, if you didn't, doesn't matter. But there's this concept. It's like an Excel spreadsheet that sense. It's called a matrix, and into those matrix go your what we talked about it last week. Who remembers who you put in the cells of your matrix, your tokens, your tokens. A token is what. It's a word or a phrase in your training data set. Now you assign or pytorch, actually, which is we're going to get into now, pi torch assigns a number. And how it does that. I'll go over it with you. It's kind of mathematically intense. We'll go through one time. But pi torch assigns a numeric value
each token.
Well, that's the reason why. For example,
King and President would have numerical values. Those words would have not exactly the same, but they would sort of be in the same cluster of meanings, because they would both be, you know, somebody who's in charge of ruling a large domain. And the older religions, they used to call that Gematria, by the way, there's this book you can read called the Bible Code, in which some very top level cryptologists studied the Bible, and they figured out that you can drop the words of the first five books of the Old Testament into a computer algorithm, and then you can ask questions, and apparently those questions will be answered in a correct way. Nobody quite knows why it is. But one example I heard when I was listening to this lecture, they asked about yiks ravine, who was the scientist who had encoded the first five chapters of the Old Testament into this numeric matrix. They asked about the exact ravine, probably pronouncing it wrong, but anyway, and the answer they got from their algorithm was the assassin shall assassinate. And it gave the guy's name, which I can't remember. It even the city in the date. So it predicted correctly. And these mathematicians, they were working for the CIA, they contacted Israeli intelligence, who told this, this leader, this act, ravine, and his answer as well, if it's God's will for me to die, I'm going to die. And he did so exactly the way it predicted. There. What does this have to do with anything? This is getting to the outer rings of what I'm going to talk about in this class. But at some point you might be curious about the philosophy, or where does this knowledge come from that's embedded in language models. I personally believe, and I have a blog article called conversational intelligence where I developed this. But it seems that when we put human language doesn't have to be English, can be Arabic, Chinese, any of these old languages, if you put them into an embedding and you ask questions, you will get meaningful answers, and that's actually going to be your assignment one. We're going to see how pytorch and TensorFlow do that. But it seems like there is intelligence baked into human language. Anyway, let's carry on now. Let's go to our lab book and get started. You
Yeah, so there's actually the book I was talking about, if you're curious, and spend a couple of bucks on Kindle. Oh, it's not out on Kindle. Well, anyway, you can get the hard copy. But I don't know, to me, this is very interesting stuff. Some people are okay to just work the procedures, but I think at some point there's more to life than just making money and spending money. So it seems as AI application developers were kind of getting close to the core of that, right? Very good. Let's flip over to our lab book now. You.
I just want to close that.
So you guys got the link to your today's lesson book right there, January, 20, so go and join me over there. Well, thank you for letting me know. Yeah, because I appreciate if I say something is there, because sometimes I don't know it doesn't get updated, or it's hidden, whatever. So yeah, just let me know that you can actually see it. That's great. All right. Now, let me get my clean screen cleaned up a little bit here.
Oh, by the way, today's review questions. I'm not going to go over them right now, unless anybody has any particular questions, but if you want to see kind of what my answers would be, what are the key differences between object oriented and probabilistic, right? So you can go and read through that and bring your questions later if you have them. If we get into this, this could wind up taking an hour or 90 minutes, and I got other stuff because I'm actually starting to think now about getting you started on the assignment. So we need to get certain stuff in place so we can get going on that. But this is linked. This is called the AI model test questions bank. It is linked on day one of your class journal. If you can't find it, let me know. How do the concept of tokens, weightings and embeddings relate to the architecture and performance of AI language models? So once again, make sure you know this stuff is going to be very, very foundational. I mean, it's for the midterm. But honestly, I don't really agree with this thing of just studying for the tests, because if you will, if you're interested in this stuff, you should want to learn everything about it. And if you're not, then go and find something else you're interested in, because there's going to be a day when you wake up and realize all that time I thought I had I don't have it anymore. So make sure the years of your life you spend doing something interesting. So here is a question on the kinds of things you might talk about. When we get into working on our our assignment, we'll come back and talk on these concepts, but this is something we didn't exactly cover, but you should be able to put together the concepts that we've talked about. So we did talk about tokenization. We've done labs the last two weeks, embeddings, right? We have a section on embeddings, not deeply, but we've introduced it. Neural networks. I asked you to watch my video on ans and GaNS. I know from my YouTube Analytics who did watch it? I'm trying to stare accusingly to make the guilty criminals, um, nervous. Anyway, uh, training your chat bot, that's going to be your assignment, and then challenges and solutions we'll get to later. So anyway, make sure You read over that.
I Today is January 20, not the 21st
so a concept now which is pretty much a cross cutting theme, we essentially only have one main overarching topic in this course, which is building AI models to support that main topic we have, as I described up there some support topics, cloud DevOps, building a CICD pipeline, understanding the role of big data and conversational memory for the AI model. But the one main thing is building AI models, you're gonna get really, really practiced at doing that, just like if this was a database. Course, you know, you get pretty good at making a database with tables. If this was a C sharp course, you get pretty good at writing a C sharp program with classes. So by the end of this, if you do everything I tell you, you read everything, I tell you, you do, you're definitely going to have the confidence to walk into that interview for an entry level dev ops position. And I know because I follow them on LinkedIn. I follow my students, and I see students I've had in previous terms. So you know, I got hired at T Bank. I got hired at World Bank, Government of Canada, Statistics Canada, right? So those are big, high quality employers. You all like to work for good benefits, good pay, good ongoing professional career development, so on. So that's where we're going to be hitting it here. So this workbook is designed to provide you with hands on introduction building models, and the primary focus this week. Last week, we briefly introduced pytorch. Now we're going to do a side by side comparison, and my hope is at the end of today's class, you're going to be pretty by one. but you're going to have a start to at least develop a sensitivity towards what job is, the right one for each tool, and how they compare and contrast so learning outcomes to familiarize students with the basic concepts and tools required to create simple AI generative language models. Because I said on day one that there are several kinds of generative models. There's models that can generate video or images, business processes, mathematics and engineering formulas, I'm going to be focusing 90% on generative AI language models. I'll work a little bit of image generation into it, but that's pretty much when we talk about generative AI. We need generative AI text models, which, to be honest, in business, those are, those are kind of the main things they want right now anyway, like the Royal Bank on its customer website isn't really going to have a generative model for generating images or video. So if we get ourselves going to the language model here, we should be pretty good. This lab will address learning outcome in terms of your outline. Learning Outcome 8.6 applying machine learning models for big data problems using Python libraries such as NumPy, pandas, matplotlib, Scikit, learn and TensorFlow. If that keeps happening, I'm going to flip this over to an in screen or an in class screen sharing thing. But right now, we're just reading the lab book anyway, so you just read your own screen. I'll take this for a break in a couple minutes anyway. So you need to get some coffee. So learning outcome nine. Learning
Outcome nine, select and
apply appropriate artificial intelligence, machine learning algorithms, libraries, techniques, algorithms to meet the needs of specific business domain. Now, another thing that I love about unified process, compared to the older methodologies like waterfall and one of those other stupid ones that used to have. Can you remember their names? It's a way. Their names? It's a waste of time to know them. But unified process starts by studying the business domain, so we get the experts. In fact, unified process. Remember, we talked last week, and if you read my PowerPoint slides, you saw it there as well. Unified Process grew out of what, what was the problem the unified process was meant to solve.
Guys, don't do this to me. You're making me sad. Unified Process grew out of the software
crisis of the 1970s software products were Jesus Christ. Software products were built after the thing after the break. I'm gonna send out a zoom link for us, but yeah, so during the 70s, they tried to make software engineering look like civil engineering, not being sensitive to the differences between building a bridge and building a piece of software and unified process. Said enough. That wasn't really there, but I guess they probably said that. And they said, Let's start our software development effort by studying the business domain. So that's user stories, right? We gather all the users, all the stakeholders, we put them in a room, and we get them to tell us how they think the system should work, and that's user stories. And then we capture all of that. We generate use cases which are business process discovery mechanism. So we learned the operations of the business by asking the people we're using it, unlike the way they did it before, which is the programmers would make this stuff up as they went along. Programmers didn't understand how the business works. So hence the software crisis. And then after we get use cases that we forward generate, we make UML Unified Modeling Language was designed to support the Operation Unified Process. And then once we have our system designed in UML. The last piece of the thing we do is traceability papers. So anyway, the point of my story now is it all started by studying an in depth study of the business domain. Now in your midterm exam, you're going to get a question asking you to describe the upcoming software crisis, which I predict is probably going to become recognized in about five to six years, which is, I'm not going to tell you, I'm going to drop little dress up as we're going along here. You have to put it together yourself. I also have a blog article about in LinkedIn. You can go and read it there, but your question is going to be to describe the emerging software crisis. And I'll give you a hint. It's going to relate to not using AI properly. Just like the original software crisis, people try to apply software engineering practices that were out of alignment with the tools they had available. Same thing can happen. Now, remember what I said about AI hallucinations? I was talking because I also do some corporate training and I do some talks and so on. I was talking to the leader of, actually, was Bank of Montreal. What was that stupid? Basically, whenever somebody has, like, a works for a division that has services in the name, you should probably know it's something stupid, useless. But he was strategic, financial, Strategic Services outsourcing, and he was ripping into me about, oh, yeah, yeah, I'm gonna be letting go of all my programming staff within the next couple of years, because we're gonna shift over to an AI centric model. By the way, whenever anybody says something incredibly stupid, never get mad at them. Don't because they're probably doing the best they can. So just smile and nod like they're saying the wisest thing you've ever heard. So I know that's pretty good. Are you familiar with the concept of AI hallucination? He was like, what's that you need to worry about? Good luck if you're replacing your programs with AI thing anyway. Yeah. So that all of that is wrapped up in learning, outcome nine, identifying, developing. In fact, I'm going to exercise my secret power here to change the course outline developing appropriate Software Engineering methodologies and attendant software project management processes. I'm
I deliver AI appropriate, yeah, solutions. I guess this was good thing. So the lab will be conducted using Google collab, which we saw last week. Next week, we'll start to move ourselves into putting face spaces. So introduction AI and AML, introduction to these libraries, and discuss what these concepts mean. So let's start here. seconds. Let's take a break. It's 205 so let's give ourselves a break until 225,
at that one.
Let me You, them?
Laptop From Here,
you. You Can Ask AML,
It
It three magazine.
I The
status,
I it.
These are am
to take it.
I in Crossing a if also My clients
And the
Good job,
guys, you job, Guys, You
It user, you okay, Yoga, so you
Do come here.
So Ah,
Security, Okay,
it Okay. Hey guys, we'll pick it up in Two Minutes. I
But Then Plastic i
Yeah, just take off. So, yeah, I'm sorry to hear that. Take off and we'll catch you up next week. Yeah, we're going to talk about some general concepts, but yeah, just go, go and take care of your family, and we'll catch you up next week. I hope everybody will be okay,
alright, guys or guys now, because this projector is sort of dorked up and I don't feel like calling it again,
I've kicked out a link in your news forms. Just click on there. I mean, most of it's going to be reading the lecture notes off your own screen anyway, but if you want to share my screen, then That's where It is. You
100 boys and girls, ladies and gentlemen, children of all ages. Let's get started. Gonna share my screen here. And yeah, that's what I'm going to Do. That's about it. I guess you
now I don't know if this is going to work. It's the first time I've done it, but I had one student who just had a family emergency, so I'm going to see if this Google meet is going to record the meeting. I'm going to try it might work.
It might not, but anyway, it might help somebody who, um, sometimes you want to watch the stuff again later or something. All right, so I gotta share my screen and we'll begin. All right, so you guys who just joined us, I'm checking out a screen sharing link now to make up for our sick and absent projector, and let's go. So you Oh, I see, I see people are jumping in there. So that's good, all right, guys. So now I found, from experience, the best way to teach this is to put, you know, the old saying, don't put the cart in front of the horse. So we're going to put the horse in front of the cart, meaning I'm going to give you
some accountabilities,
which you need to do, and then I'm going to teach you the theory and so on of how to make it work. So we're already starting now to converge. Here we are in week three, and our assignment is due in week seven, right? So we've got about another month or so, another four classes, but they're going to go pretty quickly. So let's. now on the assignment, and I'll tell you what the assignment will involve doing, and you won't really understand it too well, but over the next couple of weeks, we'll build it up in little bits, and then by the time we get there, then you'll be pretty strong on it. So let's start with that right now. So I will start by kicking this out to your your Moodle page, and then we'll go.
So I guess it works easier for people. I've been told if I make it to be like a little assignment dropbox, so you don't know where to find it. Let's go and set that up. I
how they made the mistake of giving me time off during Christmas. I was away for a few weeks, so I forgot Now to do this, so hopefully I can get it back in a couple back in a couple seconds. So add a, add a, add an item here, where's the Add Item thing?
Ah, add an activity resource. And we will make this to be an assignment.
And this is going to be the one. It's assignment one, but it's actually the only assignment in this course, building and deploying
grown AI, and you can do this in group of up to four, if you want. That's what the college says, maximum four people in a team. So what a lot of people do, because the assignment is going to build up, it's going to
be a stepping stone up to the project. So it would probably make sense. I mean, assuming you can get along with each other, and nobody gets into a fight or something, to start with the same team and go through all the way to the project. So building and deploying your own AI language model, and your instructions are going to be right here. I'll go over more detail later. I'll talk about the grading rubric and how to hand it in and all that. But for right now, let's just start to talk about the theory behind
how you build an AI language model.
And the due date. I'll just put something for right now, because I have to put something, I think we said it was due after the break, or something, the reading week break. I'm just going to set it to um, because I have to set something here. I'll set it to the
maybe first week of
March. But I think it was actually a week or two after that, after the reading week break. I can't remember off him. So I'll just make it like March 1 for right now. But that was subject to change.
So anyway, the point is you can jump into there and get to this instruction sheet, which I'm now reading off of, oh,
it's doing this again. It was doing this last time. That's so irritating. It says I have to put a name there, but I actually have put a name. Ah, why are there so many problems to deal with? Okay, that is a title. Why is it telling me I don't have A title
because it doesn't like the colon. I
Okay, I guess I'll just
make a news farm announcement for right now, like figure out what's going on with this.
But last time, even though it said it didn't save, it actually maybe it did save even though it was giving that nonsense here, I.
No, I guess not,
right? I'll just make a stand alone page for it for right now. So I
Okay, I'll park it way up at the top so everybody can see it. I
All right. So anyway, the point of this is you can now get to this instruction sheet through there, so I'll start dumping stuff into there. So the way this is going to work now you can work in teams of up to four. You're
going to be applying the method studied in
class, you will build
an AI language model and train it on some training corpus of your choice, right? Some people might have a hobby, right? They like art or something, so they can build a model to have discussions about art. I've had students who've trained it based on, you know, maybe their religion, you know, their their holy books of their their religion, or so on, or basically anything that's interesting to you, you will build and train an AI model
on a training corpus of interest to you, I
you will build it based on a
foundational model, so you're not going to be reproducing what they did for chat GPT, right? Because that took, I think, like, three or four years. Cost about a billion dollars, and they had 1000s of programmers and business analysts involved and so on. So you're going to take a foundation model, maybe Vicuna, maybe chat GPT four, or there's dozens of them out there. I personally like Vicuna. Think it's very extensible and easy to work with. I'll be demonstrating that, but I've had students in past terms who go and find something else, research and do it. So again, I'll demonstrate that later on.
So chew you will choose some foundational model to build on top of.
And where are you going to get these models?
I hear you asking, well, they're available on model gardens, yeah, that's a thing. Model gardens on hugging face spaces, you will choose some foundational model to build on top of. You should start from now to become familiar you
with hugging, with the model garden on hugging face spaces. Now you can also, you don't need to do that. You can also just, if you're working in Python, there's a number of other models in different locations, but I think hugging face spaces is a great tool to become familiar with. So one of the
learning outcomes I want you to walk out of this course with is to be quite familiar with what we can do with that amazing platform. And it's free, right?
I pay a couple of bucks a month. I have a pro membership so I can share models and stuff out with students. But the stuff I'm telling you to get you you can you don't have to pay for any but, right? It's all freely available.
Sorry, there's something wrong with lean back here.
In fact. Let's go and take a look right now. Let's go and everybody make their own account on hugging face spaces. So that is hugging face.co. I think not.com.co think yes. So everybody now go to right here. Take a couple seconds and do it now, hugging face.co.
I think the origin of that was that there was a group of I think this was actually created as a student project at one of the universities in the US. And one of the girls the group was really into sending emojis. You know, this, this like hugging emoji and stuff. So I don't know, they just chose that as a name, but anyway, go and get that, and
let's go and set up an account. Where's my I
so go and make an account for yourself
over there. Give you a couple of minutes to do that. You're going to be setting up several subscribe, several accounts on different systems. My recommendation is you use your personal email, because the college email is going to go away when you graduate, and some of this stuff you you might think, well, that's pretty cool. I want to keep doing it. And even if your first job is not being an AI developer, right? You just get some job that you can get. So you got to pay for your food and rent,
but continue to be an AI hobbyist and just continue to, you know, publish blogs and fool around that on your own, and within a year or two, you'll get what you want, because the demand for this stuff is going to be going up, not down, believe me. So
I'll give everybody a couple of minutes of quiet so you can go and look, there you
And once you're logged in, so you can get data spaces. You can set up spaces. You can read their documentation, write spaces. Whether you're going to be doing that actually for your assignment, in your project, you're going to be creating a space for your team. You can see data sets. You can pull there. But for right now, take a couple of seconds and go to the model garden, and just Look at the models that are available right there.
You You guys are really lucky. I mean that because you're coming into this stuff,
I actually got into, I think I told you the story last week. I got into IBM. When the internet, the web, started to become a thing, nobody had the slightest clue what it was. So you have the first mover advantage, but honestly, that's going to be like a firecracker compared to, like the big Canada Day fireworks, comparing to this AI stuff, this is such an interesting time to be one of the first movers, One of the first
new entrants into this field. I
so you're basically going to be making an AI model
and deploying it to the hugging face spaces as your model server. Now you got four people in your team, so you got four accounts whose account you use to deploy it to. Well, just choose one person, but I still recommend strongly the other three of you still deploy it to your own account, because you're going to have a thing up there in the internet, right? You can post it in your LinkedIn and your resume. You can make a portfolio, and example of what your work is capable of
doing. So while you're there, also
take a look at the various data sets as well as spaces and read the documentation. So that's your little bit of homework for next week to get into that.
You so there's a good general model. I'd say 90% of students actually do this for their assignment, their project, they make a document, answering question. I'm. Imagine how powerful it is. Instead of reading a book, Imagine if you could have a conversation with a book, and you could ask it, because something I probably told you these stories before, because I tell them all the time, but I was very interested in when I was in university, I was very interested in history, except it didn't really pay, so I didn't do it. But now a thing that's fun for me to do is to ask my AI, you know, take causative
factors from this, you know, follow the Roman Empire, correlate them to what's going on in modern Western society. And if you can come up with a question, the valuable thing, the currency, the gold in the future, as a cognitive systems
trainer, is you don't need to know all the answers, but you do need to know enough to posit good and meaningful questions, because if you can posit a good question, the AI can always answer it. So here's some stuff, right? If you want to get into making an image generator, go for it. There's text generation. And here's models, right? You can look at the different model cards and see what's going on with them. So yeah, just spend maybe 20 or 30 minutes clicking around here and see what it has for you. There's your data sets, medical data, right? Industrial data, supply chain data, anyway. Now when I talk about the reason for doing this was so when I talk about a model server and a model garden, you'll have a little connection, right? You won't be sitting there confused when What's that? So let's go back to our instruction sheet now. Now one thing to bear in mind, hugging face spaces deployment, thank you, is a bit more tricky in the sense that with Google collab, you just open the Jupiter notebook and you start typing to get your stuff in right to get your code into
hugging face spaces, you need to commit it
to get and Then get a GitHub repository is your
gateway to get into hugging face spaces, not particularly difficult. And if you don't know how to use GitHub, we'll cover maybe even we'll spend the last couple of minutes today, I'll get everybody set up on being able to synchronize your local and your you know that Git is a protocol, right? What is Git? Git is a protocol like HTTP.
Is a protocol. Git is a protocol that attaches,
essentially a photograph or a record of changes to snapshots of your code. So you have Git and then you have github.com. GitHub is a server, just like you have HTTP and you have a web server. So I'm going to show you how to get to the point where you can make a GitHub repository on github.com then you can make a local Git clone, and then your changes locally, you can push and pull them up with the server. And once you can do that, then when you want to make your model in hugging face spaces, you just copy and paste in the name of the the GitHub repository that you want to clone any GitHub. So anyway, now you've seen that. So the assignment is to build and deploy a model to hugging face spaces. So as Judge, I was just where we left off, as I was starting to say, you should become familiar with the model garden and hugging face spaces, and you should start to think about what foundation model
you want to
foundation your model. You're probably most people use Bucha, because that's the one I demonstrate, fine, but you should at least be aware there's other ones, and potentially for your use case, there might be better ones
called foundation models, as you build on top of them, I
and then next you're going to train it so you're going to use methods such as we're going to practice today, in fact. And next you will train your model based on your training corpus,
your training set of text.
In fact, we're going to find out probably, yeah, we're actually, we're going to start to talk about it today, one of the elements of training is creating
and embedding. I asked you last where, not really, I mean,
I suggested, but it's not a very hard suggestion. If you really want to get into the philosophical background, you could read the book I showed you called lightness is dream. You. In which he was trying to build a calculus of human
thought, which is what today we would call an embedding. He didn't have the there was not enough support available in his time in terms of the math and so on. So he failed. He
only invented calculus when he was trying to invent AI. But that was still a pretty good accomplishment. So
as you progress with training your model, you'll be creating an embedding
to store model tokens and weights, and you will use Bayesian training methods in the course of doing this. Now, you don't have to know how to do this yourself. You just call the pytorch method. But I believe it's a good idea to have a intuition and intuitive sense of understanding, of feeling what's going on you
amazing training to do this. Now I have some lecture notes, which I'm going to kick in here with my strong encouragement for you to read
before next week, so you'll be a little bit more familiar with them. So let's go and get them. So embedding and
yeah, I've got a bunch of these. This is probably as good a one as any to get started with. There's some YouTube videos you can watch and so on. They're not my videos, but for videos which aren't mine, they're still not bad.
So building and embedding with an artificial neural network,
And also I have one on basing and training. I
This was actually the assignment from about two or three
sessions ago of running the course. But it
does. Do a pretty good job of talking about embeddings and
