Hey good. I'm looking forward to shoveling at lunch. Yeah, I'm not. No I'm not either.
Like I hurt my Bye buddy. I hurt my Achilles tendon a couple of days ago so I feel like I won't be very efficient at shoveling. No
sounds like other people in your buildings problem. Maybe
I'll do I'll go do a little bit I like to do the street a little bit so that people can walk by but yeah, I just like I wore old shoes and it just like rubbed up. I walked like six miles and like just up against the back of my ankle. And so it just like it hurts. Weirdly.
That's terrible.
It's so stupid. I was like, this is the stupidest thing if I just like thrown away the shoes. Yeah,
I've been there. I'm definitely done there.
Hi, guys. Hey. Nice to see you, Serena.
Nice to see you too.
I know Jenna's not in so I'll be in charge of recording the meeting. So don't let me forget. I'll just give everyone a minute or so. More to join
you drooling. Oh, you are gross. You're talking to me. Oh, is drooling
capsules? Not gross. It's adorable.
It is pretty adorable.
It is. My cat always rolls that's like the standard state of being. So
I've got some serious cat withdrawals. I can't I can't have a pet because I've spent half the year away. So every time I see when I'm like Elmira
you got to have like a share. Adopt with another person
I'll get jealous. If the cat likes the other person.
That's true. That is true.
All right. Do we think this is just us? Oh,
my goodness. What's on the agenda today?
All right. Um,
I think you're you have a presentation but like no one signed up to do the redwood release planning which could probably just be an inform.
Yeah, I guess I'm happy to present for the sake of the recording if you guys are interested. And then I can just make a note about Redwood release planning
I think it's up to you like is it fair? My hair's a mess. Do you think it'd be better as a discussion
um, I feel like I just want to get it out there. I want people to like go to the document and add their comments. So to me the sooner it gets out there, the better but so I'm happy to walk you guys through it. Serena you may have already looked at it but
oh, did I
maybe I shared it with you on Thursday night so maybe not.
I skimmed it but no, I didn't read it very closely.
All right. If you guys are down, I'm down here. All
right. Yeah, works for me. And work. Some use otter pilots is on so there'll be nice notes taken as well.
Yeah, there's notes on all the like cat drooling and stuff. Cut out notes from above this point. Are you guys okay with me recording this meeting? Awesome. All right.
All right. So the agenda for this meeting is pretty short. But I'm going to walk you through the product requirements document for the aspects v one release. And then just a quick inform that Redwood release planning sort of has started. There's a link in the agenda to this. The notes from the first Redwood Redwood release planning meeting, which was actually great, it was a pain storming exercise. So a brainstorm for the pain points with the existing release process. So definitely take a look if you're interested. And then just a note that there is this community release plan sheet as well. That sort of has a Gantt chart for our release planning moving forward those features that are being planned for 2024 and 2025. So definitely take a look at that as well. But without further ado, I'm happy to to get into this presentation. About aspects view on release. Let me stop sharing this and change the tab. Okay, alright. So I'm just going to quickly walk through this aspects v1 Product Requirements Document. I have a work in progress label on it, mainly because I'm still working with our engineer on just those technical requirements. We do have a technical approach drafted, but he's still finalizing it. So once that's linked I'll remove this sort of work in progress label. I'm definitely open to any comments, feedback, definitely take a look. Ask any questions, make any suggestions that come to mind and there's a link to this in the meeting notes. So without further ado, so what is this document? The document is really those product requirements for our v1 release of aspects which the goal of the release is really to deliver an MVP experience for our instructional designers and course teams to access a curated or out of the box set of reports and visualizations needed for analyzing course level data. So I definitely understand that there needs to be sort of cross course as well as whole instance level reporting down the line. But in the effort of sort of focusing in on one need before sort of going super broad. I did a lot of discovery on what is this course level? What are these course level reports look like before we zoom out and sort of add to those in future releases. So any questions before I move into key use cases? comments, concerns? All right. So this key use cases section I won't go through it in detail because these are literally user stories. Essentially, what are all the needs? What are all the specific requirements for each of these generalizations? But more broadly, these, these use cases really sort of a lot of them won't be surprising if you're familiar with aspects. They'll cover enrollment data for the course. Learner Engagement data for the course. One thing to call out, that's different from aspects is one need that came up in user research was really the need to be able to add a glance look at drop off for the course. So there is an additional report here that I'm hoping sort of gets to the crux of that question. So it's really showing the number of learners who are engaging with content over the span of the course at both the section and subsection level. So you'd be able to select if you want to look at folks who are looking at the pages in the section or the subsection, if you want to look at the number of learners who have looked at who viewed part of or a whole video and a section or subsection sort of across the span of the course. And the same for attempting a graded or ungraded problem as well. And then, again, performance metrics, nothing too surprising here, very similar to what we have in insights. But the big thing that is new and different with this aspects view on release, and this is something that came up in the research. The user research that was conducted is really can I look at information for an individual learner, someone who maybe comes to me with a question or an issue with their grade? Can I see a sort of snapshot of that learners progress through the course their engagement with the course. Or if I'm at a more high touch environment? Maybe I need to see maybe I'm meeting with a student and I want to see a snapshot of information about that individual learner. So this would be a new sort of intro. Something that's introduced with this V one release that is very different from us from insights would be a individual learner dashboard that shows information about the learner their first name, last name, username, first name, last name if provided, user name and email, their enrollment track enrollment date, date of the last visit to the course. Their current course grade, and then their engagement and performance data for the course. So similar to what we provide sort of for all learners, but just for that one learner, and then also a similar dashboard for a group of learners. Can I look at information about learners that may be I deem at risk? Now I my hypothesis is that it would be extremely useful to be able to configure a learner group based on certain criteria that's important to me that I consider at risk. In my instance, but for the sake of this V one release, we'd show essentially the same information that we show for individual learners for a learner group. One preset learner group, defined by items defined by criteria that may mean this learner is at risk that sort of surfaced in our research. So can I see this information for the a learner group of at risk learners defined by learners who have enrolled in my course who have not yet passed the course? Who have not visited the course in seven or more days? And a learner who has done something other than signed up for the course. So, here I have, they visited a course other than the course homepage. They've watched a video, they or they've graded, they've answered a graded or ungraded problem, or they've posted in the course discussion forum. So this would be sort of allowing you to sort of hone in on a specific group of students and look at engagement and enrollment performance data for that group of learners. Any questions about that?
Yeah, how often do the learner groups like refresh because presumably somebody will get out like those criteria won't apply to them anymore. Make a learner group that's like my definite like if I make another learner group with a different definition I that might change week on week. Absolutely.
It's a great question. And one I actually should ask, Brian Messick our engineer on this to confirm because at least my understanding is that the goal of aspects and this like new new set of technologies that he's using to build aspects would be to present data as real time as possible. What that actually means is a great question in the context of this criteria here. So I can actually add that to the open technical questions section of this document because it's a great one. I will
Yeah, I could actually see multiple answers being quote unquote, correct. Because I might want to see later in the day, the same group, even if somebody I've contacted has moved on, because maybe I still want them in that set. So I can like, make sure they're headed to the right track. Right. So if it refreshes every five minutes, that might not be correct, but it might be correct. I don't know. I don't I have no opinion. It would just be good to know.
Right? Absolutely. And it's a great question. One, one thing that may be helpful here instead of well building out the like ability to save a learner group in V. One is that one report that I think would be useful for this group is sort of the contact information and names, usernames, emails, as some sort of downloadable report. So presumably, you could like save a learner group but in a very low tech way by downloading the report and saving it and naming it but it is a great question I can. I'm going to take that down. So I have it in my discussion with Brian later today. So beyond these use cases, I think something that would be important to just call out are the out of scope. items as well as future direction. So in building out these requirements for the v1 release, of course, there had to be some things on the chopping block so that we could get this thing out the door. So the in scope items are out of the box, course level enrollment engagement and performance reports and visualizations sort of discussed in the use cases section of the document. And then the out of scope bit are those cross course and whole instance reporting for instructional designers program admins, instance admins, you name it. Those will be out of scope for this release. Mainly because we want to do more discovery work before really defining what those reports look like and visualizations look like. One thing that is sad, especially given how important demographic information is for a large set, of course authors and instructional designers is the need to sort of revisit learner demographic data. So access to demographic data is important to many course teams, but will involve not insignificant amount of heavy lifting sort of on the back end to enable. So there is a future direction section of this document that outline specific demographic use cases that came up over and over again and user interviews. I've captured them in the future directions section of the document so we don't lose them while we figure out how to enable these sort of on the back end. So the ability to so in scope, the ability to view contact enrollment engagement information for individual and learner groups. And then the out of scope would be the ability to configure a custom learner group or an elegant mechanism for contacting all learners within a group. We do provide email addresses in this release. But contacting and nudging at risk learners or groups of learners is definitely something that came up in user research, and may be worth exploring. potentially even outside of aspects. Maybe this belongs elsewhere on the platform, but it is something that did come up a lot in talking with instructional designers and course teams. So that would be out of scope for this release. Another thing that may be out of scope is embedding those dashboards within Open edX LMS. For now, the idea would be to present these visualizations and reports within superset. But in the future, potentially exploring how do we bring these reports elegantly and in a way that they can be seen? large and in charge on your screen from inside the Open edX LMS. And then, information about time on site time and course. Time on page data for all learners, learner groups and individual learners is out of scope for this release. The reason I call it out is it has been requested. However, it's something that would be extremely hard to calculate. So that is out of scope for this release. This just divvies up the features. Potentially that would be an involved in creating this release and what the requirements are. I just want to call out one non functional requirement here. The idea would be that this release can support instances up to 2 million learners. That's the goal. Anyway, and then this is an italics this bit about language because I'm still working with our translations working group to really define how many languages are going to be supported out of the box and then what are the goals sort of long term, and then the future direction section sort of paints, what I'll be focusing on beyond this initial release, or what we'll be focusing on. So this goes without saying but of course conducting discovery to inform what are those cross course, reports and visualizations that program administrators as well as course authors want to look at in order to compare courses across one another. Compare enrollment across courses or programs. What are those visualizations, for instance admins that need to be surfaced? And then beyond that, further improvements would include these are really detailed level but these are requirements that were in our V one release, but upon further sort of technical investigation got sort of de prioritized so as I called out the the needs around demographic data would certainly be something that I would like to support or explore in a future release if possible. Other things are embedding these reports within the Open edX LMS. And then there are a few others but definitely take a look at the document. For more detail, and definitely add comments and questions sort of as you go through it. Any questions? All right.
Just one question. So has V one has it been released or to release it okay.
This is to be released. So there is a proof of concept or a beta in superset that you can view to sort of see what superset looks like with similar reports. However, this v1 release would be really building out the reports sort of as defined by user research, so we did I think it was 11 or 12 user interviews and then combed through our discussion forums and slack to look at the needs of course authors to really define what are those? Out of the Box reports we need to support in superset for workhorse authors, and those reports need to be built. Good question.
I think there was something at the end of the document that said, what was released in v1 and then I was like, what everyone's been released? Oh, you're speaking to your future self on what you do. At that point?
Yes, exactly. That's the what happens next. Yeah. All right. Any other questions?
I'll just add for the sake of people watching this. I do know a little bit about translations. And my understanding is the translation working group is working to support some number of like, officially supported languages. I don't know what that number is or what the languages are. And then I think beyond that, there are people who say have an interest in a language we don't support, who are free to do the translations themselves, and they can pull in those translations. But just because someone gets a translation up to 100%, doesn't necessarily mean we'll support it officially. We've had the problem in the past for like one dedicated person goes in gets the language up to 100% and then never does it again. So that language drops to 80% 7060. So I think the translations working group is is focused on making sure that a set of languages is always up to date.
Got it? Yeah, I do have so I don't know how up to date the document was that I looked at from Eden but I have I just want to make sure this is right before it being like firmly in this document, but I have 19 but as soon as that's sort of confirmed, I will make this not italics and kind of make it less wishy washy here.
Yeah, that makes sense. 19 is definitely more than I know of so very cool that there. If I assume that that means that they've expanded the number of languages. So that's pretty cool. Yeah, it
was definitely on like a translations Working Group document. So yeah, no, that's exciting. Awesome. I'll stop sharing. Here. Anything else? I think that's it for the agenda.
Did we want to just make a nod to the red wood release planning? Yes, yes. Which I think is just we're planning to meet monthly from here on out. Those meetings are on the shared working groups calendar, and we have a Slack room. Do you know the name of that slack room offhand? If you just look for Redwood, Slack, you'll find the slack room so anyone's welcome to join the meetings. Join the slack room, even if you're just curious and you just want to follow along
Yes, it's Redwood underscore release underscore planning. And I'll throw it in chat to
Ooh, good. Question, Ryan. Um, I am not 100% Certain on that. I can ask Jenna. I have a meeting with her later today. She's actually out this morning. So I will. I'll see what she says. And if there's an update, we can throw it in the the core or the product Working Group channel. If that helps me I can I can figure to Okay. correctly. No worries. All right. Anything else?
I would just say Ryan if you're gonna ping Jenna, can you do it in the overall chat room? Because I am also curious about the answer to that. Sure.
All right. Well, if we don't have anything else, have a good rest of your day.
Thanks for the presentation Chelsea. Beautifully done.
