Bristol Public Schools

N | Bristol Public Schools\_Warren Pre-Proposal Call Transcript\_4-1-2026 (00:00:04.633) Aaron Lyles: Um, yeah. So I think you guys caught the background of the name. (00:00:08.633) Carly Fortin: Yeah. (00:00:10.233) Aaron Lyles: No longer serious (00:00:11.533) Carly Fortin: And you got to meet Debra and you know her role then. Okay, (00:00:15.033) Aaron Lyles: great (00:00:15.333) Carly Fortin: Great. Unless you met before. (00:00:17.233) Leszek Ward: Yep. I'm here for real this time, not (00:00:19.833) Debra Vitale: I'm like (00:00:20.333) Leszek Ward: A combination of my phone and computer (00:00:22.133) Carly Fortin: And (00:00:22.533) Leszek Ward: wherever (00:00:23.033) Carly Fortin: I was (00:00:23.233) Debra Vitale: thinking (00:00:25.333) Aaron Lyles: It (00:00:25.633) Leszek Ward: It worked out. (00:00:28.933) Carly Fortin: And so after our last meeting, I had some thoughts about... I did have some questions about how we can make this work to capture student voice in the process that we built. One of my primary concerns was really around discouraging teachers from taking a more informal approach in the future as part of their regular continuous improvement work because of a reliance on a tool that they might find very useful. But as I was thinking about that, where I do think we have a need is for us as a district to gather a district-wide perspective that leads off some of the focus work that we are doing and the questions that we're asking. So last year, just to give you some things and some context, this year we've been focusing on student discourse and we want teachers to explore the student experience and discourse by observing, by doing some reading and research, and by talking with students. They started the year with three focused questions and discourse and concerns that came from teachers themselves. So an example might be what grouping structures or scaffolds best support students who might be uncomfortable in discourse or who might require accommodations or modifications. That was a combination of the idea that we want to improve discourse and concerns from teachers about, "I don't really know what I could do to better support all of our students." That concern from teachers came from a district-wide survey. I don't know how really representative it was; I think we only had about 100 teachers respond out of about our 600\. They're using writing to share their ideas. There's a qualitative nature to the teachers' perspective in addition to student experience that might help us formulate questions that would be relevant for primary grades K through two, intermediate, and middle. I thought that, and then it's again like helping us as a district really set the direction. Part of it, honestly, is the fact that it's working in partnership with an outside agency. It will help our... (00:03:26.935) Aaron Lyles: teachers (00:03:27.135) Carly Fortin: believe that this is not just Carly's manipulation (00:03:29.235) Aaron Lyles: to get (00:03:29.635) Carly Fortin: them to answer the question. We want them to answer, I think for a lot of reasons, it makes sense. I think about it from a district perspective, but those are some of my initial thoughts that (00:03:38.735) Debra Vitale: I was sharing it with you right now. (00:03:39.635) Aaron Lyles: He said, just send it out to you. Love it. Yeah. (00:03:43.735) Leszek Ward: And (00:03:44.435) Aaron Lyles: What are the age groups you know you're talking about? What comes to (00:03:46.635) Carly Fortin: Students, is it just 9 through 12 or is it? So we have a K through 12 system, and so I'd be looking for that. One of the challenges that we have encountered is trying to get more information about the student experience from K through 2\. You mentioned in the last time some kind of video chatting app, and that was really appealing to me to think about how we can use that to really capture the students' experience. Is there a way that we could feedback to teachers in a way that might be unique and meaningful and motivating? (00:04:16.635) Aaron Lyles: For them to think about their practice, yeah. Um, (00:04:21.335) Leszek Ward: I'm gonna try (00:04:22.535) Aaron Lyles: To pull that out too. I think I still have that to show. Um, yeah, we worked with second grade (00:04:28.435) Carly Fortin: class (00:04:28.935) Aaron Lyles: In Guildford and did video-prompted questions with audio responses back with no PII. And then the audio responses were obviously gated and (00:04:39.335) Carly Fortin: in (00:04:39.835) Aaron Lyles: A credential kind viewing basically for the principal, the three teachers involved. So they had access to hear the audio, they couldn't download it, they couldn't keep it. It stayed and we get to see who logs where, so (00:04:55.935) Carly Fortin: like very (00:04:56.635) Aaron Lyles: Like protocols, when it's that type of media. Everything's HIPAA compliant so there's no concern. It's (00:05:03.135) Carly Fortin: more about (00:05:04.535) Aaron Lyles: Just the logistics of how that works. So, you know, that was a fun one to kind of go in and I was the facilitator, so it wasn't the teacher, you know, so the kids felt comfortable and then they literally started interacting with the tools. (00:05:19.635) Carly Fortin: if it was (00:05:20.035) Aaron Lyles: a (00:05:20.135) Carly Fortin: person (00:05:20.635) Aaron Lyles: and it was my (00:05:21.135) Debra Vitale: daughter. So (00:05:21.635) Aaron Lyles: It was like a really cool experience. Um, so yeah, that's something we can do. It's a bigger lift with the younger ages, so the sooner the better as far as really diving in and figuring out its components. For the older demo, it's much easier. The writing component tends to be what most people click, but as you guys saw, I think in that last demo, you can answer with video, audio, or text. So if you're more comfortable just audio noting, or if you want to get on camera and do selfie style, whatever kind of caters to your preference. (00:05:53.935) Carly Fortin: Caters to the participant's preference. So, great, (00:05:57.935) Aaron Lyles: Our high school age students, that's how—yeah, that's great. Okay, so while I pull this up, I'll pull a few things to share, but any other questions or comments that you have? (00:06:12.035) Carly Fortin: Had, and thank you for the content. It's super important. I think one of the things that we're anticipating—and I don't mean to do all the talking, so let's check in. Debbie, you could jump on that. It's useful because the humanities and the STEM person here—writing is so broad as a topic. So I'm anticipating a lot of struggle in trying to narrow down what we're curious about regarding student experience with that. Are we really thinking about writing in terms of how students are formulating their thinking, or how they're sharing the polish of their thinking? (00:06:44.335) Debra Vitale: in the (00:06:44.535) Carly Fortin: end and, and our some area? So that is more about modeling and being able to communicate your ideas through what you're writing or drawing. And so that's where I'm like, there could potentially be a ton of information that we're going to need to try to figure out how to make it in a more usable, targeted way for (00:07:07.235) Aaron Lyles: what we're looking (00:07:08.835) Leszek Ward: to do for next year. Yeah. And I wonder, I'm wondering out loud, Carly, once you start describing some of those, to what extent—and the answer might be none at all—we want to at least pre-name some of those buckets, right? So that either the tool or the questions can start to feed into areas that we've already thought about, right? So, even like ready to communicate versus writing to learn. Like, do we want somehow to fork and guide responses into those, not into those two areas (00:07:42.335) Carly Fortin: like collect both or something. (00:07:44.735) Leszek Ward: Um, (00:07:45.635) Carly Fortin: And I was thinking that that's part of the (00:07:47.335) Leszek Ward: the (00:07:47.535) Carly Fortin: Initial setup, just given what we heard (00:07:49.635) Leszek Ward: about (00:07:49.835) Carly Fortin: The form in the past, like, I think that that kind of work with us could help some of that initial framing because we'd have to put it (00:07:57.935) Debra Vitale: Out in a way for people to gather that information. Yeah, I feel like writing in mathematics, in particular especially in the younger grades, might be an equation or an expression or manipulatives with something near it to communicate their thinking. It's not what it's going to look like in (00:08:18.235) Leszek Ward: Language arts. Let's say (00:08:19.535) Carly Fortin: Look very different and now there's parts of me that are just maybe focused on the communication of ideas in production, right? Something like something along those lines because it is (00:08:32.335) Debra Vitale: When I'm (00:08:33.435) Carly Fortin: using the framing of modeling, that's (00:08:35.435) Leszek Ward: what I'm (00:08:35.735) Carly Fortin: Thinking about it as a visual representation of the way in which you're trying to communicate your (00:08:40.435) Leszek Ward: radius. So (00:08:41.135) Aaron Lyles: I Yeah, and the tool and this is where, you know, the discovery will basically shape that path like full confidence that those things get ironed out and Discovery in the tools fully capable and I might be using you but it sounds very similar to what we do with archetype typing. So there's the two ways that the tool kind of approaches that and one is you go in saying, you know, we have these archetypes of our audience that we at least know where we think and then all The tool goes in and it on Earth's archetypes either (00:09:17.335) Carly Fortin: dispels or (00:09:19.035) Aaron Lyles: Confirms the ones you already had and then unearths new ones. So it's a good way to at least really figure out the cohorts of your students and your audience. (00:09:27.535) Carly Fortin: And (00:09:28.335) Aaron Lyles: Then as far as frameworks are concerned, I actually just did one yesterday. I'm working with Nesolagus, and we're doing some work there and there's a specific framework around racial and social justice. So, the tool knows that framework and what those four buckets are, so when it goes (00:09:47.835) Carly Fortin: to (00:09:48.135) Aaron Lyles: The analysis, it's actually using that framework. (00:09:50.635) Carly Fortin: So (00:09:51.635) Aaron Lyles: It's super customizable. However, you know, we need to do that, so, (00:09:55.135) Leszek Ward: um, (00:09:56.135) Aaron Lyles: Yeah, I think, you know, once everything looks like it's good, it's a (00:09:59.435) Carly Fortin: Goal. Like we can (00:10:00.035) Debra Vitale: Dive in and (00:10:00.835) Aaron Lyles: The tool (00:10:01.435) Debra Vitale: Do a lot of that work. Oh, sorry. No, go ahead. I was just thinking about, like, I don't know how this fits in, but kids that are in classrooms that are using, like, building thinking classrooms. (00:10:14.835) Carly Fortin: Their (00:10:15.535) Debra Vitale: Communication is based on collaborative nature. So how do we pick that part out? If, like, their framework around how it is compared to being in isolation and looks really different and kind of discriminating among classrooms, are you utilizing that structure in multiple subject areas so that communication could feel different to kids? But I don't know that. (00:10:44.735) Leszek Ward: I think some of it too is I don't know if it's going to come before or after the data collection that we're talking about with Aaron. Some of it is for us to also decide what we mean, like where we're drawing lines around it, right? So, like, we could take two paths and we say, like, we want it to mean, (00:11:06.535) Carly Fortin: like, (00:11:06.635) Leszek Ward: Writing, like written words with letters to the greatest extent possible, maybe bottoming out like K12 because that's where that bottoms out. But we really want to focus on a writing-intensive focus, to the greatest extent possible in all content areas outside that, or we could define it more broadly and say we don't think that makes sense. We mean physically visible representation (00:11:30.535) Carly Fortin: of ideas (00:11:31.035) Leszek Ward: and that much broader. (00:11:32.835) Carly Fortin: Exactly, so clear. (00:11:34.235) Leszek Ward: All right. That's not what I'm struggling with, because I feel like that gets way too fast if writing means anything from my essay to my whiteboard and building in classrooms. (00:11:47.335) Carly Fortin: I wonder if it loses any meaning. Like, I think there are two things. I do think that writing to learn and the modeling piece of it do strengthen (00:11:56.435) Leszek Ward: the way (00:11:56.935) Carly Fortin: That students are seeing connections between ideas and helps them see new possibilities. So, I think that is an area where I do want to pursue that. So maybe it's like framing those two kinds of categories and buckets, and then even the questions that schools can choose to do or teachers can choose to pursue. (00:12:16.135) Leszek Ward: Could be targeted under each one of those paths. And I think like one in either case, I guess I'm not quite (00:12:24.335) Carly Fortin: sure how (00:12:24.635) Leszek Ward: Much defining we want to do up front, how much we want to do after the fact. I'm cognizant of whatever we do before will then also shape the responses we get. That's what I'm struggling with, but I'm also just thinking about things like what people are not limited. (00:12:41.535) Carly Fortin: I did (00:12:42.135) Leszek Ward: not limit like the overall (00:12:42.835) Aaron Lyles: information. (00:12:45.335) Leszek Ward: Right. Based on what people don't know, they don't know, right? So like if, in (00:12:52.235) Carly Fortin: either (00:12:52.535) Leszek Ward: direction, in the hypothetical is always talking about. If I'm like third grade teacher and, or fourth grade, fifth (00:12:58.435) Carly Fortin: grade, teacher (00:12:58.735) Leszek Ward: Departmentalize the math. I'm like, writing, I got nothing, right? That doesn't apply to me and we think there might be some universe in which it does. I don't know at what point that matters or vice versa, like if I'm high school English and I'm only thinking about writing. (00:13:14.535) Debra Vitale: in terms of (00:13:14.935) Leszek Ward: like the essay and in revision, and that's my only filter for how I'm (00:13:21.135) Carly Fortin: gonna respond (00:13:21.435) Leszek Ward: to this. Maybe that's what this helps us unearth, right? Is it, just is that fourth grade math season 0, use case for writing, as we define it (00:13:30.035) Carly Fortin: and (00:13:30.235) Leszek Ward: high school English only sees this narrow use case, maybe that tells us something about what we want to do with that. But that's what I'm thinking about, like that. How to not let people's current getting out, like what change are in the future in we building? Some way (00:13:45.951) Carly Fortin: limit. Um, okay, so I'll share the screen information. That's and then I'll, like, one of the things that I think could be appealing about the go through way this. It's which the system is working. Because I a from lot. So what I (00:13:55.451) Aaron Lyles: again, (00:13:55.551) Debra Vitale: recall (00:13:56.151) Aaron Lyles: on, (00:13:56.351) Leszek Ward: just Debra (00:13:56.651) Carly Fortin: hasn't (00:13:56.651) Leszek Ward: say (00:13:56.751) Carly Fortin: She hasn't seen it yet. (00:13:57.251) Leszek Ward: Up, so (00:13:57.451) Debra Vitale: She's at (00:13:57.951) Carly Fortin: Like, (00:13:57.951) Debra Vitale: Any (00:13:57.951) Carly Fortin: Point, I don't (00:13:58.251) Leszek Ward: and (00:13:58.251) Debra Vitale: I (00:13:58.451) Carly Fortin: Promise. (00:14:01.151) Debra Vitale: Like, and let me, (00:14:01.951) Carly Fortin: I (00:14:01.951) Debra Vitale: I will (00:14:02.151) Aaron Lyles: take (00:14:02.351) Carly Fortin: No offense, imagine to it. It will (00:14:03.351) Aaron Lyles: take, (00:14:03.551) Carly Fortin: We'll fix it, okay? All right, so the response all back, right, so when could I help create people dig surveys a little bit deeper within the tool? And so my initial reaction is like, no, that doesn't apply to me, but we're like, no, a discovery tool. (00:14:15.151) Leszek Ward: Actually, so when we (00:14:15.751) Carly Fortin: Sit in a room, you could get these questions ahead of time based on conversations. I think that maybe this is some of the back and forth that goes deeper into the objectives and better and (00:14:23.551) Aaron Lyles: pain (00:14:23.751) Carly Fortin: than (00:14:23.851) Aaron Lyles: The points survey of the project, kind of, and data that audience we typically and what collect. We're, yeah, working on. And then I'll say, if when we you think meet things for on that, the workshop, I think what you're doing right now, like this noodling through, this is like, that's the good stuff and this is depending on how you want to structure it. What I love about and why I built this was to not do that. Well, we gotta get it all done in this one survey. The point here is we can effectively do this through multiple touch points that are more digestible, that have kind of their own mission. And if we do that route, the first one is kind of this, right? Like it's a conversation that we have to figure those things out, like how is the math department showing up and the English department, what does that look like to then? Okay, now we know collectively how people are comfortable, where they're going to show up, what that's gonna look like now. We launched a little bit more substantial survey, which gets to the core. (00:15:18.551) Leszek Ward: mission of (00:15:18.951) Aaron Lyles: The, our objective that we got out of discovery, right? So that's probably what I would think would make the most sense to kind of figure some of those things out. Also, the way to your point, Carly, about the, you know, when people answer, there is kind of a crafted response of like, oh, that's interesting, can you tell me why you said that? So there's also those branching mechanisms built in that. (00:15:42.651) Carly Fortin: allow (00:15:43.451) Aaron Lyles: the user to offer something to you. And unless I think (00:15:47.751) Leszek Ward: you said (00:15:48.051) Aaron Lyles: like to the blind spot, like we don't know, we don't know. And we don't want to force people into a box, and really just the way the questions are structured gets around a lot of that. Anyways, so the methodology is very dense as far as how to shape questions in a way that really yields exactly what you're pointing out. So I think there's a little comfort in the fact that the (00:16:14.451) Debra Vitale: tool does (00:16:14.951) Aaron Lyles: that by nature anyways, and Debra, I'll show you. So it makes it (00:16:19.451) Debra Vitale: I'll probably feels very (00:16:21.951) Aaron Lyles: Obscure, right? Yeah. So, and it's funny because again, not to keep mentioning any, but it's in this education world, so I think it's relevant. But there's a transition of leadership at the top, and that's kind of getting announced today. So for the people that are in the room hearing that, we created one of these conversational surveys with just five questions that do branch to say, how are you feeling right now? Like, when this person, when this transition happens, what's that going to mean for you and your department and this and that? They really get a true pulse and temperature to help ease this transition, which is going to be very big within the organization. That will then help us because we need to work with all these leaders in the work we're carrying on to do with the conferences and not the workshops. And one more quick example that we did with the Westover (00:17:13.551) Leszek Ward: school. (00:17:13.851) Aaron Lyles: Was we used this survey tool to get the sentiment from the student body. So then my DIB consultants could go into a workshop with, we'll go through all those questions together, and the discovery tool is kind of taking all that information, builds a technical brief that then gets uploaded into the survey tool that uses the methodology and all the context of discovery to create a really thoughtful question set. So, imagine all that already happened. Then we land in here, and this is the back end of what these surveys look like. And the question branching I was mentioning to you is here. So this is the note I think I showed you guys before, but this is, you know, multiple choice questions. So, nice to meet you. Name for students, we use names, but you know, typically it would be trying to have a personal conversation with somebody. Let's start simple. When you think about surveys, what's the first word that comes to mind? Based on every single answer option, there is a branch that speaks directly to that. So you feel like your response was listened to, and then you can continue in the path. It's all sorts of ways to archetype that. So what that looks like on the user side is, and we can kind of test this in tab, tablet, desktop, or mobile, it's set up so it looks like a text. So you're asking questions in a way that feels very familiar for the participants. So everything here is customizable down to the look and feel and to how we are using language, and each client has their own vernacular on what that tone needs to sound like. So this is another thing that comes out of discovery. We figure out who is the host, what does it sound like? (00:20:10.851) Carly Fortin: And, (00:20:12.251) Aaron Lyles: Sometimes in schools, it's not you guys. It is like us, who is on the outside, so that way there is that feeling of comfort while you're taking this. Everything's consent-based, so there's always an off-ramp, and then depending on what we ask and when, sometimes there's additional consent that just clearly states, like, hey, do we have your permission to do XYZ if there's something else. So I'm going to show you some question types in here, just to give you an idea. So this is what you saw on the background that looked like a chaotic mess of rabbit holes. This is what it looks like on the user side. So, nice to meet you. Aaron, let's start simple. When you think about surveys, what's the first word that comes to mind? If I choose boring, it gives you your little thought bubble for a second, and then it pulls up, I hear you, let's see if we can change that, and there's a meme. All these GIFs are interchangeable, (00:21:07.851) Leszek Ward: you (00:21:08.251) Aaron Lyles: can get them out, but it just makes it a little (00:21:10.451) Carly Fortin: bit more. (00:21:10.851) Aaron Lyles: fun, especially with your audience. And this demo one is just to kind of show you question types, but there are scaled ones that have emojis and, you know, there's a numbered one, so it really is the limit on what type of questions. The methodology is specific, though, as far as you're kind of easing into the heavier open-ended. So these lighter touch points are taking temperature and building trust. (00:21:37.251) Carly Fortin: and then (00:21:38.051) Aaron Lyles: Also, like reciprocating conversation to get to the more substantive questions. This is a multi-select here. What frustrates you most about the way feedback is collected? I just have to imagine this is your content and not survey-related stuff, but I hear you, Aaron. Now, let's narrow it down. So those kinds of little, you know, acknowledgment messages are deliberate, and they're in here, so it does feel natural. This is a ring choice question, so you can drag or (00:22:11.951) Carly Fortin: click (00:22:13.351) Aaron Lyles: This is a really good application, a lot of different ways to kind of see the (00:22:17.451) Leszek Ward: hierarchy (00:22:17.851) Aaron Lyles: Of thought. And then great, that helps me understand where you're coming from. Let's keep going. It's going to go through a few more question types. You can see the video component, show you what the data looks like on the back end, and then I'll quickly kind of show you what we did in Guildford with the little ones. (00:22:37.451) Leszek Ward: and then, (00:22:39.151) Aaron Lyles: Seconds. Any questions while I get to the video? Okay. All right. Now for the part that most surveys get wrong: open-ended questions. So this is kind of like a little blurb, and I think to what you guys are saying too, there's an element of storytelling. So, it's important that you guys tell your story as well of what are you trying to do? What do you (00:23:09.051) Carly Fortin: stand (00:23:09.351) Aaron Lyles: for? Like what you know, all that kind of like (00:23:11.451) Carly Fortin: Mission (00:23:11.851) Aaron Lyles: Vision comes through and it helps because it's kind of like you're giving vulnerability when asking for vulnerability, and it really helps with the engagement. This is why I love the conversational format here. So this is our multimedia question type. So obviously, you know, (00:23:35.951) Carly Fortin: I don't think you (00:23:36.451) Aaron Lyles: can hear that. I'm sharing the full (00:23:37.851) Carly Fortin: window but (00:23:38.451) Aaron Lyles: I'm saying things and (00:23:40.551) Leszek Ward: Nothing interesting. Clearly. (00:23:44.651) Aaron Lyles: Tells us our times. (00:23:46.951) Debra Vitale: Is incredibly valuable. (00:23:47.451) Aaron Lyles: We got. Thank you, you could read it. Um, so everything's captioned, and then the participant has the option to respond either by video, audio, or text to what we were saying earlier. So, if they do choose video, it will get set up here. You can write down what you're going to say here, so there's that feature if people aren't comfortable just doing it off the cuff. Once they're ready, they hit record. There are some permissions on your device that you'd have to agree to before this, of course. Then you would go ahead and record. You'd see that countdown based on how much time we've allowed. (00:24:26.951) Carly Fortin: Allowed. (00:24:27.351) Aaron Lyles: participants (00:24:27.751) Carly Fortin: to do, (00:24:28.151) Aaron Lyles: That's a great shot. If I hit yes, it sends. If I hit no, you have the option to rerecord, you could play it back, and then once you say I've finished recording, that gets logged into the table. Then you're going to see a consent piece here. One more thing about your recording regarding your response: I'd love to share your story. Would you be okay with using your response publicly? This won't be applicable for students, of course, but just (00:24:57.051) Carly Fortin: showing you (00:24:57.551) Aaron Lyles: Kind of, we always have consent in mind when we build. So now I'll go out and show you the data. Any questions on the way? (00:25:04.051) Leszek Ward: This survey tool kind of builds surveys, and basically, what you were, (00:25:08.951) Debra Vitale: then (00:25:09.351) Leszek Ward: Suggesting for the younger grade is that we can live almost entirely (00:25:13.551) Aaron Lyles: in that video response world, right? And just yeah. So what I can do actually is I'll just jump out now. (00:25:20.151) Carly Fortin: Um, (00:25:20.751) Aaron Lyles: so this is (00:25:21.251) Debra Vitale: what we (00:25:21.551) Carly Fortin: do for (00:25:21.851) Aaron Lyles: Gilford. (00:25:22.351) Debra Vitale: So (00:25:22.651) Aaron Lyles: There was this, like, you know, into your unique number. So what we did, there was no PII, every student was given, and we didn't link up with Google schools or anything. We just gave a random number, and I had a physical copy of that. So I would go in and I would know. (00:25:43.551) Carly Fortin: that (00:25:44.451) Aaron Lyles: Student blah blah was this person, um, and then. (00:25:48.251) Carly Fortin: as soon as that entered we started (00:25:52.651) Aaron Lyles: Um, so again, you're not going to hear this, but that's my daughter. (00:25:56.251) Carly Fortin: and (00:25:56.651) Aaron Lyles: She's asking questions and then the students would start engaging in the space. So we did a quant call where the student would say, yes, let's get started. Chromebook. Um, and then, you know, (00:26:12.251) Debra Vitale: once they've touched that they start (00:26:17.451) Aaron Lyles: How are you feeling about being at school today? Pick the emoji that best fits how you're feeling, so they would click it and then because this isn't open, like it's not an open LLM. This is all predetermined, so we would have generic responses back but still thoughtful. So it's like, oh, tell me why you feel that way. So this is where the student clicks record your answer, and they have the only option now is audio. So video is gone, text is gone, and then they would just click this and they tell their (00:26:48.951) Carly Fortin: story (00:26:49.351) Aaron Lyles: And it was fascinating to hear what somebody—it was really amazing just to hear the clarity and the honesty in the responses. It surfaced a lot of really interesting information. So that's what it would look like for the (00:27:03.351) Leszek Ward: youngers who are just the video. (00:27:05.151) Aaron Lyles: Audio, Bali. Um, and then on the back end, this is kind of what the data starts looking like when it comes in. The dashboards are designed so anybody, regardless of experience with data, could come in here and make sense of things. We have our basics on the project overview page, which is the summary of the project, the purpose to our hosts, or this is (00:27:29.651) Leszek Ward: kind of. (00:27:29.951) Aaron Lyles: for another day, but this is the scoring. So we score every survey we build and then we look to see if it hit all the objectives of our methodology, but also your objectives as the client. Are we making sure these things are considered? We use a really specific framework on this one, as you can see, and we missed a couple of (00:27:52.251) Carly Fortin: them. Right. So that (00:27:53.051) Aaron Lyles: was important to call out and, you know, I (00:27:56.551) Debra Vitale: would love (00:27:57.051) Aaron Lyles: everything and be 100%, not always. So we want to see where we can do better on that kind of stuff. So everything is in here, or for anyone who needs (00:28:04.751) Debra Vitale: to go (00:28:04.951) Aaron Lyles: down that specific rabbit hole and then we go into the student experience. So, for this one, this is a boarding school in Connecticut that is going through a transition. So they have given a lot of student aid, which now, they are kind of in a (00:28:21.951) Leszek Ward: conundrum. (00:28:22.551) Aaron Lyles: where they're having financial issues at the school and they've also had a much more diverse ninth grade class come in, and they have an all white staff. So there is this divide culturally and they want to get it right, and they really want to understand their student body. But also, like, how do we do this? And the tool, in this one, we didn't give it archetype hypothesis, and it actually went in and found these archetypes of thriving and connected, seeking voice, peer anchored, residential, challenge at risk, culturally navigating, and for each one of (00:29:00.551) Leszek Ward: them, it shows (00:29:01.551) Aaron Lyles: you your key indicators here. Like code switching was an issue that came up, they adjust their (00:29:07.651) Leszek Ward: ton of (00:29:08.151) Aaron Lyles: behavior. There were mixed rules of (00:29:09.651) Carly Fortin: perception. (00:29:10.351) Aaron Lyles: It'll give you a simple quote insight, so a lot of really (00:29:15.751) Leszek Ward: interesting qualitative feedback that comes (00:29:18.051) Aaron Lyles: through (00:29:18.451) Leszek Ward: Here. I'm stopping for a second. Yeah. So you said if you scroll up a little bit, you said those archetypes were discovered by the system, right? So it created driving (00:29:29.851) Aaron Lyles: connected and (00:29:30.551) Leszek Ward: Peer anchored as categories of students. Mmm, cool. And then those are the indicators that it used to do that. So under cultural navigating as code switching, yes. Was there a preloaded question specifically for code switching like (00:29:44.551) Aaron Lyles: was that (00:29:44.951) Leszek Ward: something that (00:29:45.351) Aaron Lyles: was (00:29:45.651) Leszek Ward: there? I'm (00:29:46.651) Carly Fortin: trying to understand like yeah, what (00:29:48.251) Aaron Lyles: was pre-built and what was not framework right there. Yeah, it's a framework. and in (00:29:53.151) Leszek Ward: each (00:29:53.651) Aaron Lyles: project is there is bespoke, like each project has its own? Yeah, when we worked with the Arts Council, there was a scoring system we created where if someone was a donor in the past or a current donor, they'd give an XML and they'd gone to one event. Like, you know, there were things that, depending on how they answered, it would (00:30:12.951) Leszek Ward: create our (00:30:13.451) Aaron Lyles: types based on that, right? So there was more of a scoring for this, the objectives of Discovery. And the framework that the tools learning on is how it developed this. Because it knew those, the things (00:30:26.751) Leszek Ward: I (00:30:26.851) Aaron Lyles: Just laid out right, like that divide, culturally different, all that, and then it used the framework that we gave it, the Muhammad framework, and then our methodology to come up with these. (00:30:39.351) Debra Vitale: Indicators. (00:30:39.851) Aaron Lyles: So it's super intelligent in the way that it—good question. And then all the transparency around how we're doing a lot of stuff is here as well. So I have an application out for the actual transparency initiatives to make it official, but we do it anyway. So this is kind of showing you all the data compliance pieces. This is the logic of how the tool is actually using logic to create that. And then you can always download the framework here and get the full, the full 60-page methodology. So you can always look at that. And then, for the insights, we have your key insights. Every question is broken down in really easy-to-read graphs that you can also change if you want to go deeper. There are cross-table capabilities in here as well, and that is somewhat built differently for each client. For the narrative data, each question that asks an open-ended question gives you the opportunity to see every single response given. Some people wrote a lot and others didn't, but everything is there. For the reports, I think the last piece of show and tell here is depending on what you guys need for the outcomes and reporting for this. We had a case study, a listening tour report, and then the facilitator guide because this is helping Surrey and Eric go and actually run that workshop. So it actually created a facilitator guide out of it. For the listening tour, this was kind of the overview of what was collected. The thematic analysis is in here. There's a pull quote here about how a girl really felt alone and how a teacher became like a motherly figure—just really awesome insight there. These are just examples of when adults didn't help. Here, (00:32:45.951) Leszek Ward: Here are some (00:32:46.251) Aaron Lyles: key things that came out of that. Pull insights, curriculum, real-world connections. This was just physical spaces as well, like how these physical spaces provided opportunities for kids to see themselves, and that came out of the data. (00:33:04.351) Leszek Ward: as well, (00:33:04.551) Aaron Lyles: which isn't something that we kind of predetermined. How people self-describe. So this is an interesting one; people were able to kind of click but also write in, and how nuanced they got on how they identify themselves was really interesting. So people didn't get put in a box that they didn't belong to. So anyways, guys, a limit on this is what the rank choice responses look like. It's just endless. And obviously, we will focus this for what your needs specifically are. But the tools (00:33:37.251) Leszek Ward: capable (00:33:37.851) Debra Vitale: of doing (00:33:38.151) Leszek Ward: a lot of stuff, (00:33:43.751) Aaron Lyles: Yeah, so that's my little spiel and show and tell if that's (00:33:48.551) Carly Fortin: Helpful and happy to answer any questions. (00:33:56.151) Debra Vitale: I don't (00:33:56.451) Leszek Ward: Have any more questions. Yeah, me neither. Seeing that (00:34:01.651) Carly Fortin: Seemed (00:34:02.251) Leszek Ward: Like seeing the video and the options. That is really interesting though. It does definitely change the feel of the data collection. (00:34:10.251) Carly Fortin: being (00:34:10.751) Leszek Ward: Able to like put a human on there, talking to you. And it's different, right? It's just, neither one's better or worse. It's just way different to have your daughter asking versus their principal asking. (00:34:21.251) Carly Fortin: versus (00:34:21.651) Aaron Lyles: me (00:34:22.651) Leszek Ward: Asking. Those are three very, very different things. (00:34:25.251) Debra Vitale: Yeah. (00:34:28.951) Aaron Lyles: Yeah, we had a couple kids who were like, you know, I really don't want to (00:34:32.151) Leszek Ward: feel (00:34:32.551) Aaron Lyles: Bad, but, and then they would say it. Yep. You know, they aren't gonna say that (00:34:36.951) Carly Fortin: without (00:34:37.451) Aaron Lyles: that outlet and that's the kind of stuff for me, like to see students open up in a way that they were harboring otherwise. So that's, to me, some of my biggest metrics that I look for: are we unlocking people to be themselves and feel okay saying these things? So yeah, it's designed for that and I think it's been effective so far. Yeah. So I think, you know, that component, like I said, has a couple different things, so obviously it can be my daughter. It depends on age groups, it depends on, you know, that kind of thing. And that came out naturally—she's close enough in age, she felt like a mentor-ish. But here, um, you know, and like they've seen Ruby (00:35:23.451) Carly Fortin: for (00:35:23.751) Aaron Lyles: so like the personality works. So make sense. Obviously not going to put crew in every project so we'd have to figure that out like who is the person asking? So some of the logistical things we adapt to figure out. But other than that, It's, you know, they would be very (00:35:41.051) Leszek Ward: similar (00:35:41.451) Aaron Lyles: There. We'd probably like to set it up the same way, and then, like I said, the older grades are much more just how (00:35:47.351) Debra Vitale: the (00:35:47.651) Carly Fortin: tool (00:35:47.751) Aaron Lyles: Naturally, the tool works. So I think what I can do is compile notes and lay out what this could potentially look like. I think what I would need is, excuse me, a timeline, budget constraints, all the details of the actual, not as fun things to figure out, to actually just go and start working, and I (00:36:12.251) Carly Fortin: just (00:36:12.651) Aaron Lyles: I'm a little blind there. (00:36:13.851) Leszek Ward: So (00:36:14.151) Carly Fortin: You gotta just. (00:36:15.651) Aaron Lyles: Well, we (00:36:17.651) Carly Fortin: might be, we might be a little blind too because (00:36:19.751) Aaron Lyles: I'm gonna write out he needs to pay. (00:36:21.251) Debra Vitale: To pay. (00:36:21.551) Carly Fortin: For it. (00:36:21.851) Debra Vitale: All right. (00:36:22.151) Carly Fortin: I agree. (00:36:25.051) Aaron Lyles: won't be (00:36:25.451) Debra Vitale: on that (00:36:25.851) Aaron Lyles: call. Great. (00:36:30.751) Carly Fortin: Yeah. Yeah, and our timeline. I know I kept saying for next year, really we're aiming for, like, I would say by November to have a complete picture of the information, the kind of questions we want schools to be going after. So having all of this information, like September or October, would be helpful. (00:36:51.851) Debra Vitale: The kind of (00:36:52.351) Carly Fortin: data (00:36:52.651) Aaron Lyles: that you're just sharing with us. Sorry, in our, you know, video aside, our timelines are very short. So, you know, we can turn around a full project in a matter of weeks. So I'm on your timeline, but calling that out because there's something (00:37:09.551) Carly Fortin: that (00:37:10.551) Aaron Lyles: before graduation that needs to go to trial (00:37:13.651) Carly Fortin: balloon (00:37:14.051) Aaron Lyles: or, you know, you need to set up for next year with the current student body. Like that's in the cards of my hand completely, it's not on yours, but I just want to call that out. Like it is. (00:37:23.851) Carly Fortin: Possibly need to start things sooner. (00:37:26.151) Debra Vitale: But... (00:37:26.451) Carly Fortin: Yeah, great. So the next step is that you're going to be sending more information. (00:37:32.251) Debra Vitale: And then... (00:37:32.851) Carly Fortin: I'm forwarding (00:37:33.251) Aaron Lyles: it (00:37:33.451) Carly Fortin: to right now. (00:37:33.851) Aaron Lyles: I think it's really (00:37:35.051) Carly Fortin: what I heard? (00:37:35.451) Aaron Lyles: Yeah. Oh, what I'm gonna do is. (00:37:40.151) Carly Fortin: Thank God. (00:37:41.251) Aaron Lyles: There's no Notetaker here. I'm gonna compile all my notes and I will kind of just put up a sample of ours. (00:37:46.251) Carly Fortin: If I (00:37:46.751) Aaron Lyles: heard you correctly. This is kind of (00:37:47.951) Leszek Ward: what the (00:37:48.351) Aaron Lyles: Project looks like. And this is how that would kind of roll out and just put the main components that we could start scoping around, and I'll, yes. (00:37:57.851) Carly Fortin: And then (00:37:58.251) Aaron Lyles: Right out, and you guys can hear what that (00:38:00.551) Carly Fortin: looks (00:38:01.151) Aaron Lyles: Like from a business perspective. But I think just, if you have an insight now it sounds like you guys have some ideas. Like, is this five surveys? Is this one survey that goes to four different cohorts? Any idea around that that helps me scope? (00:38:19.551) Carly Fortin: Without having more conversations, yeah, I think I would be looking probably for four to five different audiences because I think there would be, and I don't know. We'd have to, I would want to talk a little bit more about that. I think a 6-12 could work, grade 6 through 12 and intermediate, 3-5, K-2, and a teacher voice group I think would be important. So I don't know without really, like, I think our team needs to do some more work around framing out some of the more critical ideas and what we want to explore, and then figuring out how different the information will likely be from those different audiences, right? So I think we'd have a better sense after we have more of a conversation on, do we want it super broad, more discovery, or (00:39:09.851) Leszek Ward: do (00:39:10.151) Aaron Lyles: Do we want to try to tailor it to one direction? Okay, and these are four different surveys, because your K through 2 is not going to do what your 6 to 12 is, and your teachers will be different. So these are (00:39:23.251) Carly Fortin: Four audiences, four surveys. Same topics, same driving question that we're going to go after, so potentially same content. Probably (00:39:37.451) Aaron Lyles: Formatted in different ways. Yep. And it's not like I've built this (00:39:42.551) Debra Vitale: to (00:39:43.051) Aaron Lyles: Not spike costs (00:39:44.751) Carly Fortin: if we (00:39:45.151) Aaron Lyles: Need to make it that diverse, like that's by design. There will be some, but it's not crazy, so that is helpful for me to understand. Like, are we looking at data of one thing across (00:39:59.651) Carly Fortin: Four different (00:40:00.251) Aaron Lyles: Groups. (00:40:00.551) Carly Fortin: There's four things, four groups, you know, (00:40:02.151) Aaron Lyles: Is there four groups now, okay? And is there any parent play in here? Is there anything that you (00:40:09.951) Carly Fortin: Need parent feedback? (00:40:12.851) Aaron Lyles: The right answer. (00:40:13.351) Leszek Ward: Right answer. (00:40:13.851) Aaron Lyles: Should be yes or no. I'm sick of hearing compared. Let me do my job. (00:40:19.151) Carly Fortin: Compared. Let me do my job. (00:40:27.751) Leszek Ward: I mean, I think the answer is yes in that it's always globally good, but I just don't know how much insight a parent really has. (00:40:36.351) Carly Fortin: has into what we're talking about right now? (00:40:37.651) Debra Vitale: we're (00:40:37.751) Aaron Lyles: talking (00:40:38.251) Debra Vitale: about (00:40:38.551) Aaron Lyles: right now? (00:40:39.151) Debra Vitale: Yeah. I have to get an observation at 12:30, so I'm going to excuse myself so I can make it there in time but happy to continue the conversation. (00:40:50.451) Aaron Lyles: So thanks. (00:40:51.651) Debra Vitale: It was nice. (00:40:52.251) Aaron Lyles: Meeting you. (00:40:52.651) Debra Vitale: Aaron (00:40:53.051) Aaron Lyles: Nice being you as well, thank you. Bye. Take care. Um, yeah. No, (00:40:59.051) Leszek Ward: no problem. (00:40:59.451) Aaron Lyles: I'm not. I'm pushing (00:41:01.251) Carly Fortin: that. (00:41:01.351) Aaron Lyles: I'm just one of them and on, like, I mean, if you want to play with it on the Nesolagus website, there's an estimate thing at the top. You could actually go on and say, I want four surveys, I want this many people, and like (00:41:16.451) Carly Fortin: it'll (00:41:16.751) Aaron Lyles: kind of give you a ballpark estimate. These are, you know, I get more custom and nuance when it comes to pricing, but it's pretty close to that. So I mean, just to get you guys in a world where, okay, this is what we're (00:41:28.751) Carly Fortin: looking at. (00:41:29.251) Aaron Lyles: This is what they asked might be just you A (00:41:32.951) Carly Fortin: Runway (00:41:33.151) Aaron Lyles: of what those conversations need to look like, it could be helpful at any point, (00:41:39.751) Leszek Ward: you can also (00:41:40.351) Aaron Lyles: be like, hey dude, we are not close. Call me. All right. Let's figure it out. Like I'm not that is not my concern. Like I need (00:41:48.751) Carly Fortin: to (00:41:49.151) Aaron Lyles: feed my child. Sure, but I want to make your projects. I want them in the world. I want them doing things. That's my ultimate goal, so I'm just calling that out. So if you do play with it (00:41:59.351) Carly Fortin: And something doesn't look right. (00:42:00.251) Aaron Lyles: Right. Just reach out. Great, but cool. So that's my homework. I will get you something that's shareable that gives at least a broad scope of what this can be, and then we'll go from there. I think the next step would just be to figure out budget and green lights, and you need to approve. (00:42:18.951) Carly Fortin: And that's for next year. Sounds good. Sounds good. And I know we have a meeting as part of the Real Impact, which is the grant that I believe is going to fund a portion of this, next Thursday. So if we had that (00:42:30.751) Aaron Lyles: kind of (00:42:30.951) Carly Fortin: Information before going into next Thursday, it would give (00:42:34.051) Aaron Lyles: a chance to be able to talk with others about it. Awesome. What would be best for you to share? (00:42:40.851) Carly Fortin: Because I'm happy to put a deck together for this. I don't think I need to be. I think so. Right, Aaron, I'm familiar with your work. So I think it would be more of us just talking about (00:42:51.251) Aaron Lyles: what (00:42:51.551) Carly Fortin: we're intending to be the scope of the project and then the information (00:42:55.351) Aaron Lyles: will be able to gather from that. So I don't think you have to go (00:42:57.551) Carly Fortin: Through. Crazy. Okay, (00:42:58.851) Aaron Lyles: cool. (00:43:00.251) Carly Fortin: Awesome. All right. I will have something to you before that. (00:43:02.851) Aaron Lyles: sounds good. Sounds (00:43:04.351) Carly Fortin: good. (00:43:04.451) Aaron Lyles: Thanks for the time. (00:43:05.451) Leszek Ward: Yeah. Thank you. Take care. Take care. And thank you.