So good afternoon again and welcome to today's Nonprofit Net Zoominar on why evaluation matters. I’m Dennis Boyer the president of the Nonprofit Net board of directors. Speaking of evaluation, we'd like to have your comments on today's program because it helps us plan future programs and it's important feedback for our speakers. There's one more important way that you can help Nonprofit Net. We need your financial support to keep our zoominars open to everyone without charge. Of course we have expenses to bring each session to you, so we hope you'll consider supporting us with a suggested gift of ten or twenty dollars. Maybe even more. I'll be sending you links to our feedback survey and our donation page through the zoom chat in just a bit. On now to today's presentation. The key to improving operational efficiency and effectiveness is evaluation. There are three components to that: measuring impact, communicating the impact, and the tools to use to apply the results to your own organization. And here to lead us through all of that is Nina Selvaggio. Nina. Hello everyone. Thank you for joining. Happy Tuesday. I’m sitting in Boston if folks want to put their name pronouns where they're sitting in the world, what organization you are representing in your chat, please feel free. Welcome. I use the she set of pronouns and run my own consulting firm creatively named Nina Salvaggio Consulting. So, I also will take suggestions for more creative consulting firm names in the chat too. So, welcome everyone. I’m excited to be here with you all today. My core passion has been as a nonprofit professional in the sector for 20 years is really about creating goals, metrics, data systems that integrate and allow for improved ways of operating so that our talent can really focus their time skills energy on bringing a mission to life. So that's how I … that's the frame in which I come to all this work. One second, it's not letting me… there we go. All right, so that's a little bit about me. I have a lot of bubbles popping up with people wanting to be admitted that dennis is taking care of. So today the agenda, like Dennis mentioned, we're going to talk about measuring and impact. First, we're going to talk about what is. Impact. How do we decide what impact is and why do we measure it. Then we're going to talk about a tool that I have used in several organizations to measure impact and lastly, we'll talk about communicating out our measured impact. I’m really screaming it from the treetops, so people know how and what we're doing. So how we can learn here today together? I wanted to tell you what I’m going to offer you. So, I’m going to offer energy, practical advice tools, and best practices. I’m going to also try and not tab too quickly. I’m going to offer time for questions, small breakout rooms. I’m going to occasionally poke fun at myself and maybe others and then give you all some real-life examples. And how I’m hoping you all can support me is come on screen, chime in, use the chat, ask questions, share your advice and tips, laugh and hopefully take some deep breaths and just really be present. All right. Let's get right into it. Let's talk about measurement and why we measure impact. Ultimately before any organization could do that, we need to talk and folks need to have a firm grasp of what your organization's impact is right and then you're going to articulate it out and measure it and then share it, share it internally and externally. So, every organization hopefully here has a mission statement or if you should that's probably a different training that you need to sign up for at some point. A mission statement - and we're going to start really basic a mission statement - declares an organization's purpose and why they exist. It offers a general description of your organization, your function and your objective. And you'll see that at the top of this triangle; it definitely works to inform your vision. And I know, I know, a lot of you already know this but we're going to make sure that we're all on the same page. Your vision - and a lot of organizations don't have these, but some do - your vision is really your inspiration it's the framework for your strategic planning. It's what you imagine your organization will be known for in 50 years. If you achieve that mission statement, right, the world will be better because of the work you did, and the world will be better in this specific vision of life. Your theory of change, which you'll see is really sandwiched in the middle, and it's your secret sauce but not really secret at all because you write it in all your grant proposals and talk about it to all your funders. And we're going to talk about your theory of change which is critical to this entire measuring impact concept in a few minutes. It's your unique selling proposition, right? There are a million organizations that seek to end hunger, but this is your theory of change, the specific way your organization wants to solve and end hunger. It gives you the ability to compete in our very competitive funding landscape, right, and to better serve the folks you seek to serve. So, your theory of change is the ways in which you see your problem being solved uniquely. It's the foundation of your organization's ability to know and then communicate out your impact. And that is ultimately your strategic management plan. So, let's talk about your theory of change. A theory of change is an organization's roadmap that outlines the steps by which you plan to achieve your vision and therefore the problem you're attempting to solve. It helps you define whether the work is contributing towards achieving the vision and mission, right? So, are you doing what you're supposed to be doing? And successful organizations have theories of change that actually affects change so that they can realize their vision and mission. So, your theory changes all those levers you're pulling to hit on your vision and mission. It obviously needs to have your initiatives, project, programs, aligned with it, right, so you're not going to do anything that's counterintuitive to your theory of change. And the most important component is to monitor your outputs and outcomes and then track those metrics that let you know like are you being effective. So, if you think an early intervention with regards to reading, and I'll give you a strong one here, if you think for example the YW Boston, their theory of change is a great one. So, they say through our diversity, equity and inclusion services as well as our advocacy work and youth leadership and empowerment program, we help individuals and organizations change policies practice attitudes and behaviors. So, they're telling you exactly what levers they're going to pull, right? So, they're going to work on diversity, equity and inclusion services. They're going to focus on advocacy work and they're going to do this work in this special way to see to achieve their mission statement. Are there any questions on that theory of change? I’m sure you're all familiar and I didn't include a logic model slide here because that seems painful. But a logic model really helps a lot of organizations in the beginning articulate their theory of change. But it's really the ways in which we figure out what are those indicators so that we can then measure it. All right, so we're going to do breakout rooms, and we are going to spend some time talking about your organization's theory of change. So, because it's so fundamental to how and if you are making impact, you need to be sure that you're clear on what it is, that you can then figure out what to measure. So, I’m going to give all the breakout rooms … how many folks? We have 48. Folks I’m going to give everyone 12 minutes. Dennis was going to create some breakout rooms and folks will have about two minutes each. And really talk about your theory of change and what needs to happen within your organization to hit your mission but then that helps articulate and shed light around what you're going to measure. I hope everyone had good conversations does anyone want to share out anything they talked about. Well, I’ll share here. I don't know whether - Anyway we collect - I think most of us find that it's really hard to measure the impact that we create. In part just the definition of the beast that we're a part of is it's hard to measure. And secondly is that, well, at least my add would be, it's hard to measure the impact if in fact you don't design measures into your program initiatives. Absolutely. 100%. And we're going to talk about that. Oh good, good. I think that that is right. I was in a couple breakout rooms, and we were also talking about the challenge of articulating our true impact. And I think at times it feels really slippery and we don't always know how we can talk about what we do and sometimes it feels unrealistic. Folks want us to do pre-tests or post-tests or longitudinal studies, right? You're just like, the folks I serve are transient, or the folks I serve - you know, it's just very difficult. And everyone wishes they had more data or different data. So, my suggestion to folks is always to work with what you have, right? So, if you have – you probably have too much information, too much data, some of which you never use. So, take a look at it and figure out what you have that could be meaningful and could tell your story in a better or different way. And there may be data that you wish you had. Put it on a on a wish list and figure out how to work towards getting it. Sometimes that takes systems investments, and people, right? Like hiring someone to do that evaluation. But just know that everyone struggles with that and we just have to do the best we can with what we have essentially. But there are little nuggets everywher. Like everyone has something good they can talk about we just have to figure out how to talk about it in a way that other folks see that. Other thoughts or questions that came up during breakout rooms? All right. I can share a little bit about my group. We only had three people in my group. and we all work for small non-profit organizations. So, and we also measure and talk about, you know, first we don't have personnel in to do the monetary evaluation and the measurement. And secondly, also we don't have a tool. And that's the purpose that we joined this workshop. We want to learn. Yeah, and also, we mentioned about why we want to learn. Because when we apply for grants, they always ask how do you measure your success, what's your measurement tool. So, we don't have any. Maybe we really did a great job, have a great impact, and you know, but we really don't know how to present those. Yeah. That’s the reason we wanted to join the workshop. Thank you, great. Yes, thank you, I - that's exactly right, we're going to - I’m going to give you an example of a tool that I love, and we can talk about how to adapt it to different organization size. And capacity is a real issue for sure. So, paring it down so it's realistic. Miranda, go ahead. I'm just wondering, and if you're planning to go over this later then then that's really fine to talk about it later. But I’m wondering if you have any thoughts on people who believe everything you're saying but aren't a director or an executive director. Like how do you suggest, how do you suggest, what's the word I’m looking for like, like yeah just bringing this into - I’m looking for a word that reminds me of off gassing, but that's not what I mean. Just like slowly penetrating this into the space in a soft but direct way. Yeah, I mean one of the things that I think is really powerful is the how do we define success. So, when you're sitting around with colleagues and team members like how do you, how do you define success. So, as you're talking about doing something say great one year from now let's say we do this new program - one year from now - how do we know we did it? Well, because you know one of the things I joke about and I made up a word is like you can do anything like shittily right? Like just because you did something doesn't mean you did it well. So, defining success is often the first thing I ask people clients to say. Like this is how we know we did this well. And one of the interesting things in a breakout room was what if we have some pet projects that don't really fit in our theory of change. And my response to that is your theory of change is your road map to achieving your mission statement. If something isn't on that road right in that theory of change, you're not working towards your mission statement. So, you shouldn't be doing it. There shouldn't be a pet project that doesn't fit on that road, right? So, to me, defining success and measuring your impact is really about how do you say no to stuff, how do you say you know what, that's a great opportunity. We just can't this year. There's only three of us on staff. We would love to do that that, sounds great, but we just don't have a capacity. Because our theory of change dictates we do things we believe, right? And your theory changes the hypotheses like obviously if any of us knew how to solve the world's problems, we wouldn't have jobs. So, it's a hypothesis but there's a million different ways you can go. So, stay focused on your hypotheses and sometimes you got to say no. So, we're going to talk a little bit about a tool that you can use. But I believe and I love is the guiding force to showing your funders that you are measuring what you do and how you do it that. You're being thoughtful and strategic but more than anything it's about how to stay focused, right? There's always going to be another grant that's appealing there's always going to be, you know, someone who has a great idea. Tere's always going to be the wily founder who's a super visionary. He's like let's just do this and all the staff sitting around are like oh my goodness you can't do one more thing. That's always going to happen. So just staying grounded in a tool and measuring that impact I think is key to eventually being successful and delivering on some of your goals. Miranda, that was a super long extraneous answer, but I think defining success right off the top is often a way to start infusing the organization with that notion. Last thing I'll say before we move on is the nonprofit sector very much around measuring impact is like a person who goes to like a new class at a gym, right? They decide, oh, you know, I've never tried Pilates but I’m going to go to Pilates and then the next day they wake up and they're like oh my gosh like I think I’m having like an appendicitis like weird parts of my body are hurting. And it's like oh no you've just never worked those muscles before and measuring impact is like that for nonprofits. For some nonprofits, you've never worked those muscles before and so laying down some of these foundational conversations like what is our - how do we measure performance. Doing that right off the bat is a way to start to work Yeah, this is a perfect time, yep. Hey Andrea. Oh, Andrea's on, it's a perfect time during my webinar. She wasn't interested. All right, let's keep going and hopefully she'll go into another room or realize she's not muted, all right. So please hold. Now my I've lost my screen. Okay. All right this is 199. When Saved by the Bell and Girbaud jeans and all kinds of fashion wrongs were obviously very hot and uptrend. But it wasn't the only exciting thing that was happening in the world in 1992. Robert Kaplan and David Norton were working at Harvard and created something called a balanced scorecard as a management system and tool and their goal was to motivate breakthrough improvements and innovation. They focused primarily on the fortune 500 Companies. They believed Coca-Cola, for example, could improve their profits if they monitored customer satisfaction and their own internal workings. So, this balanced work card tool that they started to launch was really the first strategic planning and management system that was adopted within these orgs, it was all haphazard and thrown around in lots of little ways. Much like you all do, right? So, like what's our output? How many people do we serve? Are people happy, right? They were doing the same things. They created this tool to align their organizational activities with their vision, mission and theory of change, right, through strategy of your organization needs to be aligned with your external and internal communications, and then they monitored it, right? So, then they're like okay let's pay attention to this stuff, this stuff's important. Then in the 2000s when denim and Britney and Justin were very popular, the nonprofit sector’s like a light bulb went off. They started to recognize the need to be more rigorous. And funders and individual philanthropers thought, well, I understand that you change hearts and minds, but we want to know like what part of the heart and what was the percentage and what was the pre-test and the post-test of the brains that you changed, right? It was much more rigorous. You couldn't just say we did this and people doled out cash anymore. So savvy nonprofits in the early 2000s started to really adopt a balanced score card too as the foundation and pinnacle of their strategic management performance. This is what a balanced scorecard looks like today. It's been through lots of iterations. There's lots of different templates online. If you were to google it, you'd come up with a whole host of visuals. It's a management tool that helps us monitor and measure our performance. Okay. So, this does not create anything new in terms of work to be done or processes to implement, right? Like this is not some - this is not a project or a program unto itself. It's a new way to simply organize our work and our data and how we talk about it. So, I’m going to explain how it works. So, aspect, you'll see that all the way to the left. Aspect runs along the y-axis as a bound scorecard always. This is a tool that is follows this formula. The aspect category is the work that you do that shows your priorities, okay, and it never changes. So, for Coca-Cola, they are not going to have impact. And this is - it's the exact same tool for Coca-Cola. They're not going to have impact, they're going to have customer, right? So, for sustainability and operations, maybe they have operations but instead of sustainability, they have profit, something like that. And then talent is your foundation, your people, right? Okay, so moving on from aspects go to objectives. Your objectives are pulled directly from your strategic plan your annual goals. These are the rows on the scorecard that align with the aspects right that you strive to meet so for impact. You have an objective which you want to make an impact. You want to improve academic gains for young children living in poverty. That's your objective. That's what you're striving to have an impact on, okay? These are tangible short term three-to-five-year objectives and measurable. You're not going to change these every year because lots of us know you don't see significant changes year after year. You need to build, get some momentum. Leslie, I love your face on my camera. You're nodding along. It makes me so happy. Thank you. Next, we have measures. Okay the next line. The next column. Measures are the statements identified as meaningful and valid ways that correspond with those objectives. Okay, so you want to improve academic gains for young children living in poverty. Your organization is going to look at what are the ways in which we do that, right? Our theory of change is that we are going to read to kids in classrooms. So, let's say we are going to saturate high need areas in the geographic areas we serve and increase the classroom served, right? That's that we're going to measure, that we're really going to make a concerted effort to saturate and do more and go deeper instead of being all over the place. This organization wanted to really focus. These are changed annually or can be changed annually. Sometimes they may stick because you may decide, oh we're getting some – like, we need more time, like, we have some momentum here. But they can be changed annually because again like our theory of change it's a hypothesis. So maybe we put a measure on there that we realized didn't really do anything to help us move forward. And we're going to take it off the next year. Then last but not least - and this is an example of three columns of metrics but some organizations only have one or two - and your last column is your metrics and data. And this is an actual percentage or number based on careful analysis or sometimes a stab in the dark, if you don't have careful analysis available, of past data and goals or industry standards or historical performance, right? You say, last year we were in 604. So, we've been trending up every year, or actually we don't know. We've never collected this data before. So, we're going to say let's increase by one percent. And then a goal would be - that's a result - the goal would be five, actually five rather, but then let's say a stretch would be seven. And let's put some stretches there. Some of the stuff is stabbed in the dark. Some of this stuff is to push the organization and be ambitious. But ultimately all of these things define success. So Miranda, it's like how do you have these conversations? It's like someone says, hey, you know what? We should start to really concentrate our efforts in one geographic area. It's like okay, well, what would that look like? How would we know we were successful in doing that? And so, start to adhere and stick data onto some of these conversations, and that will hopefully help us drive our actions. Because the measures which we're going to talk about next are going to drive our actions. Questions, thoughts, comments before we move on to some of the measures work? Nina, this is Dennis. Now we have two questions in chat. They appear to be from the same person. Pardon me. One is talent - does talent include basically anyone who makes it happen or just staff or whatever? Great question. Talent is your staff. Talent is your aspirational staff or your actual staff that you have accessible. I mean, I guess I should say volunteers for some organizations should be included in that, so it's the people that make the work happen, but it's not, you know, it's not program affiliate directors. It's not anyone who you can't have a strong impact on, right? Does that make sense? So, it's not – no, Logan, so, for example, if you're working at, you know, you’re running a program at a YMCA, you probably don't have a ton of impact on how the people who work at the actual YMCA, how they experience their day-to-day and or how they impact your mission or goal, right? But if you have volunteers who come in and are actually executing and working towards your mission, you have the ability to impact them. Thank you for clarifying that. That makes sense. That example made sense. Thank you. Perfect. So, with talent, talent is, you know, at the bottom, appropriately so, because they are the foundation to all the work that we do. So, you know this organization is retain high performing high, potential staff, so they had a retention issue. So, they want to work to retain them. So, you put this here and that's great but now you actually have to do work to deliver on retaining high-performance staff. Which is why it's important the definition of doing that well would be 85% of who you identify as high performing, high potential staff stay. It drives behavior, right? You cannot change what you don't measure. Okay, we have two more. What does stretch mean? Stretch is like, this is like, oh, this could hurt a little bit, but I think. If we could do it, it would position us even better. Like this may require us doing things differently or looking at it differently. Or investing differently or trying a little bit harder than we have in the past in different ways. So, a stretch goal is really like it would be amazing to accomplish and to hit too. But we know probably based on historical data or industry standards, it's not super realistic. But we're going to like give it a college good old try. Okay, and how often should an organization revisit a balanced scorecard to track impact effectively? Oh, my gosh, who asked me that question? Seriously? That is - yeah, well, I was just going to give them a shout out because yeah because that's a great question. Because we're going to talk about that. That is exactly part of the implementation because that is key to watch it, monitor and adjust it. So yeah, we are going to talk about. That is from Antonino. Oh, Antonino, we're going to talk about that in a hot second so thank you. Other questions and then we'll get into the measures because the measures are the things you can change annually. They're really the bulk of this tool. So, as you have grant requirements for your impact, promises that you need to keep it might go into one of those measures. But the next year that grant is only a one-year grant and you're going to - you're not going to continue to do that work in that exact same way or you're going to find some of these measures are ineffective or not the right things to look at and that's fine. You're going to change them next year. People can jump in as we move forward. Sorry, I keep losing my cursor. Okay, so those measures that take up the vast majority of your balance scorecard, you leave them there. You set them annually. You do not change them. Like no cheating, no adjusting them, right? You set them. You put a metric and you're going to focus on them for a year. So, the measures are really important to holding everyone accountable within the organization to get the work done and move the needle towards your your mission and your annual goals. So, they should be transparent so everyone in the organization and/or outsiders so funders, key philanthropy individual givers, right, like whomever you care to share this information - it should be very transparent. It's really the cornerstone of how you run and manage your organization, these measures, and like I said before it helps you stay focused. So, you know when to say no, you're like you look at the balance scorecard tool. You think these are the measures; these are the things we're focused on. Which by the way I didn't even mention this, you should never have more than 11 measures on a balanced floor card. Because that's 11 buckets of initiatives or work or or actions that are going to tease out until like a lot of different things and so you should just focus no more than 11. 9 to 11 max. And then we use it to monitor and adjust our performance and behaviors that impact those measures. So, if you realize behaviors are being executed differently you need to figure out how to tweak them in real time so that you can actually be successful. So, these are some of the high-level ways that ideal what your ideal measures should be. So shared value. So, if your organization has value statements, they should align with your mission statement. Nothing about them should be counter-intuitive or shocking to people if they know your organization. When they see your measures, they should reflect those high standards of performance sometimes include some of those stretches that are a little bit harder to achieve. You want to represent future accomplishments, so you want to put things on there that maybe you haven't achieved yet, but you hope to someday achieve and this year you're going to give it a go. They should conjure up an image or a picture like - I love this one - like when you read a measure, you should be able to envision the kids sitting around partaking in this activity and being successful, right? It's like your theory of change is working, like you can see it. It should be that unifying theme. So, one of the things we talked about in the - oh sorry in my workout in my breakout room - was the unifying theme. So, if your organization constantly is coming back to something, some word some concept, it should all line up. They can be tracked. They are linked to the impact. They are very specific so that there's no if, ands, or buts about it. And they can be challenging but achievable. My best. The organizations that are most rigorous and hard on themselves tend to want to put things on a balanced scorecard or measures or even data points that are so hard. And I - this is not meant to be a challenge activity. This is not meant to like make people cry, right? Like this is meant to really bring your organization into alignment with all that you're doing and be rigorous but ultimately better serve the folks you seek to serve in a very concerted way without feeling like you stink and you're not doing a good job. So, you want things to be achievable, like this isn't - like no one should come to work and cry. Any questions on measures? Okay, so the balanced scorecard really takes the organization on this journey, which is why we went through like the vision, mission, theory of change so quickly. Because that's really the foundation for this. It's where we are now right? it's the challenge you're seeking to solve and the reason why you exist is because you believe you have a solution, that if folks pay attention to it, i.e., give you dollars, donate, volunteer, you can take action and make a difference. But without knowing your impact and then being able to measure it and then share it out, you aren't able to impact those who most need the services because you could be doing things that aren't really working, right, but no one knows and no one no. one's talking about it. The goal is really to create that innovation with your theory of change. Like you want to make improvements on how you're serving whomever whatever population and trying to eradicate the problem you seek to eradicate so it's really think about it as a pathway to action. You're starting in one spot, and your goal is to end up somewhere else but it's a process. Question spots. All right, Nina, I had a question for you. So, going back to the pathway - pathway to action slide - you mentioned the theory of change. So sorry can you just clarify like is the pathway to action - are the pieces that are in this slide kind of what we're going to be using to come up with the theory of change? No, your theory of change is where we are now. Your theory of change is part of your starting point. It's, you know, why you believe your organization should start. Because you have something special to offer the problem. You think that you have a solution that's not already out there I mean. There's lots of duplicative work happening out in the sector but ultimately you think that you have something special or unique to offer and you see this problem you're going to dedicate resources, people, time, energy to solving that problem. In that way, that's your theory of change. Whatever - like I said it's like early intervention reading, it's, you know, then once you have that foundation established, now you need to start measuring what you're doing because theories of change are theories right? Like we don't necessarily know they're the most effective things. I mean there's lots of research out in the world that tells us certain things work better than other things with different problems. Like there's definitely research out there to show our theory of change can be effective or not. But you want to constantly measure what you're doing to ensure that you are in fact being effective. And you know that might mean you have certain organizational barriers. That might mean you don't have the right people in the right place. You might be lacking policies procedures. Maybe you don't have enough money. You're in the wrong places, right? So, you're trying to figure that piece out so ultimately you can improve or change the story. You can solve the problem in a different way. Cool. That’s helpful? It is. Thank you. Other questions? Again, there's one in chat. Can you use stories and testimonials as a measurement? No. Not on the balanced scorecard. So not as a data point. So, you're never going to have on the balanced score card like create a strategic plan, right? You're never going to do something that's like a check mark yes or no. You're going to have that data point, that metric, that percentage. You're going to have something that tells you whether you did it well because like I said, we can do a lot of things. That doesn't mean we do them well. So, you can have a quote. You can have a quote, a testimonial. You can have a story. Those are fantastic. You want to have those to complement the actual data, right? So, you want to have something that tells the story of how fantastic your services are. Like a real live human utilizing your services or being a recipient to your program etc. But ultimately you need to be able to say for example like if you have one testimonial that like 92 percent of the participants who came through your program believed the same thing right, to show the power in that one voice. You want to be able to say it's not just this one voice. Literally it's not just this one voice. 92 percent of the people believe that or something like that. Like you want you know and that's why sometimes folks get really balled up and how rigorous the data has to be or how thorough, longitudinal, and how many people. And it's really about looking at what you have. Is it a three-question survey when participants first come to your program? Is it an evaluation right after? Is it like what are the ways in which you can do this that make it easy for you with the resources, people, power that you have? And that you can easily implement right? Like some people are like, we would love to have this data. It's like right, that takes like 30 ipads qr scan and like all this stuff. Like we can't do that, so you got to back it up a little bit. Maybe build up for that. So, love testimonials but you're not going to be able to just have testimonials. Sorry, I'll go faster. Is that it, Dennis? Yes, that's it from the chat. Perfect any other questions from the non-chat folks? All right, we're going to take a quick - can I do a time check? What time, is that all right? We are going to skip our reflection. And last but not least we are going to talk quickly about articulating our strategy and then sharing it out. So, this is really the key part, the final step to now you've established what you're going to do, how you're going to do it. You're going to figure out how to measure it. But then you need to communicate it and engage not only the staff but external stakeholders in it. So, you know you've articulated the strategy, you've adopted a balanced scorecard. You're going to monitor your activity. You're going to manage your activity. You're going to have strategic success and then you're going to communicate it out externally and internally. Sorry I’m just trying to move my mouse to here. So, the how is really like that balanced scorecard right? Figuring out what your measures are that you're going to look at for a year. Where you're going to communicate it out, that really depends on your organization. So, board meetings, staff meetings, funder engagement meetings. I was a managing director for a national organization, and part of my role was to communicate out the balanced scorecard. So once a month I held an open call and anyone from the organization could just call in and listen to me give an update. So, whatever the forums are that you have available. Annual meetings to members etc. You share this information out because you're proud of it. Because even if you're not hitting some of your goals you're still working towards it and you're pivoting and making changes. Why. We understand the why now right? Hopefully that it's obviously the - it's the funding landscape. It's because we want to serve people, because we want to serve people the best we can and do right by them. So, who's who communicates out this information? Again, it depends on your organizational structure. It's your ED, it's a managing director, it's an impact director, it's a program director, right? It's someone who's going to have this as part of their job responsibilities to really watch these measures. Again, why you don't want more than nine to eleven, okay? The what is a balanced scorecard tool. Something where you're condensing all the core goals. And then when how often do you do this? So, monthly depending on the organization size, quarterly depending on the audience, right? So, board meetings - if you have a quarterly board meeting, you share it out quarterly. It really will depend on the venues. Again, but this isn't something you create, put in the drawer, or leave on your desktop and never look at again until the end of the year. You're not going to have success. You have to keep it for at the forefront of your brain. So, I would encourage you to look at it more than more than even quarterly. So, I know we are close to time. Does anyone have last questions? So, what did you skip in reflection? Can you tell us in one sentence? Sure, I was going to have you jot down what are some of the measures your organization should be measuring or is measuring. Okay, thank you. Can we have your slides sent to us? I’m sure. Let me talk to Dennis about that. Okay, thank you. And there are lots of different versions of a balanced scorecard tool. This is the most modern or current iteration of it. Some of them were circles so it really - and you know some people pare it down - so it's really about making a tool that works for you. But this premise is really powerful with regards to following through and true to your impact and then really figuring out what are those measures to drive behaviors, and then how you define success with that data. Yeah, I think we just started having the tools like that and communicate them in our board meeting. Yes, that's fantastic, yeah. You know, and then that's where like those anecdotes come in and you're like telling those mission moments, right? Where you're giving that data and then you're saying, oh, this actual human right? Right. Those two things paired together are really powerful. Thank you, Veena. Thank you. All right. I will stay on for a few minutes if people have questions. But thank you all. Dennis, do you have a wrap up that you need to do? I certainly do and it begins with thank you Nina. That was so much wonderful information and done in a very you know wonderfully compact way and so we all thank you. Thank you, and as we kind of have your words echoing in our heads we're going to turn ahead to next week's zoom pardon me next month's zooming out. The words diversity equity and inclusion and their acronym DEI are everywhere right now, and as businesses and organizations struggle to express support for these principles it's not always clear what these words even mean. It can be a special challenge to meaningfully apply them. So, our may speaker is a recognized expert in the field, Leora Norwich, the Executive director of the network for social justice that's Tuesday May 10th from 1 to 2 pm. And just before you take off a reminder your feedback on today's program is very important to us. We're going to email you a link to our satisfaction survey. And also, please remember that hundreds of organizations including yours have benefited from our seminars. And the way we can keep them open to everyone without charge is with your financial support so please consider making a gift of ten twenty dollars or even more we'll be emailing those links and survey links to you so thank you once again and we hope to see you in May.