Speaking the Truth with Conflicting Data

The C H Smith & Associates team joined with our former client representative Dr. Susan Zelman of the Ohio Department of Education to present at the 2018 annual American Evaluation Association conference in Cleveland, Ohio on November 3.  In a presentation entitled “Speaking Truth with Conflicting Data”, we shared with other professionals in the evaluation community about our experiences of evaluating the first two cohorts of the Ohio Straight A Fund (RIP).  So while the Straight A Fund is no longer alive, we did not want its lessons to be left in the graveyard. Our aim was to start a discussion in which all of us could acknowledge some challenging aspects of the field and get unstuck in our search for solutions.

The truth is that even when data are collected effectively and appropriately, the story they tell are not always clear.  We illustrated this through sharing our encounters with messy, complex, and conflicting data in the world of K-12 education evaluation.  For this problem, we relied on a storytelling solution.  We made a decision that it was appropriate to let the conflicting data co-exist to give a complex answer on project efficacy that arose from data triangulation.  In practice, this approach meant detailing the school district’s data dilemma in an insert to the full technical report, with a goal to build empathy in the client for the sub-grantee.

Besides providing the typical “lessons learned” from the perspective of the evaluator, one aspect of our discussion that was notable was the participation of our client, who commented on her reaction to the challenges that came up and the solutions that we chose.  It seemed that the audience of evaluators found this perspective illuminating, with a few attendees making comments about how these views related to their own work.

It is so often such a positive experience to look back on past work with colleagues and to share knowledge within a professional community.  Thanks to Dr. Susan Zelman for partnering with us and to all the other generous presenters at AEA2018!



Posted in Data Driven-Decision making, Program Evaluation | Tagged , | Leave a comment

C H Smith & Associates Team Member’s Article on Advertising and Nutrition Published

Dr. Andrew Menger-Ogle is our talented research and evaluation associate here at C H Smith & Associates. Recently his research article “Television advertising of food during children’s programming in Nepal” was published in the scientific journal Nutrition.  The study found that 55% of commercials during children’s programming in Nepal focused on food and that 70% of the food advertised had excessive amounts of saturated fat, sugar, and/or sodium.   The World Health Organization recommends marketing practices that would decrease  children’s exposure to these types of commercials. A video compilation of the commercials analyzed in this article  is above and can be found on our YouTube channel.

You can find the full research article at this link free of charge until August 15: https://authors.elsevier.com/a/1XI4C3CdHsfS0X

C H Smith & Associates is committed to strategically delivering data to inform our actions as individuals, organizations, and communities.  We look forward to building off of Andrew’s expertise and expanding our work in public health and nutirition.


Posted in Data Driven-Decision making, Market Research, Nutrition, Program Evaluation, Public Health | Tagged , , , | Leave a comment

IT IS What IT IS: A Planning Approach

As I was preparing a presentation to fellow small businesses for the Greater Cincinnati/ Northern Kentucky African-American Chamber of Commerce, I was reviewing the strategies that C H Smith & Associates has deployed with various clients and in its own business planning. The phrase, “It is what it is,” came to mind.  We have turned this phrase on its head and instead of being a term of resignation, it is now a term of empowerment.

The C H Smith & Associates “IT IS what IT IS” planning process looks at the current state and desired state of an organization through these principles.

I:  Intention

T: Truth

I: Inquiry

S: Sustainability

In the initial phase of implementing the process, organizations reflect on their current state and aspirations:

  • What is the Intention, (hope or desire) we have for the future?
  • What is the Truth about where our organization stands today in pursuit of that intention?
  • What are the lines of Inquiry we should pursue to further understand how to make the intention happen?
  • What are our Sustainability concerns around this intention?

To answer these questions, C H Smith & Associates works with clients to look at the data about the organization, its industry and markets, and capacity.  With this new information in hand, we facilitate a process to flesh out “…what IT IS” for an action plan:

  • How should the Intention be articulated in a specific goal?
  • What is the Truth about the specific objectives and tactics that need to go into action?
  • How should we establish active Inquiry for a continuous improvement process?
  • What are our plans for Sustainability of this work and how will we secure and manage the supports and resources to reach our goal?

Using “IT IS what IT IS”, one can reinforce a frame of mind that is positive and success-oriented for the organization’s culture.  If you would like to learn more or explore applying this approach and its detailed processes for your organization, reach out to me at chs@chsmithassociates.com.

© C H Smith & Associates, 2018

Posted in Data Driven-Decision making, Non-profit Organizations, Organizational Performance, Strategic Planning | Tagged | Leave a comment

How the North Pole Works

For the holiday season, enjoy this story explaining how Santa “makes good things happen” …

Christmas time was nearing, and Mae had been a very good girl this year. Mae discussed her Christmas wish list with her mother.

“Mommy, will Santa bring a puppy, remote control car, a tablet, and a walking doll for me this year?”

“Well, dear Mae,” says her mother.  “Santa, with his varying appearances and languages, hears every child’s request. He and his helpers do their very best to bring all the joy a child needs, although he may not bring all the toys a child wants.”

“Why can’t Santa bring me everything on my list? I have been very good this year,” wondered Mae.

“You have been a very good child, Mae.” her mother replied.  “I should explain how Santa and the North Pole work to a good and smart girl like you.”

“I want to know how the North Pole works!” said an excited Mae.

Her mother started to explain, “Santa counts on mommies, daddies, grandparents, and other kind people to give money to keep his shop running. Santa does quite a bit of fundraising to get all of the supplies and permissions to make the toys.”

“Why does Santa need permission to make toys?” asked Mae.

“The toys you see in the store were designed by people who have to give permission to Santa to make the toys they design. To recognize them for giving him permission to make the toys, he pays them,” her mother continued the story. “His handiwork, international delivery, and –most importantly, his joyous spirit are free. He needs money, however, to build toys and hire some of his contracted special helpers.”

“Mommmmy,” Mae said with doubt. “What is a contracted special helper? I have never heard about any contracted special helpers!”

“Santa can’t make puppies with his hands, and Santa can’t get all the toys made in time when the children don’t get to sit on his lap until a few weeks before Christmas.” said her mother. “His contracted helpers help get the gifts that his workshop cannot make and work with the shop to make sure everything arrives on time.”

“In fact, I am a contracted special helper,” whispered her mother. “You don’t hear about us because we like to stay behind the scenes.”

“You are one of Santa’s helpers!” said Mae in wide-eyed amazement.

“That’s right Sweetie,” said her mother proudly. “In fact, I talk with one of the elves every day in the weeks right before Christmas to make sure we create the right amount of toys that meet Santa’s quality standards and that we deliver to the warehouses on time for Christmas.”

“What are warehouses?” Mae queried.

“This is where the toys are stored across the world so that Santa can pick up the toys on his journey,” explained her mother. “His sleigh would be too heavy if he tried to carry every child’s presents all at once.”

“Mommy, can we go see the toys in the warehouses?!” asked Mae.

“Unfortunately, not,” her mother said. “You don’t want to ruin the surprise for Christmas Day!”

“Alright,” said Mae disheartened but fascinated by this new discovery. “What else can you tell me about Santa and the North Pole?”

“It is amazing,” her mother continued. “Santa heads a global team of helpers from the elves to little kids like you.”

“Little kids like me?” asked Mae. “How do I help Santa?”

“Do you remember that big bin in the store for needy kids?”

“Yes,” replied Mae.

“You can buy a toy from the money in your piggy bank to place in the bin for children whose parents can’t afford to pay Santa for toys,” explained her mother. “Those toys are sent to Santa for him to deliver with his special love.”

“I like helping other kids, Mommy,” said Mae. “I can’t wait until Christmas!”

“I know, Sweetie,” said her mother giving her a big hug. “No matter what gifts we give or receive on Christmas, we will all be together. And we have Santa and his many helpers to thank for making good things happen.”

Posted in Organizational Performance, Uncategorized | Tagged , , | Leave a comment

Using Evidence-Based Practices in Intervention Pilots

It’s a familiar story to those involved in education or social service program delivery: a new intervention has been piloted to enhance outcomes, data have been collected to varying degrees of success, and soon the funder or sponsor will expect to see a final report.  The team is hopeful that data will show the intervention’s impact, but demonstrating impact within a short window could be difficult. Hiccups may have hindered the collection of outcomes data.  Still another measurement complication might have been if the logic model for the intervention didn’t even plan for demonstrable impacts until after the final report is due.

CHS&A has come across this scenario many times in working with clients. Most recently, we worked with a state agency that had dispersed funding to 20 grantees to engage in innovative strategies so that students with disabilities would more successfully transition from school to independent living and to a job or higher education.  Our task was to measure the impact of those pilot interventions, and our evaluation team was facing the two challenges described earlier.  First, we did not have input into the data collection and reporting plan of the individual grantees in the beginning of project implementation, and we were left hoping that the data collection protocols were sufficient to yield answers to our research questions.  Second, so many of the intended impacts of these interventions were scheduled to take place after the reporting period.  Despite our keen awareness of these complications, it was early in the process when our concerns where alleviated by a key project design criterion: the client had required that the grantees base their interventions in evidence-based practices and predictors (EBPs).

The movement to adopt EBPs in health and education has been on the rise in recent decades in education and health sectors, and for good reason.  Many well-meaning service-providers have promoted ideas for interventions based on their past experiences, intuition, or the opinions of their peers and trainers, but these don’t always prove effective.  EBPs are the result of putting these ideas to a rigorous test to determine which are reliably effective.  This means that service-providers who use EBPs aren’t enacting an intervention solely because they believe it will work, but because there is empirical evidence demonstrating results in past applications.

CHS&A was pleased to find that our task was to evaluate interventions based on EBPs.  It meant that even though we only had limited access to output and short-term outcome data, and though the medium and long-term outcomes were still too far off to measure, there was still a line in the research literature that connected the interventions in question to specific longer-term outcomes.  For example, we wouldn’t know the graduation outcomes of freshmen students who received the intervention for at least three more years, but we did know that there was evidence suggesting that the EBP led to increased graduation rate among the target population.  Essentially, EBPs allow some degree of extrapolation into the future when it comes to outcomes.  To draw this link effectively, research resources should be used to ensure that the EBP-based intervention is implemented with fidelity to the original EBP description.

If you are preparing a proposal for a pilot intervention in education or health, consider basing that intervention on EBPs, especially if the reporting window is for only a year or two.  Your proposal preparation efforts are more straight-forward since the theoretical rationales for EBPs tend to be robust.  At the end of project, you will also be able to speak more confidently about potential longer-term outcomes even if you aren’t able to demonstrate them.

Information on EBPs is widely available. In order to increase the use of EBP in practitioner communities, a multitude of research and professional organizations have provided clearinghouse websites to help search EBP databases.  See the listing below for some example resources that might be a fit for your projects.  The first listing is the one that CHS&A drew on for our evaluation in post-secondary transitions.


Health/Mental Health

Posted in Data Driven-Decision making | Leave a comment

Data Driven: Menu Calorie Labeling and its (Missed) Opportunities

Last month I sat down for lunch at a midwestern diner chain and asked the server what items were popular. She rattled off a couple of dishes, among them a seafood platter. It sounded tasty if a bit heavy, but then I was shocked to see listed on the menu that this platter would run me between 2,130-2870 calories! No thank you—another equally enticing dinner platter had a full 1,200 calories less!

Politico reported that in early May the FDA had delayed the implementation of a rule requiring restaurants and other food vendors to list that caloric content of items on their menu. This delay drew attention because it came just four days before the rule would go into effect, meaning that businesses had already invested time and resources into ensuring compliance.  While there was widespread agreement among those interviewed that an unstable regulatory environment is bad for business, there are broader-ranging views on whether requiring restaurants to list caloric content is worth the trouble. I argue, based on the emerging research on food choice decision-making, that calorie labeling on menus is important for creating a decision architecture that empowers consumers and encourages competitive innovation, both of which result in a benefit for population health.

Empowers consumers

Most U.S. consumers use product attributes such as cost, taste, and convenience as primary dimensions by which they compare foods and ultimately choose what to buy and eat. One thing these attributes have in common is that people can easily and immediately understand how pricey, tasty, or convenient a food is during that dining experience. Because these attributes can be so motivating, some researchers of marketing and decision-making have called them search attributes. But there are other dimensions that are relevant to consumers’ food choice—called credence attributes—which are not immediately, experientially obvious. The nutrient profile of a food is one of these credence attributes. Unfortunately for consumers, these hidden attributes are not incorporated into our mental food-choice calculus unless they are made visible in some way. As Daniel Kahneman is fond of saying in his book Thinking Fast and Slow, “What you see is all there is.”

The impact of this tendency of human thinking is that people often make decisions without taking into account all of the relevant information. In the aforementioned Politico article, the author Helena Bottemiller Evich gives a scenario in which a convenient store pizza sells for only $1 per slice. While the search attributes of convenience, cost, and to an extent taste initially paint an appealing picture, the mental calculation is incomplete until you also realize that each slice contains between 560-620 calories. This data point may make the pizza not so appealing for some consumers, and a reasonable intake for others based on their goals and lifestyles.

Of course, I’m glossing over the complex ways in which individuals are more or less likely to seek out nutrient information based on their values and knowledge. But the basic premise holds true: consumers are most empowered to make their best choice when they are given access to credence attributes through techniques like caloric labeling. This happened for me when I used caloric labeling to avoid the seafood platter.

Encourages competitive innovation

In the diverse food marketplace, it’s no secret that some food companies are better poised to benefit from menus displaying calorie information, namely those whose brands are associated with being healthy, natural and fresh. Bottemiller Evich points out that Panera Bread has been voluntarily displaying calorie information on its menus since 2010. How much better for them if competitors are forced to do the same? Surely Panera Bread is betting that the more salient health and nutrition become to the consumer, the more likely those consumers will be to seek out Panera above its competitors.

This description may raise the hackles of free-market true believers, since requiring calorie labeling for foods might appear to push consumers towards healthier products and doom some restaurants in the process. But this regulation does not pick winners and losers. Instead, it amplifies existing pressures in the marketplace to provide a healthier product. Consumer sensibilities do evolve over time, and the most successful companies adjust to that evolution. Take McDonald’s Corporation for example. News articles from 2011, 2013, 2015, and 2016 have detailed McDonald’s efforts to keep their menu relevant and appealing to consumers by moving towards healthier items or reformulating existing items. Even as they do this, fast-casual competitors who emphasize higher-quality ingredients and fresher options pull away more market-share. Calorie labeling increases the salience of healthy eating and is likely to speed food consumers’ drift towards a health-orientation.

Benefit for population health

Calorie labeling can empower consumers to make fully-informed decisions and calculate the true value of a food purchase. As markets react to these developing consumer trends towards health, they work to make their foods more attractive by delivering what consumers want. It is a data-driven decision ecosystem on full display. Now, the delay of the labeling rule means that the public will have to continue waiting for this systemic change to build momentum. In the meantime, we’ll all have to take greater individual initiative to make informed decisions about our caloric choices.


Dr. Andrew Menger-Ogle is a psychologist who has a researched the influence of food labeling and marketing on decision-making.  He currently provides methodological and analytical expertise at C H Smith & Associates, LLC.

Data Driven is a series of blog posts by C H Smith & Associates to focus on the use of data for everyday decision-making.


Posted in Community Engagement, Data Driven-Decision making, Nutrition, Program Evaluation, Public Health | Tagged , , | Leave a comment

Credit When It’s Due: Food for the Journey

Good things are happening! Ohio just won a $500,000 grant from the Lumina Foundation for the Credit When It’s Due Initiative!

The Credit When It’s Due Initiative has 12 states and multiple funders seeking to increase the “reverse award” of associate degrees to students who transferred from a two-year institution to a four-year university and earned the relevant credit for an associate degree en route to a bachelor’s degree. I was happy to be involved in writing this grant and look forward to working with Ohio’s colleges and universities so that more students get the degrees they deserve.

For many students, especially adult learners, the road to a college degree is a complicated journey. The Credit When It’s Due Initiative can be food for this journey– awarding a credential  to students by pairing credits from the community college and university to award an associate degree. There are many real challenges for students: balancing work, family, and school; scraping up enough financial aid without creating too much of a debt burden; transferring between schools and programs; and keeping confident that you can get a degree. There are thousands of students in Ohio who are so close to a degree- they have 45 credits from a community college, transferred to a university, but have no official degree in hand. With  just one more semester at the university, these students could be eligible for associate degree.  About half of the students fitting this description are over 25 years old– adults who need to know that their efforts are not in vain, particularly if the storms of life strike and they need to stop out of college or they need a confidence boost to keep going amidst trials to obtain a bachelor’s degree and more. Awarding an associate degree to students who acquired the required credits, even though a student may still be pursuing a bachelor’s degree, is a meaningful milestone and recognizes the work put in by students.

Congratulations again to the Ohio Board of Regents and participating colleges and universities. In the next two years, I hope to hear many congratulations to thousands of graduates with college degrees in hand.

For more information on Credit When It’s Due, visit



Posted in Uncategorized | Leave a comment

Face the Facts, But Keep the Faith

As Hurricane Isaac hit the Gulf Coast this week on the seventh anniversary of Hurricane Katrina and the resulting failed levees, I reflect on a powerful lesson I learned about human character and organizational development in 2005 when I was placed as a staff member on loan to help Louisiana establish what is now called the Foundation for Louisiana.   I worked for six weeks in the Louisiana capitol to set up basic building blocks for the foundation while staying in an RV with co-workers and the homes of generous Baton Rouge residents. At the same time, my sister, Jewel, a Metairie, La. resident, stayed at my home with my husband in Ohio after riding out Katrina at the New Orleans airport and getting one of the first flights out when air travel resumed. Among the many impressions and lessons, one particular mantra sticks out: Face the facts, but keep the faith.

In the classic Good to Great by Jim Collins, he refers to this as the Stockdale Paradox.  Admiral Jim Stockdale was the highest-ranking United States military officer imprisoned at the “Hanoi Hilton” prisoner of war camp for eight brutal years. He was the first three-star Navy aviator to win the Congressional Medal of Honor. Stockdale said to Collins, “You must never confuse faith that you will prevail in the end– which you can never afford to lose —with the discipline to confront the most brutal facts of your current reality, whatever they might be.”

While the challenges of starting the organization were no match to the challenges many families had for rebuilding their lives, we too had to be reminded of the Stockdale Paradox. In trying to establish a philanthropy, we were faced not only with the challenges of immediate community needs from the Katrina disaster, but also with the frustrations of complex politics, an upheaval in community networks, and oh yes, another hurricane- Hurricane Rita. We had to keep steady focus on executing the tasks within our control and believe that our work was necessary for the community to overcome the devastation of the disasters.  We could not just hope that everything would work out; we had to work it out. We had to confront the facts of nature, human errors, political miscues, and tenuous infrastructure.

Since the beginning of time, generations and generations of people, communities, and organizations have survived based on their ability to confront their reality, find practical ways of making the best of what is in their control, and maintaining hope for a better future.  In your most challenging times as an organization, you need to:

  • Face the Facts: Don’t discount the outcome data; don’t trivialize the survey results; don’t pass on the feedback like a hot potato that you never want to touch again. Analyze information with your staff and board members when appropriate. Tune your ear so that you know the truth when you hear it.  Pay attention to the external factors affecting your work and adjust. If you are truly facing the facts, you will create a plan for dealing with your organizational reality. As you outline and purse this plan, expect bumps in the road. Have a strategy for dealing with more adversity if it comes.
  • Keep the Faith: You must understand and believe in the value your organization is providing. Maintain confidence that this work will continue with successful outcomes because of the value it has in the community. Stoke the passion for the organization to not only survive but thrive in its efforts.

My thoughts and prayers are with those affected by Hurricane Isaac.  May individuals, families, and the organizations that are working to meet their needs have the strength and resources to face the facts and keep the faith. May they prevail against life’s storms.

Posted in Non-profit Organizations, Organizational Performance, Voice of the Customer | Tagged , , , , | Leave a comment

Passing the torch: Welcome to Making Good Things Happen

As the world is watching the 2012 Summer Olympics in London, I kick off the Making Good Things Happen blog of C H Smith & Associates.

I hope that this blog is useful to professionals in schools and colleges, social service organizations, community development corporations, government agencies and more. This blog will promote conversation and share lessons so that organizations can better carry out their work and serve their communities with excellence.  At times this blog will serve as a…

  • Coach: Sharing best practices and trends in the field that are demonstrating promise and success.
  • Sparring Partner: Challenging you to think and act differently about the work that we do and welcoming your challenges to the assertions of the blog so that we can all improve together.
  • Cheerleader: Keeping you encouraged as a professional. Letting you know that you are not alone in the challenges you face.
  • Teammate: Proposing ways that C H Smith & Associates can be a part of your team  to help you with a winning organizational performance.

I look forward to connecting with many of you. Thank you for following!


Posted in Uncategorized | Leave a comment