Search

Volume 32, Issue 2, Fall 2008 - Forrestal;Issel;Kviz;Chávez

Loading...

Validating a Children’s Self-Report Plate Waste Questionnaire

Sarah G. Forrestal, BA; L. Michele Issel, PhD, RN; Frederick J. Kviz, PhD;
& Noel Chávez, PhD, RD, LDN

ABSTRACT

Purpose/Objectives
The National School Lunch Program is well situated to address the vulnerability of lower income children at increased risk for both under- and overnutrition.  Evidence suggests, however, that a significant amount of food served in the program goes uneaten.  One way to monitor this problem is through children’s self-reported plate waste (PW), but no validated measure exists.  Therefore, the purpose of this study was to develop and validate a children’s self-report PW questionnaire.

Methods
Lunch trays were collected from 54 sixth grade students after lunch.  Remaining foods were weighed to determine the percent wasted.  The same children completed a PW questionnaire later in the day.  To assess validity, students’ self-reported PW was compared to the weighed quantities.  The questionnaire also asked about a series of 15 potential reasons for PW.  Reliability of these items was analyzed to assess internal consistency.  Twenty-four participants completed the procedures a second time so that test-retest reliability also could be determined.

Results
Significant correlations were found between weighed and self-reported PW (p<.05), indicating criterion validity.  Test-retest reliability on eight of the 15 PW reasons was significant (p<.1); an additional four were invariant for at least one of the two timepoints.  The series of items was also internally consistent (Cronbach’s alpha = .71).  Further research is needed to evaluate the instrument across a wider variety of school settings and lunch menus.

Applications to Child Nutrition Professionals
Child nutrition professionals may use this valid and reliable self-report measure of children’s PW to monitor menu performance in federal child nutrition programs.  This instrument may be used to assess changes made to the meals served, which can assist in reaching Healthy People 2010 goals for school nutrition.

INTRODUCTION

The National School Lunch Program (NSLP) is designed to provide low cost, nutritious lunches and snacks to children in over 95,000 schools (United States Department of Agriculture [USDA], 2007).  The NSLP is mandated to provide one-third of children’s Recommended Dietary Allowance of vitamins A and C, iron, calcium, and protein (USDA, 2007) – nutrients that support physical and cognitive development (Birch & Fisher, 1998; Bryan et al., 2004).  In 2007, 30.5 million lunches were served each day, 59% of which went to children whose family income qualifies them to receive the meals for free or at a reduced price (USDA, 2008).

The NSLP is especially important for low-income children, who may have limited access to nutritious foods.  These children are at increased risk for both over- and undernutrition.  In the past three decades, the number of overweight children in the United States has risen dramatically, with prevalence doubling among children age 6 – 11 and tripling among 12- to 19-year olds (National Center for Health Statistics, 2007).  Currently, 17% of children nationwide are overweight or obese (Ogden et al., 2006), and lower income children are 2.9 times more likely to be overweight than their higher income peers (Strauss & Knight, 1999).  Lower income children are also more likely to be food insecure.  Compared to children from higher income families, low-income children are five times as likely to experience food insecurity (Nord, Andrews, & Carlson, 2007).

Despite the importance of the NSLP, evidence suggests that the meals may not be meeting children’s nutritional needs.  In a national survey of school cafeteria managers, respondents estimated that up to 12% of the calories served in the meals were wasted, with nutrient-dense fruits and vegetables being discarded the most (Buzby & Guthrie, 2002).  Increased attention is being given to the overall nutritional quality of foods and beverages available at schools.  The Institute of Medicine (2007) recently published a report identifying standards to ensure that school foods are consistent with the 2005 Dietary Guidelines for Americans.

One criterion to evaluate menu performance in the NSLP is plate waste (PW), defined as the amount of edible food served that goes uneaten.  PW can be measured a variety of ways, including direct weighing, visual estimation, and self-reports.  While direct weighing is the most accurate approach, setting up and using a scale in a cafeteria can be disruptive to foodservice operations, and data collection can be messy and time-consuming.  In contrast, self-reported measures of PW are quick and easy to administer.  However, to be useful substitutes for direct weighing, valid and reliable self-report measures are needed.  A literature search by the authors did not find any descriptions of self-reported PW measures validated with children.

The purpose of this study was to develop and validate a children’s self-report PW questionnaire.  Specifically, the study was designed to answer three questions: (1) are children’s self-reports of PW consistent with actual behavior?, (2) are self-reports consistent over time?, and (3) are self-reports internally consistent?

METHODOLOGY

The study was conducted in two phases with sixth grade students at one public middle school.  The study targeted sixth graders because their level of cognitive development is likely sufficient for handling the response tasks required by the questionnaire, especially portion size estimation (Livingstone, Robson, & Wallace, 2004).  In Phase I, a questionnaire draft was developed and revised based on feedback obtained in a focus group.  In Phase II, the revised questionnaire, pre-coded with the day’s lunch menu, was validated by comparing children’s self-reports to their weighed PW.  The Institutional Review Board of the University of Illinois at Chicago approved all materials and study procedures, including parental consent and child assent, prior to data collection.

Phase I: Questionnaire Development
Key concepts from the school lunch PW literature were organized into a framework that guided questionnaire development.  Items inquired about school, cafeteria, and individual factors.  Children were asked: (1) when school begins and ends, and when they eat lunch, (2) how often they eat school lunch, (3) how much of each school lunch item they ate on the target day, (4) how much school lunch they would eat under several conditions that may contribute to PW, and (5) demographic questions.  Twelve items asked about the reasons for PW.  Respondents were asked how much they agreed that each contributed to wasting food.  The items were drawn from surveys of upper elementary students (Meyer, 2005) and cafeteria managers (United States General Accounting Office, 1996).

A draft questionnaire was presented to five students in a focus group.  The children were a convenience sample drawn from one of the three groups of sixth graders in the school.  Participants completed the instrument and then discussed it.  They were presented with three alternatives to measure school lunch consumption: bars to be shaded up to the appropriate fraction (Figure 1a), semi-quantitative amounts arrayed in a grid (Figure 1b), and pie charts representing fractions eaten, ranging from none (100% shaded) to all consumed (0% shaded; Figure 1c).  The first approach resembled one used in a Summer Food Service Program PW study that did not use self-reports (Cotugna & Vickery, 2004).  The pie chart was adapted from a hospital menu performance study (Connors & Rozell, 2004).

Figure 1. Alternative plate waste self-reporting methods presented to the children in Phase I: a) bar version, b) grid version, c) pie chart version.

 a) 
JCN&M V.32 I.2 - Valid... Img-a 

b) 

Hardly Any or
None Eaten
One-fifth Two-fifths Three-fifths Four-fifths All  or Almost All Eaten

a. Entree

1

2

3

4

5

6

b. Vegetable

1

2

3

4

5

6

c. Fruit

1

2

3

4

5

6

d. Dessert

1

2

3

4

5

6

e. Milk

1

2

3

4

5

6


c)

 JCN&M V.32 I.2 - Valid... Img-c 

Phase II: Validation
Fifty-four children, none of whom had been in the focus group, participated in Phase II, in which the questionnaire was validated against weighed PW.  The convenience sample consisted of students in the other two groups of sixth graders in the school.  To minimize reactivity, participants were not told the study purpose.  Prior to lunch, each was given an identification number.  After lunch, participants were asked to return their trays to the researchers without discarding anything.  As children returned their trays, they provided their identification number so that weighed PW could be matched to the appropriate questionnaire.

The school served pre-packaged foods.  To determine the quantity served, five of each item was weighed to the nearest gram on a digital scale (Ohaus Model CS-2000, Pine Brook, NJ), with the weights averaged.   To determine the quantity wasted, uneaten items left on students’ trays were weighed.  The observed PW percentage was calculated by dividing the amount left on the tray by the average amount served.  Self-reported PW was calculated by subtracting the fraction reported eaten from 100%.

Two hours after lunch, participants completed the questionnaire in class while questions were read aloud; administration required 15 minutes.  All students, whether or not they had eaten school lunch, were asked to complete the questionnaire.  A subset of 24 participants repeated the procedures one month later when the menu was served again.  Lunch was identical except that apple were served the first time, and bananas the second time.

Table 1 summarizes Phase II participants’ characteristics.  The sample was multi-ethnic and predominantly male at both assessments.  Eighty-five percent of the children reported receiving free or reduced price lunch, which exceeds the national average of 59% (USDA, 2008).  Forty-four percent reported eating school lunch on the day of the first assessment; for the second assessment, 53% of the children ate school lunch.

Table 1. Demographic characteristics of the full sample and the subset who underwent repeat assessment.

Time 1 (n = 54) Time 2 (n = 24)
Mean Age (SD) 12.2 (.5) years 12.4 (.7) years
Gender 39% Female
61% Male
38% Female
62% Male
Race/Ethnicity 41% Hispanic
26% African American
7% White
4% Asian American
22% Multiracial/Other
38% Hispanic
23% African American
4% White
4% Asian American
31% Multiracial/Other
Lunch Price 75% Free
10% Reduced Price
15% Full Price
67% Free
17% Reduced Price
16% Full Price

 

Data Analysis
SPSS (version 15) was used for all analyses.  Validity was assessed with Pearson correlations between self-reported and weighed PW.  Reliability of the PW reasons was assessed using two measures. Cronbach’s alpha was calculated to determine internal consistency among the statements.  Kendall’s tau-b was calculated to determine test-retest reliability among the children who completed the instrument twice.

RESULTS AND DISCUSSION


Focus Group Feedback
Participants’ feedback guided revisions to the instrument prior to Phase II.  Specifically, the children preferred shading bars to report PW (Figure 1a), saying the method was easiest to understand.  The PW reasons were revised to include three additional considerations the children identified (dietary restrictions, open food packaging, and foods past their expiration dates), and the response options were changed to ask about quantities eaten: all, some, a few bites, none, and not applicable.

Questionnaire Validity
The children wasted large amounts of the school lunch, likely exceeding the 12% of calories found in previous studies (Buzby & Guthrie, 2002).  The entrée, pepperoni pizza, was wasted the most (78% of the amount served), and milk the least (19%).  The children were asked to identify their least liked school lunch food, and pizza was the most frequently named (52%), though this may at least partially reflect the item’s salience rather than its true unpopularity.  Thus, participants’ observed behavior is consistent with their self-reported preferences.  The unpopularity of the pizza was serendipitous, as it provided a good test of the questionnaire.  The substantial amount of waste available for measurement made comparisons to self-reports feasible; better liked foods may have been eaten entirely, making PW validation impossible due to a lack of any leftover foods for weighing.

Pearson correlations between self-reported and weighed PW ranged from .55 to .98, and were significant (p<.05) for all items (Table 2).  This supports the questionnaire’s validity as a measure of children’s PW.  Bananas had the largest correlation.  The children generally either ate all or none of the fruit, making it easy to report accurately.  Milk had the smallest correlation.  Because it was served in an opaque carton, the packaging likely prohibited an accurate assessment of its consumption.

Table 2. Pearson correlations between weighed plate waste and reported intake for specific menu items (* p<.05; ** p<.01; *** p<.001).

Menu Component n Mean Percent (SD) Measured Wasted Mean Percent  (SD) Reported Wasted r
Pepperoni Pizza 24 78.1 (38.4) 79.1 (31.5) .85***
Apple 9 54.7 (35.9) 62.1 (40.2) .68*
Banana 6 49.1 (51.0) 38.2 (49.4) .98***
Fruit Juice 21 34.2 (37.7) 34.2 (40.9) .67***
Cookie 13 21.2 (38.9) 37.4 (45.0) .61*
Milk 22 18.8 (31.8) 33.4 (40.9) .55**

 

Questionnaire Reliability
Reliability was evaluated with two different measures.  Test-retest reliability was calculated on the series of PW reasons, using data from children whose reports could be matched from both assessments.  With alpha set at .10 due to the small sample size, Kendall’s tau-b was significant for eight of the 15 items; five of the eight were significant at p<.05.  Four items were invariant for one or both of the assessments (Table 3).  Further research is needed to assess whether the invariant or non-significant items should be deleted, or if the lack of significance for some items may be related to the small sample size achieved in this study.

Scale reliability was also examined.  For the same series of items, Cronbach’s alpha was calculated to determine whether the questions were reliable across children.  With a value of .71, the series demonstrates moderate internal consistency.

Table 3. Test-retest reliability of measures of plate waste reasons (– = responses were invariant at one or both assessments; * p<.1; ** p<.05; *** p<.001).

Item n Tau-b
When I am not hungry 14 .16
When I do not like the way the food looks 15 -.11
When I do not like the way the food tastes 12 .67**
When I do not like the way the food smells 14 .78*
When the school lunch packaging is open 13 -
When the expiration date has passed 12 -
When I do not think the food is healthy 12 .71**
When I am allergic to the food item served 6 -
When I am not supposed to eat the food item served 12 -
When I have never tried the food item before 14 .50*
When the amount of food served is too much for me 13 .77***
When I also bring food from home to eat 11 .54**
When the cafeteria tables are not clean 10 .69**
When I pay more attention to my friends than eating 11 .51*
When there is not enough time to finish eating 10 -.03

Study Strengths
The qualitative and quantitative procedures in the two study phases provided different types of evidence for validity.  In Phase I, children’s feedback provided evidence of face validity; that is, their comments indicated that the questionnaire was generally relevant and comprehensible.   After their suggestions were incorporated, Phase II offered further evidence of validity and reliability.  Significant correlations between reported intakes and weighed PW for all tested menu items demonstrate criterion validity.  A measure of test-retest reliability suggested that children’s reports of their potential eating behavior under a variety of circumstances were stable over time for two-thirds of the statements tested.  Finally, Cronbach’s alpha was high for these same PW reasons, providing evidence for internal consistency.  It is noteworthy that findings were significant despite low statistical power due to the small sample size available for the study.

Four additional methodological aspects of the study contributed to its strength.  First, students were asked about what they ate for lunch shortly after the meal, minimizing the difficulty of the recall task.  Baxter et al. (2004) documented a similar effect.  Fourth grade students were observed eating school lunch and were later interviewed about their intake during the preceding 24 hours.  Students who were interviewed in the afternoon had fewer omissions and intrusions for school lunch items, as well as better overall accuracy compared to students interviewed later in the evening or the following morning (Baxter et al., 2004).  Querying students shortly after the target meal guards against fading memory.

Second, the school served pre-packaged foods.  Students were asked to report only the amount eaten, not the amount served.  Estimating portion size is a difficult cognitive task even for adults, and may be beyond the ability of children younger than 11 (Livingstone, Robson, & Wallace, 2004).  Thus, asking students only about the fraction they consumed, rather than an absolute quantity, is less challenging.  However, many schools do not serve pre-packaged lunch items, so using the questionnaire elsewhere may require students to estimate both the amounts served and eaten, due to greater variability in the portion sizes served.

Third, sixth grade students were deliberately chosen for the validation study because their level of cognitive development was likely sufficient for handling the response tasks (e.g., memory recall, portion size estimation, and abstract reasoning).  Borgers, de Leeuw, and Hox (2000) reviewed children’s capabilities as survey respondents within the framework of Piaget’s cognitive developmental stages.  By age 12, children are generally in the formal thought stage, where they are competent in logical operations and abstract reasoning.  Participants in this study were 12 years old, and therefore likely had sufficiently developed cognitive skills to provide valid self-reports, though this was not directly assessed.  It is likely that the questionnaire would perform at least as well, if not better, with older subjects, given their greater cognitive development.

Finally, the recall task was simplified by pre-coding the questionnaire with the day’s lunch menu.  Recognizing foods from a list is a superior recall strategy compared to having to remember on one’s own.  In a study eliciting children’s dietary recall strategies, participants reported reading the lunch menu facilitated memory retrieval (Domel, 1997).  Listing the food items on the PW questionnaire may have served a similar function, making it easier for the children to recall additional information about quantities consumed.

Study Limitations
Although the findings are promising, the study has two limitations.  Participants comprised a convenience sample from one school.  The study was not intended to be generalizable to the national population of sixth grade students.  Further replication is needed in a variety of school settings to draw stronger conclusions about the instrument’s validity.  Second, only one menu was tested.  Although this simplified the validation procedures, more work is needed to assess whether the questionnaire is reliable with other foods.  It is possible that children may have a more difficult time reporting their consumption of entrees such as hamburgers or chicken sandwiches, which can be more easily separated into their component parts (e.g., bun, patty, etc.) than pizza.

Applicability of the questionnaire itself may be limited by the particularities of the foodservice setting.  Cafeterias implementing the offer versus serve provision pose a greater challenge to its utility.  Students’ freedom to choose or decline items may hamper the usefulness of a pre-coded instrument, given the potentially greater variability in items selected.  On the other hand, the offer versus serve provision was implemented to reduce PW (Buzby & Guthrie, 2002), so the need to reduce waste in these settings may not be as pressing.

CONCLUSIONS AND APPLICATIONS 

This study has demonstrated that: (1) 12-year-old children may accurately report what they consumed in the school lunch when queried in a developmentally appropriate manner, (2) the questionnaire is a valid and reliable measure of PW in the NSLP, and (3) it represents an easy way to assess children’s PW without resorting to more resource-intensive methods like weighing.

Although the questionnaire requires further evaluation, it has the potential to be used in both basic and applied research in different settings, so long as respondents are cognitively able to complete the response task and standardized portions are served.  Such settings may include the Summer Food Service Program or for afterschool programs participating in the Child and Adult Care Food Program. 

The questionnaire could be used empirically to identify additional factors that contribute to PW in child nutrition programs.  For example, the role of competitive foods in children’s PW patterns could be explored (neither vending machines nor a snack bar were present in the study school, so these factors could not be assessed).  Anthropometric measures could be combined with self-reports to examine the relationship between adiposity and underreporting food intake, a phenomenon that has been documented previously, including recently by Savage and colleagues (2008).  Child nutrition professionals, including foodservice personnel, nutritionists, and researchers, may use this questionnaire as a rapid, low-cost assessment of menu performance or an evaluation tool in school foodservice programs.  Thus, the questionnaire may be used to help reach Healthy People 2010 Objective 19-15: “Increase the proportion of children and adolescents aged 6 to 19 years whose intake of meals and snacks at school contributes to overall dietary quality” (United States Department of Health and Human Services, 2000).

ACKNOWLEDGEMENTS

The authors thank the faculty, staff, and students at the participating school for their assistance and participation.  Thanks also to Meghan Benson and Moji Owolabi, who aided in data collection.  The study was funded by a University of Illinois at Chicago Provost’s Award for Graduate Research.

REFERENCES

 

Baxter, S. D., Smith, A. F., Litaker, M. S., Guinn, C. H., Shaffer, N. M., Baglio, M. L., et al. (2004).  Recency affects reporting accuracy of children’s dietary recalls.  Annals of Epidemiology, 14 (6), 385-390.

Birch, L. L., & Fisher, J. O. (1998).  Development of eating behaviors among children and adolescents.  Pediatrics, 101 (3 Pt 2), 539-549.

Borgers, N., de Leeuw, E., & Hox, J. (2000).  Children as respondents in survey research: Cognitive development and response quality.  Bulletin de Méthodologie Sociologique, 66, 60-75.

Bryan, J., Osendarp, S., Hughes, D., Calvaresi, E., Baghurst, K., & van Klinken, J. W. (2004).  Nutrients for cognitive development in school-aged children.  Nutrition Reviews, 62 (8), 295-306.

Buzby, J. C., & Guthrie, J. F. (2002).  Plate waste in school nutrition programs: Final report to Congress (Report No. E-FAN-02-009).  Washington, DC: United States Department of Agriculture, Economic Research Service.

Connors, P. L., & Rozell, S. B. (2004).  Using a visual plate waste study to monitor menu performance.  Journal of the American Dietetic Association, 104 (1), 94-96.

Cotugna, N., & Vickery, C. E. (2004).  Children rate the Summer Food Service Program.  Family Economics and Nutrition Review, 16 (1), 3-11.

Domel, S. B. (1997).  Self-reports of diet: How children remember what they have eaten.  American Journal of Clinical Nutrition, 65 (4 Suppl), 1148S-1152S.

Institute of Medicine (2007).  Nutrition standards for foods in schools: Leading the way toward healthier youth.  Washington, DC: The National Academies Press.

Livingstone, M. B., Robson, P. J., & Wallace, J. M. (2004).  Issues in dietary intake assessment of children and adolescents.  British Journal of Nutrition, 92 (Suppl 2), S213-S222.

Meyer, M. K. (2005).  Upper-elementary students’ perception of school meals.  Journal of Child Nutrition & Management, 29 (1).  Retrieved October 28, 2005, from http://docs.schoolnutrition.org/newsroom/jcnm/05spring/meyer/index.asp

National Center for Health Statistics (2007).  Prevalence of overweight among children and adolescents: United States, 1999-2002.  Retrieved March 7, 2008, from http://www.cdc.gov/nchs/products/pubs/pubd/hestats/overwght99.htm

Nord, M., Andrews, M., & Carlson, S. (2007).  Household food security in the United States, 2006 (Report No. 49).  Washington, DC: United States Department of Agriculture, Economic Research Service.

Ogden, C. L., Carroll, M. D., Curtin, L. R., McDowell, M. A., Tabak, C. J., & Flegal, K. M. (2006).  Prevalence of overweight and obesity in the United States, 1999-2004.  Journal of the American Medical Association, 295 (13), 1549-1555.

Savage, J. S., Mitchell, D. C., Smiciklas-Wright, H., Symons Downs, D., & Birch, L. L. (2008).  Plausible reports of energy intake may predict body mass index in pre-adolescent girls.  Journal of the American Dietetic Association, 108 (1), 131-135.

Strauss, R. S., & Knight, J. (1999).  Influence of the home environment on the development of obesity in children.  Pediatrics, 103 (6), 85-92.

United States Department of Agriculture (2007).  Child nutrition programs: National School Lunch Program.  Retrieved March 7, 2008, from http://www.ers.usda.gov/Briefing/ChildNutrition/lunch.htm

United States Department of Agriculture (2008).  National School Lunch Program: Participation and lunches served.  Retrieved March 7, 2008, from http://www.fns.usda.gov/pd/slsummar.htm

United States Department of Health and Human Services (2000).  Healthy People 2010: Understanding and improving health (2nd ed.).  Washington, DC: United States Government Printing Office.

United States General Accounting Office (1996).  School lunch program: Cafeteria managers’ views on food wasted by students (Report No. GAO/RCED-96-191).  Washington, DC: United States General Accounting Office.

BIOGRAPHY

Forrestal, Issel, Kviz, and Chávez are, respectively, Doctoral Candidate, Clinical Associate Professor, Professor, and Associate Professor at the University of Illinois at Chicago School of Public Health.


 Average 5 out of 5
Loading...

Contact Us Advertise on SNA Site Map Media Center Privacy Policy

© 2000 - 2014 School Nutrition Association, All Rights Reserved