FSA Quality Assurance Toolkit

6. Case Studies

FSA Quality Assurance Toolkit

Last updated: 03 March 2023

6.1 Case study 1 – Focus group (Curtis et al., 2017)

Background

The most prevalent category of determinants and risk factors for childhood obesity (for example, dietary behaviour, physical activity, and sedentary behaviour), begin within the family environment, where children consume around two-thirds of their daily food intake. Children are dependent on their parents and carers to provide food that is conducive to both a healthy weight and development. Hence, family-based approaches are now well recognised in the childhood weight management literature, where they are considered the ‘gold standard’ for improving children’s weight status and overall health. Despite this there has been a lack of understanding regarding exact parental influences on children’s dietary behaviours within the context of the obesogenic environment and consequently, how to directly target parents in weight management interventions. This study aimed to explore parents’ capability, opportunity, and motivation towards portion control behaviours with their children, drawing on the Theoretical Domains Framework (TDF) and the COM-B model.

Methods

Focus groups facilitated interaction among participants, stimulating rich data for analysis, with the researcher playing an active role in guiding the discussions. A purposive sampling approach was used to recruit participants for the study. Eligible participants included three sub-groups of participants to allow for triangulation of data. Participants comprised: (1) family weight management case workers working with families with overweight children, (2) parents with overweight and or very overweight children and (3) parents with healthy weight children ≥5 years. This supported the convergence between multiple sources of data to generate themes, validate findings, improve credibility and acquire greater overall understanding of the phenomena. Participants were recruited through emails distributed to the local public health department, community family weight management groups and a university. Ethical approval for focus groups was obtained from the University of Warwick Research Ethics Committee. Consent forms were administered and signed before the focus groups began.

Six focus groups with case workers (n = 4), parents with overweight children (n = 14) and parents with healthy weight children (n = 8) took place at university and community settings. All focus groups were facilitated by one moderator. The conversation was guided by a pre-specified set of questions. The focus group with caseworkers took place at the university and lasted 120 min, while focus groups with parents took place at local weight management programme locations and the university, lasting 60 min. With participants’ permission, focus groups were audio recorded and transcribed verbatim and the raw data were coded using thematic analysis. Braun & Clarke’s (2006) six stages of analysis was used to explore the data. The reliability of the data analysis was further enriched by the use of an additional trained qualitative researcher who independently coded 10% of the data in order to establish inter-rater reliability.

Results

All COM-B model components (except physical capability) were identified as important for supporting parents in achieving the target behaviour. These components aligned with nine TDF domains: Psychological Capability: Knowledge; Memory; Attention and Decision-Making Processes; Skills; Automatic Motivation: Emotion; Reinforcement; Reflective Motivation: Beliefs about capabilities; Beliefs about consequences; Social identity; Physical opportunity: Environmental context & resources and Social Opportunity: Social influences.

There was consensus among parents regarding their lack of knowledge of appropriate adult and child portion sizes.

”Until I came here, I didn't really know much about portion sizes at all.” (Parent, FG4).

Some parents and case workers agreed that parents tended to use their own portion sizes of food as a guide for measuring their children’s portion sizes. Consequently, they may not differentiate between adult and child portions.

”For me, I find it particularly difficult dishing out the correct portion size for children and for adults, I suppose. I just tend to give everybody the same amount” (FG3, parent).

Case workers described how grandparents in particular, can make it difficult for parents to ensure their child is eating healthily and that they may ‘undo’ parent’s good work.

”mmm..and on a positive notes..um..some of my families..they are really trying to make this change but Grandma..they go over to Grandma’s and Grandma is giving them ALL THIS STUFF!” (Case worker, FG1).

Focus group discussions repeatedly underscored parents’ fears of causing their child to feel anxious about their weight if they attempted to discuss it with them.

”We might try and tackle it a little bit, try discuss it with him..but we don't want him to go the other way and you know..have anxieties about that” (Parent, FG3).

Discussion

Findings suggest that parents’ internal processes such as their knowledge and skills, emotional responses, habits and beliefs, along with social influences from partners and grandparents, and environmental influences relating to items such as household objects, all interact to influence portion size behaviours within the home environment.

Limitations of the study include the use of a small purposive sample, with the majority of participants being Caucasian females. Consequently, the identified views on the facilitators and barriers to parental provision of a healthier diet for their children may be less representative of fathers and male caregivers and other ethnic groups. The use of focus groups also involves limitations. There is always the potential for some participants to feel intimidated and dominated by other group members which may impede their ability to share their opinions and ideas, which may also reduce generalisability of findings.

6.2  Case study 2 – Survey study (Mead et al., 2021)

Background 

Urban agriculture (for example, the growing of fruits and vegetables in urban and peri-urban areas) may represent a solution to ensure a sustainable food system, also improving health, well-being, and food security. Urban agriculture encompasses a broad range of informal and formal food production operations, from urban allotments and home/community garden growing, to commercial urban farms. Historically, urban agriculture has been relied upon to mitigate food shortages during crises, such as war. More recently, evidence suggests that urban agriculture may help improve diet quality and reduce food inequalities. There is some evidence that engagement in urban growing is associated with a healthier diet; however, the mechanisms that may account for this relationship are not well-understood. Identifying the drivers of this relationship is important as a means of informing changes in policy and good practice if urban agriculture is to be supported as a food systems solution. Theoretically, urban agriculture may be associated with i) greater perceived access to fruits and vegetables, ii) increased connection with nature, iii) lower psychological distress, and iv) increased health and ethical food choice motivations, which may in turn promote improved dietary quality and health. However, to our knowledge, there has been no empirical investigation of these potential pathways linking urban agriculture with healthier diets. Addressing these gaps, the current study used a cross-sectional online survey to test if proximity to and engagement with urban agriculture is associated with better diet quality, and what mediates this relationship. 

Methods 

Participants were recruited from the Qualtrics participant panel and by an opportunity sample of members of the general public. We aimed to recruit 595 participants. A sample size calculation indicated that 475 participants were needed for 90% power at alpha .05 (H0, Root Mean Square Error of Approximation (RMSEA) = 0.01, H1 = RMSEA = 0.08). We increased this by 20% to allow for attrition. We aimed to recruit 400 participants via the Qualtrics participant panel, plus 195 participants from the general public to ensure that our sample contained a mixture of participants who did and did not have experience of urban agriculture. Participants were eligible to take part in the study if they were aged 18 years or over and based in the UK. 

A series of questions to measure participants’ proximity to and engagement with urban agriculture on a continuous scale. Scores reflected the number of examples of urban agriculture participants engaged in and the frequency they engaged in this. Cronbach α value for the measure was 0.851. Diet quality was assessed with a short food frequency questionnaire, which has been used in previous studies. A higher total score represents better diet quality. Cronbach α value for the measure was 0.683. 

The survey was delivered through Qualtrics via a weblink. Participants viewed the Participant Information Sheet and provided informed consent at the start of the study. Survey completion took approximately 20 min. Participants were offered entry to a prize draw at the end of the study to win a share of £250 shopping vouchers as thanks for their time. The study was approved by the University of Liverpool Health and Life Sciences Research Ethics Committee. Data were collected in July – August 2019.  

A structural equation model was fitted in MPlus, with a Satorra-Bentler correction for non-normal data. Model fit criteria were assessed, including the standardised root mean residual (SRMR), the comparative fit index (CFI), the Tucker-Lewis Index (TLI), and the root mean square error of approximation (RMSE). Unstandardised regression coefficients and 95% Confidence Intervals (CI) are reported. Data from participants who were missing values for any of the key variables of interest were removed from the dataset. The study protocol and analysis plan were preregistered on Open Science Framework.  

Results 

Six hundred and twenty-four participants reached the end of the survey. Following removal of incomplete and improbable responses, the sample available for analysis was N = 583. Participants had a mean age of 42.75 years (SD = 15.71, range 18–86 years).  

The structural equation model provided an acceptable fit of the data (SRMR = 0.070, CFI = 0.964, TLI = 0.942, RMSEA = 0.058). The direct effect of urban agriculture engagement and proximity on diet quality was not significant. However, greater urban agriculture engagement and proximity was associated with greater perceived access to fruits and vegetables, health-related food choice motivations, ethically-related food choice motivations, and nature connectedness. Contrary to expectations, however, greater urban agriculture engagement and proximity was associated with greater psychological distress. 

Discussion 

This study explored the association between proximity to and engagement with urban agriculture and diet quality, and whether this relationship is explained by one or more mediating factors. Results indicated that greater proximity to and engagement with urban agriculture is associated with greater perceived access to fruits and vegetables, health and ethical-related food choice motivations, nature connectedness, and, unexpectedly, greater psychological distress.  

The cross-sectional nature of these data limits our ability to make any causal inferences regarding the relationships between urban agriculture, diet and mediating factors. We acknowledge that without longitudinal, intervention-based assessments, this evidence base is still lacking and our study is limited in its ability to address methodological shortcomings. Furthermore, we used a brief, self-reported proxy measure of dietary quality and the Cronbach α reliability score for this scale was not optimal. Explicit assessments of food intake and food choice are needed to confirm such causal relationships. We also developed our own measure of proximity to and engagement with urban agriculture as there was no pre-existing measure in the literature that would be suitable for our analytic approach. Further work is now needed to validate this tool to ensure it is an accurate reflection of participants’ proximity to and engagement with urban agriculture. Finally, our sample is comprised of mostly white, female participants who are in some form of employment. This limits the generalisability of our findings and future work should seek to include a more diverse sample of participants.

6.3 Case study 3 – Behavioural intervention trial (Breathnach et al., 2022)

Background 

Excess weight increases morbidity and mortality. Biological, behavioural, societal, and environmental factors interact leading to positive energy balance and excess weight. Prompting consumers to swap their initial food and drink selections for lower-energy alternatives while shopping may help bring energy intake into line with public health recommendations. Swap-based interventions have been tested in experimental settings and the results show reductions in both the saturated fat and salt content of grocery baskets. More recently, offering swaps was shown to reduce the energy content of snacks and drinks ordered in an experimental online canteen. This study also found that accompanying swaps offered with physical activity calorie equivalent (PACE) information, indicating the amount of energy contained in a food or drink and the amount of physical activity that would be required for it to be expended (for example, “How about a swap? Save [x] calories = [y] min walk”), significantly increased the likelihood that a swap offered would be accepted when compared to offering swaps with no specific information (for example, participants were simply asked: “How about a swap?”). The provision of PACE information also increased intervention acceptability ratings. These findings indicate that providing easily interpretable or tangible information when offering lower-energy swaps for snacks or drinks increases their acceptance. 

However, little is known about (a) whether lower-energy swaps offered across a full canteen menu, including items such as hot meals or sandwiches, would be accepted; and (b), if swaps are accepted, whether consumers immediately compensate for energy reductions (for example, by ordering more items and thus more energy) across their whole meal. Although field trials are considered the gold standard method of investigation, they are costly and challenging to conduct. Given the lack of research on swap-based interventions in canteen settings, we decided to use a field-lab hybrid study to perform an initial investigation of the potential effectiveness of the interventions which could be used to inform a future field trial. The aim of this study was to test the effect of (i) offering lower-energy swaps, and (ii) offering lower-energy swaps with a PACE message on the total energy of items pre-ordered for lunch within the context of an experimental online workplace canteen. We hypothesized that the Swap + PACE intervention would be more effective than offering swaps alone. 

Methods 

Study design and setting: Field-lab hybrid studies are hypothetical choice experiments usually delivered via online platforms that mimic real-world plausible scenarios. While they are not as tightly controlled as traditional lab experiments, nor do they test ‘real’ choices like field trials, they permit the testing of variables that would be difficult to examine in a field trial due to the pragmatic constraints that real-world settings inevitably impose.   

This pre-registered (AsPredicted ref: 56358), three-arm, randomised controlled trial was conducted in an experimental online canteen developed using REDCap, a web application for data collection. The website was designed to simulate an online pre-ordering system for a real-world workplace canteen. An online canteen pre-ordering system is a website which displays the canteen’s menu and allows employees to place their lunch order in the morning for collection later that day. Participants were able to hypothetically order their lunch from 6 menus containing a selection of main hot meals (n = 3), jacket potatoes (n = 10), soup & sandwiches (n = 15), sweet snacks (n = 18), savoury snacks (n = 20), and non-alcoholic drinks (n = 18) based on the menus of a real-world workplace canteen with whom we partnered. In the real-world canteen, main hot meal options (n = 3) change on a daily basis. Participants were randomly assigned to view and choose from the main hot meals for 1 of 5 different days to reflect this. The CONSORT checklist was used in the design and reporting of this study. 

Participants: In February 2021, participants were recruited through Prolific Academic, an online participant sourcing platform. To be eligible for the study, participants had to be ≥18 years, a UK resident, speak English fluently, and be in full or part-time employment. Those following restricted diets, for example, vegetarian or dairy-free, were ineligible, as this would affect the acceptability of swaps offered. Interested participants were asked to confirm their eligibility, read the information sheet, and provide consent. 

Randomisation and blinding: Simple randomisation (1:1:1) was performed using Predictiv. Participants were randomised to both a trial arm (1 of 3) and a menu (1 of 5), meaning that participants were evenly allocated to 1 of 15 groups. To do this, the platform allocated eligible participants a random integer between 1 and 15 representing the 15 conditions. While investigators were not blinded to condition, they were not able to manipulate any study parameters following the initial study set up, as all study procedures were automated. 

Online ordering task: Following randomisation, participants were directed to REDCap where they were asked to indicate their current subjective feeling of hunger. Participants were then asked to imagine they worked for a company that had a pre-ordering website for their canteen and to order their lunch for the day using the website. They were asked to make choices that were in keeping with what they would typically have for lunch during their working day. Lower-energy swaps were automatically offered for originally selected menu items, if a suitable alternative was available. Participants placed one order only and did not pay for this order. Upon completion, participants were debriefed and reimbursed with £0.50.  

Swaps offered were pre-determined by the research team using the criteria outlined in this section. The criteria for main hot meals differed to the ones in all other menus. Regardless of the menu, to qualify as a swap, the alternative had to contain at least 50 kcal less than the originally selected item, because a minimum of 50 kcal reduction per-person per-day has been identified as being clinically relevant. Participants were randomly allocated to one of the following groups: 1) Control: No swaps offered; 2) Swaps: Swaps offered were accompanied by the message: “How about a swap?”; 3) Swaps+PACE: Swaps offered were accompanied by the message: “How about a swap? Save [x] calories = [y] min walk”. 

Measures: After placing their lunch order, participants completed a brief exit survey to explore the acceptability of the intervention and record participant information (sex, age, ethnicity, and education along with height and weight for the calculation of body mass index (BMI). The Scottish Physical Activity Screening Questionnaire (Scot-PASQ) is a validated scale and was used to assess whether participants were meeting physical activity guidelines. A shortened 3-item version (Cronbach’s alpha = 0.81) of the Dietary Intent Scale, was used to measure dietary restraint. Acceptability was assessed by asking participants how acceptable they felt it would be for their employer to (a) implement a pre-ordering system for their workplace canteen and (b) offer them swaps for their food choices (only those in the intervention groups). Response options were on a scale from 1 (completely unacceptable) to 5 (completely acceptable).  

Primary outcome: The primary outcome was the total energy (kcal) of items ordered by each of the three groups, controlling for the energy content of the first item ordered. 

Sample size: We aimed to recruit 2,214 participants. With 80% power, this would allow us to detect a 35 kcal difference at an alpha level of 0.05 (an uncorrected analysis) or a 40 kcal difference at an alpha level of 0.016 (Bonferroni). We applied the Benjamini-Hochberg (BH) correction where the alpha level required was between these two bounds. While a 50 kcal reduction would be a clinically relevant energy reduction for adults, we powered the minimum detectable effect size to 35-40 kcal, because we expected the relative effect between the experimental groups to be smaller than the effect between the experimental groups and control. Baseline energy estimates (mean = 423 kcal, SD = 236) were taken from a pilot randomised controlled trial conducted in 6 workplace canteens across the UK. 

Statistical analysis: Participants had to order at least one food item, not order from all menus (because this was deemed as an implausible lunch order), and checkout to be included in the analysis. The 19 participants (1%) in the intervention groups who were not offered any swaps, because they selected the lowest energy menu items in all the categories they ordered from, were included in the analyses. 

The primary outcome (energy ordered) was analysed using analysis of covariance (ANCOVA). This analysis was pre-registered as ANOVA but ANCOVA was used to control for the energy content of the first item ordered because that was the baseline value of our dependent variable. We controlled for the energy of the first item a participant ordered because we wanted to control for the initial choices participants made but after the first choice their subsequent choices may have been influenced by previous swaps offered. 

Results 

Invitations were sent to a random subsample of a pool of 17,773 eligible panel members. Of those invited, 2,477 participants consented and were equally randomised to 1 of the 3 groups. Of those, 2,150 (86.8%) participants followed the instructions, completed the study, and, thus, were included in the analysis. Participants were on average 36.8 (SD = 11.6) years old. Just over half (54.3%) were female, 80% identified as white, and 51% had completed tertiary-level education.  

Participants ordered on average from 3 (SD = 0.91) menus. The average energy content of lunches ordered was 781 kcal (SD = 315 kcal, range: 226 to 2,226 kcal). The average energy content of final lunch orders was significantly lower in both intervention groups when compared with control [control mean = 819 kcal]: swaps -47 kcal [95%CI: -82 to -13, p = 0.003]; swaps + PACE -66 kcal [95%CI: -100 to -31, p < 0.001]. The difference in the average energy content of final lunches ordered between intervention groups was not statistically significant [-19 kcal, 95%CI: -53 to 16, p = 0.591]. 

Discussion 

The aim of this study was to test the effect of (i) offering lower-energy swaps, and (ii) offering lower-energy swaps with a PACE message on the total energy of items pre-ordered for lunch within the context of an experimental online workplace canteen. Offering lower-energy swaps significantly reduced the energy content of lunches pre-ordered compared with not offering swaps. Although accompanying swaps offered with PACE information significantly increased swap acceptance relative to when this information was not provided, it did not significantly reduce energy pre-ordered. 

Strengths and limitations 

This study used a randomised design and recruited a large sample of employed adults that broadly matched the distribution of the UK population in terms of sex, ethnicity, and education. Participants were randomised to see 1 of 5 different menus, meaning that swaps were offered for 15 different main hot meals. This menu variety helps to increase the generalisability of our findings. By partnering with a real-world company and simulating a pre-ordering website using their canteen menus, this study was able to test the effect of offering lower energy swaps for lunch time meals in a similar manner to how choices would be made when using an online canteen in real life. Qualitative research with employees of the partner organisation informed swap choices and intervention delivery.  

The primary limitation of this study is its hypothetical nature. Participants made imaginary choices and were not required to spend their own money. The experimental nature of this study means that effect sizes observed in real-world settings may be smaller than those reported here. Given the nature of the sample (professional survey takers), it is possible that the results may not entirely reflect the behaviours of the general population. While the demographic characteristics of panel members sampled broadly matched those of the general UK population, little is known about the generalisability of the study findings outside of the UK. Self-reported height and weight measures to calculate BMI may also have been influenced by social desirability bias. Although the measure of dietary restraint was based on a validated scale, due to time constraints, a shortened unvalidated version was used, which still maintained a high Cronbach’s alpha (α = 0.81). Our analysis was in available cases. Although imputing data for non-completers may have slightly attenuated our estimates, the proportion of missing data was relatively small (13%) and therefore any such biases are unlikely to affect the interpretation of the results. Finally, total energy intake at baseline may have been an effect modifier, but we did not measure it due to well-known limitations of existing methods. However, we did not find any evidence that the effect depended upon hunger, dietary restraint, physical activity level, or BMI (a reasonable proxy for energy balance), so such effect modification of total energy intake, if it exists, is not likely to substantially modify intervention effects.