Designing a Sensory Study for Product Development: Tests, Panels and Statistics

Designing a sensory study: what it is designed to answer

Every product change raises the same commercial question: will the consumer notice, and will they like it? Designing a sensory study means turning that into a testable decision, such as “is the reformulated sauce detectably different?” or “can we say it is preferred?”. Without a defined decision the data describe the product but do not answer anything, and a claim built on them carries regulatory and commercial risk in both markets.

Human assessors are the measuring instrument, so most of the design is about controlling what else influences them: expectations from packaging or product names, the order samples are tasted in, carry-over between samples, fatigue and the influence of other people^[2][9]. FRL’s guide to challenges in the sensory evaluation of food covers these bias sources in practice.

Choosing the right sensory test

Sensory methods fall into three families, defined by the question each answers: is there a difference, what is the difference, and how much is it liked^[2]? Our overview of sensory evaluation methods for food and beverage products compares them in more detail.

Discrimination tests (triangle, duo-trio)

A discrimination test asks only whether a perceptible difference exists. In the triangle test each assessor receives three coded samples, two of which are identical, and picks the odd one out, so the chance of a correct guess is one in three^[1]. The three samples can be served in six orders, which are balanced across assessors. In the duo-trio test assessors match one of two coded samples to a reference, so chance is one in two; it is easier to perform but statistically less efficient than the triangle test^[1][9]. Results are judged against the binomial distribution. A difference test does not say which attribute changed or by how much, and it is of limited use for products with strong carry-over or lingering flavours^[1].

Descriptive analysis and QDA

In descriptive analysis, trained assessors agree a shared vocabulary, calibrate it against reference samples and score the intensity of each attribute on line scales over replicate sessions, as in quantitative descriptive analysis (QDA)^[3][9]. ISO 13299 gives general guidance for establishing a sensory profile^[3]. The output is an attribute-by-product intensity profile that shows what differs and by how much.

Affective and hedonic consumer tests

Affective tests measure liking or preference. Consumers typically rate each product on the 9-point hedonic scale, from “dislike extremely” to “like extremely”, in a central location or a home-use test^[6][9]. Preference mapping then links the descriptive profile to liking scores and shows which attributes drive acceptance, which tells a formulator what to change^[9].

Table 1. Sensory test methods, the question each answers and how it is analysed

Building and screening a sensory panel

A sensory panel is only as valid as its recruitment. Trained descriptive panels are small, often around 8 to 12 assessors, selected for sensory acuity, ability to describe and reliability, then trained and monitored^[4][9]. ISO 8586 sets out how to select, train and monitor assessors^[4], and ISO 11132 describes how to measure a descriptive panel’s discrimination, agreement and repeatability^[5].

Consumer groups are recruited to match the target buyer by age, usage and region. For a US and EU programme, treat each market as its own group, because liking can differ between them. Record the recruitment criteria, because they define who a result applies to. Our article on EU consumer trends through sensory data shows how regional panels change the picture.

Sample handling and test conditions

Samples carry random three-digit codes, are served at a controlled temperature in a balanced order and are separated by palate cleansers. This limits expectation, order and carry-over bias^[2][9]. ISO 8589 gives guidance on test rooms: individual booths, controlled lighting and ventilation, and neutral surroundings^[7]. Pungent or spicy products need longer rests and fewer samples per session, because heat builds and lingers.

Formulator’s note. Write the decision rule before the first sample is tasted. Fix the test, the number of assessors and the pass criterion in advance, because changing them after seeing the data invalidates the stated significance level.

Statistics that make results valid

The number of assessors sets how much evidence a result needs. Figure 2 shows the smallest number of correct answers in a triangle test that is significant at the 5% level for panels of 12 to 60. With 36 assessors, 12 correct answers are expected by chance; 18 correct has a one-sided binomial probability of 0.028 and is significant, while 17 gives 0.058 and is not (calculated by FRL from the binomial distribution, p = 1/3)^[1].

Two design choices matter most. First, decide whether you are testing for difference or for similarity. Failing to find a difference with a small panel is not evidence that two products are the same; a similarity test fixes a small β-risk and a maximum acceptable proportion of discriminators, and usually needs more assessors^[1]. Second, fix α, β and that proportion before testing, so the panel size follows from the risk you are willing to accept rather than from convenience^[1][9].

ANOVA, significance and panel performance

For descriptive data, analysis of variance (ANOVA) tests whether products differ on each attribute while allowing for assessor and replicate effects^[9]. Panel performance checks on discrimination, agreement between assessors and repeatability show whether the data can be trusted^[5]. For consumer tests, the number of people depends on the size of difference you need to detect: in Hough and colleagues’ worked example, detecting a difference of 10% of the scale at α 0.05 and β 0.10 needed about 112 consumers^[10]. A significant p-value shows a difference is unlikely to be chance; it does not show that the difference matters commercially.

Sensory standards in the US and EU: ISO and ASTM

ISO sensory standards are market-neutral and used on both sides of the Atlantic: general methodology (ISO 6658), the triangle test (ISO 4120), assessor selection and training (ISO 8586), sensory profiling (ISO 13299), panel performance (ISO 11132), hedonic testing (ISO 11136), test rooms (ISO 8589) and sensory quality control (ISO 20613)^{[1][2][3][4][5][6][7][8]}. ASTM International’s Committee E18 on Sensory Evaluation publishes methods widely used in the US, including E1958 on sensory claim substantiation^[12]. What differs between the markets is claims and labelling law.

Sensory claims: evidence that holds up in the US and EU

Comparative and improvement claims, such as “preferred to Brand X”, “tastes as good as the original” or “new recipe, less bitter”, are objective claims that need substantiation. Vague subjective statements such as “great taste” may be treated as puffery in the US, but any taste claim that can be measured should be backed by evidence^[13][14]. ISO 20784 gives guidance on substantiating sensory and consumer product claims^[11], and ASTM E1958 covers the same ground for the US^[12].

In the US, the FTC requires advertisers to have a reasonable basis for objective claims before they make them^[13]. In the EU, food information must not mislead the consumer, including about the characteristics of the food^[15], and claims are also judged under the Unfair Commercial Practices Directive^[16]. In both markets, choose a fair benchmark, an adequate sample of the target consumers, balanced presentation and pre-specified statistics, and keep the raw data. Natural flavour wording is a separate labelling question, covered in our natural flavour labelling: FDA vs EU guide.

Linking sensory data to product decisions

Set decision rules before testing, for example “no significant difference in liking against the benchmark at the 5% level”. Illustrative example (not client data): a sugar-reduced sauce for both markets is first checked with a triangle test of 36 assessors. If 18 or more pick the odd sample, a descriptive profile locates the sweetness or heat shift, and a consumer test in each market then checks whether liking has held.

The same logic applies across the cluster. Use these tests to confirm a defect when a complaint arrives (see off-odour and taint recalls), to check whether a new trend flavour from our flavour trends 2026 report suits your consumers, and to validate a flavour change, as in the flavour validation case study. Common regulatory questions are answered in our sensory testing regulations FAQ.

Table 2. Applications and the test combinations that suit them

Limitations and considerations

Results apply to the product, panel and conditions tested. Panel drift, seasonal or batch variation in the product and cultural differences between US and EU consumers can all change outcomes, so replicate where a decision is costly and re-check panel performance over time. Sensory data are strongest when paired with instrumental measurements, such as texture or volatile analysis, that explain why a difference exists.

Conclusion

Sensory evaluation works only when the method matches the decision. A sound study uses the right test, a screened panel, controlled conditions and statistics fixed in advance, and it records every choice. For claims, documented design and raw data matter as much as the result itself.

For brands & formulators

Food Research Lab, based in Chennai, India, designs sensory studies for US and EU brands, from the decision rule and test choice to lexicon development, panel recruitment and statistical analysis. Panel sessions are run in-house or arranged through accredited partner laboratories, including consumer tests in the target market. See our sensory research and analytics service, build the results into your next launch with new food product development, and check market-specific claims with regulatory and scientific research.

Contact us with the decision you need to make and the products involved.

In this series

• Flavour trends 2026: swicy and the flavour directions US and EU shoppers are buying.
• Natural flavour labelling: FDA vs EU: how the two systems define and label natural flavourings.
• Off-odour and taint recalls: how food taint becomes a withdrawal or recall in the US and EU.
• Designing a sensory study: tests, panels and statistics that hold up (you are here).
• Flavour validation with sensory panels: a worked example from brief to decision.
• Sensory testing regulations in the US and EU: common questions answered.
• Sensory research and analytics: FRL’s sensory service for product teams.

1. International Organization for Standardization. (2021). ISO 4120:2021 Sensory analysis: Methodology: Triangle test. https://www.iso.org/standard/76666.html (cited for the triangle procedure, chance probability of 1/3, difference and similarity testing, its efficiency relative to duo-trio and its limits with lingering flavours)
2. International Organization for Standardization. (2017). ISO 6658:2017 Sensory analysis: Methodology: General guidance. https://www.iso.org/standard/65519.html (cited for the three families of sensory methods and general test controls)
3. International Organization for Standardization. (2016). ISO 13299:2016 Sensory analysis: Methodology: General guidance for establishing a sensory profile. https://www.iso.org/standard/58042.html (cited for descriptive profiling)
4. International Organization for Standardization. (2023). ISO 8586:2023 Sensory analysis: Selection and training of sensory assessors. https://www.iso.org/standard/76667.html (cited for assessor selection, training and monitoring)
5. International Organization for Standardization. (2021). ISO 11132:2021 Sensory analysis: Methodology: Guidelines for the measurement of the performance of a quantitative descriptive sensory panel. https://www.iso.org/standard/76669.html (cited for panel discrimination, agreement and repeatability checks)
6. International Organization for Standardization. (2014). ISO 11136:2014 Sensory analysis: Methodology: General guidance for conducting hedonic tests with consumers in a controlled area (amended 2020). https://www.iso.org/standard/50125.html (cited for hedonic consumer testing)
7. International Organization for Standardization. (2007). ISO 8589:2007 Sensory analysis: General guidance for the design of test rooms (amended 2014). https://www.iso.org/standard/36385.html (cited for test-room design)
8. International Organization for Standardization. (2019). ISO 20613:2019 Sensory analysis: General guidance for the application of sensory analysis in quality control. https://www.iso.org/standard/68549.html (cited for sensory quality control programmes)
9. Lawless, H. T., & Heymann, H. (2010). Sensory evaluation of food: Principles and practices (2nd ed.). Springer. https://doi.org/10.1007/978-1-4419-6488-5 (cited for duo-trio chance probability, descriptive panel size and training, the 9-point hedonic scale, ANOVA and preference mapping)
10. Hough, G., Wakeling, I., Mucci, A., Chambers, E., IV, Méndez Gallardo, I., & Rangel Alves, L. (2006). Number of consumers necessary for sensory acceptability tests. Food Quality and Preference, 17(6), 522–526. https://doi.org/10.1016/j.foodqual.2005.07.002 (cited for the worked example of about 112 consumers to detect a 10% difference at α 0.05 and β 0.10)
11. International Organization for Standardization. (2021). ISO 20784:2021 Sensory analysis: Guidance on substantiation for sensory and consumer product claims. https://www.iso.org/standard/69080.html (cited for sensory claim substantiation)
12. ASTM International. (2026). Committee E18 on Sensory Evaluation: Fact sheet 2026. https://mcsdocs.astm.org/committee-documents/E18-Fact-Sheet-2026.pdf (cited for the committee’s scope and its key standards, including E1958 on sensory claim substantiation)
13. Federal Trade Commission. (1984). FTC policy statement regarding advertising substantiation. https://www.ftc.gov/legal-library/browse/ftc-policy-statement-regarding-advertising-substantiation (cited for the reasonable-basis requirement for objective claims)
14. Federal Trade Commission. (1983). FTC policy statement on deception. https://www.ftc.gov/legal-library/browse/ftc-policy-statement-deception (cited for the treatment of subjective puffery)
15. European Parliament & Council of the European Union. (2011). Regulation (EU) No 1169/2011 on the provision of food information to consumers (Article 7). EUR-Lex. https://eur-lex.europa.eu/eli/reg/2011/1169/oj (cited for the rule that food information must not mislead)
16. European Parliament & Council of the European Union. (2005). Directive 2005/29/EC concerning unfair business-to-consumer commercial practices. EUR-Lex. https://eur-lex.europa.eu/eli/dir/2005/29/oj (cited for the unfair commercial practices framework that applies to claims)