Semantic Guidance Fusion Network for Cross-Modal Semantic Segmentation

Leveraging data from various modalities to enhance multimodal segmentation tasks is a well-regarded approach. Recently, efforts have been made to incorporate an array of modalities, including depth and thermal imaging. Nevertheless, the effective amalgamation of cross-modal interactions remains a challenge, given the unique traits each modality presents. In our current research, we introduce the semantic guidance fusion network (SGFN), which is an innovative cross-modal fusion network adept at integrating a diverse set of modalities. Particularly, the SGFN features a semantic guidance module (SGM) engineered to boost bi-modal feature extraction. It encompasses a learnable semantic guidance convolution (SGC) designed to merge intensity and gradient data from disparate modalities. Comprehensive experiments carried out on the NYU Depth V2, SUN-RGBD, Cityscapes, MFNet, and ZJU datasets underscore both the superior performance and generalization ability of the SGFN compared to the current leading models. Moreover, when tested on the DELIVER dataset, the efficiency of our bi-modal SGFN displayed a mIoU that is comparable to the hitherto leading model, CMNEXT.

Embodied Cross-Modal Interactions Based on an Altercentric Reference Frame

Accurate comprehension of others' thoughts and intentions is crucial for smooth social interactions, wherein understanding their perceptual experiences serves as a fundamental basis for this high-level social cognition. However, previous research has predominantly focused on the visual modality when investigating perceptual processing from others' perspectives, leaving the exploration of multisensory inputs during this process largely unexplored. By incorporating auditory stimuli into visual perspective-taking (VPT) tasks, we have designed a novel experimental paradigm in which the spatial correspondence between visual and auditory stimuli was limited to the altercentric rather than the egocentric reference frame. Overall, we found that when individuals engaged in explicit or implicit VPT to process visual stimuli from an avatar's viewpoint, the concomitantly presented auditory stimuli were also processed within this avatar-centered reference frame, revealing altercentric cross-modal interactions.

Impulse perturbation reveals cross-modal access to sensory working memory through learned associations

We investigated if learned associations between visual and auditory stimuli can afford full cross-modal access to working memory. Previous research using the impulse perturbation technique has shown that cross-modal access to working memory is one-sided; visual impulses reveal both auditory and visual memoranda, but auditory impulses do not seem to reveal visual memoranda (Wolff et al., 2020b). Our participants first learned to associate six auditory pure tones with six visual orientation gratings. Next, a delayed match-to-sample task for the orientations was completed, while EEG was recorded. Orientation memories were recalled either via their learned auditory counterpart, or were visually presented. We then decoded the orientation memories from the EEG responses to both auditory and visual impulses presented during the memory delay. Working memory content could always be decoded from visual impulses. Importantly, through recall of the learned associations, the auditory impulse also evoked a decodable response from the visual WM network, providing evidence for full cross-modal access. We also observed that after a brief initial dynamic period, the representational codes of the memory items generalized across time, as well as between perceptual maintenance and long-term recall conditions. Our results thus demonstrate that accessing learned associations in long-term memory provides a cross-modal pathway to working memory that seems to be based on a common coding scheme.

Effect of Piperine on Saltiness Perception

Chemical irritants, like piperine, have the potential to increase human perception of tastes and odours, including saltiness. This cross-modal interaction could help the food industry develop new salt-reduced food products that maintain their salty taste. The objective of this study was: firstly, to determine the detection threshold of piperine (n = 72), secondly to evaluate piperine's influence on saltiness perception in model solutions (n = 78), and lastly to identify piperine's effect on sensory perception of low sodium soup using temporal check-all-that-apply (TCATA; n = 75). The group mean of the individual threshold was 0.55 ± 0.15 ppm. Piperine increased the saltiness perception of the model solutions, but it also increased the bitterness and decreased the sweetness of the solutions. The piperine significantly increased the saltiness intensity of the soups (evaluated using a generalized labelled magnitude), but during the TCATA task, the salty attribute was selected less for the soup with piperine than the control (based on the average proportion of selection). The TCATA indicated that the peppery attribute dominated the participants' perception of the soup with piperine. More studies are needed to assess piperine's cross-modal interactions.

Recent advance in technological innovations of sugar-reduced products

Sugar is crucial as an essential nutrient for humans as well as for providing texture, sweetness and so on to food. But with the rise in people's pursuit of health, it is becoming increasingly clear that excessive consumption of sugar can locate a load on the body. It has been that excessive sugar is associated with many diseases, such as dental caries, obesity, diabetes, and coronary heart disease. Therefore, researchers and industries are trying to reduce or substitute sugar in food without affecting the sensory evaluation. Substituting sugar with sweeteners is alternatively becoming the most traditional way to minimize its use. So far, the sweeteners such as stevia and xylitol have been are commercially applied. Several studies have shown that technological innovation can partially compensate for the loss in sweetness as a result of sugar reduction, such as cross-modal interactions that stimulate sweetness with aroma, nanofiltration that filters disaccharides and above, enzyme-catalyzed sugar hydrolysis, and microbial fermentation that turns sugar into sugar alcohol. This review summarizes these studies to enhance the safety and quality of sugar-reduced products, and will provide some theoretical frameworks for the food industry to reduce sugar in foods, meet consumers' needs, and promote human health.

The Impact of Vanilla and Lemon Aromas on Sensory Perception in Plant-Based Yogurts Measured with Static and Dynamic Methods

The application of cross-modal interaction is a potential strategy to tackle the challenges related to poor sensory properties, such as thin mouthfeel, in plant-based yogurts. Thus, we aim to study the influence of aroma compounds possibly congruent with sweetness on the perceived sensory profile. Descriptive analysis and temporal dominance of sensations (n = 10 × 4) with a trained panel were conducted with and without a nose clip. One unflavored sample and samples flavored with either lemon or vanilla aromas were included (vanilla; 0.05%; 0.1%; lemon: 0.025%; 0.05%). Odor intensity, thick, sticky, and melting sensation, sweetness, and grain-like flavor were evaluated on an unstructured 10-cm line scale with anchors and reference samples. The results demonstrate how vanilla and lemon aromas suppressed grain-like flavor and enhanced odor intensity and sweetness. The following order was detected among samples in perceived sweetness intensity: unflavored < lemon < vanilla. The two sessions with and without nose clip differed statistically in sweetness, highlighting that the aromas impacted the perceived sweetness but not the mouthfeel in vanilla samples. The study suggests that congruent aromas could modify the perceived sweetness in plant-based yogurts; however, aroma or perceived sweetness does not impact the mouthfeel in plant-based yogurts. While the odor−taste interaction in such products is evident, the study highlights that aroma compounds alone do not modify mouthfeel.

Odor-Taste-Texture Interactions as a Promising Strategy to Tackle Adolescent Overweight

The adolescence period is characterized by a considerable risk to weight gain due to the high consumption of food rich in sugar. A promising strategy to reduce sugar consumption may lie in exploiting the ability of our senses to interact to each other (cross-modal interactions). The aims were to investigate the cross-modal interactions and gustatory function in normal-weight and overweight adolescents. Fifty adolescents (25 overweight and 25 normal-weight) were involved. Subjects rated liking and attribute intensity in pudding samples obtained by adding vanilla aroma (0.1%; 0.3%), butter aroma (0.05%; 0.1%) or a thickener agent (1%; 1.5%) to a base formulation. The gustatory function was also measured through the “taste strips” methodology. Overweight adolescents were found to have a significantly (p < 0.001) worse ability to correctly identify all tastes. Cross-modal interactions occurred differently according to their body mass index, with a significant increase (p < 0.05) in sensory desirable characteristics (e.g., sweet and creaminess) due to aroma addition, especially in overweight subjects. Furthermore, butter aroma significantly increased hedonic responses only in overweight subjects. Tricking our senses in the way of perceiving sensory attributes could be a promising strategy to develop innovative food formulations with a reduced sugar amount, which will lead to a potential decrease in caloric intake and help to tackle the obesity epidemic.

Development of tropical mixed juice with low added-sugar content: Sensory and nutritional aspects

The aim of the present study was to develop a mixed tropical fruit juice of cashew apple, acerola and melon with low added-sugar content considering sensory and nutritional aspects. Five formulations were developed varying the concentration of the different fruits. A total of 172 consumers rated their overall liking using a 9-point hedonic scale and described their sensory characteristics using a Check-all-that-apply question. In addition, the functional properties of the juices were evaluated by analyzing vitamin C, total phenolic compounds and antioxidant capacity. Results showed that the addition of sugar and fruit composition affected the sensory characteristics and consumer liking of the formulations. Samples without added sugar and with higher melon concentration showed lower liking scores (p < 0.05). In addition, the highest concentration of acerola had a positive effect on nutritional characteristics. The formulation with 50% of fruit pulp (60% of cashew apple, 30% of acerola and 10% of melon), 47% of water, and 3% of added sugar achieved the best results. The effect of aroma on sweetness perception was investigated by adding identical to natural aromas of melon, apple and pineapple to the selected formulation. However, aroma did not significantly modify sweetness perception, evaluated using a rate-all-that-apply question.

Sweetness Enhancement by Aromas: Measured by Descriptive Sensory Analysis and Relative to Reference Scaling

Sweetness enhancement by aromas has been suggested as a strategy to mitigate sugar reduction in food products, but enhancement is dependent on type of aroma and sugar level. A careful screening of aromas across sugar levels is thus required. Screening results might, however, depend on the method employed. Both descriptive sensory analysis and relative to reference scaling were therefore used to screen 5 aromas across 3 sucrose concentrations for their sweetness-enhancing effects in aqueous solutions. In the descriptive analysis, samples with added vanilla, honey, and banana aroma were rated as significantly sweeter than samples with added elderflower or raspberry aroma at all sucrose concentrations. In relative to reference scaling, honey aroma significantly increased the sweet taste compared with samples with added elderflower or no aroma at low and medium sucrose concentrations. Banana and raspberry aromas also increased the sweet taste significantly compared with the sample with added elderflower aroma at medium sucrose concentration in the relative to reference scaling. This demonstrates that the cross-modal effects observed by the 2 methods were different. In terms of the methods applied, relative to reference scaling was generally found to result in a decrease in the measured sweetness enhancement by aromas. In the descriptive analysis, the cross-modal effect of aromas on sweet taste perception was found to be significantly higher at 2.5% and 5.0% w/w sucrose compared with 7.5% w/w sucrose. These results highlight the importance of considering how references are employed in sensory analysis and how they affect cross-modal interactions.

Rapid modulation of spoken word recognition by visual primes

In a masked cross-modal priming experiment with ERP recordings, spoken Japanese words were primed with words written in one of the two syllabary scripts of Japanese. An early priming effect, peaking at around 200ms after onset of the spoken word target, was seen in left lateral electrode sites for Katakana primes, and later effects were seen for both Hiragana and Katakana primes on the N400 ERP component. The early effect is thought to reflect the efficiency with which words in Katakana script make contact with sublexical phonological representations involved in spoken language comprehension, due to the particular way this script is used by Japanese readers. This demonstrates fast-acting influences of visual primes on the processing of auditory target words, and suggests that briefly presented visual primes can influence sublexical processing of auditory target words. The later N400 priming effects, on the other hand, most likely reflect cross-modal influences on activity at the level of whole-word phonology and semantics.

The influence of task-irrelevant music on language processing: syntactic and semantic structures

Recent research has suggested that music and language processing share neural resources, leading to new hypotheses about interference in the simultaneous processing of these two structures. The present study investigated the effect of a musical chord's tonal function on syntactic processing (Experiment 1) and semantic processing (Experiment 2) using a cross-modal paradigm and controlling for acoustic differences. Participants read sentences and performed a lexical decision task on the last word, which was, syntactically or semantically, expected or unexpected. The simultaneously presented (task-irrelevant) musical sequences ended on either an expected tonic or a less-expected subdominant chord. Experiment 1 revealed interactive effects between music-syntactic and linguistic-syntactic processing. Experiment 2 showed only main effects of both music-syntactic and linguistic-semantic expectations. An additional analysis over the two experiments revealed that linguistic violations interacted with musical violations, though not differently as a function of the type of linguistic violations. The present findings were discussed in light of currently available data on the processing of music as well as of syntax and semantics in language, leading to the hypothesis that resources might be shared for structural integration processes and sequencing.