This study aimed to investigate the link between reinforcement learning and structure learning. Reinforcement learning is a framework where humans learn to make decisions by interacting with their...Show moreThis study aimed to investigate the link between reinforcement learning and structure learning. Reinforcement learning is a framework where humans learn to make decisions by interacting with their environment, receiving rewards or punishments based on their actions, with the goal of maximizing cumulative reward over time. While structure learning is the cognitive process by which individuals acquire and internalize the underlying organizational principles or structures of information. It enables individuals to perceive patterns, rules, and relationships, allowing for effective organization and comprehension of knowledge. In the context of the environment, structure learning involves the representation and understanding of stimulus or action-outcome associations within one's surroundings. By recognizing and learning the environmental structure, individuals can better navigate, anticipate, and respond to stimuli, optimizing their interactions and adapting their behaviours accordingly. The link could be shown by the presence of a cognitive module, between reinforcement and structure learning. The cognitive module refers to the mental processes involved in acquiring, processing, and using information. Participants completed three tasks, as they are a good representation of reinforcement learning and structure learning, as well as, learning and decision-making, and have been shown to be reliably linked to e.g., specific neural correlates. The tasks are the two-stage bandit task, the weather prediction task, and the credit assignment task, Participants performed above chance in all three tasks. Interestingly, we found significant correlations between central (performance) metrics between tasks. The main analysis using Pearson correlation revealed significant correlations between the credit assignment task and the two-stage bandit task, as well as between the credit assignment task and the weather prediction task. There was only a marginal correlation between the weather prediction task and the two-stage bandit task, which disappeared after controlling for shared variance with the credit assignment task using partial correlation. The findings indicate that reinforcement learning and structure learning share common variances and behavioural metrics, such as reaction time, accuracy, performance, and outcome, suggesting a link between the two forms of learning and supporting the presence of a shared cognitive module underlying these processes. This has implications that structure learning seems to be a promising link between different learning tasks, highlighting its importance for understanding learning and decision-making across different contexts and across individuals.Show less
Anxiety is known to affect cognitive processes and learning. Studies have shown that people with anxiety tend to be biased towards negative feedback and learn better from it. This phenomenon has...Show moreAnxiety is known to affect cognitive processes and learning. Studies have shown that people with anxiety tend to be biased towards negative feedback and learn better from it. This phenomenon has been studied in reinforcement learning, but its effect on less known structure learning has not yet been studied. Structure learning refers to inferring the structure of an ambiguous environment to which reinforcement learning adds reward values on. This research paper aims to investigate if bias towards negative feedback caused by high level of trait anxiety found in reinforcement learning is also present in structure learning. We measured reinforcement learning and structure learning capacities with three well-known and validated tasks and trait anxiety with the STAI-T questionnaire in 48 students. Negative feedback bias was quantified via the win-stay-lose-shift behaviour. Using simple linear regression analyses, we found no significant effects of trait anxiety on negative feedback bias in reinforcement and structure learning tasks. Using moderation analysis in exploratory data analysis we found no significant results of the level of trait anxiety moderating the relationship between performance in the tasks and various independent variables. This could be due to the limitations of this study, for example using a behavioural instead of a neural measure to measure negative feedback bias, which raises a question of whether behavioural measures are sensitive enough to measure negative feedback bias in trait anxiety.Show less
Reinforcement learning allows people to maximize their gains by seeking rewards and minimize losses by avoiding punishments. However, the ability to appropriately learn from positive and negative...Show moreReinforcement learning allows people to maximize their gains by seeking rewards and minimize losses by avoiding punishments. However, the ability to appropriately learn from positive and negative feedback is altered in individuals with social anxiety, particularly dependent on the social context. While high socially anxious individuals are more responsive to negative compared to positive feedback in non-social contexts, there is a lack of consensus on their feedback learning pattern when being under social scrutiny. Our study investigated the relationship between different levels of social anxiety and learning from positive versus negative feedback in social versus non-social contexts by using the SELF-Symbol paradigm, a probabilistic learning task. Participants with different levels of subclinical social anxiety (N = 123) had to learn the differing probabilistic accuracy contingencies of Japanese symbols through negative and positive feedback. In the standard condition, participants were alone in a room, while in the social condition, participants were observed by an examiner. The results did not yield any significant findings, indicating no difference between different levels of subclinical social anxiety in learning more accurately from positive versus negative feedback in the non-social as compared to social condition. The results are discussed with reference to the continuous nature of social anxiety symptoms and the representativeness and research design of the current study. Addressing current limitations can lead to more hopeful future research and advancements in recognising social anxiety also in non-clinical samples.Show less
For an optical filter consisting of four cascaded optical cavities, two different methods of locking the four cavities to resonance have been investigated. The first being the classical dither...Show moreFor an optical filter consisting of four cascaded optical cavities, two different methods of locking the four cavities to resonance have been investigated. The first being the classical dither locking technique, making use of lock-in amplification to provide PID feedback upon a modulated cavity length. This method has been experimentally implemented using a microcontroller. The second method is a non-linear Q-learning approach based upon dither locking, which has proven capable to lock at least two cavities in simulations. The First method would require the use of polarising beam splitters and waveplates between the cascaded cavities to measure uncoupled reflections, while the second method would require only the total coupled reflection.Show less
Learning by trial and error is a very fundamental type of learning. In cognitive sciences, this concept is called reinforcement learning and is widely used for research, since it allows comparably...Show moreLearning by trial and error is a very fundamental type of learning. In cognitive sciences, this concept is called reinforcement learning and is widely used for research, since it allows comparably easy operationalisation. In order to translate this into computable data, models of reinforcement learning such as the Rescorla-Wagner learning rule are used. These models usually neglect disturbance in learning, which is called computational noise and reduces the precision of neural computations during the learning process. Recent research has looked further into exactly this parameter of learning and found that it indeed plays a big part and should not be neglected (Drugowitsch, Wyart, Devauchelle & Koechlin, 2016). Besides replicating a study by Findling and colleagues (2019), this study specifically looks into the role of noradrenaline in the reinforcement learning process and learning noise. Noradrenaline excretion was triggered by transcutaneous vagus nerve stimulation, while a two-armed bandit task was used as learning task. We did not find significant effects of this neurotransmitter on the learning process, however, successfully replicated the research of Findling et al. (2019). These results imply that computational noise during the learning process is still a crucial factor in reinforcement learning and should therefore be further investigated. Noradrenaline might not impact the amount of learning noise to the expected extent.Show less
Changing people’s behavior is sometimes of vital importance. However, it seems some people are not always willing to adapt, no matter how many valid reasons are provided. These observations may...Show moreChanging people’s behavior is sometimes of vital importance. However, it seems some people are not always willing to adapt, no matter how many valid reasons are provided. These observations may indicate the effect of a confirmation bias at play. This study aimed to research existence of confirmation bias in a reinforcement learning environment, while using gender as its main predictor to explore possible differences with the intent to better understand behavioral change resistance. The study had a 2 x 2 within-subjects design. 112 participants performed an instrumental learning task involving factual and counterfactual reinforcement learning, in part derived from a previous study by Palminteri et al. (2017). Following exclusion criteria, 99 participants were included for analysis (age = 25.29, SD = 13.27; 81.82% females). Contrary to expectations, statistical analysis showed no evidence of a non-zero mean confirmation bias in the population (Z = -1.260, p = .208, r = -.127). Additionally, no relation was found between the ability to adapt and confirmation bias (rs(97) = .197, p = .051), strengthening the previous finding. A difference in confirmation bias between men and women could not be supported by this study (U = 717, p = .913, r = -.01). Though these results were unexpected, they exposed opportunities to improve confirmation bias testing by controlling for autocorrelation of choice, metacognition and mental states. Finally, though results may have been influenced by effects from the current COVID-19 pandemic, it also provides a once-in-a-lifetime opportunity to study these effects by replicating this study afterwards.Show less
