This work details an exploratory analysis aimed at identifying a specific learning circuit within GPT2-Small hypothesised to recognize and enforce the grammatical rule that words following a full stop are capitalised. The analysis focuses on the model's ability to predict the correct case of a pronoun following a full stop in a structured sentence format: "{name} is a … ."
More details of the analysis can be found here