How accurately does a Transformer store integer intervals?

I wonder how we might study how LLMs 'count' in the residual stream. Normal arithmetic doesn't necessarily get to this, because it uses the notation system to offload capacity onto tokens e.g., asking 4903283094283 + 473834482378 can be largely solved through heuristics around digit manipulation. So instead, can we look at something like: Assume that the alphabet is a ring. What is N letters after C? Of course, one could break this down into numbers, then use digit manipulation. But if we ask the model to answer without thinking tokens, could we look at the hidden states and see counting happening from layer to layer in the residual stream?