Yeah, I assumed Jim joined the army instead of going to college, but I'm not sure why. It's sort of implied in "Brother's Keeper," but not conclusively. Or perhaps he went to school while he was in the army? Meh--Jim's backstory is pretty much a mess anyway.

You're right--now that I look at the dialogue, it seems more likely that it's an official big-brother thing. So what, Jim has known Danny since Danny was like eight? No wonder he was so upset.