I like Get Your Goat, but I vastly prefer 50,000 B.C - love it, in fact. No album with pop songs as good as "Survival," "Saddest Day of My Life," and "Man Who Rolls" deserves the "completists only" label.
GYG and 50,000 are both simpler than Pony Express Record, composition-wise, but one critical difference between the two is Nathan Larson, whose distinctive guitar playing and composition style had tons to do with the evolution on PER and hadn't joined the band until after GYG. The other is that, while GYG is simpler by virtue of being a lead-up to PER, 50,000 is a deliberate pulling back. It's their attempt at making a power-pop album using some of the bizarre tricks they perfected on PER.
Actually, the album closest in style to PER is Mind Science of the Mind, Larson's semi-solo album - it's kind of the midway point between PER and 50,000 with more of a blue-eyed soul bent (and no Wedren on vocals, of course). That might be harder to track down, though.
Knowing Jake's tastes a bit, I'm thinking he'd also really like their soundtrack for First Love, Last Rites, which is far more straightforward than 50,000. They snagged a ton of great guest vocalists (Robin Zander, John Doe, Jeff Buckley, Liz Phair, the folks from Low, etc.) and seem to be having fun with genre-hopping.