There is another thread that has a bit to do with this one: What airplane do you fantasize about flying?
Flying modelers don't just fantasize about flying an aircraft, they either build or buy one and go out and fly it.
I've flown an X-32, Viggen, Me-163, a Sea Dart off water, a Space Shuttle, and there will be many more airplanes in the future. Flying models is fundamentally different from full scale flying and flying modelers don't necessarily want to fly real airplanes.
First off, unless you've had a charmed and well planned life and successfully climbed the steep pyramid to become a fighter pilot, you will never come anywhere near piloting any of the high performance aircraft you fantasize about. Most likely you will wind up having to make do with a garden variety Cessna or Piper. I do not think you can call flying in an airliner flying that you would want to do for pleasure and it is only a step above a Greyhound bus and the bus has the advantage that you can ask the driver to let you out to get away from stupid unruly people. I have only enjoyed full scale flying when I could see out the windshield or point out something interesting to the pilot and say "Let's fly over there."
The other advantage of being the pilot of a flying model is that there are very small consequences for crashes. You go and pick up the pieces and get another airplane and go back to flying. Most pilots take several airplanes to the field. If one airplane has an issue or incident that can't be readily fixed, you grab another airplane and fly again.
In full scale, crashes have dire consequences. Two of the pilots I've flown full scale with have died in crashes. A couple of weeks ago at a nearby airport, a 70 year old who spent 7 years building a Sonex homebuilt ended his maiden flight and life with a crash in a parking lot within 2 miles of the airport. I may not be the brightest guy in the room, but I'm smart enough to avoid the fatal mistakes of others.
I've lived long enough to be an old punk, and I don't feel lucky. A man's got to know his limitations.
I'll stick to flying models and hope to die many years from now at my workbench building my next airplane to fly.