Working on your genera fitness is great for most people. This would include your average working out like running, lifting weights, or doing yoga. But general physical preparedness (GPP, as we call it in the biz) should only be the baseline for those who are training for a specific sport or event. GPP focuses on health, but not on performance. And while you typically need to be healthy to perform at the highest level, being healthy does not necessarily translate to athletic achievement.
For example, in most sports, it helps to have one or more of the following attributes: strength, explosiveness/power, flexibility, agility, aerobic and anaerobic conditioning, etc. You can easily work on all of those at the gym by yourself. But no matter your strength or conditioning level, you still need sport-specific training to work on your skills if you want to excel at your sport. It doesn’t matter how many times you hit the weights or run, those are not going to make you run a better route, use better technique, or improve your hand-eye coordination. Being more agile will not make you a better shooter, swimmer, golfer, etc.
Always work to have GPP. But if you are competing in a sport, make sure you take the time to focus on your sport-specific skills. Most of the time, if someone is highly skilled and in decent shape, they will outperform someone with low-to-moderate skill but is in great shape.