My guess is that he would have been even more productive.

There is a mistaken prejudice that to be a great physicist you have to devote every moment of your life to the field. I know many great physicists, and that is certainly not the case. Nothing helps more for success than to have a happy and loving relationship with family. When I was the group leader at the Lawrence Berkeley Laboratory in astrophysics, I always insisted that the physicists who worked for me put family first. I think this contributed to the huge success of our laboratory in becoming a leader (if not the leader) in the new field of experimental cosmology. (Previously LBL was known primarily for particle physics.)

There are still the lone wolves, those who have "no life", who are occasionally productive. But they are the exception, not the rule.

There is a misconception about the life of physicists based on (I think) the fact that many high-school nerds are behind in their social development. That is not the case for successful physicists. You'll find they are enormously social, truly integrated into the larger society, and usually at the core of a very successful family.

My experience in mathematics is much more limited, but my brother-in-law Bill Thurston is considered one of the greatest and most productive mathematicians of the past 50 years, and he was a devoted family man. He loved children, and really enjoyed such things as teaching my daughter how to count binary on her fingers.