Physician you'll need some math but lots of chemistry and biology. For mathematician, pay attention in algebra. There are just about no mathematician jobs, BTW.

You haven't said *why* it's really hard to learn.

You will need to narrow down your "best field". Go to the Library and take a Vocational Aptitude Test. it will help you narrow your field to the one you are best suited for, the one where you have the built in natural talent, the one which you will enjoy the most and make the greatest contribution.