My life has never been easy. In a way, I'm grateful for that. One thing I learned over the years is that overcoming challenges makes you better. This has been coined in the phrase "What doesn't kill you makes you stronger". It sounds grim, but it actually has some real truth. When you work out with weights, you literally have to push yourself and lift until it hurts. After a few days, you do it again, but your body has made an adjustment in the form of muscle growth, and you should be able to lift the same amount with less effort. So now you have to increase the weights and do it again. Eventually of course you reach limits where the body just can't reasonablly add more muscle, but most of us won't ever hit those limits, so there is usually room for growth.
The same goes for your brain. If you want to become smarter, you have to push yourself. You need to continue to learn. One software developer guidebook that I read early in my career suggested that you needed to learn a new programming language, framework, or methodology every six months. I've been fortunate in my career that I've often worked in places where learning something every six months was pretty much a basic requirement. It was often painful and stressful, and there have been times when I even questioned whether software engineering was even the right career path for me. The problem is that when I started reading books like What Color is Your Parachute (which is actually a good read if you are having those thoughts that you might be on the wrong path), I just really couldn't come up with what I would rather be doing. Technology is the thing that excites me the most, even if it is sometimes the thing that can also frustrate me a lot. Eventually I realized that you just had to work through that learning curve and eventually you would coast down on the other side.
There is always the possibility that you might be in the wrong career, but you shouldn't infer that from those times when you are facing new challenges that you have never faced before. If you start doing that, you will continually doubt yourself because when you are starting something for the first time, it will always be harder than something you have been doing for a long time. A lot of us know how to walk around, but some of us don't know how to juggle (I'm still not really able to do that even after trying for hours with scarves). Is it because juggling is so much harder than walking? Not really. In fact, any kind of motion with our bodies that we aren't familiar with can be difficult at first, but your body develops a "muscle memory" and eventually you get to a point where you can do something without even actively thinking about the details. You will notice that when jugglers are doing their job, they aren't focused on the items in their hands; instead, they are focused on the items in the air so that they can adjust if some object doesn't fall quite where expected. Once the objects land in their hands, they instinctively know how to throw the object to the other hand and back into the air.
Even if you do feel you are in the right career area, there is a possibility that you might need a shift in direction. Being a doctor doesn't mean that you are meant to be a neurologist or a cartiologist. You might have to try a few different things to find your speciality in that area. There will be tasks that fall in the general category, but those things will become routine and will not be the big consumers of your mental energies. The big consumers are going to be the tasks that come along that put you outside your comfort zone. If they come along too often and it doesn't get better over time, that could be a sign that you are in the wrong speciality. You should face these challenges as they come, and hopefully they get easier as you learn (and then you are ready to face new challenges).
The main point is to not burn yourself out facing challenges that rob you of the enjoyment of what you do overall. It is ok to have a limit and say to yourself "this is the wrong direction for me". It can be humbling to do this, because we can feel like it is a weakness, but skills are not a single dimension scale where every person can be compared fairly. Some people are much better at memorizing or reacting in short amounts of time or learning things quickly. The key is to find the skills that you excel at, and once you have found those, home in on them and make them your core. Continue to work on other skills by challenging yourself, but think of it more as investing in the future rather than trying to measure up to some non-existent standard. Setting goals for yourself that are realistic can be a useful tactic - you can always set another goal to raise the bar higher, so keep it reasonably set so that you can periodically achive the goal and then set new ones.
I sometimes forget techniques like this and will set goals that are far too optimistic, and this will quickly lead to frustration. The more frustration you feel, the harder it is to motivate yourself to continue facing challenges. You also have to give yourself a break once in a while. I'm often hard on myself, saying "I should be able to do this!", and feeling like a failure, discounting all the amazing things that I have learned to do in the past. Sometimes, you will fail (i.e., not achieve some goal in the expected timeframe), and instead of taking that as a reflection of yourself, you need to take a step back, evaluate whether the goal was realistic, and think about other possible ways of achieving the goal. In short, you get knocked down, but you can get up again, and as long as you do that, they will never be able to keep you down.