I am currently a bit worried that I may be the weakest in the cohort
or that it'll take me a very long time to understand simple things and
that I may feel incompetent and like I don't deserve to be there.
If this is the mindset you're carrying into the doctoral study you're going to have a very bad time. Because even evidence is not pointing to that direction, you can still be subconsciously looking too hard into clues that match your expectation. I think you may have some introspection to do.
While competitions exist, doctoral study is closer to an individual backpacking than a 5k race. You chart your route, pack your stuff, get onto the track, deal with contingencies, take rest, enjoy, get lost, find new way, and perhaps enjoy again. Your supervisor would be like that shop keeper at a travel gear store giving you different levels of advice/service, or be like your GPS; most of the time, you're responsible for yourself.
But what of those horror stories about adviser pitching their students to fight over resources? About those shaming experiences for those who didn't get student grants? Or corridor gossips about so and so being dumb? Yes, they may happen as well, but these are largely out of your control. What I found useful to deal with most of the adversity are two things: the will to grind and move forward, and a set of internal personal values.
Familiarize yourself with the school/program expectations. Attend workshops and read up about opportunities. Build professional networks. Work with your advisor, lay out plans and milestones, and work towards completing them. If things go wrong, come up with some suggestions and consult the advisor.
Don't outsource all your sense of success or accomplishment to others. Think larger goals: focus on what career you'd like to craft, what kind of students you'd like to produce; not if you're the top pick of the faculty among your peers. Draft and keep working on a personal mission statement, and be very clear to base them on your own values: leave the world a healthier place, make education more equitable, etc. And not "people would think that I am the best this or that..." I can't give a whole introduction here, but do search online on how to craft a personal mission statement. Do that soon if you don't have one; it's super useful as a personal compass.
Up play your strength: e.g. as you have worked as an educator, you may also consider applying for teaching assistantship. Being a TA would allow you a great chance to review the materials, pocketing some money, and enriching your post-graduate teaching portfolio.
There will be stellar candidates, and plenty of people brighter than you (which, is exactly the reason you should be there to begin with.) If they are nice, befriend them and perhaps you just made the first step of a life-long research collaboration; if they are not nice, learn from their strengths and move on. Be aware that the playing field will be unfair: some will have more experience, more resources, more popularity; and that's exactly how the rest of the system works. You worked before, you'd understand.
While I don't want to trivialize your worry, the fact is that this move opens a lot more opportunities to development, expansion, enrichment as it does disappointment, anxiety, and frustration. In other words, you have a lot more important things to plan and be prepared for. Take your head out of the birdbath, stop drowning yourself, because if you turn around, you'll see that you have a sea to chart.
I'd recommend a couple titles that you may find useful:
"The Professor Is In" by Karen Kelsky: This title discusses many things on how to be academically marketable. Kelsky, a seasoned admission director, shares a lot about getting ready to be a professional scholar.
"F*ck Feelings" by Michael Bennett and Sarah Bennett: This, or any other similar title that had recently conquered the Self-help book shelves, sometimes provides useful approach to deal with feelings. While it did not change my life, I did find seeing feelings through a comedic lens soothing.
"The 7 Habits of Highly Effective People" by Stephen Covey: Classic oldies that you can probably get a used one for 25 cents. It serves to give a good framework of seeing what success is. It's also one of the earlier titles (that I know of) which mentions the idea of "personal mission statement".
"Networking for People Who Hate Networking" by Devora Zack: I hate networking or pitching myself. This book puts much of my anxiety into perspective.
I wish you best of luck. Take the dive and really enjoy the time. Move forward.