Probably the easiest way to get started building planes these days is with foam board models, such as those popularised by FliteTest. Foam board is a 5mm thick non-beaded polystyrene foam sheet with a layer of paper on either side for added strength. You can buy sheets from craft stores for a dollar or two. It's easy to cut with a sharp knife (small box cutters are ideal) and it can be glued with hot glue or UHU POR (but some other adhesives will melt the foam). The paper layer makes it easy to paint, with almost any paint.
You can build a good starter plane with two or three sheets of foam, and with practice it only takes an afternoon or so. Your first attempt may be a bit scruffy but you will get better with practice, so plan to make several. Smallish, light models are fairly tough and easy to patch up, but knowing you can completely rebuild for very little cost is a huge confidence boost and will help avoid the common story of buying an expensive model from a shop, crashing on your first flight and giving up.
Download and print out a 3-view drawing of your favourite plane and stick it to the foam. Cut out a profile of the fuselage, and the plan view of the wings and tail. Stick them together, add some nose weight so it balances about 25% from the front of the wing and it will probably fly. Experiment with the nose weight, the angle of the tail and dihedral. You'll also learn a lot about the location and weather required for flying.
Next find a plan for a beginner RC plane in foam board, such as the FliteTest Tiny Trainer, Simple Cub or Scout. These models are relatively light and simple, and all have the wings angled up (called dihedral) which causes them to self-level, vastly increasing your chances of success.
These all have free plans under the 'resources' tab in their shop. Print them out, lightly glue the paper to the board and cut them out. Feel free to simplify the shapes slightly as they were drawn for a laser cutter.
Now for the electronics. There are some toy planes under $20 that are quite fun for a 10-year old under adult supervision but they're quite limited and not a good way to start the hobby. You don't have much control and they're very slow, so will blow down wind. You also can't reuse the gear in anything else so they're not a good investment.
A basic radio will cost at least \$30, but you'll want to upgrade that within a year so it's not really a good investment. For \$40 you can get a cheap computer radio with multiple model memories (for when you have more than one plane) and channel mixes (for flying wings, deltas, V-tails or flaps) - that's all the features you really need for a lifetime in the hobby. Most people will spend more for a 'nice' transmitter, but you don't need to. Transmitters generally come with a receiver.
The recommended sizes for the various bits are listed with the plans. You'll need to do a bit of research to work out what all the numbers mean.
Next you'll need a motor and Electronic Speed Controller (ESC). These are \$5-10 each. You will break props, so get some spares.
9 gram servos are about \$3 each. You'll need 2 initially, but it's worth having a couple of spares for crashes or more complex models.
You'll need multiple batteries, as flights are typically 10 minutes or so.
Basic chargers start at about \$20. Get one that can handle a range of sizes and cell counts, and it'll last you for years.
You won't get that lot for \$50 but you should manage to keep it under \$100. I suppose the stuff that's in the plane should be under \$50, as the expensive bits are the transmitter, charger and spare batteries!
The good news is that the gear can be used on multiple models. When one gets too damaged, or you want to try something new, just move the gear across. The transmitter, charger and batteries can all be shared between multiple models.
It's worth shopping around a bit but avoid anything that's super cheap with no reviews.