I'm going to give you the answer first:
Yes, it is worth calculating the approximate range if you're building a (very) long-range aircraft. However, theoretical calculations will usually be such a crude approximation of reality that you can't rely on them, and the actual maximum range will depend on a myriad of factors besides what you can consider in the approximation.
If you're flying short-range, near obstacles (or inside of them!), doing acrobatic maneuvers, etc, the local conditions, attitude of your drone, its proximity to things, etc at each particular moment will affect the performance of your video link orders of magnitude more than any theoretical range. Things such as where your antennae are pointed and how directional they are, what kind of noise reduction your receiver has, and where you are standing suddenly become extremely important and intimately connected to the configuration of the building or other environment that you're flying. Sometimes, just moving your receiver several centimeters to the side could make a huge difference in the clarity of the received signal, or lowering your transmitter's power can make the link perform better due to less multipath interference.
So what people usually do is they have some known values of range and general performance for certain combinations of gear, and they consider how much a certain change to that gear will affect this performance. As in, swapping a 3dB antenna for an 11dB option might give you an 8dB stronger signal, which corresponds to 4dB (~2.5x) more range in open air versus the alternative under the same conditions; or it might give you the ability to punch through that wall (while your less-directional omni antenna covers the rest of your surroundings). Maybe upping the VTX power twofold could give you sqrt(2)=1.41x more range even through that wall. Or it might give you more interference instead. So after making that hypothesis, you implement the change, go and test it and see if it really works this way; It might or it might not!
Okay, now comes the nerdy part.
I don't know the exact formula you're speaking about, but I'm going to reason my way through this just for kicks.
So, our goal is to have a signal at the receiver that's higher than its sensitivity (which represents the weakest signal that the receiver can detect and demodulate). Our signal path is as follows:
Original signal -> transmitter's amplifier gain -> (feedline loss) -> TX antenna gain -> path loss from travel between antennas -> RX antenna gain -> (preamp?) -> receiver.
(scroll it).
The signal is multiplied at every step except where it is reduced while traveling through the air, so we need the product of all gains multiplied by the loss factor to be higher than the sensitivity.
So our max-range equation is:
AmpGain*Gain1*Gain2*PathLoss = RxSens.
In the simplest, ideal case, where we assume that there's nothing around except the two antennas (free space), the losses from transmission are inverse-square:
FSPL = wavelength/(4Pi * distance²), from which we get:
AllGains*wavelength/(4Pi * Distance²) = RxSens;
4Pi * Distance² = AllGains*wavelength/RxSens;
MaxDistance = sqrt(AllGains*wavelength/(4Pi*RxSens)).
And here comes the caveat.
We don't live in an ideal world.
We're not transmitting in free space. There might be buildings or trees or rain or anything else in the way, too. Worse, it doesn't even have to block your line of sight, as objects near the line will cause diffraction of your signal which can interfere with your reception if they are within the fresnel zone. The ground itself will often be the biggest source of such interference at larger distances, sometimes reducing the propagation factor to an effective inverse-fourth-power law.
Also, consider that the max-gain vectors of your antennas are not always pointed at each other; while you can use an antenna-pointing device on the ground, your aircraft will pitch and roll, reducing the transmitting antenna's gain in your direction.
The earth's atmosphere can affect your signal significantly as well, and it will do so differently each day, both in terms of how much and in terms of how.
For all of those factors, while you can predict roughly how they are going to affect your range, there's a lot of things about them in each particular location and at each particular time that you simply don't know. And since signal-modifying factors stack by multiplying, the discrepancy can get pretty big pretty fast.
Your receiver isn't ideal, either. It will probably start to corrupt the signal in various ways as you approach its sensitivity level. You're unlikely to ever see this, however, because long before it reaches that critical stat of your receiver, your precious signal will drown in...
Noise.
Noise is the worst enemy of anyone working with high-gain systems. It's the third law of thermodynamics personified. Thousands of manmade and natural radiation sources around the planet bombard everything around with random radio waves. Even if you exclude those, outer space is actually quite noisy as well. Your receiver itself generates noise internally; in fact, for most gear including your receiver sensitivity itself is defined by that internal noise level. Still, external noise tends to be orders of magnitude more powerful, and whatever the source, you don't want any of it.
The total strength of noise as received by your gear is called the noise floor, and that floor is lava. No matter how sensitive the receiver, the moment your signal level dips below the noise floor, it's gone: still out there, but you can't tell it apart from the noise to decode it properly anymore.
Depending on the nature of your signal, it can even become totally unintelligible before it reaches the noise floor, at several dB above it. Usually, the higher a signal's bandwidth (which is closely linked to data transmission rate), the sooner it's overcome by noise. Some advanced digital transmission modes can extract a message from well under the noise floor, but in order to do that they transmit several bytes per minute.
Video is a notoriously high-bandwidth type of data, both in the internet and in radio. Analog video especially so, as it has zero compression. So it gets disrupted by noise rather easily. On the other hand, zero compression implies lots of redundancy in the data, and redundancy allows for error correction. Do you know what's one of the best signal processing and error correction machines in the world? The human brain! Even when a video signal is almost entirely eaten up by static, we can still see shapes in it. So for video, the intelligibility threshold seems to be just at the noise floor mark, give or take a couple dB.
You can fight noise in various ways. Usually, if most of it is coming from certain sources on the ground, just using a directional antenna helps by attenuating signals from unwanted directions. Sometimes, if the noise is just the regular background entropy kind, it doesn't. A good frequency selectivity of your receiver (i.e. its ability to ignore signals that aren´t the target frequency) also helps tremendously.
An interesting way of fighting noise is signal-stacking, as done by receivers like RapidFire or True-D X: Instead of simply switching to the antenna with the highest signal, they use signals from both and merge them into one. Since the useful signal is the same and noise is random, the signal adds up while the noise mostly cancels out. Astrophotographers use the same technique to get stunning clear pictures of the sky, except that they have to stack hundreds of extremely noisy photos. The digital communication modes I mentioned earlier also work this way by sending multiple copies of the same transmission.
Video is still a high-bandwidth signal, however, and you've only got two copies of it to stack, so the gains from the likes of Rapidfire are far from dramatic, but it'll win you a couple of additional dB over the noise floor. Beware, though: such receivers depend on a special sync signal in the video to do the adding correctly, and when the noise eats that... You'll probably get a blank screen.
That's a common caveat with any technology that does the error correction for you, by the way, including digital video: it gives a cleaner and more pleasant signal while it does, but will probably give up before you do. So this kind of error correction might not be the best for very long-range flights.
Final word
To appreciate the sheer expanse of outcomes that you might possibly encounter once you start pushing the range envelope, it's probably best to look at ham radio operators.
On one end, they will casually use 50-watt transmitters to just talk locally within a range of no more than several tens of kilometers, and sometimes use kilowatts to punch through unfavorable conditions. On the other end, they can literally talk to the other side of the earth on fractions of a watt and bounce signals off of the Moon. They've been doing all of that crazy stuff for decades and had invented things that were thought impossible.
And the reason for all of those achievements is lots and lots of experimentation. Without testing what works and what doesn't, a theory is useless. So, if you decide to jump down this rabbit hole, start to think like a ham. Make that estimate to the best of your ability, assemble what you think should work best, and then go out and fly it. See how it works in reality, and how it differs from your expectations. Make hypotheses. Go out on different days and in different places to see how the conditions change your outcome. Then change something in your system and see what happens. Write it down. Make your own theory.
And, most important of all,
Have fun!
