I like to think about this by comparing three pairs of hypothetical scopes, each aimed at a single astronomical target, and each with an identical camera sensor.
Pair I: two 100mm-aperture telescopes, one is f/4 (400mm focal length) and one is f/8 (800mm focal length)
Pair II: two 800mm focal length telescopes, one is f/4 (200mm aperture) and one is f/8 (100mm aperture)
Pair III: two f/4 telescopes, one is 100mm aperture/400mm focal length, one is 200mm aperture/800mm focal length
When comparing the first two, they clearly collect the same total number of photons from the target, because they have the same aperture. The f/4 has a shorter focal length, so it produces a wider view. The wider view concentrates the light of the target over a smaller number of photoreceptors, so it requires less exposure time. If you do the math, the f/4 scope image covers 1/4 as many photoreceptors, so each photoreceptor gets 4 times as many photons per unit time, and the image requires one quarter of the exposure time.
In the second pair, the focal lengths are the same, so the incoming photos get spread across identical numbers of photoreceptors on the the camera sensor. The 200mm aperture is 4x the area of the 100mm aperture, and admits 4x as many photons per unit time, so it exposes four times faster.
The third pair is where it all comes together: the 100mm admits 1/4 as many photons per unit time compared to the 200mm aperture, but concentrates them on 1/4 the number of photoreceptors due to it's wider field of view from it's shorter focal length. The two factors end up cancelling each other out, and both scopes end up requiring exactly the same exposure time. Focal ratio is the sole determinant of exposure speed.
Now - the obvious question is "why bother with a huge heavy scope if focal ratio is all that matters, I'll just buy a tiny refractor with a fast focal ratio!"
The answer is resolution; when you gain exposure speed by increasing field-of-view and concentrating the light over a smaller number of photoreceptors, you produce a lower-resolution image. The only way to get both fast exposure speed and high resolution is to combine long focal length and fast focal ratio...which gives you a resulting big aperture, and something like a big, heavy 10" f/4 reflector.
I hope that's helpful. The relationship between speed, resolution, focal ratio, aperture, and focal length is, in my experience, not very intuitive.