On bortle 2-3 that gain does not exist. I'm not using IR filters, maybe that's the problem.
You are expecting something that isn't logical.
If your skies are already dark enough to see something, then NV is not going to suddenly change it to something else.
If your conditions and equipment is such that you can already see the spiral arms of the Whirlpool galaxy, NV can make it a bit brighter, but it is not going to suddenly reveal things that are not there to begin with.
In my estimate, under dark skies, NV will give about double the aperture in benefit. Now you can already see M51 from very dark skies with a 4" scope, so seeing it with NV is only going to be maybe the equivalent of seeing it with an 8" scope. Yes, it will be a bit brighter, but it isn't going to suddenly look like a 2 hour image capture.
I use the example of the telescope observations in the Deep Sky Observer's Guide. They will often give an eyepiece impression of a subject from different size telescopes so that an observer can consider what they might be able to see under excellent conditions. Here are the impressions for M100 from two different size scopes.
4/6" Scopes-75: Messier 100 is considerably bright in small telescopes. Its elliptical halon elongated 4' x 3' ESE-WNW, and its core quite luminous, small, and round.
8/10" Scops-100x: Messier 100 has a bright, small core embedded in a 5' x 4' ESE-WNW oval halo The halo is generally smooth and its edges diffuse, but some variations in brightness can e seen near the center of the core
12/14" Scopes- 135x: This is a fine, face on galaxy with a 6' x 5' ESE-WNW halo surrounding a bright core....................................
The point here is that all three telescopes can already see M100 but what happens is that as the aperture is increased, the extension generally increases slightly and things are a little brighter.
And this is all you should expect from NV. If you can already see it, then the best NV will typically do is to show it with a bit more extension or perhaps a bit brighter and with a bit more structure, similar to they way going from the 4" scope to the 8" scope does.
If you are already seeing a galaxy as large as the catalogs generally list it, then what else would one expect from night vision. If I can see the full extension in a given scope, then NV can't make it bigger than it is.
My own experience is that NV will give an increase in galaxy extension (if the galaxy is not already seen at full extension) equivalent to what one would get using conventional telescopes with double the aperture. If you can see it, you can see it, but now the question is how much of the faint extension will you see with glass vs image intensifier.
So, the reason why NV does not produce the same result under dark sky as it does under lighter skies is that you are already seeing much of the galaxy, so there is not nearly as much room for improvement as with galaxies under brighter skies, where the long pass filter reduces light pollution allowing the galaxies to be seen they way it would appear from much darker skies.