I can think of a few reasons the distance is set that way, and as Hribek pointed out earlier, the value that says "Distance NPC's appear/disappear" is in Freelancer's executable or a .dll file, so it is possible to change the value.
Anyways, a few of those reasons are to prevent additional sever lag/cut down on the operations FL Server has to do and to make outrunning NPC's possible (if you didn't notice, their thrusters go faster than players' thrusters)
My vote on this is to keep the NPC view distance the way it is, as it's already annoying enough being chased down by miscellaneous NPC's that have unlimited CD's.
But if the value were to be increased, make it only like 3.5 K or something close to the original value.
