If you can get a proper decompiler, it should be possible to get the Flserver source code in order to improve it. But I haven't seen any decent decompiler that's up to the task. Other than that, you can dissassemble it, but as the word says, you get the source in assembly language. ugh.
It would be easier to make a new server by sturying its behaviour instead. But such a project would not be easy, and will require an entire dev team to do it.
(If you find any mistake in my English, please let me know via a PM)
(Really, I speak terrible English, so please, tell me if I make mistakes. I'd like to improve it a bit )