In 99 cases out of 100, you won't be able to hand craft assembly better than a good compiler can - partly due to compilers being much better and partly due to the skill level required. 20 or 30 years ago compilers weren't as good and a reasonably competent person could craft more optimised assembly but these days compilers are pretty damn good and you need some extra level of ability to best the compiler.

However, there's still that 1 time out of 100 and given how resource intensive ffmpeg is, it's worth spending that extra time to hyper optimise the code because it'll pay off massively.