Seriously. I remember first getting into Deity and realizing it's basically just exploiting intimate knowledge of how the AI works. The actual max difficulty is Prince, where the AI doesn't get bonuses, and it's so terrible at actually pursuing an agenda it's not very challenging.
I am a bit hopeful that VII's decoupling leaders and civs will force the AI to be a bit more generally good. At least make it so you don't know exactly what sort of tactics to use from the first turn you meet it.
I think there are some exceptions. Like Kitfox publishing Dwarf Fortress. Taking weird little indies and giving them an art / usability budget to become more accessible and, in turn, make the OG devs a bunch of money. Nobody loses.