Modric, 33, won his third successive Champions League in May before helping Croatia to a first World Cup final.
Former Brazil and AC Milan forward Kaka, in 2007, was the last player other than Messi or Ronaldo to win it.
Ronaldo, who joined Juventus from Real Madrid in the summer, came second.
Barcelona forward Messi finished fifth, with Atletico Madrid frontman Antoine Griezmann and PSG attacker Kylian Mbappe – both of whom won the World Cup with France – third and fourth respectively.
Modric was one of eight Real Madrid players in the top 30, including Wales international Gareth Bale in 17th.
Lyon and Norway striker Ada Hegerberg, 23, won the inaugural Women’s Ballon d’Or while Mbappe, 19, became the first winner of the Kopa Trophy – awarded to the best under-21 player and selected by former Ballon d’Or winners.
The top 10
1. Luka Modric (Real Madrid and Croatia)
2. Cristiano Ronaldo (Juventus and Portugal)
3. Antoine Griezmann (Atletico Madrid and France)
4. Kylian Mbappe (Paris St-Germain and France)
5. Lionel Messi (Barcelona and Argentina)
6. Mohamed Salah (Liverpool and Egypt)
7. Raphael Varane (Real Madrid and France)
8. Eden Hazard (Chelsea and Belgium)
9. Kevin de Bruyne (Manchester City and Belgium)
10. Harry Kane (Tottenham and England)