The majority of multi-agent reinforcement learning (MARL) methods for solving adaptive traffic signal control (ATSC) problems are dedicated to maximizing the throughput while ignoring fairness, resulting in a bad situation where some vehicles keep waiting. For this reason, this paper models the ATSC problem as a partially observable Markov game (POMG), in which a value function that combines throughput and fairness is elaborated. On this basis, we propose a new cooperative MARL method of fairness-aware multi-agent proximity policy optimization (FA-MAPPO). In addition, the FA-MAPPO uses graph attention neural networks to efficiently extract state representations from traffic data acquired through visual perception in multi-intersection scenarios. Experimental results in Jinan and synthetic scenarios confirm that the FA-MAPPO improves fairness while guaranteeing passage efficiency compared to the state-of-the-art (SOTA) methods.