Even I thought so bro. But won't partial pressures change in such a way that Kp is a constant.
p1 + p2 + p3 = P
When Cl2 is added, total number of moles n increases. n3 for chlorine increases. so p3 increases. But at the same time, n1 and n2 are same, but n increases, so p1 and p2 decreases.