Modern societies depend on a growing production of scientific knowledge, which is based on the functional differentiation of science into still more specialised scientific disciplines and subdisciplines. This is the basis for the paradox of scientific expertise: The growth of science leads to a fragmentation of scientific expertise. To resolve this paradox, the present paper investigates three hypotheses: 1) All scientific knowledge is perspectival. 2) The perspectival structure of science leads to specific forms of knowledge asymmetries. 3) Such perspectival knowledge asymmetries must be handled through second order perspectives. We substantiate these hypotheses on the basis of a perspectivist philosophy of science grounded in Peircean semiotics and autopoietic systems theory. Perspectival knowledge asymmetries are an unavoidable and necessary part of the growth of scientific knowledge, and more awareness of this fact can help avoid blind and futile struggles between scientific perspectives, and direct efforts toward more appropriate ways of handling these fundamental knowledge asymmetries. Concretely, we show how different kinds of scientific knowledge, expertise, disagreement and learning can be correlated to the perspectival structure of science, and propose how polyocular communication based on (second order) observations of the observations made by specialised perspectives can be used to handle such perspectival knowledge asymmetries. This can help overcome the observed problems in carrying out cross-disciplinary research and in the collective use of different kinds of scientific expertise, and thereby make society better able to solve complex, real-world problems.