We explore further the suggestion to describe a pre- and post-selected system by a two-state, which is determined by two conditions. Starting with a formal definition of a two-state Hilbert space and basic operations, we systematically recast the basics of quantum mechanics - dynamics, observables, and measurement theory - in terms of two-states as the elementary quantities. We find a simple and suggestive formulation, that ``unifies'' two complementary observables: probabilistic observables and non-probabilistic `weak' observables. Probabilities are relevant for measurements in the `strong coupling regime'. They are given by the absolute square of a two-amplitude (a projection of a two-state). Non-probabilistic observables are observed in sufficiently `weak' measurements, and are given by linear combinations of the two-amplitude. As a sub-class they include the `weak values' of hermitian operators. We show that in the intermediate regime, one may observe a mixing of probabilities and weak values. A consequence of the suggested formalism and measurement theory, is that the problem of non-locality and Lorentz non-covariance, of the usual prescription with a `reduction', may be eliminated. We exemplify this point for the EPR experiment and for a system under successive observations.