The model of the one-class support vector machine (voc-SVM) is based on the "origin separation approach", i.e., to add a sample at the origin to the training data for the second class and apply a maximum margin separation as known from the C-SVM. This has been proven only for hard margin separation but a clearly defined relation between the voc- SVM and the classical SVM (C-SVM) is not yet existing. In this work, the origin separation approach is analyzed in more detail. The approach reveals to be a more general concept to relate binary and unary (one-class) classifiers. We prove how its application to the v-SVM, a variant of the C-SVM, directly results in the voc-SVM. Furthermore, we apply this concept to the C-SVM and other related methods (balanced relative margin machine, regularized Fisher's discriminant analysis, online passive-aggressive algorithms) to derive entirely new classifiers. This includes variants that can be updated online which allows the application on large datasets or on systems with very limited resources.